MCLoader & XPort

[Joe Rocci]

Does anyone have any experience using Mecanique's MCLoader to bootload a PIC over a network by using a Lantronix Port at the PIC end of the link?

I can get the loader to work fine with a direct serial connection, but MCLoader seems to bail out as soon as I try to load over the network. However, I can get Hyperterminal to communicate with the PIC over the network just fine. Could the problem be that MCLoader doesn't have any tolerance for network delays?


Joe

[Charles Linquis]

I have tried the combination that you are using, and gave up. I think you are correct in your belief that the timeouts in MCLoader are too short.

[Joe Rocci]

Charles,

Thanks for the quick reply!

Do you know of a bootloader combo that WILL work through an Xport over a network? The 'network' will eventually include the internet.

I'm using a PIC18F2620.

Thanks
Joe

[Charles Linquis]

I know of no bootloader combo that will work over an XPORT. I tried uChip's but that doesn't work because it requires a third wire to trigger it. I suppose that with a little re-coding it might be possible to use it.

When I tried MCLoader, I used a serial -> Ethernet utility to telnet into the XPORT. I'm supposing that is the method you tried as well.

My guess is that the only way to make MCLoader work with this setup would be to disassemble it and modify the timing loops. In an attempt to avoid that mess, I contacted Mecanique and asked if they would be interested in modifying their code, or licensing the source so that I could do it. They said they weren't interested in either one.



Check my private msg to you.

[Jumper]

Hi,

If size if of less importance you can always consider to write the loader your self in PBP. As long as it is for the 18-series it seems to work.

/me

[Joe Rocci]

Jumper,

Actually, I made an inquiry about that very idea on this forum about a week or so ago, but I don't recall getting any suggestions.

If you can point me to an example of how someone else did this, I'd really appreciate it

Thanks Joe

[Charles Linquis]

http://www.oshonsoft.com/pic18bootloader.html

[Jumper]

The end result after doing this is that we will have a loader in the high part of the memory (in this example at adr DEC 60160). Every time the PIC starts it will jump to the loader, there we check if we want to enter the loader or jump back to resume the application software.

This is ONLY tested on 18-series (PIC18F4620).

1: We need to get the library out of the way since PBP usually puts this in the low part of the memory. The easiest way to do this is to modify the PBPPIC18.lib file. Look at these original lines.

ifdef LOADER_USED
LIST
ORG RESET_ORG+8 <---- modify this line
NOLIST
endif

Here i modified it to point to the address where I want my loader to be.

ORG 60160 <---- now all code will be from this adress (Block 470)

Then by using DEFINE LOADER_USED 1 in the LOADER SOFTWARE this part will be activated. So Any software written with this define will be programmed at this address. Not just 4 words in as the usual function. Make sure the loader adress is a multiple of 128, (look at the Erase block below)

By doing this modification the normal use of this DEFINE is hijacked so it will no longer work as the manual sais.


2: Where to put the application code, this is usually adr 8 for normal loaders.

Since the 18-series needs to have the flash memory erased before we can program it again there are som things to think about. We can only erase entire blocks of 128 BYTES at the time, if the adr in the ERASECODE command points inside one block ALL bytes in it will be erased, EVEN positions prior to the address if they are a part of the same block. Blocks are multiples of 128. In the first 3 positions of Block 0 our loader software has written the GOTO instruction and we do not want to loose that. One way is to use READ code and read the information before you ERASE it but this still leaves the danges that something might go wrong and we end up in a position where we no longer can goto the loader.

Plan B: We use DEFINE RESET_ORG 64 in the APPLICATION SOFTWARE to move the application software to start from Block 1. Then we never have to erase block 0. Sure, we lost the space for 28 instructions but if size is a big problem, a loader in PBP is not the right way to go anyway.

To return to the APPLICATION software just use @GOTO 0x0040 (goto adress 64)

Loader code:

DEFINE LOADER_USED 1

pause some
check if we want to change software
if not jump back

for as long as we need
check address so it is not above 60160 and overwrite the loader
Erase block
program block

END


Application software:

DEFINE RESET_ORG 64

and then it is just as always


When developing the APPLICATION SOFTWARE you can just use a normal programmer to program the code if you dont want to use the bootloader. But as soon as the BOOTLOADER is put in the PIC we must use that way of programming otherwise the programmer will erase the goto intruction and we will never enter the loader.

I know it is not a working program example. I use this loader to recieve an encrypted data file that the loader will unpack and write to the flash. The HEX file you want to put in the PIC has to be parsed by either the Loader or by a VB software, I use a VB software to parse and encrypt the HEX file and the loader just decrypts it and write the data.


If there are any questions please let me know but this should at least be a start.

/me

[Joe Rocci]

Charles,

Great insights and recommendations!

I'm still digesting the implications of this as it pertains to actual coding, but right now I have a very basic question. You said a couple of times in your narrative that the minimum erase and programming block size for 18F flash is 128 bytes. In reading the 18F2620 datasheet, it says the minimum block size is 32 words/64 bytes. Can you reconcile this seemingly conflicting info?

Also, Jumper referenced the Oshonsoft loader. On their web site (http://www.oshonsoft.com/pic18bootloader.html) they provide Basic code for a loader written in their variant of PicBasic, which is somewhat different. One of the differences is that they begin the loader program with a "StartFromZero" directive, which seems like it does something similar to what you described. Can you comment?

Thanks again for the great suggestions.

Joe

[Jumper]

Hi,

It should of course be 64 bytes (remark to my self: never post anything before lunch!!!) This is why we have our application offset to 64 (the 128 number came from an other part of my project)

Different Basic usees different ways to put the software in the desired addresses. This way by changing in the PBP18LIB file is maybe not what the designers had in mind but I can not find any better way. PBP always puts the library first (probably because older PICs have pages and stuff inside) even if you use an assembler ORG to try to move it. We can not share the library with the application program since it changes depending what functions are used in the application code. So we need to relocate the library.


PBP supports the use of a loader, but there is less support to write one.

It is really not that hard to make the program, just remember that we always goto the loader, then in the loader we decide if we want to program the PIC or start our application software.

In the application software we only need to add the DEFINE RESET_ORG 64 and then we can forget that we will use a loader later. If the PIC has the loader inside or not doesn't matter for when we have started the application software we have passed that stage.

If you want you can of course jump to the loader from your application software but then you would need to have either en EE-prom flag or an address in the flash to indicate that the loader should enter the programming phase. (PIC starts up -> goto loader --> check EE-prom -> enter or jump back and restart the application software)

More questions? Just post them....

/me

[Jumper]

This might help, rename it to .bas

/me

[Joe Rocci]

Jumper,

A few basic questions about your file:

1) "DEFINE LOADER_USED 1 'Sets loader at address defined in Library file"
Does this refer to a modified library file as explained earlier in this thread?

2) "RB6 var PORTB.6; RB7 var PORTB.7"
I assume these are just 'flag' bits to tell you which mode the chip is in?

3)"IF temp=$FFFF then STOP"
What happens after the 'STOP'?

4) "@GOTO 0x0040"
Why 0x0040?'

5) "'Erase before writing and write full blocks 64 bytes"
Do you suggest writing the erase and program code in assembly, or can it be done in PBP?

Joe

[Jumper]

1: YES, just like that, put in the address where you want to loader to be in the LIB file.

2: Rb6 and Rb7 are connected (same for both): RB6 ---> 470 ohm ---> LED ----> GND or any other pin can do the same.

So when they are high the led is turned on, just for fun. (and to see what is happening)

3: STOP makes the PIC to STOP dead as I don't want it to jump to an "empty" address and then run thru the entire empty program memory and end up in the loader again. If you want to you can go back to INIT to loop inside wait for the byte.

4: Because your application software should start 64 bytes into the program memory. This is done by DEFINE RESET_ORG 64 in your application software. This is so we don't have to erase block 0 and therefore can never end up in a position where we can no longer access the loader. DEC 64 = HEX 0040 and the @GOTO must be a hex number.

If you dont want to do this you can set the LOADER_USED line back to the default in the LIB file and then your code will end up starting from address 8 as it probably do today by DEFINE LOADER_USED 1. But then you need to store the data you want to keep from block 0 before erasing it and then write this data back together with the new data. This will take more than 28 intructions so you will lose less space if we start from adr 64.



5: ERASECODE address will erase the entire block that the address variable is a part of.
WRITECODE will write a BYTE or WORD variable to a position in the memory but you have to write 32 WORDS or 64 BYTES for it to be transferred to the memory.

So get the data from somewhere (an array is usually good) and write it with a for-next loop. I don't think WRITECODE will accept an array as a valid argument so you better load it in a temorary variable and use the temorary variable in the WRITECODE command.

Check the manual about these commands. There is no need for any assembler at all in this project, just use pbp!

I am not really sure how assembler commands will work with this setup, you might end up in the wrong RAM bank or something. I have not tried it and I dont' plan to either. As long we stay with PBP commands it should be safe.


/me

[Joe Rocci]

Thanks Jumper, this should be an interesting project. I'll work on it as I get time, and I'll let you know how things are progressing.

Thanks again!

Joe

[Joe Rocci]

Jumper,

Just had a look at the .lib file. It seems to that, instead of permanently redefining an existing DEFINE (which would have the effect of making existing projects incompatible), I could just create some new DEFINEs to do this job. Is there any problem with that?

Joe

[Joe Rocci]

This thread, which I find very interesting, has really gotten off-topic from the title. Does anyone know how to split this off into another thread, maybe titled something like "Implementing a Bootloader in PBP?

Joe

[Jumper]

ifdef MAKE_LOADER
LIST
ORG 60160 ; type the adress for loader here
NOLIST
endif


I added this Define in the LIB file and changed in my code to :

DEFINE MAKE_LOADER 1

and it works just fine. Now we don't have to hijack any other defines. So now we have a new DEFINE in PBP.

If someone brilliant can find a way to send the address variable from PBP it would be great. I would like it to look similar to the:

DEFINE RESET_ORG 64 where you include the value to be compiled.

Like:

DEFINE MAKE_LOADER 60160

then there would be no need to change in the LIB file if you wanted an other address for the loader. Is that possible????

/me

[Joe Rocci]

Jumper,

I made the .lib modification and modified your code to read back 16 memory locations starting at 60106. Data coming back from the read is all FF's, so it looks like there's no program located there.

Comments?

Joe

================================================== ======

DEFINE MAKE_LOADER 1 'Code will begin at 60160


DEFINE HSER_RXSTA 90h
define HSER_TXSTA 24h
DEFINE HSER_BAUD 9600
define OSC 8
OSCCON = $FF
TRISB=%00111111
TRISC=%10111111


RB6 var PORTB.6
RB7 var PORTB.7

TX Var PORTC.6
RX Var PORTC.7
'************************************************* ********************************************
I VAR BYTE
x var byte
Flash var word
Temp var word

'************************************************* ********************************************

Goto INIT


'************************************************
jump_back:
'Flash=64
'READCODE flash, temp 'Read the value at Flash adr
'IF temp=$FFFF then STOP 'If it is $FFFF we have no application code in the PIC so STOP
'@ GOTO 0x0040 'Start application
'*************************************************

'Here we decide if we should enter the Loader or not receiving a byte.

INIT: Pause 200 'wait for things to settle
Rb6=1 'indicate we are waiting for a character
HSERin [x] 'if nothing after 5 sek go to jump_back
if x="L" then
PAUSE 100
Goto Loader
ENDIF 'If it is what we want to get then enter the laoder
goto Jump_Back 'everything else jump_back

'************************************************* ****************

Loader:
FOR I = 0 TO 15
READCODE (60610 + I),X
hserout [$0D, $0A, ":", HEX(X)," "]
NEXT I
RB7=1 'Indicate we have entered the Loader
RB6=0
pause 200

'Here we can recieve new data and write it to the FLASH
'Erase before writing and write full blocks 64 bytes
'

goto Jump_Back 'When finished goto
END
'************************************************* ********

[Jumper]

Check this part:

FOR I = 0 TO 15
READCODE (60610 + I),X <--------------- wrong
hserout [$0D, $0A, ":", HEX(X)," "]
NEXT I


should be READCODE (60160+i),X

But there is something else too, I need to check some things..... gimme half an hour

/me

[Joe Rocci]

Jumper,

Well, while I fail to see any syntactical or other error in the way I wrote the line, I did cut-paste your suggested line into my program - no change

Am I failing to understand something about how READCODE works, or is there no code at the memory locations where there should be code?

Joe

[Jumper]

Try the original HIJACKING way since I couldn't get it to work with our own DEFINE the second time I tried it.

If you are using MPLAB you can look under view and program memory to see what your code looks like.

Try the original approach:

1: Change the LOADER_USED line in the LIB
2: DEFINE LOADER_USED 1 in your code
3: Compile
4: Check program memory, there should be code in position 0 and 2 then FFFF until $EB00

Syntax in PBP can be a little bit tricky. Some functions don't really like that you have a formula as argument. They want to have a WORD or BYTE only.

/me

[Jumper]

DEFINE LOADER_USED 1
DEFINE HSER_RXSTA 90h
define HSER_TXSTA 24h
DEFINE HSER_BAUD 9600

This is with the DEFINE LOADER_USED in the lib file like this

ifdef LOADER_USED
LIST
ORG 60160 ; Own loader address
NOLIST
endif

Any luck now??

/me

[Joe Rocci]

Jumper,

It seems like that's better. Here is dump result:
:4
:0
:9E
:AA
:FD
:D7
:AE
:50
:D8
:80
:E1
:EF
:75
:F0
:E9
:50

But we should be able to do this with a custom DEFINE, no??

Joe

[Jumper]

I didn't really look into this because I never use any other loaders but maybe this forum can come up with an answer.

This will not be solved today since we have passed Midnight a long time ago here.

Maybe moving this to a new thread might be a good idea after we have solved these problems. Then all our misstakes will be forgotten......

/me

[Joe Rocci]

Jumper,

Looking at the .lst listing. the code appears to start at $EB00, even though the ORG directive specifies 60160

joe

[Jumper]

EB00 is 60160

Remeber some numbers are in Decimal and some in HEX

/me

[Joe Rocci]

I might be doing the math wrong, but by my calculation EB00 is

E(15) * 4096 + B(11) * 256 + 0 + 0 = 64256

Joe

[Joe Rocci]

Ooops!!!!

how stupid of me...of course E is 14 decimal and you are correct!

So the code is assembling at the correct location....hmmm

Joe

[Joe Rocci]

Jumper,

I went back to the other method (your special DEFINE), and here's what the (partial) assembly code looks like. It apears that something's putting another ORG after the one we defined. I wonder if it's another INCLUDE file that's doing it:

00037 LIST
00EB00 00038 ORG 60160 ; type the adress for loader here
00088 LIST
00089 ; Oscillator is 8MHz
01140 LIST
000000 01141 ORG RESET_ORG ; Reset vector at 0
01150 LIST
000000 EF66 F000 01151 goto INIT ; Finish initialization
02058 LIST
000004 02059 HSERIN
02060 ifdef HSER_CLROERR
02061 btfsc RCSTA, OERR ; Check for overflow error
02062 bcf RCSTA, CREN ; Toggle continuous receive to clear error
02063 bsf RCSTA, CREN

[Joe Rocci]

Jumper,

OK, I think I have it. In the .lib file, there was another ORG statement further down that was apparently overriding the special ORG DEFINE that we put in. It looked like this:
ORG RESET_ORG ; Reset vector at 0

I commented it out, and all seems to work OK now, org'ing at 0 if nothing is specified and ORG'ing at 60160 using your special DEFINE to select the desired starting location.


Joe

[Jumper]

We need the DEFINE RESET_ORG in the lib file when we write the application software to move the code to a suitable address (I like 64).

Have you tried if that part works?


/me

[Charles Linquis]

I think this is a worthwhile project, but don't forget those of us that use parts with more than 64K of code space!

[mister_e]

I agree. Funny... those i did with more than 64K, as now, don't use the bootloader, just ICSP. Case by case i guess.

[Jumper]

Sure it would be nice if all PIC's could be using this loader but:

How do we address the memory above 65535 ($FFFF) in READCODE and
WRITECODE when PBP only have 16-bit variables. It seems to me that Microchip has outgrown PBP, 32-bit variables in PBP would be nice.

ICSP is probably the way to go, unless it is done in ASM.

Let's start with the 64K devices and see if we run into any mines. We are still far from finished.....

/me

[Joe Rocci]

Good morning all!
(well morning my time...where is everyone else from?)

An update on the project so far:

1) We have a custom PBP DEFINE to specify where our bootloader should be in memory (needs to be checked further per Jumper's previous)
2) I've written and compiled test code with the new DEFINE, and the resultant program truly does locate where it should be and run properly
3) I wrote a little program that resides in 'remote' loader memory (ORG'd at 60160d) and writes a block of data to 'low' memory space starting at 0. This data block consists of 256 incrementing values starting at 0h and ending in FFh.

I'm having a small problem with the last part. The data that I read back from memory is correct for the most part, but every 8th memory location has a "0" instead of the correct value. I don't have the code in front of me to attach, but I'll send it when I get back to my office later this morning. Basically though, the sequence of events is:

A) Dump the 256 byte memory block to Hyperterminal using READCODE
B) Do a 64-byte block erase using ERASECODE
C) Rewrite each byte in the block just erased using WRITECODE
D) Repeat for a total of 256 bytes
E) Dump the re-written memory values to Hyperterminal

If you want to cogitate on this for a while and speculate on what I might be doing wrong, I'll get you with some more details later. I think that once we get through this, we're on our way!

BTW, in regard to programming bigger (>64K) parts, maybe we can fall back to using a little bit of embedded assembly code. Check out the OshonSoft web site for an example of how simple it can be:

http://www.oshonsoft.com/pic18bootloader.html


Joe

[Joe Rocci]

Ok, as promised, here is the code and the result that I described earlier:

CODE:

INIT: HSEROUt [$0D,$0A,$0D,$0A] 'Make space between display dumps
FOR I = 0 TO 255 step 16 'Dump 512 memory locations to screen
hserout [hex4(i), ":: "]
for j = 0 to 15 '16 bytes per line
READCODE (i+J),X
hserout [":", HEX2(X)," "]
next j
hserout [$0A, $0D] 'Line break after each line
next i
HSERin [x] 'Start on "L" from keyboard
if x="L" then
Goto Loader
ENDIF
'************************************************* ******************************
Loader:
for j = 0 to 255 step 64 'Erasecode does 64 bytes per block
erasecode j 'Erase a block
for i = 0 to 63 'Write 64 bytes
writecode i+j,i+j 'Write the block that was just erased
next i
next j 'Keep going until done

goto INIT 'Print out the result and wait




and here is a partial of the screen dump:

0000:: :00 :01 :02 :03 :04 :05 :06 :00 :08 :09 :0A :0B :0C :0D :0E :00
0010:: :10 :11 :12 :13 :14 :15 :16 :00 :18 :19 :1A :1B :1C :1D :1E :00
0020:: :20 :21 :22 :23 :24 :25 :26 :00 :28 :29 :2A :2B :2C :2D :2E :00
0030:: :30 :31 :32 :33 :34 :35 :36 :00 :38 :39 :3A :3B :3C :3D :3E :00
0040:: :40 :41 :42 :43 :44 :45 :46 :00 :48 :49 :4A :4B :4C :4D :4E :00
0050:: :50 :51 :52 :53 :54 :55 :56 :00 :58 :59 :5A :5B :5C :5D :5E :00

[Jumper]

This is a BAD example because we Erase the first block and can never enter the loader again after RESETTING the PIC.

But atleast this code writes what it should.


for i=0 to 4 'erase first 5 blocks
Flash = n*64
ERASECODE Flash
next i

for n=0 to 255
FLASH=n
WRITECODE FLASH, n
next n


I didn't really look into why, but your software does not write correctly. As I have said a million times:

DO NOT USE FORMULAS INTO READCODE AND WRITECODE

It just doesn't seem to work (no arrays, no cute stuff, nothing except as below!)

use ONLY a WORD variable for the address and a BYTE or WORD variable for DATA


/me

[Joe Rocci]

As I have said a million times:
DO NOT USE FORMULAS INTO READCODE AND WRITECODE
-------------------------

Sorry Jumper,
I wasn't around the first 999,999 times you said this. However imlementing it only in the WRITECODE part did seem to fix the problem.

Joe

[Jumper]

Glad it worked out, PBP has is hidden features :-)

Have you tried to write an application software starting from 64 yet? It would be fun to know if that part works with the new DEFINE.


/me

[Joe Rocci]

After fooling around all day trying to figure out why WRITECODE sometimes works properly and sometimes doesn't (see my other WRITECODE thread), I think I'm finally getting somewhere.

Using Jumper's suggestion for using DEFINE RESET_ORG 'X', I built a tiny program that loads into memory location 64 (40h). I then copied the object code from the hex file that resulted from compiling this little program and I inserted it into DATA statements in my loader program.

Also per Jumper's idea, I modified the LOADER_USED define in the .lib file so that it located the loader program high up in memory - at 60160 to be precise. My loader program reads the program bytes out of the EEPROM where DATA put them, and WRITECODEs them into their intended executable Flash memory locatinos starting at 64 (40h). It then uses an "@ GOTO 64" to jump to the "downloaded" program.

Eureka - It works!

So now the basic mechanisms are working. What has to be done next is to work out a data transfer protocol and write a small VB application to send the .hex file contents to the PIC. This won't be trivial I think, because each line of the hex file has to be parsed and a 'padding' scheme will have to be developed because not all lines of the .hex file are equal in length.

Thanks for the help Jumper. My available time to work on this will decline after this week but I fully intend to pursue it until it's complete

Joe

[Jumper]

I downloaded VB 2005 Express and it is way better than any other VB i have used. Support for COM port is back again and it is really easy to read HEX files either into byte arrays or to read the lines into a string array.

ReadAllText and ReadAllLines are really usefull functions

Since each line after the colon has the number of data bytes we can easily get the data bytes from the string. Just remember that they are stored LB, HB in the HEXFILE. Rearrange that part and send them out as a string to the comport.

In the pic collect them with HSERIN [HEX4 x] and they are ready to be written to the FLASH.

Do this for all lines until the next line in the HEX file has an address that is different from 16+last line's address, then we know we have gotten all data.

It can be done in any VB but in 6.0 and 2005 there is a comport object, 2003 does not have that.

and it is free

/me

[Joe Rocci]

Jumper,

Thanks for the insights.

I'm a VB6 sorta guy and I have the complete MS Visual Studio, so that's what I'll do the application in.

My previous comments about parsing the .hex file lines and padding the short ones might have been premature. I was thinking about how you'd implement this if the PIC was coded in assembler, in which case I believe you generally write 64 byte blocks. In that case, the file lines would have to be read, buffered, parsed, and disassembled & reassembled into the 64 byte chunks the PIC likes to work with before transmitting to the PIC. I still haven't figured out all the limitations of WRITECODE, but it seems to me that, assuming the intended write block has been ERASECODE'd, you can write one byte at a time? In this case, it's quite simple as you said, and data can be fed to the loader in chunks as small or big as it can buffer in RAM. Do you agree??

Joe

[Jumper]

Now here is where the 128 (in the beginning of the thread came in)

I get a string from the PC, erase the blocks and write the data. When this is ok I send a command to the PC to show we are ready for the next data chunk.

Then we can fiddle around in the PIC without missing any data.

So I just gathered the start address for the HEX file (in our case allways 0040) and the collected the data from the hex file and arranged it so it will fit the HSERIN HEX4. Then I feed out the characters from the string in as big pieces I want to have.

At the same time you can calculate a checksum for each chunk that the PIC will report back before sending next data. It is also possible to encode the string so only your loader will understand the data. With the code protection turned on the application software can not be copied (unless they break the code of your encrypted file, just pick any algorithm from the internet and it would't be easy to do)

Or invent you own "super safe" algorithm as the DVD people did :-) (not that good idea)

But at least we have solved the basics of how to trick PBP into making some kind of loader. Too bad we had to hijack the old define but, with our new loader who needs that define anyway. Then we can hope the really clever people in this forum might figure out a way to include our own defines.


/me

[Joe Rocci]

I got a little time today to work on the coding for the PIC end of this loader system. Programing entirely in PBP (with the exception of one line of assembler for the final 'GOTO" that starts the downloaded application), it looks like this loader will fit into about 600 bytes of PIC18 memory space - not as bad as I expected.

The down-side is that there are other loaders, coded in assembler, that are as small as 100 bytes. The up side is that this one's written entirely in PBP, is supportable by virtually anyone, and is therefore infinitely scalable for any desired level of functionality by anyone in this community.

In the coming days, I'll try to find time to write the VB6 host application that drives the loader. Once something that functions is available, I'll try to find a way to post the sources so that this can become an open-source, community supported tool.

Joe

[Jumper]

If you got all of the loader including library to fit inside 600 bytes I am impressed. I had expected much more.


/me