when 128k is not enough

[richard]

Some files to ponder on . done on a pic16f1825 8 MHz , two led flashy blinky things with a pointless int_int

mikro c 181 rom words 11 bytes ram int latency 19 us
pbp 3 324 rom ram ? int latency over 58uS

when I look at the hex dump both versions end at rom 00b8 so i'm not sure how to really compare the size

[richard]

On the other hand, doesn't that make for unpredicatble latency, ie one interrupt it needs to save 5 variables, next interrupt it needs to save 10? I'm asking, not arguing by the way ;-)

turns out to be quite true but still near 3 times faster on average

mikroc latency for the previous example ranges from 16-21uS
pbp is not rock steady either there I a 1-2 uS variation

[richard]

when I look at the hex dump both versions end at rom 00b8 so i'm not sure how to really compare the size

I'm wrong again was looking at wrong file . the pbp version is 341 words c version 181.
also the pbp version has a redundant for/next loop removing it saves 28 words
surprisingly the int latency has increased by 4uS by removing it ???

[longpole001]

this is very interesting reading , cheers richard

[richard]

updated files and arduino mega328 version too , there were some minor errors but the results are unaffected

arduino int latency 10.75uS code size 1794 bytes

[HenrikOlsson]

Hi Richard,
I started writing a reply based on the previous files you posted, in which I found what looked to me quite substantial differences but now you've changed them so I'm just going to ask instead.
Are all three versions functionally equivalent, ie. are they doing exactly the same thing?

It still looks to me as if there are functional differences between the C version and the PBP version - I'm only comparing the versions for PIC, haven't looked at the Arduino code. It's my understandning that the follwing two snippets should be functionally equivallent:
C-version:

while (1){
     if (d_mode)
     pins_on=va+vb;
     else
     pins_on=vc+vd  ;
    for (x=0;x<5;x++){
        latc =   latc | pins_on  ;
        delay_ms(200);
        latc =   latc & all_off ;
        delay_ms(200);
        }
    if (!d_mode)  shuffle();
       d_mode= !d_mode;
 }

PBP-version:

action:
  pins_on=a+b
  for x = 1 to 5
    latC =   latC | pins_on
    pause 200
    latC =   latC & all_off
     pause 200
   next x

   gosub shuffle
  
   goto action

In the C version there's a d_mode variable which you're checking, I don't see that in the PBP version - what am I missing. (I truly suck at reading C code!)
It won't have an impact on the Interrupt latency (at least not for PBP) but if we're comparing compiles size we're going to have to make the code as functionally equal as possible or the results won't say much, no matter which way it swings.

/Henrik.

[richard]

they are now . I had a few copy paste errors where the second delay fell off the c version and the pbp still had a unnecessary loop in it as well as the led initialising to 0 line misplaced into the loop .I just saw the leds flashing and assumed all was equal . the outcome is largely unaffected anyway . I was really just quantifying the interrupt latency for my own edification , I still don't think the code size comparison is valid a fairer test probably needs a few serout2's and a lcdout and a onwire or two to be meaningful .
the mikro c is now 193 bytes and the pbp 315 other than

[Ioannis]

Richard,

since you do not have PBP variables in ISR, you can set this:

INT_HANDLER  INT_INT, _TL, PBP,yes

as this:

INT_HANDLER  INT_INT, _TL, ASM,yes

and see the difference.

Ioannis

[richard]

In the C version there's a d_mode variable which you're checking

your right of course
that's the way I eliminated the redundant loop in the c version
looks like I skipped that step for pbp

code size now 336 and functional equivalence restored
thanks for noticing that

*  Name    : UNTITLED.BAS                                      *
'*  Author  : [select VIEW...EDITOR OPTIONS]                    *
'*  Notice  : Copyright (c) 2014 [select VIEW...EDITOR OPTIONS] *
'*          : All Rights Reserved                               *
'*  Date    : 29/11/2014                                        *
'*  Version : 1.0                                               *
'*  Notes   :                                                   *
'*          :   pic12f1822                                                *
'****************************************************************
#CONFIG
cfg1 = _FOSC_INTOSC
cfg1&= _WDTE_ON
cfg1&= _PWRTE_OFF
cfg1&= _MCLRE_ON
cfg1&= _CP_OFF
cfg1&= _CPD_OFF
cfg1&= _BOREN_ON
cfg1&= _CLKOUTEN_OFF
cfg1&= _IESO_ON
cfg1&= _FCMEN_ON
  __CONFIG _CONFIG1, cfg1

cfg2 = _WRT_OFF
cfg2&= _PLLEN_OFF
cfg2&= _STVREN_ON
cfg2&= _BORV_19
cfg2&= _LVP_OFF
  __CONFIG _CONFIG2, cfg2

#ENDCONFIG
 
DEFINE OSC 8 

include "dt_ints-14.bas"
include "REENTERPBP.bas"
asm
INT_LIST macro
      INT_HANDLER  INT_INT, _TL, PBP,yes
      endm
      INT_CREATE
      INT_ENABLE INT_INT
ENDASM
    osccon=$70    '8 mhz
    anselA=0        'dig i/o 
    ANSELC=0
    TRISC= %11110000
    TRISA= %011111
    APFCON0.7=0
     OPTION_REG.6=0
  a  var  byte
  b  var  byte
  c  var  byte
  d  var  byte
  pin_mode var bit
  d_mode var bit
  x var byte
  all_off var byte
  
  pins_on var byte
  led var lata.5
 
  
  
  
  a=1        ;portC.0
  b=2         ;portC.1
  c=4         ;portC.2
  d=8         ;portC.3
  pin_mode=0
  d_mode=1
  all_off=~(a+b+c+d)
   led=0
  latC =   latC & all_off
  
 action:
  if d_mode then
  pins_on=a+b
  else
  pins_on=c+d
  endif
    
  for x = 1 to 5
  latC =   latC | pins_on
   
  

  pause 200

  latC =   latC & all_off
  
  pause 200

  next x
  if !d_mode  then  gosub shuffle
     d_mode= !d_mode
  goto action
  
   shuffle :
   pin_mode = not pin_mode
  if pin_mode then
  swap a,d
  swap b,c
  else
  swap b ,d
  swap c ,d
  endif
  return
  
TL:
 led=!led 
@ INT_RETURN

[richard]

INT_HANDLER INT_INT, _TL, ASM,yes

int latency now same as or even slightly better than c around 16-20uS

but its not always an option

[HenrikOlsson]

No, as long as the C compiler also always handles all needed context saving (ie SFR and "system variables") automatically the fair comparison in this case is to have the handler defined as PBP.

I'm pretty sure that a big chunk of the 336 bytes is comming from DT-INTS and I think that part of the compiled/assembled code will be constant while (and I'm guessing) in the C version, due to the intelligent context switching (as the call it in the MikroC manual), as you add more code to the ISR and/or Main() it'll need to do more context saving/restoring and therefor needs more instructions (=more latency) and more RAM (of course).

I think that DT-INTS always saves everything, for good and bad - obviously.

As I stated earlier, comparing these things isn't easy and is never going to be exactly fair.

[richard]

I have notice the c latency can be as low as 4us I first thought that was a glitch but it happens fairly often now that I'm looking , so the odds are that increased complexity in the isr will spread the range further.

there is a thing called chipkit-pi a pic 32(28 pin dip) clone of the arduino , its designed to plug directly into a raspberry pi and has a free ide based on the arduino ide . it has some limited arduino library compatability .
that's my next toy , perhaps thats the step up past 128k ,midrange c gets you nowhere extra in that regard

[longpole001]

this thread has been a bit hijacked although interssting

having alook at the 18f87k22) 80 pin chip , it tells of the eternal memory - section 8.5 addressing feature for both 8 and 16 bit mode , this option appears to be afar better one for accessing an 40- 128k of flash surely than changing horses ???????

[richard]

as the complexity increases I'm always expecting this result

[pedja089]

It's easy to add external memory, but how you would compile code for whole project, and then split code in 2 segments?
Only way I see this could work is to use only ASM, so you could manually enter address of each subrutine. But would MPASM allow you to assembly code larger than 128K?

[Demon]

this thread has been a bit hijacked although interssting

having alook at the 18f87k22) 80 pin chip , it tells of the eternal memory - section 8.5 addressing feature for both 8 and 16 bit mode , this option appears to be afar better one for accessing an 40- 128k of flash surely than changing horses ???????

When they mention external flash, do they mean an SD card?

Robert

[longpole001]

thats intersting question , i expect not , it seem to me more like adding an external flash chip is liuke adding eeprom , great but how o use it more than just storage

[tumbleweed]

When you use the external memory bus of the 87K22, you connect it up to a regular 16-bit parallel flash device and the operation is totally transparent to the program. Program code < 128K uses the internal flash, and addresses above that go to the external memory chip... up to 2M bytes

Assuming that the compiler doesn't have any internal program size limits, there's nothing special you have to do when writing your program... you just end up with a really big hex file.

There are some limits/things to note:

- external memory is slower, so you have to reduce the system clock. The only reference I've found in the 87K22 datasheet is a note that says"Fmax=25MHz in 8-bit external memory mode". Using a 16-bit wide memory may ease this a bit, but there's nothing in the data sheet to back that up.

- you'll likely have to have a bootloader that can program both the internal and external flash devices since the regular ICSP programming tools like the pickit won't work to program the external flash.

- likewise, you can't use the ICD debugging tools, and the MPLAB simulator doesn't support the EMI either.

- there's an errata w/the 87K22 for the EMB. The CE chip select pin won't be extended if you use wait states, so you have to make sure the flash is fast enough to run in a single cycle.

[longpole001]

The only reference I've found in the 87K22 datasheet is a note that says"Fmax=25MHz in 8-bit external memory mode". Using a 16-bit wide memory may ease this a bit, but there's nothing in the data sheet to back that up.

this would mean the whole chip has to run at 25mhz , assuming that you external mem/ addressing is used for compiled code by the sounds of things

Assuming that the compiler doesn't have any internal program size limits, there's nothing special you have to do when writing your program... you just end up with a really big hex file.

i am not so sure that it does , i have always though it was tied to the chip selected when compiles , as such the chip size limits would apply and give errors in the compile , not warnings and thus not compile ???

[towlerg]

i am not so sure that it does , i have always though it was tied to the chip selected when compiles , as such the chip size limits would apply and give errors in the compile , not warnings and thus not compile ???

I guess you'll need to edit BLOCK_SIZE in the device file.

George

[HenrikOlsson]

I'm not saying there isn't a way to make it work but I'm pretty sure PBP doesn't support it natively.
You'd need to somehow fool the compiler into thinking the device has more memory than it has (if that's even possible). Then the compiler and assembler generates a single .hex file which you'd need to split at the correct location, program the first half into the PIC itself and the second half into the external memory device using some other type of device programmer.

I don't know the format of the .hex file, if there's any sort of header and/or checksum or anything like that. If there is, then simply cutting the file into two most likely isn't going to work and some sort of software would need to be written to repackage the large hex into two with the correct format.

Again, I'm not saying it isn't possible and I look forward to hear about any experiments. All I'm saying is that I don't think it's going to be straight forward.

/Henrik.

[Demon]

I'd head over to the MeLabs forums for something so technical.

Robert

[HenrikOlsson]

I've just tried a couple of things....

In MicroCodeStudio I selected an 18F1220, which has 4k of FLASH.
I started to add code and compile untill I got close to 4k (I got to 4095 bytes).
At that point I expected to get an error message if trying to add more code but I didn't. Since I knew I was at the limit I added started to add @ NOP, one by one, each adding two bytes to the compiled/assembled size as shown in MCSP. Finally, at a reported size of 4107 bytes, it no longer seemed to get "any bigger". However, the compile and assemble process finnishes without errors and the actual .hex file keeps growing.

I then started MPLAB, selected a 18F1220 as the device and imported the .hex file. Needless to say, the last line displayed is at adress 4094
I then found this blog post which discusses the format of the actual .hex file.
What it tells me is that, as expected, it's not just a matter of simply splitting the file at the right location and I have no idea if the compiler/assembler process actually generates the correct adresses etc for calls outside of the chips "normal" memory.

In MPLINK helpfile there's a short section on how to set the linker up to build C and assembler Project for use with external memory. How that could be applied to PBP (if at all) is beyond me at this point.

I'd definitely drop MeLabs a line but considering you've already mentioned you're up against the 128k limit at their forum I would've expected them to suggest using external memory if at all possible.

/Henrik.

[longpole001]

ill have a check with melabs on this possible option to use extended flash , but the chip running at only 25mhz to support the extended memory hanging from it , would be a killer , as i do need the speed of 64mhz

but if charles can comment on this option when using pbp , would be good , as its not something that i think has been asked before ?

[longpole001]

re reading the datasheet , section 31-1 - note1 Fmax= 25Mhz in 8 bit External Memory mode - FOR VDD VALUES 1.8 TO 3V ,

so i would think that as long as VDD is higher than 3V then Fmax = 64Mhz

[longpole001]

also the CE for wait states on external memory bus has been fixed in C5 release of silicon

[longpole001]

APPLICATION NOTES ON EXTENDED MEMORY BUS
http://ww1.microchip.com/downloads/e...tes/00869b.pdf

[tumbleweed]

I probably should have been clearer when I said there's nothing special you have to do when writing your program.
There was talk of having to use ASM and split the program into multiple sections, manually enter addreses, etc.

To enable MPASM to override the device size so you can use external memory you need to add a __MAXROM statement to the device .INC file, like so:

__MAXROM 0x1FFFFF		; 2Mbyte

I don't use PBP much anymore, so I just did that with an old 18F6680 (64K) program I had and built a 1.2MByte program using PBP 2.47


As you've all mentioned, the trick here is what to do with it once you're done. You have to get part of the hex file into internal memory, and the rest into the external flash chip. If you somehow want to program the devices using an external programmer, there are tools you can use to split the hex file into the needed segments. I'd recommend the hexmate utility that you can get for free w/some of the Microchip tools.
Personally, I think you'd be better off using a bootloader program running in the internal flash and just let the device itself program everything.


some other comments...

also the CE for wait states on external memory bus has been fixed in C5 release of silicon

Thanks for that. I didn't realize it had been fixed.

re reading the datasheet , section 31-1 - note1 Fmax= 25Mhz in 8 bit External Memory mode - FOR VDD VALUES 1.8 TO 3V ,
so i would think that as long as VDD is higher than 3V then Fmax = 64Mhz

I think you're mis-reading that statement. There's a period in that sentence that you've deleted.
In Figure 31-1 the note reads:

Note 1: FMAX = 25 MHz in 8-Bit External Memory mode.
For VDD values, 1.8V to 3V, FMAX = (VDD - 1.72)/0.02 MHz.

That puts a limit of 25MHz on FMAX when using 8-bit external mode. Period.

If you look later in the datasheet at Figure 31-6 it shows an external 8-bit bus cycle. At cycles Q3 and Q4 it shows fetching a byte from each of the byte-wide chips in a single Q period. At 64MHz, each OSC Qn is 16ns! That would be one fast flash chip!

Unfortunately, the datasheet doesn't show a fetch using a 16-bit device. App note AN869B shows a 16-bit fetch, and the timing is a lot more reasonable since it uses Q3 and Q4 together to fetch a single 16-bit value. If you combine that with a wait state (Figure 12) then you'd have a bus cycle of ~120ns, which would match up with the speed of most flash IC's.

The thing is you would still only have roughly a single cycle (16ns) to latch the multiplexed address info. I doubt that the PIC IO pins would actually be able to keep up with this speed. 25MHz would give a single Q time of 40ns, and a bus cycle of ~160ns. You could probably run a bit faster than this, say perhaps 32 or 40MHz before the IO pins quit working properly, but as I said there's nothing in the datasheet to guarantee that.


This is all pretty advanced stuff. You'd have to decide if it's worth all the bother or not. Of course, porting 120K of existing PBP code to something else is a lot of work too!

[pedja089]

I just did quick test.
Compiler doesn't mind compiling code larger than 128K. Actually, there is no way that compiler could know hex size, before assembling. So far so good.
But after assembling, assembler returned error that generated hex is too large. After adding __MAXROM 0x1FFFF to P18F67K22.inc, hex file is generated without any error.
I have done "secure" bootloader for PIC18F67K22, so when I need more memory, there is simple solution.
Thanks, tumbleweed!

[longpole001]

thanks for that ,
i been trolling the microchip site and this has been asked before using c , with various results , 2 threads i read eventualy gave it up.

16bit mode would be the only option to consider , and i have a raised a ticket with microchip to pin down timing on the 16bit option for the bus , but the timing you mention seems likely ( microchip also give an example - in C of setup for the addressing , but it was not of much help

but the underline factor is the speed drop of the cpu the 25mhz- or even say 32mhz to allow for the extended memory bus and that not really an option for this project. 64Mhz it works well and i really cant drop it back now , just running out of code space


so again its looks like either is porting the code/ learning new compiler / c probablyor Microbasic maybe but i dont hear good things so far / new hardware chips / probalby go ARM

OR

having 2 cpus / split off some of the hardware / then develop and complete interface for the cpus with interupt handing / some varable transfering / sub routines indexing interface so the one cpu can call each others routines and what at this point looks like a lot of duplicated code in both cpus and work

but it seems is the only way forward with PBP .

the code has taken most of this year to write / test , and even though porting would be alot easier than starting again , i really dont want to , it would take another 3-4 months to learn new script language and then port everything

EMB option looks so good an option over the above options if PBP can be made to use it well , but droping the cpu speed down to use the EMB is not really an option ,


this really sucks some days

cheers

Sheldon

[tumbleweed]

If Microchip gives you any good information, please post it here. I'd be very interested.

Sounds like you're between a rock and a hard place. Splitting a program up to run on two uC's doesn't necessarily make things easier, or save code space. It certainly doesn't make it more reliable, that's for sure. Quite the opposite.

If you're finding a lot of duplication of code then that's not a good sign. Try and split things up so that more of the work can be offloaded to the other cpu. If they have a lot of functionality/data to share things might not get any better.

Good luck!

[longpole001]

one interesting thing i was thinking about in relation to i/o pin speed for 18f67k22 it allows the system clock at 64mhz full speed out on Reference clock output , signal is bit unclean and puting through a buffer helps a lot ,

microchips reponce for 16bit EMB operations would be the key

[longpole001]

and i can see a way to only slow down the EMB operations without slowing down the system clock ???

[Demon]

one interesting thing i was thinking about in relation to i/o pin speed for 18f67k22 it allows the system clock at 64mhz full speed out on Reference clock output ...

Good idea, except you'd end up with code that runs at 64MHz when it's on the PIC, and 25MHz when it's from the external source no?

Robert

[longpole001]

thats true , how ever i could ensure that certain code / process are run in the area , and the lower priority / execute time perhaps ???, but it would slow down the overall performace of those fuctions

but i dont think there is away to control the speed of the EMB apart from system clock rate and the amount of wait states for access

[tumbleweed]

but i dont think there is away to control the speed of the EMB apart from system clock rate and the amount of wait states for access

Correct.

how ever i could ensure that certain code / process are run in the area

If you're talking about dynamically changing the clock rate I have a hard time seeing how that would work. I'm not sure how you'd locate certain functions in the external memory space, but beyond that you'd have to watch out for what happens during interrupts. ISR's (and all the peripherals) would run at two different speeds depending on when the intr occurred. You'd almost have to disable interrupts when executing code out of the slower region.

[longpole001]

yep i forgot for moment about interrupts , the bottom line is what ever speed you can run the EMB , then thats the top speed of the whole pic from that point.


i might say adding EMB is not something you really want to do, unless you really really had to ,

for starters the chip is a 80 pin chip , its costing about $6 , for under 50 units plus then 16bit flash at 90ns , at about $2 each , plus interface glue logic , and add to opcb design , manufacture etc etc

also if you needed some of those 80 pins for microprossors i/o use and they are also used for the EMB interface some addtional interfacing is also required.

its clear you will reduce your speed down of the pic to 25mhz , perhaps 32mhz if your lucky ,( waiting on MIcrochip responce for 16 bit EMB spec )

so the 128k limit of 8 bits chips , from the options covered here in this thread , you are far far better to

A. Do Not Use PBP -
if you ever suspect or likely ever to expand a project and there a chance you need more that 128k of internal code space for the project, and want to use the chips 64mhz speed , . then drop any plans to use pbp from the start ,
as the there are very few ways you can go forward , and for the most part your project will be higher cost / effort than choosing another platform , even if you know pbp well ,
there is no upward way forward and there is not likely to be so for sometime.

B. if no other choice and you find your self at the 128k limit and you have used PBP , then you seem to have 2 options

1. You poor sole - its time to consider the longer road and learn C coding in the end it more transportable , port your code to another plateform and never look back but loose a lot of time on the project your on now

2. stick with PBP but add 2 or 3 CPUs , split the fuctions/ supported hardware over the cpus , however your not ever going to get a double / triple of extra space
the overheads are large depeding on the fuctions / interface code overheads on each cpu and that maintaing more cpu's coding make for lot more work , updating etc , your only stalling the above options but it may be enough
to get you over the line, and time to reconsider ever going down the PBP again for project that has the chance to get past 128k.


as said this sucks some days

will update when microchip gives a meaning full reply for Fmax on the EMB at 16bit interface
cheers sheldon

[Demon]

This is pretty much why I think modular from day #1 on a project. When you consider the low cost of PIC chips, I don't mind having a PIC that scans a keyboard, another slave outputs to a LCD and have a master send/receive data via USART at 115,200 using Darryl's interrupts. It's robust, proven and works over long cable lengths.

I used I/O expanders when I first started but it's not even worth it any more. Today, a PIC costs less, runs at 64MHz, has waaaay more pins, can use complicated I/O logic (charlie-plexing) and can still do any of the features available on that model PIC. Much more bang for the buck.

[longpole001]

H1 guys ,

well further feedback from Microchip on the EMB of the 18f87k22 series , indicate that at 16bit which they do not state the the Fmax speed in the Datasheet , is that the device will run the EMB at 64mhz (16Mhz instruction rate internal)

Silicon C5 fixes the wait state Errata to allow for upto 3 wait states on the chips connected to it., however code running in the EMB would be slower than on the PIC as a result of the wait states

Microchip states that the expected chips would need if running at 0 wait states a total access time < 21ns ( electrical spec 167 ( address to data valid time in datasheet figure 31-7, table 31-11) ,including the glue logic

This can only be done by using the Sram 16bit ( most are 10ns ) , with glue logic at about ( 5ns max) , where the code would be loaded from serial Flash into Sram before running

this method on surface may allow for the expansion of available ram to the PIC as well , but i am waiting on confirmation of this and some other timing clarification on the above tables in the datasheet

The points highlighted in the thread indicate that the generated hex file would need be edited at the 128k code boundary , sub divided into 2 hex files manually, then loaded in to the PIC program area , then into external flash , then from external flash into the sram for use at startup .

It should be noted that Sram of speed and glue logic is not cheep , in fact its comparable to adding the same size PIC again

but it may be a way forward , more info still needed from microchip

regards

Sheldon

[longpole001]

MICROCHIP have confirmed that emb works at 64mhz at 16bit mode , recommends glue logic, and sram - see links

i have asked charles to confirm method would work as he is working on code using the 18f87k22 at the moment

regards

Sheldon

We have received some suggestions from another engineer for third party components. Obviously, anything suggested will need to be vetted by you for your application as Microchip can not support or verify third party products. However, here are the suggestions:

A lot matters on the choice of vcc
http://www.nxp.com/documents/data_sh...LVCH16373A.pdf - is a 3 v part
http://www.ti.com/lit/ds/symlink/sn74ahc16373.pdf is slower, but supports 5v
http://www.alliancememory.com/pdf/sr...c1026bv1.3.pdf 64Kx16, 5V
http://www.cypress.com/?docID=49420 256Kx16, 5V
http://www.alliancememory.com/pdf/sr..._8TIN_V1.4.pdf 256Kx16, 3V