when 128k is not enough

[longpole001]

Hi guys ,
well the project i am doing has now reached a point where i need to go to the next chip size , but there is not one that is supported by PBP

the easiest way seem , just add another chip and use the serial port to interface the 2 cpus' then start building software interface moduals for each of the hardware which is on each cpu

since i really only need more code space , another 128k would be nice ,

but its expected that 40% overhead of duplicated code would need to reside in the 2nd cpu for interface services.

how have others gotten around this problem , when you know the project exceeds 128k
i already have a lot of tables , off board serial flash etc , and only what cant be put in serial flash is on the program space

has any one gone down the path of converting pbp to mikrobasic that the compiler supports larger chips ?

cheers

Sheldon

[longpole001]

or if placing 2 cpus on the board , what signaling was used , would spi bus be better interface with interrupt handling ?

[richard]

if you need two cpu's I'd make one a raspberrypi , honestly multi gigabyte sd cards ,usb sticks, wifi ,Ethernet 512meg of ram and the elegant simplicity of python scripts and sql databases.
bang a pic on the side for lowdown nuts and bolts real time interfacing . join them together with rs232,spi,i2c what ever you like and bobs your uncle. multitasking (threading ) with python is just wonderful and have a look at pigpio by joan for some lowlevel stuff if you like (pretty amazing),but I still prefer bit-banging with a PIC.
why struggle when you can overkill it to death

[towlerg]

Longpole, the conversion to Mikro Basic might not be as easy as it sounds, mB is procedural like Proton24. Have you considered Proton24? Assuming that that there is an unsupported, appropriate 8 bit device with more ROM, have you tried manually editing the PPI etc.?


Richard, I think the idea is to migrate without re-writing all the code.

multitasking (threading ) with python is just wonderful

I guess that only applies if you know Python. If you don't it looks like Forth or something.

George

[Demon]

I like to use PBP, another PIC and I2C at 115,200.

Robert

[HenrikOlsson]

Sheldon,
Can you describe a little bit what this device and code consists of, what it's doing and so on? Perhaps a block diagram?
That will probably help others, including me, to come up with more relevant suggestions.

For example, does it have some kind of user interface, buttons, LCD or perhaps a console interface of some sort to speak to humans? If so, you could offload that to a secondary PIC and then have some less verbose Communication of those settings etc between the UI controller and the main controller.

You know that things like HSEROUT and ARRAYWRITE etc litterally EATS away codespace, something like 6bytes for each and every byte/characters. Using something like the "strings in codespace" routines discussed on this forum might be Another way (if you're not already doing that). And, if you're NOT doing any of that stuff what the heck ARE you doing that takes up 128k?

Again, it's not easy to come up with suggestions without any sort of background information. It's not like there's a magic compiler directive that just shrinks the code ;-)

With that said, at some point or another the limit IS of course reached.

/Henrik.

[richard]

updated files and arduino mega328 version too , there were some minor errors but the results are unaffected

arduino int latency 10.75uS code size 1794 bytes

[longpole001]

this is very interesting reading , cheers richard

[richard]

when I look at the hex dump both versions end at rom 00b8 so i'm not sure how to really compare the size

I'm wrong again was looking at wrong file . the pbp version is 341 words c version 181.
also the pbp version has a redundant for/next loop removing it saves 28 words
surprisingly the int latency has increased by 4uS by removing it ???

[longpole001]

yes splitting up into 2 cpus is no fun either , its a lot of work and cost either way, using EMb looks easier to implements if the codeing can be done ok , another issues is the amount of i/o ports the emb takes ,

the i/o ports cant really be used for anything else except the eMB bus, cos even if the ports / normal cpu i/o while not using the emb , any code that runs in the emb and requests hardware connected to pins that are emb bus wont work

[longpole001]

i would love to migrate , to another platform but the amount of work /time required its just not a road i want to go down , i need to finish this project , and i am very close for stage i want to be at ,

i need to add few other features , which will exceed 128k ( its currently at 110k ) and adding another chip appears to be the easiest way to use my existing coding/ knowlage and with what is written so far for the support code i have .


hendrich i am doing block diagram as its has quite a few moduals you would imagine

adding a 2nd chip although its pain , as the interface support to access and use hardware on both cpus and the common interface varables then transfer to the other cpu as needed , notification via status word for use between cpus for interrupt control notification ,etc etc

it is bit scarry at this point on the amount of extra work needed , to gain about 40k of program space , but that far less than learning a new scripting langage and new plateform i think.

ill post a block diagram of the key hardware interfaces.

I am trying to get the code smaller ,but tring to find 40k is just not likely

cheers'

sheldon

[longpole001]

the project is event timer control system using rf gates


its hard to write each of the blocks here without describing in full all the software components and their functions

but as an overview the hardware has

2 x spi buses , rf devices sd card , 32m flash
software spi for rtc ( to be changed to IC2 on next update)

usb/ serial for updates/ terminal services / other support hardware / remote displays

sound / horn control modual
4x 4 matrix keyboard
option 1 - 240-x 64 GLCD
option 2 - 240x128 GLCD with touch screen ( to be done yet)
spi 32meg flash - holds fonts / banners . terminal strings/ event dephaplts , time records of events , system controls / update temp bootcode


the hardware is simple stuff really but coding wise it takes a space for all the options anf fuctions for the rf compontes and thier many options , and trying to make t work well woth only 64mhz is also a challenge.

without access to most of varables when using the hardware from the other cpu , alot duplication is needed as well addtional interface code from what i am thinking ,

i guess thats price you pay for the 128k limit and project thats getting to big

it real shame that pbp cant make use of the larger pics , even if it were just the code space

[Demon]

Is it possible to have two versions of the board, one for each LCD?

That would make for an easy "split line", and have common code for most of the logic (easier to maintain).

Robert

[Dave]

You keep talking about code duplication, Maybe you should look at turning some of it into subroutines and save some space?

[longpole001]

i have pulled out option 2 for the display , and the touchscreen interface and support at this point , , a cutdown version of the graphics lib , to just support 240x64 display , all to save space.

and that will be added back for that version of the unit

the code is always in subroutines and i have made a lot of common called routines for support in each modual

There is seperate program for the loading and coding of the serial flash for the all fonts / strings / graphic symbols / event dephalts / menus system configuration / events menus /options / record data ,
only fonts that need to be in the main code area are.

the fact is the project has gotton bigger/ features added as it progressed , but the cpu space is limited

regardless of those type of code changes and they are on going , i wont be able to find a lot more space without removing features , or get a lot better at coding what is needed , ( always plays its part)

a new board will be designed , as this one was the prototype , and although i will use it as the first version for product testing / feedback , a lot of features will need to be shelved till i have more code space to allow for them ,

but in the end i need to have those features and some future expansion options ,

so i am looking at those now , as any new board would need to have those options , and thus the thought of adding a 2nd cpu

how to do it ???

when i say duplication of code its more to do with overhead when moving say the matrix keyboard over to the 2nd cpu, but the graphics modull / support is on cpu 1

most menus/ rf selection etc are displayed based on key pressed so code " if key x then do a lot of shit in cpu1 and use those varables in cpu1 , "
so moving the code that gets the key value is of little size moved to cpu2 , a small space saving on cpu1

but the all the varables , subroutines for hardware it calls as result need to be then sent as requests to cpu1 ,

so i cant really move all the keyboard menu statements / actions as they relate to hardware/ variables in cpu1 without a very large change on both cpu1 code for most of it would need to be copied over to cpu2 in the form of requests to cpu1.

and thats just moving the keyboard , i am looking at the other hardware items , but i most cases it not an easy task to isolate them as other parts of code do use them , and any parts that are moved to cpu 2 need to be coded to the cpu1 and 2 interface varables , which in tern duplicates the variables on both cpu's , which is no real space improvement ;(

This how i thinking of it at the moment ?

[Demon]

I think you're trying to hold a desk together with a single screw.

I'd delegate; one PIC to manage the RF peripherals, one for both LCDs, and so on.

A USB master PIC would poll each slave in turn, managing the exchange of data.

I was a mainframe programmer in COBOL. PC programmers make fun of such an archaic language but it taught me to think modular. It makes for more hardware, sometimes duplication, but it makes robust systems.

Robert

[towlerg]

I agree with Demon, try to look at the whole as a bunch of functions and locate the function on the CPU best placed to deal with a particular piece of hardware, for instance move the LCD handler and fonts etc. to a supernumerary cpu attached to the LCD's. As you are going to have 2 ? CPU's suggest SPI (a good deal quicker than I2C) slave /master. You'll need to establish some protocol, like first byte is command (in this case data to LCD), second byte is length of data, data n bytes (inevitably this will get more complex). You'll also need a mechanism to allow data to go the other way.

Actually the more I think about this, the more positive this could be. While leveraging your existing programming skills, you'll inevitably you'll get a performance hike, with all the humdrum stuff removed (to the other CPU) debugging becomes easier, if the second CPU is a separate board it makes it easier to add functions and a whole bunch of other stuff I haven't thought off.

I know the old corporate nonsense about challenges and opportunities but I think in this case viewing the situation positively could result in all sorts of benefits.

I wish you the best in your project, it sound really interesting. If possible post your progress, I for one am keen to hear where this story goes!

George

[longpole001]

this thread has been a bit hijacked although interssting

having alook at the 18f87k22) 80 pin chip , it tells of the eternal memory - section 8.5 addressing feature for both 8 and 16 bit mode , this option appears to be afar better one for accessing an 40- 128k of flash surely than changing horses ???????

[richard]

as the complexity increases I'm always expecting this result

[Demon]

this thread has been a bit hijacked although interssting

having alook at the 18f87k22) 80 pin chip , it tells of the eternal memory - section 8.5 addressing feature for both 8 and 16 bit mode , this option appears to be afar better one for accessing an 40- 128k of flash surely than changing horses ???????

When they mention external flash, do they mean an SD card?

Robert

[longpole001]

ok if i am going to break out to 2 or more cpus's

then perhaps breakup into hardware voltage use
cpu1 128K 18F 64PIN 64MHZ 5v - 18F67K22
xtl based 10ppm
cpu clock calibration control
GLCD 1 = 240 X 64 5V UNIT
GLCD 2 = 240X 128 5V UNIT

GLCD SUPPORT SERVICES -
gldc bightness control PWM / back lighting

KEY MATRIX 4 X4 / SUPPORT / KEY MENUS SYSTEM MENUS / EVENT MENUS SUPPORT
touch screen interface / support / CALIBRATIONS
cpu clock calibration

SERIAL PORT 1 - USB TTL MODUAL / USB CHANGER pwr CCT / VOLTAGE MONITOR / alarm SERVICES / USB UPDATE BOOT CODE SUPPORT / REMOTE DISPLAY RF DONGLE SUPPORT / DEBUG - 5V UNIT

RTC - software iC2 - 5V UNIT / modual 1/100th SUPPORT / ELAPSED TIMER MODUAL SUPPORT /

RGB Led for onboard status/ monitor (heartbeat) both cpus.
SPI bus 1 - level converters used - FLASH 32Mb (3V3 / FLASH SUPPORT / FONTS / SYMBOLS / BANNERS / common STRINGS / ALL MENUS / TEMP BOOT CODE STORAGE / REMOTE DISPLAY STRINGS/ OTHER DATA tables AS NEEDED
SPI bus 1 - ( same bus as flash ) SD CARD ( 3v3) / CARD SERVICES/ FILE SUPPORT CONVERTED TO NOT USE LONGS
( COMMON BUFFERS FOR BOTH DEVICES )
spi bus 2 - cpu2 interface ( level converters used ) / INTERFACE CODE / BUFFERS / INTERRUPT SERVICES CONTROL WORDS / STATUS WORDS AND PROTOCOLS

spare serial + port 3 pins

CPU2 128K 64MHZ 18F67K22 3V3

RF MODUAL 3V3 / SUPPORT SERVICES / ALL, RELATED RF ATIVITIES FOR GATES/ AND SUPPORT RF TRAFFIC DEVICES / HAND CONTROLLERS ID DATA / SYSTEMS / PROPRIETARY ENCRYPTED ENCODED PROTOCOL

EVENTS HORN CONTROL / SOUND CHIP modual 3V3 / flash - EVENT TRIGGERING AND GATES control / options settings / monitoring services

ALL EVENT RELATED DATA / EVENT GATE CONTROL / ELAPSED TIMER INTERFACE SERVICES / EVENT LOOKUP FIND ROUTINES / Floating point for distance speed caculations ( use longs )

SPI BUS 1 - 2ND FLASH CHIP 32mb - / EVENTS RECORDS DATA / boot code temp use

Spi bus 2 - cpu2 interface ( level converters used ) / INTERFACE CODE / BUFFERS / INTERRUPT SERVICES CONTROL WORDS / STATUS WORDS AND PROTOCOLS


Shared services modual

coding for bootloader programing for cpu1 , cpu2 , flash 1


well thats a staring point anyway

[Demon]

Unless you move to next-gen PICs, this is a huge step towards a much more reliable system; easier to write, debug, support and expand.

Awesome project. Looking forward to seeing your demo video.

Robert


Edit: I know it's gnawing at your guts, that it feels like a step back, another delay towards your goal.

It's the opposite, it's an investment. I was a system's analyst, from the high level down to the code. You'll be happy later when things go sideways or you add more features.

[longpole001]

hi robert

i will still progress on the current 1 cpu for the moment cos i have boards made / even if they are modified , for product testing the last 15k of code space should be enough to get the base requirements going so i can get a product feedback in jan ,

i will make use of the prototype boards for the product in its baseform , and after the feed back ill put that into the design of the 2nd pcb , which will use 2 cpus and break up the tasks as i see them at the moment .

it is a step backwards, in that if i had taken up pic design in C i would not have this as big an issue as i do now and change in platform would not be as painful and this would not be needed

but its been over 20 years since coding in C and i do like basic and PBP , but now i am seeing the many restrictions by the choice.

it is very sad that pbp can not support the larger chips even if it were for just to use the larger code space and use only 8bit fuctions ,
it at least allow some upgrade path using pbp past 128k.

It also seems likely that PBP will not progress in any functionality for some time , apart from adding a few more 8 bit chips in the next year or so ,
also given DT's passing has delayed any further release updates to PBP/ U2 programmer support etc for majority of this year.

Microchip in marketing wisdom have made that choice as well forcing you down the 16bit or 32 bit path just to get a bit more code space.

but this is where i am at so ill just have to make it work


i did not expect this to become as big as it has but to start again on new platform is not practicable given i just want to get it done


cheers

sheldon

[towlerg]

Might I ask, what device are you currently using? and which device would you like to use?

George

[longpole001]

thats intersting question , i expect not , it seem to me more like adding an external flash chip is liuke adding eeprom , great but how o use it more than just storage

[tumbleweed]

When you use the external memory bus of the 87K22, you connect it up to a regular 16-bit parallel flash device and the operation is totally transparent to the program. Program code < 128K uses the internal flash, and addresses above that go to the external memory chip... up to 2M bytes

Assuming that the compiler doesn't have any internal program size limits, there's nothing special you have to do when writing your program... you just end up with a really big hex file.

There are some limits/things to note:

- external memory is slower, so you have to reduce the system clock. The only reference I've found in the 87K22 datasheet is a note that says"Fmax=25MHz in 8-bit external memory mode". Using a 16-bit wide memory may ease this a bit, but there's nothing in the data sheet to back that up.

- you'll likely have to have a bootloader that can program both the internal and external flash devices since the regular ICSP programming tools like the pickit won't work to program the external flash.

- likewise, you can't use the ICD debugging tools, and the MPLAB simulator doesn't support the EMI either.

- there's an errata w/the 87K22 for the EMB. The CE chip select pin won't be extended if you use wait states, so you have to make sure the flash is fast enough to run in a single cycle.

[longpole001]

i am using 18f67k22 , like to use - anything bigger for code space and pbp ,

the device is holding together well with the code as it is running at 64mhz , but faster, bigger , more would be nice for some wiggle room , 2 or more cpus are the only option forward when using pbp and a chip that has 128k code space close to full

[towlerg]

I had a quick look, may be wrong but the only devices I could find with more that 128K were PIC24 and PIC32. If that is the case, don't even bother hinting to ME, it a whole new compiler.

George

[longpole001]

The only reference I've found in the 87K22 datasheet is a note that says"Fmax=25MHz in 8-bit external memory mode". Using a 16-bit wide memory may ease this a bit, but there's nothing in the data sheet to back that up.

this would mean the whole chip has to run at 25mhz , assuming that you external mem/ addressing is used for compiled code by the sounds of things

Assuming that the compiler doesn't have any internal program size limits, there's nothing special you have to do when writing your program... you just end up with a really big hex file.

i am not so sure that it does , i have always though it was tied to the chip selected when compiles , as such the chip size limits would apply and give errors in the compile , not warnings and thus not compile ???

[towlerg]

i am not so sure that it does , i have always though it was tied to the chip selected when compiles , as such the chip size limits would apply and give errors in the compile , not warnings and thus not compile ???

I guess you'll need to edit BLOCK_SIZE in the device file.

George

[HenrikOlsson]

I'm not saying there isn't a way to make it work but I'm pretty sure PBP doesn't support it natively.
You'd need to somehow fool the compiler into thinking the device has more memory than it has (if that's even possible). Then the compiler and assembler generates a single .hex file which you'd need to split at the correct location, program the first half into the PIC itself and the second half into the external memory device using some other type of device programmer.

I don't know the format of the .hex file, if there's any sort of header and/or checksum or anything like that. If there is, then simply cutting the file into two most likely isn't going to work and some sort of software would need to be written to repackage the large hex into two with the correct format.

Again, I'm not saying it isn't possible and I look forward to hear about any experiments. All I'm saying is that I don't think it's going to be straight forward.

/Henrik.

[longpole001]

yep 8bit and 128k is the largest they make and pbp only supports 8 bit

[longpole001]

i asked charles on the possible support of the 16 bit , 32 bit in limited to access code space / or full usage ,

We've attempted to complete this on several occasions, but there have been setbacks. The most recent attempt was stalled in 2013 due to issues concerning the developer that we were using for the PC-side executable.

The project is bigger than an expansion of the current PBP. Some parts of the current compiler can be reused, but it's really a matter of creating a new compiler just for PIC24. The good news is that we've already completed most of the work. The issue is frequently discussed, but no development is happening right now.

In the past, I've shared my optimism for completion dates and such. Things didn't work out. I am still optimistic, but I'm reluctant to take a guess as to what the universe holds in store. It's been a tough year.
Charles Leo
microEngineering Labs, Inc.
http://melabs.com

[towlerg]

due to issues concerning the developer that we were using for the PC-side executable.

So what part of a compiler isn't PC-side?

Anyways, even if they released a new compiler tomorrow (which they clearly are not) you'd still have to port your code to a new compiler (would you base a commercial product on version 1.0.0.0 of a compiler?) and there are already several mature options , one you've already mentioned and Proton24 (which I see supports several 256k Flash PIC24 devices). It looks like you've got to bite the bullet one way or another.

As I said before, fascinated to hear your progress.

George

[richard]

587000 floating point records extracted from a 14 million record database plotted and displayed on a hdmi screen , converted and exported to a png file , time < 15 sec
raspberry pi -python
data is fed from wireless weather station every 48sec captured by a pic 12f1822 with rfm01 wireless module and uploaded to the pi via i2c
its a good combination.
its going to leave a pic24 for dead


ps the graph is interactive when viewed live on the pi

[longpole001]

looks like the way to go in the long term and need to tack real,look at it for project but its a lot of overkill for what i need - at the moment , given that i realy need a little more code space to top things off

also it add a large cost for the product which need to be made in the 50 units at a time
i will get one to play with though

[richard]

THE OTHER OPTION
arduino mega 2650 :- 256k flash 8k ram 4k eeprom ,the sainsmart clones <$20 delivered .
big bonus 16mhz mega is about 4 times faster at floats than a 64mhz pic18 and equal or better at most other tasks, plus the free compiler is outstanding , especially in its optimisation capabilities .
floats are pain free in comparison to the microchip float library. its almost as if atmel actually want you to use their products unlike others who want $1000 for a decent optimising compiler.
most of the free libraries I have tried are very good even the colour touch screen glcd lib (try to find a free one that works for a pic), there are bigger/faster models too including a 32bit version that looks to be using the same ide . got one of these (a mega)plugged into the pi too via rs232
ps debugging
wow for $25 got the atmel studio arduino suite plugin , source level debugging at last that actually works and without any fancy (expensive ) headers either

the arduino forum has about a 100 new posts a day compared too ??

[Ioannis]

Richard, what you post seems a lot interesting. But I 'd like a more detailed reasearch. For example, same kind of code in PBP and Arduino C like lanquage.

What size code produces each compiler?

I have a strong feeling that Arduino is not near in the same neighbour. So if Sheldon has problem with 128Kb in PBP, how would this code be efficient enough for the Arduino? I think it won't.

Have seen small programs that take many KB in Arduino that could be built with less than 1 or 2 in PBP.

The other side is that is very easy and worry free as you said. Meaning there is no such thing as free meal.

Ioannis

[richard]

ioannis , what you say is correct esp for core 10 and 14 chips who's architecture is not particularly c orientated . pbp is hard to beat for small efficient code on these chips.
but for floating point work and/or complicated string parsing on pic18's c comes into its own . I firmly believe in horses for courses and lets face it the difference in price between a pic12f1822 and a 18f45k20 is not that great. as I have said before why struggle unless you are going to make and/or sell heaps of units , most of my stuff these days is one off , plus If I can get the esp8266's to behave the IOT becomes a reality . the esp8266 prg is 8.67k compiled with c on a pic . I will port it to arduino and compare. so far I can't get it to function properly with pbp3 the inherently blocking behaviour of basic functions is making life hard for that idea ,but worth pursuing for the IOT revolution (if its not all hype)

[longpole001]

APPLICATION NOTES ON EXTENDED MEMORY BUS
http://ww1.microchip.com/downloads/e...tes/00869b.pdf