Briefly, maximum number of vars?

I have a program running on an 18F26J50, and use about 30 Vars, mostly byte sized, and about 15 of them are declared as 6 byte arrays.

The code runs great and I want to include the SDFS file system code for SD Cards. I have previously used this code successfully and find it is easy to work with.

I have begun by copying in the list of variables and aliases. However, if I copy in all of the SDFS vars my program runs as normal except that some data is being corrupted. The data is held in buffers, and represents led segment patterns on a display. If I comment out, say, 20 of the SDFS variables then my code works, and if I leave them in the led display shows some garbage at exactly the same point each cycle.

Is there a limit to the number of variables which you can declare ? Knowing I would need a lot of vars I used the 18F26J50 because of it's 3k of SRAM and as far as I can tell I am probably only close to using 1k of Ram at the moment.

I've tried lumping arrays together (not that it should matter) and rearranging the order of the SDFS vars before my own, but the effect is always the same.

The effect is not specific to those particular vars which I comment out either, if I comment out any 15 or so of the SDFS vars my code begins to work again.

Chris

I'm beginning to think that this has something to do with memory addressing in PBP, could somebody make a comment if I need to do anything different to address the large data space other than rely on PBP to select the correct banks for me.

Chris

Chris,

I am not really sure, but, you seem to be on the right track thinking that PBP may need to be guided around the RAM banks.

I haven't worked on any project that uses a lot of RAM, so my comments may be totally invalid.

However, my understanding is - PBP sets the banks by looking at the starting address of a variable before accessing it. What this means, PBP knows where to find the first element of a multibyte array, but the moment the array wraps over to the next RAM page, PBP brings you back to the start of the existing/selected page.

A possible workaround would be to place large arrays in their own RAM page where they cannot wrap around without PBP knowing.

I wonder if this helps

I would suspect Array overflows.

All arrays are zero based.
It's easy to forget that a 32 byte array only goes to 31.

PBP does not check "Bounds" for arrays.
And if you go past it, something else gets corrupted.

With 18F's, banks don't matter with BYTE arrays.
WORD/LONG array's are a little different.
<br>

Quote:

Originally Posted by Darrel Taylor

I would suspect Array overflows.

All arrays are zero based.
It's easy to forget that a 32 byte array only goes to 31.

PBP does not check "Bounds" for arrays.
And if you go past it, something else gets corrupted.

With 18F's, banks don't matter with BYTE arrays.
WORD/LONG array's are a little different.
<br>

Thanks Darrel.
Part of my faultfinding included checking for overflow, as well as increasing the size of my arrays, just to be sure.

The SDFS code variables include several WORD and LONG arrays. You have said that they are handled differently, can you suggest what might be going wrong, and a possible solution ? Would it be wise to force the location of such arrays ? How would that be best achieved ? Could I locate LONG and WORD arrays at the beginning of my code to ensure those variables begin at the start of a page ?

Is it important to also point out that none of these WORD and LONG arrays are ever used in my code at the moment. I commented out all code which used them, to successfully eliminate the program itself.

TIA
Chris

Quote:

Originally Posted by Jerson

Chris,

A possible workaround would be to place large arrays in their own RAM page where they cannot wrap around without PBP knowing.

I wonder if this helps

I guess it has something to do with my understanding of PBP as much as anything else, your description of the array pointer system ties in with my own thoughts.

Darrel can confirm that the only array types which might give cause for concern are WORD and LONG. The variables which I am pasting in contain a single LONG array of 8 elements. All other arrays being byte sized. I've just tried putting the LONG array at the start of the declaration section, but do you think it would be better to ORG it to somewhere ?

What pains me is that I took a piece of working code which I have been working on for months, imported some variables which are not duplicates and which are never implemented in my own code, and I end up with non-working code.

The shift to linear memory can't come soon enough !

Chris

Hope this makes things clearer...

I have the usual var declarations (over 200). The following names are for example only.

abc var byte
bcd var byte
cde var byte
def var byte[75]
efg var long[8]
-
-
-
-
-
-
-
wxy var byte
xyz var byte

If I comment out the second last variable 'wxy' (or comment out any of the later bytes) the code runs. If I leave wxy in the list then elements of array 'def' become corrupted. At first I suspected it was something to do with using too much ram, so I reduced the size of array def to 50 elements. But the problem is exactly the same, even though I freed up 25 bytes of ram when I reduced the size of 'def' (and of course, made sure I didn't point to any of the now missing array elements, in practice I only use the first 20 elements at the moment)

Commenting out 'wxy' allows the code to run properly, and also wxy is never actually used yet at any point in my code.

This is what leads me to wonder if it is not the actual _number_ of variables which is the key, rather than the amount of space which they use. I have 211 variable names (excluding aliases), that's 211 pointers to individual addresses, and presumably that info is saved on a table. How many variables does PBP require, because if it uses 45 or more then that would cause overflow of an 8-bit table or variable.

Chris

Time for a test

Hi
To make sure PBP does not break any RAM boundries I would like you to do a small test. Take the code that does not work and add some extra code to it at the start of your program.

Code:

Dummy_0 Var Byte $FF Dummy_1 Var Byte $1FF Dummy_2 Var Byte $2FF Dummy_3 Var Byte $3FF

This will force PBP to load byte variables at the end of each RAM bank.

I am really interested in the results.

Chris,

I suggest you try loading all your BIG arrays right at the top. Then put all the little single variables like bytes or words or longs after that.

As to how many variables PBP needs, I think you can look at the generated LST file and see the variables before your own.

There is another possibility that you need to consider - is there any asm interrupt code of your own that might be disturbing the register banks.

Regards

Quote:

Originally Posted by sinoteq

Hi
To make sure PBP does not break any RAM boundries I would like you to do a small test. Take the code that does not work and add some extra code to it at the start of your program.

Code:

Dummy_0 Var Byte $FF Dummy_1 Var Byte $1FF Dummy_2 Var Byte $2FF Dummy_3 Var Byte $3FF

This will force PBP to load byte variables at the end of each RAM bank.

I am really interested in the results.

Thanks fro the suggestion but there is no change.
in keeping with the original problem, using your suggestion has increased the number of vars by 4, and if I comment out 4 of the unused vars to compensate the code runs. Changing the location of the dummy vars has no impact on the problem

Quote:

Originally Posted by Jerson

Chris,

I suggest you try loading all your BIG arrays right at the top. Then put all the little single variables like bytes or words or longs after that.

I've tried this method, it seems to make no difference

Quote:

There is another possibility that you need to consider - is there any asm interrupt code of your own that might be disturbing the register banks.
Regards

I do have an ASM interrupt.
In the interrupt I use indirect addressing to load an array (the array in question) with data received from the serial port, before it is transmitted out of the serial port again, trapping the received data for analysis and real-time display.

Before I give the details, the gotcha here is that the code works fine without these additional unused variables. but once I add a few vars in the declarations, after my own var declarations or before them, the problem occurs. At no time are those additional vars referenced in the program code.

I have looked at the PBP files and it seems that FSR0 and FSR2 are used by PBP, but not FSR1. To do the indirect addressing I load FSR1 with LFSR 1,_RX_BUFFFER, the address of the 1st byte in the RX_BUFFFER array. Then I use POSTINC1 to put each successive received byte into the buffer. I then _save_ the values for FSR1L and FSR1H, so the next time I need to load the next array element I restore FSR1L and FSR1H with the last values in the interrupt. I actually get the same problem even when I don't save the values for FSR1, which suggests that it is pointless to do that anyway

What I have not done, is to save the values in FSR1L and FSR1H when the program interrupts, but then I don't see any need because PBP doesn't seem to make any use of FSR1, plus the same code works as expected when I reduce the number of unused variables.

Chris

Quote:

Originally Posted by Chris Barron

.... The shift to linear memory can't come soon enough !

Chris

Don't count on it happening any time soon. Right now the only Microchip products that have it (4Gb linear addressing space) is the PIC32 line. :rolleyes:

Quote:

Originally Posted by Chris Barron

I do have an ASM interrupt.
In the interrupt I use indirect addressing to load an array

Just for confirmation I have reverted to a previous interrupt routine which does not use indirect addressing, does not access the FSR's at all. The same problem is happening.

Would someone be able to tell me what is the best way to force PBP to locate my variables in places which I know will all my LONG and WORD arrays to fit on one bank of memory

Chris

There was a thread awhile back where someone was having the same trouble, to many variables. Sounds like this is the same thing.

Maybe a work around would be to scale things in your code and SDFS by not declaring some of the pins as VARs.

An example would be in SDFS if you are not using the card detect

Code:

SD_CD Var PORTB.4 ' SD card detect SD_CD_TRIS Var TRISB.4 ' SD card detect direction

to comment that portion out. Probably some things in your code that could be re-done also???

Just thoughts.

BTW, are you using the SDFS from MElabs? The one for FAT16 or the one from Jeremy posted on the forum, SDFS32.

Maybe seeing your code would help too.

Quote:

Originally Posted by mackrackit

There was a thread awhile back where someone was having the same trouble, to many variables. Sounds like this is the same thing.

Maybe a work around would be to scale things in your code and SDFS by not declaring some of the pins as VARs.
Just thoughts.

I could do that, but I would rather not be limited to a smaller number of variables than I have ram locations for.

I would post the code, but it's over 40k now. Take out the data tables and it's down to about 28k, so it's still a massive post ! The essence remains the same though.

I'm using the FAT16 MELabs version of SDFS. If the FAT32 version uses less space and doesn't have the same caveats I would like to try it out, but even after that I still would like to include more variables.

As I've said, the number of ram variables declared seems to be deciding factor and the problem doesn't seem to be affected by the number of aliases. I might consider declaring some large byte arrays and then declare the current individual byte vars which I am using as aliases of the larger array.

Chris

Quote:

Originally Posted by Chris Barron

Darrel can confirm that the only array types which might give cause for concern are WORD and LONG. The variables which I am pasting in contain a single LONG array of 8 elements.

Well, they are handled a little different, but they are still no cause for concern.

The compiler simply makes sure that no single element of an array crosses a bank boundary. The array can span multiple banks, but LONGs or WORDs will always be on one side or the other, never split.

If you want to make sure that an array is completely inside a single bank ... specify the bank in the variable declaration.

MyArray VAR WORD[16] BANK3

This forces PBP to put the entire array in bank3 and will not allow it to cross a boundary.
However, I do not think that's your problem.

Can you at least post the interrupt routine.
That's the most likely place.
The second most likely is any ASM code (doesn't have to be the INT handler).
<br>

Chris

Is it possible to let us have just "all" your variable declarations so that we can test it out?

I am suspecting that the compilers' symbol table is unable to cope with so many variables.

These are my own vars.

Code:

Change_delay var word[6] Delay_set var word[6] Tube_previous var byte[6] Tube_current var byte[6] Tube_next var byte[6] maxtubes var byte Datim_buf var byte[8] Scroll_delay var byte TScroll_progress var byte Raw_tube var byte[6]; $200 RX_BUFFER Var Byte[50] Tube_sel var byte Tube var byte Tube_max var byte DP var byte[6] Delay_cnt var byte delay_cnt = 255 Delay_mode_u var bit BlanksL var byte BlanksH var byte SegsL var byte SegsH var byte Hours VAR BYTE Minutes VAR BYTE Seconds VAR BYTE Year VAR BYTE Month VAR BYTE Day VAR BYTE Weekday VAR BYTE Lastsecs var byte sec_cntr var byte: sec_cntr = 0 Sword var word ; spare word Addr var word Prog_split var byte Longest_effect var byte Dir_bit var bit :dir_bit = 1 Out_bufL var byte Out_bufH var byte Eout_bufH var byte[6] Eout_bufL var byte[6] Speed_trim_set var byte Speed_trim_val var byte Saved_Digit var byte[6] Saved_DP var byte[6] Touch_Status var byte Touch_Val var word Saved_effect var byte[6] Cycle_cntr var word :cycle_cntr = 0 Letter_mode var byte :letter_mode = 0 Letter_effect var byte :letter_effect = 0 Letter_speed var byte :letter_speed = 0 Letter_pause var byte :letter_pause = 0 Letter_time_freq var byte :letter_time_freq = 0 Letter_time_mode var byte :letter_time_mode = 0 Letter_date_freq var byte :letter_date_freq = 0 Letter_date_mode var byte :letter_date_mode = 0 Letter_temp_freq var byte :letter_temp_freq = 0 Letter_temp_mode var byte :letter_temp_mode = 0 Letter_time_cntr var byte : letter_time_cntr = 0 Letter_date_cntr var byte : letter_date_cntr = 0 Seg_progress var byte Disp_mode var byte: disp_mode = 0 ; mode 0 = serial data ; mode 1 = time ; mode 2 = date ; mode 3 = Letterplay ;EEDATA var byte ;EEADR var word obit var bit q var byte X var byte X2 var byte RX_BYTE_CNT Var Byte RX_BUF Var BYTE clk var PORTB.0 dat var PORTB.1 strobe var PORTB.2 blank var PORTB.3 Device VAR BYTE[6] M_type VAR byte M_type1 var byte SPARE VAR BYTE FONT VAR byte[6] EFFECT var byte[6] control var Byte Dig_newL var byte[6] Dig_newH var byte[6] Dig_oldL var byte[6] Dig_oldH var byte[6] Effect_pause var byte[6] Effect_var var byte[6] Outa2h var byte Outi2p var byte ESPEED var byte[6] EE_ADR var word EE_DAT var byte Effect_progress var byte[6] RX_PROGRESS var byte Tube_dat var byte[6] Last_tube_dat var byte int_dev var byte base_data var word base_data = 1012 tube_mask var byte randnum var word mode var byte

I can't post everything together because it exceeds the maximum message size, so you might want to add the SDFS vars to the above list yourself.

This is my ISR

Code:

ASM myint btfss PIR1,0 ; timer1 o/flow bra Data? ; keypress stuff here EndKey: ; end of keypress code bcf PIR1,0 bra end_int Data? btfss PIR1,5 ; is the received data flag set ? goto dot movf RCREG,w movwf _RX_BUF ; save the received byte immediately movwf TXREG ; Load the TX register immediately to pass the byte on nop x1: btfss PIR1,TXIF ; Wait for TX send to complete bra x1 ; progrss register bits 0-3 get set as each element of the header ; is received. If a bad byte is detected then progress is reset ; and the code starts waiting for another $ sign prog0 btfsc _RX_PROGRESS,0 ; has the progress,0 step been completed bra prog1 ; yes, test for progress,1 movf _RX_BUF,w xorlw '$' ; test if this byte is a $ sign btfss STATUS,Z goto dump ; not a $ sign, clear the progress register bsf _RX_PROGRESS,0 ; no, set this bit now movf TMR1L,w subwf _randnum,f goto out ; is a $ sign, don't clear progress, allow more prog1 btfsc _RX_PROGRESS,1 ; same as above, validate the received data/header bra prog2 movf _RX_BUF,w xorlw 'B' btfss STATUS,Z goto dump bsf _RX_PROGRESS,1 goto out prog2 btfsc _RX_PROGRESS,2 bra prog3 movf _RX_BUF,w xorlw '7' btfss STATUS,Z goto dump bsf _RX_PROGRESS,2 goto out ; The fourth byte to be received is the 'type of message' character ; as for prog0 to prog2, if the fourth byte doesn't meet the test ; criteria of being a valid message type this data sentence is aborted prog3 btfsc _RX_PROGRESS,3 bra prog4 clrf _M_type clrf _M_type1 typep?: movf _RX_BUF,w ; decimal points xorlw 'P' bnz typem? bsf _M_type,0 bra duntype typem?: ; message ascii digit movf _RX_BUF,w xorlw 'M' bnz typef? clrf _Disp_mode bsf _M_type,1 bra duntype typef?: ; font movf _RX_BUF,w xorlw 'F' bnz typew? BSF _M_type,3 bra duntype typew?: ; UDC write command movf _RX_BUF,w xorlw 'W' bnz typee? bsf _M_type,2 bra duntype typee?: ; effect movf _RX_BUF,w xorlw 'E' bnz types? bsf _M_type,4 bra duntype types?: ; speed movf _RX_BUF,w xorlw 'S' bnz typed? bsf _M_type,5 bra duntype typed?: ; digit delay movf _RX_BUF,w xorlw 'D' bnz typev? bsf _M_type,7 bra duntype typev?: movf _RX_BUF,w xorlw 'V' bnz typec? bsf _M_type,6 bra duntype typec?: movf _RX_BUF,w ; clock function xorlw 'C' bnz typea? bsf _M_type1,0 bra duntype typea?: movf _RX_BUF,w ; alarm function xorlw 'A' bnz typel? bsf _M_type1,1 bra duntype typel?: movf _RX_BUF,w ; Letter play xorlw 'L' bnz typez? bsf _M_type1,2 bra duntype typez?: movf _RX_BUF,w ; set total number of digits in array xorlw 'Z' bnz dump ;typenul? bsf _M_type1,3 bra duntype duntype: bsf _RX_PROGRESS,3 clrf _RX_BYTE_CNT lfsr 1,_RX_BUFFER bra out ;'; If we got to here then the message type is valid. ;'; now to pick the correct data byte from the stream relevant to this device's ;'; enumerated position in the chain, and then save it for processing prog4 ;duntype capture_bits: movff _RX_BUF,POSTINC1 ; capture all bytes of data incf _RX_BYTE_CNT,f inc_dun: btfss _M_type,2 ; Is this a UDC write ? bra typnot2 movf _RX_BYTE_CNT,w xorlw 18 bz goodmes bra out ; For all other, non- type 2 messages, count data bytes ; and when the current byte number equals this device's ; enumerated position put the current data byte in the buffer typnot2 movf _RX_BUF,w xorlw 13 ; Is this the final character of a message ? bz goodmes btfsc _M_type1,0 ; Clock command bra out btfsc _M_type1,3 ; Declare total number of tubes in array bra out btfsc _M_type1,1 ; Alarm command bra out btfsc _M_type1,2 ; FLW functions 'L' letter play bra out btfsc _M_type,7 ; Delay functions bra out btfsc _M_type,1 ; Message functions bra out ld_tube_dat: movf _RX_BYTE_CNT,w xorwf _Device,w bnz dev2 movff _RX_BUF,_Tube_dat ; Tube_dat now has the correct data byte with respect bra out ; to this device's position in the chain dev2 movf _RX_BYTE_CNT,w xorwf _Device+1,w bnz dev3 movff _RX_BUF,_Tube_dat+1 ; Tube_dat+1 now has the correct data byte with respect bra out ; to this device's position in the chain dev3 movf _RX_BYTE_CNT,w xorwf _Device+2,w bnz dev4 movff _RX_BUF,_Tube_dat+2 ; Tube_dat+2 now has the correct data byte with respect bra out ; to this device's position in the chain dev4 movf _RX_BYTE_CNT,w xorwf _Device+3,w bnz out movff _RX_BUF,_Tube_dat+3 ; Tube_dat+3 now has the correct data byte with respect bra out ; to this device's position in the chain goodmes movf TMR4,w subwf _randnum,w movwf TMR4 bsf _control,7 dump clrf _RX_PROGRESS dot: btfss INTCON,2 ;Did a timer0 overflow occur ? bra out bcf INTCON,2 ; do timer0 maintenance bsf _control,1 movlw 160 movwf TMR0L out: end_int: RETFIE FAST endasm

Here's a video of the device, when it works. This is the LED version, my main work is on the VFD tube version, versions of which can be seen in my Youtube collection. The only difference between the VFD and LED versions is the order of bitbanging data to the output SR's, and the VFD version uses CCP1 to generate 40V via a boost convertor

http://www.youtube.com/watch?v=Dke0ZyPhaaM

The VFD Version
http://www.youtube.com/watch?v=G7SeR...eature=channel

This is the version which started it all off.
http://www.youtube.com/watch?v=fz2Fn...eature=channel

Note that in the last video (the first incarnation) each display tube has it's own controller IC, a PIC16F690, where the two previous versions now multiplex 4 display elements and have independent transition effects, operated by a 18426j50 driving an output shift register.

The displays are daisychained via a serial connection and form a self arranging linear network. They take simple ascii text commands at the input and organise themselves so that the correct character is displayed on the correct tube. For example the message $B7MTest , would put the letter T on the first tube, e on the next, and so on.<$B7M> is the header used for display 'M'essages.

if they are all connected together then each element has it's own 'place' in the display array (very much like the DMX protocol uses individual addresses too) then larger arrays of mixed display media can be built, such as this
http://www.youtube.com/watch?v=ek8S4r7bBUU

I keep most of the detail at my Yahoogroup
http://groups.yahoo.com/group/smartsockets/
http://groups.yahoo.com/group/smartsockets/

Quote:

Originally Posted by Jerson

Chris

Is it possible to let us have just "all" your variable declarations so that we can test it out?

I am suspecting that the compilers' symbol table is unable to cope with so many variables.

That is what I was alluding to.
If that is the case then one workaround may be to declare a large array, called 'MY_VARS' and then declare the existing program variables to be related to that one as aliases, because the number of aliases seems not to make a difference in the same way that the number of vars do.

Chris,

Accessing your vars in your asm routine without knowing which bank they're in, or handling BSR, is most, if not all, of the problem I suspect.

If all of your vars will fit into 1 bank, just tell PBP to place them all in the same bank, then set BSR to this bank on entry to your int handler.

Quote:

Originally Posted by Bruce

Chris,

Accessing your vars in your asm routine without knowing which bank they're in, or handling BSR, is most, if not all, of the problem I suspect.

If all of your vars will fit into 1 bank, just tell PBP to place them all in the same bank, then set BSR to this bank on entry to your int handler.

Thanks Bruce. What i've assumed is that because all of my vars do fit into one bank if i put them at the start of the declarations they will be placed there. The problem with the extra vars is happening whether I place the extra vars before or after my own. it doesn't seem to matter where I put them the fault doesn't go away until I comment out enough of them to get the total number down. But I take your point about using the BSR in the interrupt. I think I have relied too much on PBP and overseen the fact that I still need to obey Microchip's laws !

You could try a really quick test to see if this is the only problem. See if you can force YOUR variables all into bank13.

Change_delay var word[6] BANK13
Delay_set var word[6] BANK13

rest here.

NOTE: You only need to do this for all variables YOU are accessing directly in your int handler, or some other assembly routines. If you have vars you only access from PBP, don't bother trying to force locations.

Then make the first line in your assembly int routine MOVLB 0XD. You might want to have a look at the .lst file after compiling just to make 100% sure PBP really is placing all your vars in this bank.

If it works, it will be the easy fix. If not, then you're going to need to know where every single one of your variables are located, and select banks accordingly. Or learn how to use some of the PBP macros like CHK?RP.

Quote:

Originally Posted by Chris Barron

The problem with the extra vars is happening whether I place the extra vars before or after my own. it doesn't seem to matter where I put them the fault doesn't go away until I comment out enough of them to get the total number down.

With 18F's, PBP sorts the list of variables first by type, then alphabetically.
Arrays are always placed at the end, in the order ... byte arrays, word arrays then Long arrays end up last and will be the first things to end up in a different bank.

It doesn't matter where the variables are place in the code.

Thanks for the tips and information Bruce, Darrel.

I'm about to take a much needed break but I'll try out the ideas when I returm

Chris

I'm looking at this again now, and wonder what the general opinion is of the following strategy.

1) Make a note of the names of all of my variables which fall into assembly routines

2) Force PBP to locate them all in Bank 13 , for example, myvar1 var byte BANK13

3) At the start of each section of assembler use 'MOVLB 0xD'

Any other advice is welcomed
Chris

I would go with Bruce's suggestion about Bank 13.

I've just tried it and it works, so respect to Bruce.

With regards to the use of PBP Macros mentioned in bruce's message, where is the best place to find information about them, the manual ?

Pleased to be going forward again !
Chris

I seem to remember Darrel posting about that macro.

You will have to read most of this to get to the good part.
http://list.picbasic.com/forum/messa...tml?1066601004