Briefly, maximum number of vars?

[Chris Barron]

I have a program running on an 18F26J50, and use about 30 Vars, mostly byte sized, and about 15 of them are declared as 6 byte arrays.

The code runs great and I want to include the SDFS file system code for SD Cards. I have previously used this code successfully and find it is easy to work with.

I have begun by copying in the list of variables and aliases. However, if I copy in all of the SDFS vars my program runs as normal except that some data is being corrupted. The data is held in buffers, and represents led segment patterns on a display. If I comment out, say, 20 of the SDFS variables then my code works, and if I leave them in the led display shows some garbage at exactly the same point each cycle.

Is there a limit to the number of variables which you can declare ? Knowing I would need a lot of vars I used the 18F26J50 because of it's 3k of SRAM and as far as I can tell I am probably only close to using 1k of Ram at the moment.

I've tried lumping arrays together (not that it should matter) and rearranging the order of the SDFS vars before my own, but the effect is always the same.

The effect is not specific to those particular vars which I comment out either, if I comment out any 15 or so of the SDFS vars my code begins to work again.

Chris

[Chris Barron]

I'm beginning to think that this has something to do with memory addressing in PBP, could somebody make a comment if I need to do anything different to address the large data space other than rely on PBP to select the correct banks for me.

Chris

[Jerson]

Chris,

I am not really sure, but, you seem to be on the right track thinking that PBP may need to be guided around the RAM banks.

I haven't worked on any project that uses a lot of RAM, so my comments may be totally invalid.


However, my understanding is - PBP sets the banks by looking at the starting address of a variable before accessing it. What this means, PBP knows where to find the first element of a multibyte array, but the moment the array wraps over to the next RAM page, PBP brings you back to the start of the existing/selected page.

A possible workaround would be to place large arrays in their own RAM page where they cannot wrap around without PBP knowing.

I wonder if this helps

[Darrel Taylor]

I would suspect Array overflows.

All arrays are zero based.
It's easy to forget that a 32 byte array only goes to 31.

PBP does not check "Bounds" for arrays.
And if you go past it, something else gets corrupted.

With 18F's, banks don't matter with BYTE arrays.
WORD/LONG array's are a little different.
<br>

[Chris Barron]

I would suspect Array overflows.

All arrays are zero based.
It's easy to forget that a 32 byte array only goes to 31.

PBP does not check "Bounds" for arrays.
And if you go past it, something else gets corrupted.

With 18F's, banks don't matter with BYTE arrays.
WORD/LONG array's are a little different.
<br>


Thanks Darrel.
Part of my faultfinding included checking for overflow, as well as increasing the size of my arrays, just to be sure.

The SDFS code variables include several WORD and LONG arrays. You have said that they are handled differently, can you suggest what might be going wrong, and a possible solution ? Would it be wise to force the location of such arrays ? How would that be best achieved ? Could I locate LONG and WORD arrays at the beginning of my code to ensure those variables begin at the start of a page ?

Is it important to also point out that none of these WORD and LONG arrays are ever used in my code at the moment. I commented out all code which used them, to successfully eliminate the program itself.

TIA
Chris

[Chris Barron]

Chris,


A possible workaround would be to place large arrays in their own RAM page where they cannot wrap around without PBP knowing.

I wonder if this helps


I guess it has something to do with my understanding of PBP as much as anything else, your description of the array pointer system ties in with my own thoughts.

Darrel can confirm that the only array types which might give cause for concern are WORD and LONG. The variables which I am pasting in contain a single LONG array of 8 elements. All other arrays being byte sized. I've just tried putting the LONG array at the start of the declaration section, but do you think it would be better to ORG it to somewhere ?

What pains me is that I took a piece of working code which I have been working on for months, imported some variables which are not duplicates and which are never implemented in my own code, and I end up with non-working code.

The shift to linear memory can't come soon enough !

Chris

[Chris Barron]

Hope this makes things clearer...

I have the usual var declarations (over 200). The following names are for example only.

abc var byte
bcd var byte
cde var byte
def var byte[75]
efg var long[8]
-
-
-
-
-
-
-
wxy var byte
xyz var byte


If I comment out the second last variable 'wxy' (or comment out any of the later bytes) the code runs. If I leave wxy in the list then elements of array 'def' become corrupted. At first I suspected it was something to do with using too much ram, so I reduced the size of array def to 50 elements. But the problem is exactly the same, even though I freed up 25 bytes of ram when I reduced the size of 'def' (and of course, made sure I didn't point to any of the now missing array elements, in practice I only use the first 20 elements at the moment)

Commenting out 'wxy' allows the code to run properly, and also wxy is never actually used yet at any point in my code.

This is what leads me to wonder if it is not the actual _number_ of variables which is the key, rather than the amount of space which they use. I have 211 variable names (excluding aliases), that's 211 pointers to individual addresses, and presumably that info is saved on a table. How many variables does PBP require, because if it uses 45 or more then that would cause overflow of an 8-bit table or variable.

Chris

[sinoteq]

Hi
To make sure PBP does not break any RAM boundries I would like you to do a small test. Take the code that does not work and add some extra code to it at the start of your program.


Dummy_0 Var Byte $FF
Dummy_1 Var Byte $1FF
Dummy_2 Var Byte $2FF
Dummy_3 Var Byte $3FF

This will force PBP to load byte variables at the end of each RAM bank.

I am really interested in the results.

[Jerson]

Chris,

I suggest you try loading all your BIG arrays right at the top. Then put all the little single variables like bytes or words or longs after that.

As to how many variables PBP needs, I think you can look at the generated LST file and see the variables before your own.

There is another possibility that you need to consider - is there any asm interrupt code of your own that might be disturbing the register banks.

Regards

[Chris Barron]

Hi
To make sure PBP does not break any RAM boundries I would like you to do a small test. Take the code that does not work and add some extra code to it at the start of your program.


Dummy_0 Var Byte $FF
Dummy_1 Var Byte $1FF
Dummy_2 Var Byte $2FF
Dummy_3 Var Byte $3FF

This will force PBP to load byte variables at the end of each RAM bank.

I am really interested in the results.


Thanks fro the suggestion but there is no change.
in keeping with the original problem, using your suggestion has increased the number of vars by 4, and if I comment out 4 of the unused vars to compensate the code runs. Changing the location of the dummy vars has no impact on the problem

[Chris Barron]

Chris,

I suggest you try loading all your BIG arrays right at the top. Then put all the little single variables like bytes or words or longs after that.


I've tried this method, it seems to make no difference



There is another possibility that you need to consider - is there any asm interrupt code of your own that might be disturbing the register banks.
Regards

I do have an ASM interrupt.
In the interrupt I use indirect addressing to load an array (the array in question) with data received from the serial port, before it is transmitted out of the serial port again, trapping the received data for analysis and real-time display.

Before I give the details, the gotcha here is that the code works fine without these additional unused variables. but once I add a few vars in the declarations, after my own var declarations or before them, the problem occurs. At no time are those additional vars referenced in the program code.

I have looked at the PBP files and it seems that FSR0 and FSR2 are used by PBP, but not FSR1. To do the indirect addressing I load FSR1 with LFSR 1,_RX_BUFFFER, the address of the 1st byte in the RX_BUFFFER array. Then I use POSTINC1 to put each successive received byte into the buffer. I then _save_ the values for FSR1L and FSR1H, so the next time I need to load the next array element I restore FSR1L and FSR1H with the last values in the interrupt. I actually get the same problem even when I don't save the values for FSR1, which suggests that it is pointless to do that anyway

What I have not done, is to save the values in FSR1L and FSR1H when the program interrupts, but then I don't see any need because PBP doesn't seem to make any use of FSR1, plus the same code works as expected when I reduce the number of unused variables.

Chris

[rmteo]

.... The shift to linear memory can't come soon enough !

Chris
Don't count on it happening any time soon. Right now the only Microchip products that have it (4Gb linear addressing space) is the PIC32 line. :rolleyes:

[Chris Barron]

I do have an ASM interrupt.
In the interrupt I use indirect addressing to load an array


Just for confirmation I have reverted to a previous interrupt routine which does not use indirect addressing, does not access the FSR's at all. The same problem is happening.

Would someone be able to tell me what is the best way to force PBP to locate my variables in places which I know will all my LONG and WORD arrays to fit on one bank of memory

Chris

[mackrackit]

There was a thread awhile back where someone was having the same trouble, to many variables. Sounds like this is the same thing.

Maybe a work around would be to scale things in your code and SDFS by not declaring some of the pins as VARs.

An example would be in SDFS if you are not using the card detect


SD_CD Var PORTB.4 ' SD card detect
SD_CD_TRIS Var TRISB.4 ' SD card detect direction

to comment that portion out. Probably some things in your code that could be re-done also???

Just thoughts.

[mackrackit]

BTW, are you using the SDFS from MElabs? The one for FAT16 or the one from Jeremy posted on the forum, SDFS32.

Maybe seeing your code would help too.

[Chris Barron]

There was a thread awhile back where someone was having the same trouble, to many variables. Sounds like this is the same thing.

Maybe a work around would be to scale things in your code and SDFS by not declaring some of the pins as VARs.
Just thoughts.


I could do that, but I would rather not be limited to a smaller number of variables than I have ram locations for.

I would post the code, but it's over 40k now. Take out the data tables and it's down to about 28k, so it's still a massive post ! The essence remains the same though.

I'm using the FAT16 MELabs version of SDFS. If the FAT32 version uses less space and doesn't have the same caveats I would like to try it out, but even after that I still would like to include more variables.

As I've said, the number of ram variables declared seems to be deciding factor and the problem doesn't seem to be affected by the number of aliases. I might consider declaring some large byte arrays and then declare the current individual byte vars which I am using as aliases of the larger array.

Chris

[Darrel Taylor]

Darrel can confirm that the only array types which might give cause for concern are WORD and LONG. The variables which I am pasting in contain a single LONG array of 8 elements.

Well, they are handled a little different, but they are still no cause for concern.

The compiler simply makes sure that no single element of an array crosses a bank boundary. The array can span multiple banks, but LONGs or WORDs will always be on one side or the other, never split.

If you want to make sure that an array is completely inside a single bank ... specify the bank in the variable declaration.

MyArray VAR WORD[16] BANK3

This forces PBP to put the entire array in bank3 and will not allow it to cross a boundary.
However, I do not think that's your problem.

Can you at least post the interrupt routine.
That's the most likely place.
The second most likely is any ASM code (doesn't have to be the INT handler).
<br>

[Jerson]

Chris

Is it possible to let us have just "all" your variable declarations so that we can test it out?

I am suspecting that the compilers' symbol table is unable to cope with so many variables.

[Chris Barron]

These are my own vars.



Change_delay var word[6]
Delay_set var word[6]

Tube_previous var byte[6]
Tube_current var byte[6]
Tube_next var byte[6]
maxtubes var byte
Datim_buf var byte[8]
Scroll_delay var byte
TScroll_progress var byte
Raw_tube var byte[6]; $200
RX_BUFFER Var Byte[50]

Tube_sel var byte
Tube var byte
Tube_max var byte

DP var byte[6]

Delay_cnt var byte
delay_cnt = 255

Delay_mode_u var bit


BlanksL var byte
BlanksH var byte
SegsL var byte
SegsH var byte

Hours VAR BYTE

Minutes VAR BYTE

Seconds VAR BYTE

Year VAR BYTE

Month VAR BYTE

Day VAR BYTE

Weekday VAR BYTE

Lastsecs var byte

sec_cntr var byte: sec_cntr = 0

Sword var word ; spare word

Addr var word

Prog_split var byte

Longest_effect var byte

Dir_bit var bit :dir_bit = 1

Out_bufL var byte
Out_bufH var byte
Eout_bufH var byte[6]
Eout_bufL var byte[6]
Speed_trim_set var byte
Speed_trim_val var byte
Saved_Digit var byte[6]
Saved_DP var byte[6]
Touch_Status var byte
Touch_Val var word
Saved_effect var byte[6]
Cycle_cntr var word :cycle_cntr = 0

Letter_mode var byte :letter_mode = 0
Letter_effect var byte :letter_effect = 0
Letter_speed var byte :letter_speed = 0
Letter_pause var byte :letter_pause = 0
Letter_time_freq var byte :letter_time_freq = 0
Letter_time_mode var byte :letter_time_mode = 0
Letter_date_freq var byte :letter_date_freq = 0
Letter_date_mode var byte :letter_date_mode = 0
Letter_temp_freq var byte :letter_temp_freq = 0
Letter_temp_mode var byte :letter_temp_mode = 0

Letter_time_cntr var byte : letter_time_cntr = 0
Letter_date_cntr var byte : letter_date_cntr = 0


Seg_progress var byte
Disp_mode var byte: disp_mode = 0
; mode 0 = serial data
; mode 1 = time
; mode 2 = date
; mode 3 = Letterplay

;EEDATA var byte
;EEADR var word

obit var bit

q var byte
X var byte
X2 var byte

RX_BYTE_CNT Var Byte
RX_BUF Var BYTE

clk var PORTB.0
dat var PORTB.1
strobe var PORTB.2
blank var PORTB.3


Device VAR BYTE[6]

M_type VAR byte
M_type1 var byte
SPARE VAR BYTE
FONT VAR byte[6]
EFFECT var byte[6]
control var Byte
Dig_newL var byte[6]
Dig_newH var byte[6]
Dig_oldL var byte[6]
Dig_oldH var byte[6]
Effect_pause var byte[6]
Effect_var var byte[6]
Outa2h var byte
Outi2p var byte

ESPEED var byte[6]

EE_ADR var word
EE_DAT var byte
Effect_progress var byte[6]
RX_PROGRESS var byte
Tube_dat var byte[6]
Last_tube_dat var byte


int_dev var byte
base_data var word
base_data = 1012
tube_mask var byte
randnum var word

mode var byte

I can't post everything together because it exceeds the maximum message size, so you might want to add the SDFS vars to the above list yourself.

[Chris Barron]

This is my ISR



ASM
myint
btfss PIR1,0 ; timer1 o/flow
bra Data?
; keypress stuff here

EndKey:
; end of keypress code
bcf PIR1,0
bra end_int

Data?
btfss PIR1,5 ; is the received data flag set ?
goto dot
movf RCREG,w
movwf _RX_BUF ; save the received byte immediately
movwf TXREG ; Load the TX register immediately to pass the byte on
nop
x1:
btfss PIR1,TXIF ; Wait for TX send to complete
bra x1

; progrss register bits 0-3 get set as each element of the header
; is received. If a bad byte is detected then progress is reset
; and the code starts waiting for another $ sign


prog0 btfsc _RX_PROGRESS,0 ; has the progress,0 step been completed
bra prog1 ; yes, test for progress,1

movf _RX_BUF,w
xorlw '$' ; test if this byte is a $ sign
btfss STATUS,Z
goto dump ; not a $ sign, clear the progress register
bsf _RX_PROGRESS,0 ; no, set this bit now
movf TMR1L,w
subwf _randnum,f
goto out ; is a $ sign, don't clear progress, allow more

prog1 btfsc _RX_PROGRESS,1 ; same as above, validate the received data/header
bra prog2

movf _RX_BUF,w
xorlw 'B'
btfss STATUS,Z
goto dump
bsf _RX_PROGRESS,1
goto out

prog2 btfsc _RX_PROGRESS,2
bra prog3

movf _RX_BUF,w
xorlw '7'
btfss STATUS,Z
goto dump
bsf _RX_PROGRESS,2
goto out



; The fourth byte to be received is the 'type of message' character
; as for prog0 to prog2, if the fourth byte doesn't meet the test
; criteria of being a valid message type this data sentence is aborted

prog3 btfsc _RX_PROGRESS,3
bra prog4
clrf _M_type
clrf _M_type1




typep?:
movf _RX_BUF,w ; decimal points
xorlw 'P'
bnz typem?
bsf _M_type,0
bra duntype

typem?: ; message ascii digit
movf _RX_BUF,w
xorlw 'M'
bnz typef?
clrf _Disp_mode
bsf _M_type,1
bra duntype
typef?: ; font
movf _RX_BUF,w
xorlw 'F'
bnz typew?
BSF _M_type,3
bra duntype
typew?: ; UDC write command
movf _RX_BUF,w
xorlw 'W'
bnz typee?
bsf _M_type,2
bra duntype
typee?: ; effect
movf _RX_BUF,w
xorlw 'E'
bnz types?
bsf _M_type,4
bra duntype
types?: ; speed
movf _RX_BUF,w
xorlw 'S'
bnz typed?
bsf _M_type,5
bra duntype
typed?: ; digit delay
movf _RX_BUF,w
xorlw 'D'
bnz typev?
bsf _M_type,7
bra duntype
typev?:
movf _RX_BUF,w
xorlw 'V'
bnz typec?
bsf _M_type,6
bra duntype
typec?:
movf _RX_BUF,w ; clock function
xorlw 'C'
bnz typea?
bsf _M_type1,0
bra duntype
typea?:
movf _RX_BUF,w ; alarm function
xorlw 'A'
bnz typel?
bsf _M_type1,1
bra duntype
typel?:
movf _RX_BUF,w ; Letter play
xorlw 'L'
bnz typez?
bsf _M_type1,2
bra duntype
typez?:
movf _RX_BUF,w ; set total number of digits in array
xorlw 'Z'
bnz dump ;typenul?
bsf _M_type1,3
bra duntype


duntype:

bsf _RX_PROGRESS,3
clrf _RX_BYTE_CNT
lfsr 1,_RX_BUFFER
bra out


;'; If we got to here then the message type is valid.
;'; now to pick the correct data byte from the stream relevant to this device's
;'; enumerated position in the chain, and then save it for processing
prog4 ;duntype


capture_bits:
movff _RX_BUF,POSTINC1 ; capture all bytes of data
incf _RX_BYTE_CNT,f

inc_dun:
btfss _M_type,2 ; Is this a UDC write ?
bra typnot2
movf _RX_BYTE_CNT,w
xorlw 18
bz goodmes
bra out

; For all other, non- type 2 messages, count data bytes
; and when the current byte number equals this device's
; enumerated position put the current data byte in the buffer
typnot2
movf _RX_BUF,w
xorlw 13 ; Is this the final character of a message ?
bz goodmes

btfsc _M_type1,0 ; Clock command
bra out
btfsc _M_type1,3 ; Declare total number of tubes in array
bra out
btfsc _M_type1,1 ; Alarm command
bra out
btfsc _M_type1,2 ; FLW functions 'L' letter play
bra out
btfsc _M_type,7 ; Delay functions
bra out
btfsc _M_type,1 ; Message functions
bra out

ld_tube_dat:

movf _RX_BYTE_CNT,w
xorwf _Device,w
bnz dev2
movff _RX_BUF,_Tube_dat ; Tube_dat now has the correct data byte with respect
bra out ; to this device's position in the chain

dev2

movf _RX_BYTE_CNT,w
xorwf _Device+1,w
bnz dev3
movff _RX_BUF,_Tube_dat+1 ; Tube_dat+1 now has the correct data byte with respect
bra out ; to this device's position in the chain

dev3

movf _RX_BYTE_CNT,w
xorwf _Device+2,w
bnz dev4
movff _RX_BUF,_Tube_dat+2 ; Tube_dat+2 now has the correct data byte with respect
bra out ; to this device's position in the chain

dev4

movf _RX_BYTE_CNT,w
xorwf _Device+3,w
bnz out
movff _RX_BUF,_Tube_dat+3 ; Tube_dat+3 now has the correct data byte with respect
bra out ; to this device's position in the chain

goodmes
movf TMR4,w
subwf _randnum,w
movwf TMR4
bsf _control,7

dump
clrf _RX_PROGRESS

dot:
btfss INTCON,2 ;Did a timer0 overflow occur ?
bra out
bcf INTCON,2 ; do timer0 maintenance
bsf _control,1
movlw 160
movwf TMR0L



out:

end_int:

RETFIE FAST
endasm

[Chris Barron]

Here's a video of the device, when it works. This is the LED version, my main work is on the VFD tube version, versions of which can be seen in my Youtube collection. The only difference between the VFD and LED versions is the order of bitbanging data to the output SR's, and the VFD version uses CCP1 to generate 40V via a boost convertor

http://www.youtube.com/watch?v=Dke0ZyPhaaM

The VFD Version
http://www.youtube.com/watch?v=G7SeR_MdfGA&feature=channel

This is the version which started it all off.
http://www.youtube.com/watch?v=fz2FnJRpXQo&feature=channel

Note that in the last video (the first incarnation) each display tube has it's own controller IC, a PIC16F690, where the two previous versions now multiplex 4 display elements and have independent transition effects, operated by a 18426j50 driving an output shift register.

The displays are daisychained via a serial connection and form a self arranging linear network. They take simple ascii text commands at the input and organise themselves so that the correct character is displayed on the correct tube. For example the message $B7MTest , would put the letter T on the first tube, e on the next, and so on.<$B7M> is the header used for display 'M'essages.

if they are all connected together then each element has it's own 'place' in the display array (very much like the DMX protocol uses individual addresses too) then larger arrays of mixed display media can be built, such as this
http://www.youtube.com/watch?v=ek8S4r7bBUU



I keep most of the detail at my Yahoogroup
http://groups.yahoo.com/group/smartsockets/
http://groups.yahoo.com/group/smartsockets/

[Chris Barron]

Chris

Is it possible to let us have just "all" your variable declarations so that we can test it out?

I am suspecting that the compilers' symbol table is unable to cope with so many variables.

That is what I was alluding to.
If that is the case then one workaround may be to declare a large array, called 'MY_VARS' and then declare the existing program variables to be related to that one as aliases, because the number of aliases seems not to make a difference in the same way that the number of vars do.

[Bruce]

Chris,

Accessing your vars in your asm routine without knowing which bank they're in, or handling BSR, is most, if not all, of the problem I suspect.

If all of your vars will fit into 1 bank, just tell PBP to place them all in the same bank, then set BSR to this bank on entry to your int handler.

[Chris Barron]

Chris,

Accessing your vars in your asm routine without knowing which bank they're in, or handling BSR, is most, if not all, of the problem I suspect.

If all of your vars will fit into 1 bank, just tell PBP to place them all in the same bank, then set BSR to this bank on entry to your int handler.

Thanks Bruce. What i've assumed is that because all of my vars do fit into one bank if i put them at the start of the declarations they will be placed there. The problem with the extra vars is happening whether I place the extra vars before or after my own. it doesn't seem to matter where I put them the fault doesn't go away until I comment out enough of them to get the total number down. But I take your point about using the BSR in the interrupt. I think I have relied too much on PBP and overseen the fact that I still need to obey Microchip's laws !

[Bruce]

You could try a really quick test to see if this is the only problem. See if you can force YOUR variables all into bank13.

Change_delay var word[6] BANK13
Delay_set var word[6] BANK13

rest here.

NOTE: You only need to do this for all variables YOU are accessing directly in your int handler, or some other assembly routines. If you have vars you only access from PBP, don't bother trying to force locations.

Then make the first line in your assembly int routine MOVLB 0XD. You might want to have a look at the .lst file after compiling just to make 100% sure PBP really is placing all your vars in this bank.

If it works, it will be the easy fix. If not, then you're going to need to know where every single one of your variables are located, and select banks accordingly. Or learn how to use some of the PBP macros like CHK?RP.

[Darrel Taylor]

The problem with the extra vars is happening whether I place the extra vars before or after my own. it doesn't seem to matter where I put them the fault doesn't go away until I comment out enough of them to get the total number down.

With 18F's, PBP sorts the list of variables first by type, then alphabetically.
Arrays are always placed at the end, in the order ... byte arrays, word arrays then Long arrays end up last and will be the first things to end up in a different bank.

It doesn't matter where the variables are place in the code.

[Chris Barron]

Thanks for the tips and information Bruce, Darrel.

I'm about to take a much needed break but I'll try out the ideas when I returm

Chris

[Chris Barron]

I'm looking at this again now, and wonder what the general opinion is of the following strategy.

1) Make a note of the names of all of my variables which fall into assembly routines

2) Force PBP to locate them all in Bank 13 , for example, myvar1 var byte BANK13

3) At the start of each section of assembler use 'MOVLB 0xD'

Any other advice is welcomed
Chris

[mackrackit]

I would go with Bruce's suggestion about Bank 13.

[Chris Barron]

I've just tried it and it works, so respect to Bruce.

With regards to the use of PBP Macros mentioned in bruce's message, where is the best place to find information about them, the manual ?

Pleased to be going forward again !
Chris

[mackrackit]

I seem to remember Darrel posting about that macro.

You will have to read most of this to get to the good part.
http://list.picbasic.com/forum/messages/5820/6532.html?1066601004