Where should I discuss SD/MMC FAT issues?

[skimask]

I've got to get away from this pooter and get some real work done. Since I found the attachments button (duoh) here a screen capture of the F_ind file option.

Actually, you're one of only a couple of other people I've seen to get PBP to work with FAT on an MMC. I got a PIC/PBP to talk to an MMC, but not talking FAT.
I'm going to have a good hard look at your code and see what (if any) major changes would be needed for FAT32 compatibility.

[JD123]

Actually, you're one of only a couple of other people I've seen to get PBP to work with FAT on an MMC. I got a PIC/PBP to talk to an MMC, but not talking FAT.
I'm going to have a good hard look at your code and see what (if any) major changes would be needed for FAT32 compatibility.

Two things. First, I have a treat for you and anyone else who's trying to understand what I've done. It was a post I put together about how to simply find a file using a file's name and FAT. I wasn't happy with the limit of the scope, but now that I've put code out there, I think some people can benefit from what I've found.

Second, man you hit the nail on the head! I went from about 55,000 Tcy to 5098 Tcy by forcing the bank locations to 0 for the variables used in the read/write loops. I was so shocked, I thought that I didn't get the data, but with each read of the F-RAM to look at it's content, sure enough all my data is there. That's amazing how much difference that made. Do you think by pushing the loop subs up to the top of the program I could see even faster speeds?

(Oops... wrong file )

[JD123]

Yeah...but everybody else seems to hate it.
(Refer to this thread: http://www.picbasic.co.uk/forum/show...t=optimization )
Check the file attached to post #29...

And just for grins...if I'd rewrite that program it would look like this:

LOL! Yep, that would be like a Ford truck I once owned. It drove so bad (hard to keep it in a strait line) that when a friend of mine tried to drive it he said "this is a one person truck". That code would also be a one person truck!

[skimask]

Second, man you hit the nail on the head! I went from about 55,000 Tcy to 5098 Tcy by forcing the bank locations to 0 for the variables used in the read/write loops

SWEET!

Do you think by pushing the loop subs up to the top of the program I could see even faster speeds?

Probably not...BUT...if you take all of the variables in that tight loop, and make them all BANK0, you might see an incremental speed increase. I can't remember if the compiler will complain once it's out of BANK0 space or not, or if you'll even know it. One thing to do would be to check the .lst file and see where they're being put.
(looking at the new doc file now)

[JD123]

SWEET!

Probably not...BUT...if you take all of the variables in that tight loop, and make them all BANK0, you might see an incremental speed increase. I can't remember if the compiler will complain once it's out of BANK0 space or not, or if you'll even know it. One thing to do would be to check the .lst file and see where they're being put.
(looking at the new doc file now)

That's what I did - ALL variables used in the loops.

I re-read the PBP manual this morning and it does say that it will issue a warning if a 'requested' bank assignment won't fix.

[skimask]

That's what I did - ALL variables used in the loops.
I re-read the PBP manual this morning and it does say that it will issue a warning if a 'requested' bank assignment won't fix.

Ok, then moving that loop to the beginning of the program might have a small-ish impact on speed for the same reason, keeping it in BANK0. And if the whole loop won't fit up there, just try to put the tightest, smallest chunk of it up there. As I found out when I built my MP3 player, one cycle inside of a tight 512 count loop, inside of a tight 32 count loop (16K counts total) makes a big difference...especially when that loop was only 12 cycles long in the first place.

[JD123]

Last night I tried coding for the READ_I2C function (my sub) and ran into a fair amount of trouble. I’m not sure, but I think it revolved around the repeated hits to the ACK sub, where I originally wrote it for writes only (see where I toggle the port looking for the ACK from the SDA on the I2C) and my not understanding (while I was head-banging the problem) that the PIC issues the ACK if it is the receiver (reading the I2C).

It was late and I didn't give it a fair chance. I'm sure I get it figured out. Still, I'm up for suggestions as to what 'should' be the I2C functions that I make. I think I've got things so fractured that it's both getting hard to follow the loops and I maybe re-writing code unnecessarily. I was thinking that I should write the functions (subs) like this:

OPEN_I2C with a R/W bit set for read/write keeping all aspects inside the sub, and I2C stays open after address until...

CLOSE_I2C will send a stop command to end single or serial byte reads and writes.

WRITE_I2C simply sends out a byte with ACK back from the I2C, I2C stays open for next WRIET_I2C.

READ_I2C simply reads in a byte with ACK from PIC, I2C stays open for next READ_I2C.

Does this sound like a logical way to breakup the I2C subs so that I can read/write both bytes and serial strings?

This would be easier if PBP had an open-ended I2C function (like CCS's C does) for long reads and writes. Not all EEPROMS have page write buffers and all EEPROMS have unlimited serial reads (less you roll it over). Think I should put a note in the suggestion box?

[skimask]

This would be easier if PBP had an open-ended I2C function (like CCS's C does) for long reads and writes. Not all EEPROMS have page write buffers and all EEPROMS have unlimited serial reads (less you roll it over). Think I should put a note in the suggestion box?

Quite frankly, I've never tried making my own I2C subroutines.
And besides, I believe PBP's I2C routines let you write multiple bytes just by the fact that you have multiple bytes in a statement (see the str modifier in the manual, Page 85 of the PBP2.50a manual, 2nd paragraph from the bottom). That should cover reading and/or writing pages at a time.
PBP is pretty tight as it is and you might not be gaining a whole heck of a lot by rolling your own subs...just banging your head....or is that 'bit-banging' your head
The suggestion might not be needed...

[JD123]

I'm trying to avoid this huge 512 byte array, which won't fit on this PIC anyway.

Speaking of the PIC, I was at the store today and picked up an 18F2050. It's not a 2025, but it should do. I loose one CCP but gain a USB. Past that I can't tell them apart.

[skimask]

I'm trying to avoid this huge 512 byte array, which won't fit on this PIC anyway.

Speaking of the PIC, I was at the store today and picked up an 18F2050. It's not a 2025, but it should do. I loose one CCP but gain a USB. Past that I can't tell them apart.

2050? Maybe a 2550...
Should do just fine. Combine that PIC with the examples from Mr.E's USB demo in another thread (or the config bit hassle that WILL follow...just seems to happen that way), and you'll be in great shape.

And the 512 byte array won't fit, but the 32 byte or 64 byte array of the EEPROM page will...

[JD123]

{getting out my reading glasses...} Yup, it's a 2550.

I don't see a "MrE, Mr. E, Mr.E or Mr E" in the member list???

[skimask]

I don't see a "MrE, Mr. E, Mr.E or Mr E" in the member list???

http://www.picbasic.co.uk/forum/show...ghlight=mister

I got this one in my hotlinks...

[JD123]

Thanks. That's a thread I wish I was around for back in 06.

[JD123]

Something is wrong with my Tcy counter. It must be rolling over. I didn't make provisions for it rolling over because I couldn't imagin it taking more than 65K Tcy to move the data off the MMC and onto the F-RAM. I guess I was wrong... very wrong.

Best condition Tcy's to move data:

gosub to MMC read - 2Tcy

get byte - 24Tcy (3Tcy per bit)

return - 2Tcy

gosub to write F-RAM - 2Tcy

write F-RAM - 117Tcy (13Tcy per bit (min. clock delay 5Tcy), 9 bits w/ack)

Ack overhead - 9Tcy if tested, 3Tcy if blind read

return - 2Tcy

total for single byte loops - 158Tcy

repeat 512 times - 80,896Tcy

I must be moving data WAY slower than I thought. Time to detect TMR1 rollover and see what it really is.

[skimask]

Something is wrong with my Tcy counter. It must be rolling over. I didn't make provisions for it rolling over because I couldn't imagin it taking more than 65K Tcy to move the data off the MMC and onto the F-RAM. I guess I was wrong... very wrong.

Make an interrupt for the overflow interrupt and increment another word based on that.
Also, I was looking thru your code again.
Whenever you multiply or divide by a power of 2, you can easily use a shift to accomplish the same thing. Saves a bunch of cycles, and some code space.
For instance, you've got:
OP = OP * 512 (at least 9 shifts, 9 double word adds + carry's, probably 100-ish cycles)
That can be rewritten:
OP = OP << 9 (9+ cycles)

repeat 512 times - 91,136Tcy

Put some of your shorter Gosub/Returns in-line with the loop that runs 512 times. For each Gosub/Return you eliminate in the loop, you save at least 2,048 cycles (about 2% for each one).

[JD123]

Noted. I'll spend a bit more time on this, but with 80K+ Tcy's, I won't come even close to using this function while recording the 8 byte data every 16.67ms. Since this function will come into play only when starting and ending a session, time won't be an issue.

By the first of next week I need to get back on track with the core of the project.

[JD123]

We'll, I've done my best and yes, about 84K Tcy is as fast as this thing is going to read 512 bytes from the MMC and write the 512 bytes to the FRAM. It's 83,968 Tcy to be exact. Last night I thought I'd give it one more go.

At the start of the evening I got my latest code loaded and checked for timer1 overflows. Yep, it was running over. The true transfer times were 166,837 Tcy. I started dissecting the code bit by bit and embedding timers to catch cycle times. Turns out I had a very inefficient MMC routine. I strait-lined the code and got it back up to speed. I tweaked the I2C as well as moving the two function subs WRITE_I2C and GET_BYTE to the top of the program. Within an hour or two I got it down to 129,837 Tcy. After doing some math, I figured I was eating 45 Tcy with every loop and couldn't account for where it was coming from. The last part of the routine to be counted was the time needed to simply perform the FOR-NEXT loop.

Bingo! Turns out using WORD sized variables in a FOR-NEXT loop can be a huge Tcy eater. The solution I used was to break up the 512 count in the for-next loop to two 256 count loops. Boom! The cycle times fell to 83,986 Tcy. So, in a few hours, I saved 82,869 Tcy, saved 400 lines of complied code and gained about twice the speeds. I'm currently at 16mhz ripping the MMC sector and writing it in a total of just under 21ms. Still not fast enough to use it as I would like, but still... that's fast I2C.

Here's the updated code:

[skimask]

We'll, I've done my best and yes, about 84K Tcy is as fast as this thing is going to read 512 bytes from the MMC and write the 512 bytes to the FRAM. It's 83,968 Tcy to be exact. Last night I thought I'd give it one more go.

Add
DEFINE NO_CLRWDT 1 'no extra clear watchdog timer instructions
to the top of your program.
Will probably save a few more cycles there.

Also, in your 'big' loop, this might help somewhat. Remove the 2 for/next loops, replace with this:
loop1:
x1 = 2 : x = 0
loop2:
gosub get_byte : gosub write_i2c : x = x - 1 : if x = 0 then loop2
x1 = x1 - 1 : if x1 = 0 then loop1

Rewriting the 2 gosubs as inline code would save you about 2K cycles at most, might get you down below 20ms, but would use up code space. No real savings there.
Also, go back and try to BANK0 as much as you can, the commonly used variables first, until something doesn't end up in BANK0 (check the .lst file).

[JD123]

Add
DEFINE NO_CLRWDT 1 'no extra clear watchdog timer instructions
to the top of your program.
Will probably save a few more cycles there.

Interesting... I didn't think I called any routines that used the WDT. I'll check the .lst and see if there are any in there.

Also, in your 'big' loop, this might help somewhat. Remove the 2 for/next loops, replace with this:
loop1:
x1 = 2 : x = 0
loop2:
gosub get_byte : gosub write_i2c : x = x - 1 : if x = 0 then loop2
x1 = x1 - 1 : if x1 = 0 then loop1
Rewriting the 2 gosubs as inline code would save you about 2K cycles at most, might get you down below 20ms, but would use up code space. No real savings there.

On the outside, this looks to take about the same or more Tcy's to run as my 2 serial loops. I'll compile the two different ways and look at the .lst to count the Tcy's and see which is faster.

Also, go back and try to BANK0 as much as you can, the commonly used variables first, until something doesn't end up in BANK0 (check the .lst file).

I've made sure that all variables used in this loop process are in bank0. Can variables not used in the loop slow the loop down?

[skimask]

Interesting... I didn't think I called any routines that used the WDT. I'll check the .lst and see if there are any in there.

PBP inserts CLEARWDT instructions whenever it thinks there's a possibility of the WDT timing out during execution. But I don't know, and I've never bothered checking, if PBP even takes a look at the config fuses to see if the WDT is enabled and/or being used. I'm assuming that PBP ALWAYS inserts the instruction whether you are using the WDT or not. Better safe than sorry type thing. At any rate, I know from my experience that in projects where I'm not using the WDT timer at all, the DEFINE saves a load of program space...or at least a fair percentage.

On the outside, this looks to take about the same or more Tcy's to run as my 2 serial loops. I'll compile the two different ways and look at the .lst to count the Tcy's and see which is faster.

And they'll probably end up being close if not identical. About the only thing I know for sure, is the nested 'manual' for/next loop will save a few bytes of program space.

I've made sure that all variables used in this loop process are in bank0. Can variables not used in the loop slow the loop down?

I wouldn't think so, but...again, keeping everything you can in BANK0 can only speed things up a bit and save a few bytes here and there.

[mister_e]

But I don't know, and I've never bothered checking, if PBP even takes a look at the config fuses to see if the WDT is enabled and/or being used.

NOPE, unless they wouldn't add this DEFINE NO_CLRWDT 1 in their list.. and there's still 2 different way to set fuses... and not a lot of user seems to use 'em anyways...


Repeat-until will execute faster than a For-Next and need less codespace. This remind me the following thread...
http://www.picbasic.co.uk/forum/show...sharing&page=2

Post 8 and++

[JD123]

Repeat-until will execute faster than a For-Next and need less codespace. This remind me the following thread...
http://www.picbasic.co.uk/forum/show...sharing&page=2

Post 8 and++

Thanks. That a good read. I've written some short programs to do the same kind of benchmarking. I didn't consider the WHILE like loops, because though I know the loop functions are faster, adding back in a counter instruction seemed to offset any gains in total Tcy's per cycler. If checking for a match (end of loop condition) does not depend on a sequential counting, the WHILE type statments would be faster, for sure. Is my understanding about this correct?

[skimask]

Thanks. That a good read.

Just so I can remember where you're going with this...
You're getting some data, saving it in an FRAM, then dumping it out to an MMC, right?
But...you need it to dump inside of 16ms...

We could go on optimizing this all day (week? month? ), saving a couple of cycles here and there, trying to shave off that last couple of uSec.
I think in the end, you'd be much better off switching over to an 18F4620 (or whatever suits you, 18Fxxxx), kick up the osc. speed a bit, and take advantage of the extra ram in the 18F. Just going to 40Mhz will get you an extra 2.5x theoretical improvement ('cause you'll probably have to add NOPs at various points to keep the timings in spec), no BANKing issues will save another bunch of cycles...and so on and so on... Jist of the story, that 20-ish ms could theoretically drop to a bit over 8ms per chunk.

[JD123]

Just so I can remember where you're going with this...
You're getting some data, saving it in an FRAM, then dumping it out to an MMC, right?
But...you need it to dump inside of 16ms...

We could go on optimizing this all day (week? month? ), saving a couple of cycles here and there, trying to shave off that last couple of uSec.
I think in the end, you'd be much better off switching over to an 18F4620 (or whatever suits you, 18Fxxxx), kick up the osc. speed a bit, and take advantage of the extra ram in the 18F. Just going to 40Mhz will get you an extra 2.5x theoretical improvement ('cause you'll probably have to add NOPs at various points to keep the timings in spec), no BANKing issues will save another bunch of cycles...and so on and so on... Jist of the story, that 20-ish ms could theoretically drop to a bit over 8ms per chunk.

You're right about moving up in the PIC. I've started reading about the I2C port on the PIC I have. It's a bit daunting for the first-time user of this port function.

So far as the function of the memory, up to this point, it's MMC to PIC to FRAM. If I could get it down to a few ms then I was considering using it as the write buffer for the collected data instead of directly dumping the collected data onto the MMC. I don't think this is going to work unless I use both the SPI and I2C ports on my PIC. Even still, I don't think I have the time in the data collection cycle to do this. The advantages of using the hardware ports is a near perfect 1mhz to the FRAM and max speed to the MMC (mmc takes up to 20mhz). I was reading that the ports clock out at Fosc/2 when it's all done and said, I think. Like I said, I'm still reading... and reading... and reading. In theory, I could move 512 bytes from the MMC in a touch over 2ms and the same data to the FRAM at 4ms, but this does not include any code overhead - it's just the bit rates.

Back to the MMC for a minute:

The only way I've found to stop a sector read that allows for the next command to be another sector read has been to either finish the original sector read or to initialize the MMC. Am I missing a command that allows aborting a sector read followed by another sector read command? I did find the STOP command, but found out that sending another sector read command following the STOP command fails to be accepted by the MMC. Seems that the STOP command would be better described as a "PAUSE" command. Even the datasheet's say it's to free up the SPI bus for other SPI functions. It doesn't say anything about aborting the read command. The only way to do all this has been by initializing the MMC, which can take quite some time to do.

One other thing skimask, I was going to post this question in the PBP forums, but since you brought it up here... How come when I use your ":" colon text editing to condense my I2C routines I get an error surrounding the "@ NOP" lines?

I'm using text lines that look like:

SDA=1:@ NOP:@ NOP:@ NOP:SCL=1:@ NOP:@ NOP:@ NOP:SCL=0:{...ect.}

The "warning" listed is #255 "Expected op code instead of 'NOP :@' ". at the end of many warnings it "fails" and aborts the assembly process.

It looks like you can not nest any lines using the colon after an ASM (@) insertion. I checked the edited text lines to make sure I didn't remove the needed space before the "NOP". Since the code is now stable, I'm just looking to compact it for ease of moving around in the program.

[mister_e]

Not a colonic master, but yeah, i believe you can't stack any ASM on the same line.

Note, a GOTO $+1 = 2 X NOP. For the 18F you have to use GOTO $+2

[skimask]

Not a colonic master, but yeah, i believe you can't stack any ASM on the same line.

Nope, you can't...and that really bummed me out...I could've saved close to 300 lines if I was able to stack 'em up....

[skimask]

The only way I've found to stop a sector read that allows for the next command to be another sector read has been to either finish the original sector read or to initialize the MMC.

I know there's an ABORT sequence of some sort in there somewhere. I'll do some digging and get back to you...

when I use your ":" colon text editing to condense my I2C routines I get an error surrounding the "@ NOP" lines?

Previous post... One @ per line, nothing else...

[JD123]

Thanks guys. This is what I thought too - only one "@" item per line. Maybe I'll just move the I2C files into a "Include I2C.bas". I'll just have to keep track of file locations as I progress from one folder (part of the project) to another.

If I include the full path name of the file, can I get PBP to find it, regardless of where the main program exists? Stupid question, I know it should be ok, just asking.

[JD123]

Not a colonic master, but yeah, i believe you can't stack any ASM on the same line.

Note, a GOTO $+1 = 2 X NOP. For the 18F you have to use GOTO $+2

Is that GOTO literal or is there a value to "$"?

[mister_e]

in asm $ represent the current address (line). using GOTO $+1 will jump on the next address, $-1 to the previous. It avoid to have to use/add tons of label when you don't really need them.

[JD123]

in asm $ represent the current address (line). using GOTO $+1 will jump on the next address, $-1 to the previous. It avoid to have to use/add tons of label when you don't really need them.

Thanks for the answer MrE.

[mister_e]

no problem, you're welcome. Once you have a newer version, just post it here, maybe we could reduce it again and again.

[skimask]

Maybe I'll just move the I2C files into a "Include I2C.bas".

You could turn all those multi-line @ NOP's into macros.
Put this near the top of your code:

ASM
nop5 macro
     NOP
     NOP
     NOP
     NOP
     NOP
    endm
ENDASM

To use it:

@ nop5

[JD123]

You could turn all those multi-line @ NOP's into macros.
Put this near the top of your code:

ASM
nop5 macro
     NOP
     NOP
     NOP
     NOP
     NOP
    endm
ENDASM

To use it:

@ nop5

That would be 9 Tcy, right?

[mister_e]

Shouldn't be more than the amount of NOP. When you call a macro it just paste the macro code, but use another name.

While it's not the best code efficient, You could also use some self expanding macro by adding an argument, so if you need 2,3 or 10 NOP, it will do it for you automatically. Not sure if you need it here.

something like

<font color="#000080">ASM
NOP?C   macro Cin
    local a
a=0
    while a&lt;Cin
        nop
a++=1
        endw
    endm
ENDASM
</font>

If you use @ NOP?C 1 it will paste 1 NOP and so on.

[JD123]

Shouldn't be more than the amount of NOP. When you call a macro it just paste the macro code, but use another name.

While it's not the best code efficient, You could also use some self expanding macro by adding an argument, so if you need 2,3 or 10 NOP, it will do it for you automatically. Not sure if you need it here.

something like

<font color="#000080">ASM
NOP?C   macro Cin
    local a
a=0
    while a&lt;Cin
        nop
a++=1
        endw
    endm
ENDASM
</font>

If you use @ NOP?C 1 it will paste 1 NOP and so on.

Thanks! I've written ASM programs, but never explored the macro functions. When I said 9Tcy I was looking at the added overhead of jumping in and out of the ASM sub. I was thinking that just to jump-to and return-from takes 4Tcy.

[mister_e]

Sometime knowing a bit of ASM and MPASM directive is handy. I learned it through Darrel's codeS and postS... it's all it's fault

[JD123]

Sometime knowing a bit of ASM and MPASM directive is handy. I learned it through Darrel's codeS and postS... it's all it's fault

If you have to be corrupted by someone, Darrel seems to be a good choice!

[skimask]

If you have to be corrupted by someone, Darrel seems to be a good choice!

Ya, no kidding, and of course, you're absolutely right about the 9 Tcyc.
A guy should never post 'from the hip', but I do it anyways...

If you use @ NOP?C 1 it will paste 1 NOP and so on.

And this little tidbit right here could be the solution to one of my ongoing 'issues' on another project...

[mister_e]

Great! But you where wrong about the 9Tcy... Read previous post again... or build an ASM project then watch the Program Memory window.