Where should I discuss SD/MMC FAT issues?

[JD123]

Hi Brian,

This is where I'm at. I can format the MMC on the PC, make a singel file using the PC, find the directory listing for the file using the PIC, find the file using the PIC, read and modify the file on the PIC and read the results on the PC via the MMC card reader.

What I now need to do is modify the directory listing to reflect the changes made to the file, so that the PC can understand the file information correctly. For example, I originally made the file "TEST123.TXT" 10 bytes long with the string "1234567890" so that it was easy to search the MMC to locate the file. As I made changes to the file using the PIC, if I went over the original 10 byte length, the data gets recorded, but the PC only sees the original first 10 bytes.

I found the file size information in the directory listing at offset 28 and this falls into what I was expecting based on that I've read. Problem is that the file size information is a 4 byte Dword and FAT specs call for the 4 bytes to START at offset 28. In reality, I would have thought that the "10" for the file size would have been in offset 31 (LSByte of the 4 bytes). I'm trying to figure out where I place the additional 3 bytes of the file size information. I can't find anything to support that the file size should be written low byte first and that wouldn't make sense anyway. So, I'm thinking that there's something wrong with my understanding of where the bytes are stored.

Anyway, this is where I was last night. I was tickled-pink just to see that I could get the PIC to make changes to the MMC that the PC could read. One thing that became real clear last night was that I'll have no choice but to add RAM to my hardware. I've got to be able to work on a sector's individual bytes and that's only going to happen if I have it in RAM and then send it back to the MMC when I'm done tinkering with the sector. I happen to have some 64KBit I2C EEPROMS handy, so tonight I'll press one of them into service on the proto board. This alone will put the breaks on FAT work for a few days (I only get a few hours a day to 'play' with PICs) while I deal with the hardware and I2C code.

I'll post updates as they come. If anyone can help me understand the offset location and data style (MSB or LSB first) of the file size data I'd appreciate reading your thoughts.

Gary

Edit: Brian, I forgot to say thanks for the links. I've been to both before and the second one was very helpful in getting me started.
--

[skimask]

I found the file size information in the directory listing at offset 28 and this falls into what I was expecting based on that I've read. Problem is that the file size information is a 4 byte Dword and FAT specs call for the 4 bytes to START at offset 28. In reality, I would have thought that the "10" for the file size would have been in offset 31 (LSByte of the 4 bytes). I'm trying to figure out where I place the additional 3 bytes of the file size information. I can't find anything to support that the file size should be written low byte first and that wouldn't make sense anyway. So, I'm thinking that there's something wrong with my understanding of where the bytes are stored.

Isn't the file size in FAT32 stored at 2 different places? And both of them have to be identical? I forget...it's been awhile since I read the FAT32 documentation...

I'll have no choice but to add RAM to my hardware.

I built an MP3 player awhile back based off an 18F4620 reading from CF cards in IDE mode. Had enough ram to buffer 6 complete sectors with room left over for a load of variables.
Also, if that's not enough, look into the F-RAM options. Serial EEPROM just like a 24C256 type, but it's ram, read/write just as fast, not slow writes/fast reads like eeprom, and practically unlimited lifetime.

understand the offset location and data style (MSB or LSB first) of the file size data I'd appreciate reading your thoughts.

Keep track of your little and big ENDIAN formats according to the FAT32 documentation. I don't know how many times that sort of thing tripped me up bad...

EDIT: Just dug out the FAT32 doc file.
Is the file you're basing your FAT32 code from called 'fatgen103.doc'?
If so, that file has a few issues in itself, wrong places for some values, some values omitted, and so on...
gain, it's been awhile. I'll see if I can't dig up the specifics.

[JD123]

Hi Skimask,

Isn't the file size in FAT32 stored at 2 different places? And both of them have to be identical? I forget...it's been awhile since I read the FAT32 documentation...

Sorry, I forgot to say that I'm using FAT16 when I format the MMC. It's the native format that the MMC's came with. I don't know what version of FAT16 I'm using. I understand there at least several versions or revisions to the FAT16 system. The MMC's I'm using are 128MB in size.

...look into the F-RAM options.

Yes, for a final in-field application this would be the way to go. As it is, I'm cheap and have the EEPROM's on hand. My application is data recording at about 100 sectors a second. For writing the data it will be sent directly to the card. The EEPROM use in my ap would only be at the first and end of the recording events (about 10-15 minutes long) and this would be just to address the FAT issues.

I'm back off to some of my web haunts where I've been reading about FAT16. Maybe I'll find the answer by going back over these sites. I do have this odd recollection of there being some part of the FAT where data is written LSByte first. Maybe it was the file size?

Gary
--

[skimask]

Sorry, I forgot to say that I'm using FAT16 when I format the MMC.

That fat32 document has a load of info in it, and specifies where it differ's from fat16...not that it matters much

My application is data recording at about 100 sectors a second.

I was able to shove a bit over 600K/sec into the CF card on my player, but that's using the IDE mode, parallel data transfer. 600K was as fast as the PIC could go @ 40Mhz.
You'll be limited to the max SPI speed, and I think you'll be lucky to be able to handle 100 sectors/second (100 sectors = 512000 bytes = 5.12mbps) and still have any time leftover to do any sort of interrupt processing.

I do have this odd recollection of there being some part of the FAT where data is written LSByte first. Maybe it was the file size?

Do a search for fatgen103.doc and you'll see what I'm looking at.
And yes, offset 28 is the file size, and yes, it is LSB first...little endian...

[JD123]

YES! I came back here to say I found this, only to see you found it too. The size IS written LSByte first. I can't tell you how many places on the web fail to state this. This is what I just now found:

data length ( =4 bytes ) 

the minimum can be 00 00 00 00 - 0 and the maximum ff ff ff ff -
 4,294,967,295
gb, mb, kb, by

this entry will state the length of data to read in.
eg, AC 3B 96 00
flip these values: 00 96 3B AC
convert to decimal: 9,845,676 bytes or 9.38MB
folders have a value of: 00 00 00 00

Thanks guys for getting me out of this ditch. It's amazing how much time I waste on some of the 'simplest' problems.

skimask, thanks for letting me know what you were able to get for speed. I'm shooting for 100, 8 byte recordings per second, but if I can make 60 I'll be happy. The recordable rate will be selectable and I was picking 100 as a top-shelf goal.

Edit: skimask, sorry... I see what caused the issue about speed. I originally ment to say ONE sector per second, not 600. 600 is the byte rate, not the sector rate. I guess I really WAS pushing the limits, the way I originally stated it!!!

[skimask]

skimask, thanks for letting me know what you were able to get for speed. I'm shooting for 100, 8 byte recordings per second, but if I can make 60 I'll be happy. The recordable rate will be selectable and I was picking 100 as a top-shelf goal.

100 8 byte recordings per second? That's only 800 bytes... I said 600KB per second, as in, 614,400 bytes per second, 6,144,000 bits per second.
But again...it was a parallel operation.

Now are you talking 100 8 bytes recordings...
or 100 512 byte (sector) recordings

[JD123]

100 8 byte recordings per second? That's only 800 bytes... I said 600KB per second, as in, 614,400 bytes per second, 6,144,000 bits per second.
But again...it was a parallel operation.

Now are you talking 100 8 bytes recordings...
or 100 512 byte (sector) recordings

Yes, I messed up. See my edit.

That should be quite reachable. The real issue is reading the PWM outputs from the accelerometers and an airspeed pressure sensor. My co-worker on this project wants to read the PWM as filtered analog resolved from a 1ms PWM frame rate. I'm arguing that I can get everything done and more in the 16.66ms that we have at 60 recordings per second. Also, using the CCP and MUX'ing the inputs I can get rock solid data with a 5000 FS range. On a +/-10G accelerometer that will be .4mG resolution - well enough for what we are doing.

Gary
--