how to store a very large amount of data

[shaiqbashir]

Hi guys!

Im using serial communications in labview. That is connected with PIC16F877A with serial interface. The labview will send a bit stream of data that will be greater than the WORD sized variable of PIC. it may be arount 256 bits at a time. I want to know a method, by which i can store these 256 bits in my PIC and then transfer them to 24C128 memory. Is there any way of doing it? the largest variable that this PIC is WORD, it means that i can save a 16-bit data simultaneously. But what when i have to store such a big data in my PIC? is there any way of doing it?

Regards,

Shaiq Bashir

[muddy0409]

I have not tried this but: If I was to need to store such a big variable I would do something like:

info VAR bit[256]


This would hold 256 bits of data which is the maximum array size for BIT variable.

Once the variable is full, or the serial in is complete I would then look at sending the data in the same sort of format to the eeprom.

As I said, I have not done this myself, but that is where I would start playing around.

Lots of luck.

[SteveB]

Since you are using a Serial Interface, I would guess that the data is being sent/received in Bytes. So you are really looking at an array of 32 bytes. Some great info can be found in this thread by Melanie (http://www.picbasic.co.uk/forum/showthread.php?t=544).

Here is the summary. Use an array of bytes like...

MyDataArray var BYTE [32]

Now, if you need to address a specific bit in that array, you can do this like so...

MyDataArray.0(bitnumber)

Remember, that arrays begin at the 0 index. So, using the previous example...

If bitnumber=0 then it will reference MyDataArray(0) bit(0)
If bitnumber=7 then it will reference MyDataArray(0) bit(7)
If bitnumber=8 then it will reference MyDataArray(1) bit(0)
If bitnumber=15 then it will reference MyDataArray(1) bit(7)
....
If bitnumber=248 then it will reference MyDataArray(31) bit(0)
If bitnumber=255 then it will reference MyDataArray(31) bit(7)

Just one way of many to get this done.

SteveB

[Charles Linquis]

If you move to the 18F series of chips, array size can be almost as large as memory. 2K bytes is no problem.

[JD123]

32 bytes should be easy to store in the 877's ram.

What's the serial format? Is it sending 256 bit (32 bytes) as uninterrupted or unformatted bits?

My current project has MMC > PIC > I2C EEPROM (F-RAM) moving 4,096 bits (512 bytes) in a shade under 21ms, or an effective baud rate of close to 200k. I don't store the values in the PIC except for a 'working byte', I just pass them through the PIC. Depending on the serial format you're using, you may not need to store the data in PIC RAM if using a FRAM I2C EEPROM. If your serial format allows for data bus holds, you can use standard I2C EEPROM's and not have to store more than 1 or 2 bytes in the PIC at any given time.

It all depends on the serial format.

[shaiqbashir]

Thank you for ur replies.

I want to tell you that im using Labview serial transmission. It is asynchoronous serial transmission. SO it can send 256 bits serially one after the other. On the other end PIC is used along with Max232. im using PIC hserin command to handle this 256 bits of data. But as you know it requires some variable to work with. I want to know that is this possible to use such a big variable that can store 256 bits with hserin command?

my basic task is to first store these 256 bits first and then transfer them to 24C128 eprom, is there any way to transfer directly to eprom without storing any data?

[duncan303]

Hi


I want to know that is this possible to use such a big variable that can store 256 bits with hserin command?

You seem to be asking two questions here but presenting them as one. As SteveB indicates take a look at Melanie’s thread on Bits bytes words and arrays in the FAQ section, and it should help you over the first problem.


it may be arount 256 bits at a time

although 256 BITS are the maximum number of elements in a BIT array. Think about
creating a BYTE or WORD array and addressing the individual BITS in those arrays. Therefore a maximum number of BITS available in an ARRAY (WORDS or BYTES) for the 16F877 is 768 BITS.



But what when i have to store such a big data in my PIC?

Do you mean you wish to take up the least code space on your device caused by creating an array to be used as a buffer?.

You could consider “preparing your data for transmission” cutting it up into chunks that you then send to your receiving PIC and them reassemble as you wish, then either store them to your external eeprom or use it. HSER has a hardware buffer of two bytes, you could use this as starting point.

Of course once you disassemble your BIT array, you will need to keep track of the sequence.

But I really do think the answer for you, as others have indicated, lies in Melanie’s Post Bits Bytes words and arrays.


I have a project which has a section that combines data contained in several variables of all types Bits bytes and words that is combined into a single array and sent through HSER over an RF link The transmission length is variable. ( I do not think there is any reason that it cannot be up to the maximum for a particular device, I havn't tried it with LONGS to break the 48 WORD barrier, maybe others have )


Anyway mine works well with no probs. Just watch out how you communicate the very first array type following a PIC reset, there have been several posts on the forum about this point.

HTH

Duncan

[mister_e]

I don't and i'm not going to use LabView... but i would hope it could send BYTES instead.. unless it's just plain stupid, unusefull and pointless.

So you just need a 256 byte sized array variable and a HSERIN STR... maybe For-Next loop+Hserin... of for-next loop+xyz=RCREG.. something easy.

You could probably get a Byte, then write it to your EEPROM.. but probably PBP built-in I2CWRITE will be too slow, You have to measure it... or try to calculate it.

Why Labview?

[sougata]

I don't and i'm not going to use LabView... but i would hope it could send BYTES instead.. unless it's just plain stupid, unusefull and pointless.

So you just need a 256 byte sized array variable and a HSERIN STR... maybe For-Next loop+Hserin... of for-next loop+xyz=RCREG.. something easy.

You could probably get a Byte, then write it to your EEPROM.. but probably PBP built-in I2CWRITE will be too slow, You have to measure it... or try to calculate it.

Why Labview?

Hi Steve,

Just curious Do you think the PIC USART would be able to handle bits without defined start and stop bits. I think until and unless it is in a standard serial transmission the USART cannot be used. Possibily Shaig is using this in a college LAB environment and I doubt his serial hardware is the PC RS232 used by Labview. Then why bits and bytes .... I am confused. If it is bytes then there are tons of example here. Am I loosing my mind.... Its 31st March and the end of another FISCAL showing I didn't do well. Shall I start selling beers .. at least have one confirm customer.

[mister_e]

Sougata, i don't know how Labview work, but i would suggest to use a SerialPort sniffer software to see what happen on the selected comport first. SerialPort sniffer software are a real handy piece of software for me, for reverse engineering or new product development.

But, there's also the naming convention... bytes, bits may sound the same if you don't know the difference between them.... well that's what i'm thinking.

[sougata]

Hi,

Being majorly a graphical development tool focused on easy acquisition, analysis, reporting and signal generation tool I don't think it would be wise to use the I/O in bit dump. Anyway you do not acquire in bit or compute.

So Shaiq please confirm the bit/byte thing. Are you sure its 256bits being dumped ? Then how ? PRZ ?

[BrianT]

1. LabView works on almost any PC.
2. LabView works with a great many vendor's Data Acquisition cards & systems - these almost universally have RS232 serial interfaces except for high speed specials like SCSI.
3. PC's do NOT have clocked serial interfaces - they ALL have byte oriented asynchronous serial ports.
4. LabView may well talk of 256 'bits' being transferred but I am 99.999% certain this will be packaged up as 32 characters each in 8N1 format.

HTH
Brian

[precision]

DasyLab is better than Labvieu, I Purchased labview for X,Y plotting, but serial port not support well, Line was disturbed, Then i purchased dasylab. Only in three steps you can capture data from pic and send data to pic. Support of labvieu is also poor (in india).

Sample of ( snap shot )dasylab include , i have maked this only in 20 seconds.

[sougata]

Well I am not sure. Dasylab was originally a measurement computing product the guys who wrote softwire. Now it has been acquired by NI.