How to receive stream of bytes using PIC USART

[unifoxz]

Hi,

I would like to know how does PIC read the stream of bytes from serial port? I was trying to read in stream of bytes at 4800 baud rate but the data crashed. The received data is not the expected data.

I welcome any idea and thank you in advance.

[mackrackit]

Welcome to the forum.

With out seeing your code it is hard to help you figure out where the problem is.

With out seeing your code I will have to point you to the PBP manual and the STR option. STR lets you read in a string to an array where it can be used for something.

[Ioannis]

Are you reading a string from GPS?

Ioannis

[unifoxz]

No, I am reading data from a SpO2 module. The module sends out 5 bytes in sequence continuously. Among the bytes sent out from module, I only need to extract two of them which provide the information of the SpO2 and pulse rate. I used software UART to communicate with the SpO2 module while hardware UART to communicate with PC UART. When I connect the SpO2 directly to the PC, the data is correct. However, when I connect the SpO2 module and PC through PIC16F877, the data all mess up. I think it's probably because of the insufficient buffer in the PIC. I never handle such problem and I failed to solve the problem after trying for the whole day. What can I do to solve this problem?

Here is the PIC code written in PICBasic Pro:

INCLUDE "modedefs.bas"
DEFINE LOADER_USED 1        
DEFINE OSC 20 
B0 VAR BYTE
B1 VAR BYTE
B2 VAR BYTE
B3 VAR BYTE
B4 VAR BYTE
B5 VAR BYTE
B6 VAR BYTE
B7 VAR BYTE
B8 VAR BYTE
PR VAR BYTE
SPO2 VAR BYTE
ID_2 VAR BYTE
ID_1 VAR BYTE
ID_0 VAR BYTE

'PB.7 = Rx
TRISB = %10000000 

'data acquisition ID
ID_2 = "0"
ID_1 = "1"
ID_0 = "1"

'request from PC, 9600 baud rate
standby:
SerIn PORTC.7,6,["P",ID_2, ID_1, ID_0]

loop:
'4800 baud rate, 
SerIn2 PORTB.7,16572,[B0, B1, B2, B3, B4, B5, B6, B7, B8]

' detect synchorous bit (the header byte in 5 bytes data)
IF B0 > 127 Then  
GoTo b0_true
EndIF

IF B1 > 127 Then 
GoTo b1_true
EndIF

IF B2 > 127 Then 
GoTo b2_true
EndIF

IF B3 > 127 Then 
GoTo b3_true
EndIF

IF B4 > 127 Then 
GoTo b4_true
EndIF

GoTo loop

' extract PR and SpO2 data
b0_true:
IF B2 > 63 Then 
    B2=64
Else
    B2=0
EndIF
B3=B3+B2
PR=B3
SPO2=B4
GoTo send_data

b1_true:
IF B3 > 63 Then 
    B3=64
Else
    B3=0
EndIF
B4=B4+B3
PR=B4
SPO2=B5
GoTo send_data

b2_true:
IF B4 > 63 Then 
    B4=64
Else
    B4=0
EndIF
B5=B5+B4
PR=B5
SPO2=B6
GoTo send_data

b3_true:
IF B5 > 63 Then 
    B5=64
Else
    B5=0
EndIF
B6=B6+B5
PR=B6
SPO2=B7
GoTo send_data

b4_true:
IF B6 > 63 Then 
    B6=64
Else
    B6=0
EndIF
B7=B7+B6
PR=B7
SPO2=B8
GoTo send_data

'send to PC, 9600 baud rate
send_data:
SerOut PORTC.6,6,[ID_2, ID_1, ID_0, "^P^", #PR, "^", #SPO2]

GoTo standby

End

[Ioannis]

I think something is not clear here.

You say that the following piece of code is requesting from PC,but I see that it is just waiting fo the string of characters to arrive.

'request from PC, 9600 baud rate
standby:
SerIn PORTC.7,6,["P",ID_2, ID_1, ID_0]

After that the next lines are executed, but are you sure you catch the beginning of the transmission? How can you be sure about that?

It is better to wait for the device to send a preample or a start character and the store the array of the 8 bytes.

instead of this:

loop:
'4800 baud rate, 
SerIn2 PORTB.7,16572,[B0, B1, B2, B3, B4, B5, B6, B7, B8]

do something like this:

my_array var byte[8]

loop:
'4800 baud rate, 
SerIn2 PORTB.7,16572,[wait("abc"), str my_array\8]

Ioannis

[mackrackit]

Another variation that might be handy in your case

SERIN2 PORTB.1,16572,[WAIT("ABC"),DEC1 VAR1, SKIP1, DEC1 VAR2, SKIP1, DEC1 VAR3]

The above will save every other character to a variable. Mix and match as needed.
Many ways to play/parse in coming strings.

[unifoxz]

I am sorry for the confusion. Actually, the following code is to wait for the PC to send the request before PIC read in the data from the SpO2 module.

'request from PC, 9600 baud rate
standby:
SerIn PORTC.7,6,["P",ID_2, ID_1, ID_0]

Once PIC get the matching ID from PC, the PIC will read the data from SpO2 module to extract the SpO2 byte and pulse rate byte from the SpO2 module. Since PC and SpO2 module are using different baud rate (which PC is 9600 baud rate and SpO2 module is 4800 baud rate), I can't put them in the suggested code line.

The PIC need to capture 2 bytes from a stream of bytes sent by the SpO2 module. The SpO2 module sends the bytes in sequence and continuously. I get the wrong data but I think there is nothing wrong with my algorithm to capture the intended data, by referring to the manual of the SpO2 module.

My guess is that the PIC's receiver buffer has overflowed and causing the data clash. How should I avoid the buffer overflow so that I could retrieve the correct data byte from the byte stream of the SpO2 module?

[mackrackit]

Sorry, but this is confusing..

How does the PC know when to tell the PIC that the time is right to receive data?

Why not have the PC work at 4800 baud also?

Seems like by the time the PC some how decides the time is right and sends this to the PIC the time would have passed for the data?

[unifoxz]

How does the PC know when to tell the PIC that the time is right to receive data?

The data from SpO2 module is sent out continuously. The updated data is sent out from the module all the time. The PC just send request whenever it wants to and the PIC will responsible to grab a few current data bytes from the SpO2 module and extract out the targeted bytes, which are SpO2 data byte and pulse rate data byte.

Why not have the PC work at 4800 baud also?

I need to send the data through a wireless transceiver module which works at 9600 baud rate eventually. For testing purpose, I am connecting the PIC directly to the PC UART instead of the transceiver.

[unifoxz]

I have tried with sending out data bytes that received from the modules to PC immediately but the data displayed on the PC is different from the data that I connected the module directly to PC. It seems like the data of the bytes stream corrupts when it passes through the PIC.

This is the testing code:

INCLUDE "modedefs.bas"
DEFINE LOADER_USED 1    	
DEFINE OSC 20 
B0 VAR BYTE

'PB.7 = Rx, PB.6 = Tx
TRISB = %10000000 

loop:
'4800 baud rate
'read byte from SpO2 module
SerIn2 PORTB.7,16572,[B0]
'send byte to PC
SerOut2 PORTB.6,16572,[B0]
GoTo loop
End

What should I do to avoid the data from crashing in Rx buffer of PIC UART, provided I cannot control over the stream of bytes received?

[ScaleRobotics]

Are you sure it's not just an inverted vs true problem? If it is a buffer problem, you could switch to the hardware uart and use something like they do for reading gps strings:

MAXDATA      CON 70       'The buffer Size for GPS Data  was 70  
gpsdata         VAR BYTE[MAXDATA]    '  70 byte array.
HSerin 200, Main, [WAIT("$GPRM"), STR gpsdata\MAXDATA\13]

[unifoxz]

I was thinking it could be inverted vs true problem too. However my assumption was found to be fault after I receive unknown characters on the screen of PC.
I also have tried similar method as suggested but the problem remains. The PIC reading is not the supposed data.

[mackrackit]

Here are several examples of serial in and out.
http://www.melabs.com/resources/samples/pbp/ser2mod.bas

Now for the

What should I do to avoid the data from crashing in Rx buffer of PIC UART, provided I cannot control over the stream of bytes received?

You are using software for serial so the only time anything gets into the PIC is when you ask for it. If you were using hardware then the buffers would need to be cleared.

'send byte to PC
SerOut2 PORTB.6,16572,[B0]

Earlier you said the PC was running the port at 9600? Did you change it for the above test to 4800?

And modifiers. The examples on the above link talks about DEC and such VS ASC||. You might need one.

[unifoxz]

Yes, I tried to set up a simple test to check what is actually happening on the bytes I receive. I use 4800 baud rate for both module and PC this time.

You are using software for serial so the only time anything gets into the PIC is when you ask for it. If you were using hardware then the buffers would need to be cleared.

I tried with PIC hardware UART to receive the bytes from module too but the result is the same. The data is not the one it should be.

May I know how to clear hardware UART buffer? I tried to reset CREN and read in the data after the reset, it is not working as well. I am not sure whether it is the way to clear the buffer.

[ScaleRobotics]

This automatically clears it, but then you can't read what it stuck on. But seeing as this data gets resent over and over, it really should be a problem. You should catch it on the next round.

DEFINE HSER_CLROERR 1                  'Auto Reset Buffer Overrun Errors

[mackrackit]

If you do not want to setup all of the defines and want to change things around on the fly here is an example that I use for testing on a particular chip. Displays the serial input on an LCD. The RCSTA.4 part clears any overruns.

Did you have a MAX232 or equivalent when you tried the hardware serial?

'******************************************
'18F6680   02/14/09  INFEED PARSE TEST BAUD 9600
    DEFINE OSC 20
    @ __CONFIG    _CONFIG1H, _OSC_HS_1H
    @ __CONFIG    _CONFIG2H, _WDT_ON_2H & _WDTPS_128_2H
    @ __CONFIG    _CONFIG4L, _LVP_OFF_4L
    DEFINE LCD_DREG     PORTG 
    define LCD_DBIT     0
    DEFINE LCD_RSREG    PORTE 
    DEFINE LCD_RSBIT    0
    DEFINE LCD_EREG     PORTE 
    DEFINE LCD_EBIT     1
    DEFINE LCD_BITS     4 
    DEFINE LCD_LINES    4
    DEFINE LCD_COMMANDUS    3000 
    DEFINE LCD_DATAUS   150
  '###############################################
    PAUSE 100 : LCDOUT $FE,1,"TEST"
    N1 VAR LONG:N2 VAR LONG
    START: N1 = 0 : N2 = 0
    HIGH PORTG.4 :PAUSE 250:LOW PORTG.4
    RCSTA.4 = 0 : RCSTA.4 = 1
    'CHANGE LINE FEED AND CARRIAGE RETURN AS REQUIRED 
    RCSTA=$90:TXSTA=$24:SPBRG=129:HSERIN[WAIT($a),WAIT($d),DEC N1,WAIT(","),DEC N2]  
    LCDOUT $FE,1,DEC N1 : lcdout $FE,$C0,DEC N2 : GOTO START

[Ioannis]

What is the exact data sent from the SpO2 module? Is there any header that can be identified befor getting data?

I mean does the module send out "Hey! Data coming" and then the measurements each time?

If the measurements are sent out continusly without any header, then your PIC is reading at random times, confusing the data frames between them.

You have to find a way to just wait for the start of each frame and then grab the next data.

Ioannis

[unifoxz]

mackrackit: Did you have a MAX232 or equivalent when you tried the hardware serial?

I don't use MAX232 for this circuit. I use a 22k resistor at Rx pin and 1k resistor at Tx pin. May I know is there any effect for not using MAX232? There is another question that I would like to know: can MAX3232 be used to replace MAX232?

Ioannis: What is the exact data sent from the SpO2 module? Is there any header that can be identified befor getting data?

The first byte of the 5 bytes data block starts with the synchronous bit of '1' while the other bytes start with bit '0'. This is the only identifier I could use to detect the the data bytes (SpO2 and pulse rate) I need.

I tried to check the bytes sent from the module since the starting of the module operation. The data might be correct for the first few bytes but the continuous bytes are all wrong.

My problem is similar to retransmitting a big file (in MB) through PIC. The file sent out from the PIC is not the same as the file before it was sent to the PIC. The PIC cannot handle a stream of continuous bytes and thus causing data went wrong.

[mackrackit]

I don't use MAX232 for this circuit. I use a 22k resistor at Rx pin and 1k resistor at Tx pin. May I know is there any effect for not using MAX232? There is another question that I would like to know: can MAX3232 be used to replace MAX232?

Yes, I used a 232. From the PIC Hardware Serial to a PC some kind of level inverter is needed. The PICs sends in true mode while PCs use inverted. It is a hardware thing that can not be fixed in soft ware . That is why we have SERIN/2, bit banging a "virtual serial poer".

I think the 3232 will also work but I have not used one to be sure about it.

If the module is always sending a 1,then use that in the WAIT part, grab what you need after that. What are the possibilities for the remaining data? 0-9-A-Z?

[Ioannis]

Dave, by 1 he means that the first bit of the first byte is "1" and the rest are first bit "0".

So wait cannot be used just like that.

unifoxz:
What is the repeat rate of the bytes? 8 bytes at 4800 and then a pause of some seconds or milliseconds?

You might just wait for the current transmission to finish, as you may have missed the begining of it, then wait for the pause state and grab the next 8 bytes.

But wait, you now say it is 5 bytes? At your first post I noticed you tried to get 8 bytes of data, right?

Ioannis

[unifoxz]

It's repeating 5 bytes. 5 bytes continued by another 5 bytes without pause, all are sent out continuously. In order to get the SpO2 byte and pulse rate byte which is at the 4th and 5th byte from the header byte (the byte with synchronous bit = 1), I need to read in 9 bytes as if the first byte I get is the 2nd byte of the 5 byte. So the header byte will be at the 5th byte and the SpO2 byte and pulse rate byte will be at the 8th byte and the 9th byte.

It's like this:
2, 3, 4, 5, 1 (header byte), 2, 3, 4 (pulse rate byte), 5 (SpO2 byte)

All the data is in hex format, no specific 0-9-A-Z.

[Ioannis]

I cannot imagine a device keep sending a blast of bytes without any synchronization!

Please give us a link for the data sheet or if small post it here.

Ioannis

[dhouston]

Many devices send continuous blasts of data.

Here's a definition for spo2 http://www.neann.com/spo2.htm. A manual for the specific device would be helpful.

[Ioannis]

I did a little search in the web and found that the modules for medical applications use the so called BCI protocol with 4800 8-ODD-1 serial setup.

Data frames are 5 bytes long and on the first byte the 7th bit is 1 while on the restbit 7 is set to 0.

So, may be you get one byte at a time and check the bit 7 only for it to be 1. Then get the rest and store in a byte array of 5 elements.

If there is not a sync byte or a significant pause between frame transmissions, PIC cannot discriminate the start byte. That is why you get confussing results.

One untested example might be:

array    var   byte[5]
index    var   byte

index=0
get_new_data:
SerIn2 PORTB.7,16572,[B0]
if B0.7=1 then
    array[index]=B0
    index=index+1
endif

if index=5 then process_data

goto get_new_data

Ioannis

[dhouston]

I did a little search in the web and found that the modules for medical applications use the so called BCI protocol with 4800 8-ODD-1 serial setup.

Where did you find the protocol description? The only reference I found indicated it uses two stop bits.

[Ioannis]

Hi Dave. I found the attached pdf that is implying the protocol.

Ioannis

[unifoxz]

The SpO2 module I use is of none parity.

I wonder the PIC have the problem to read the bytes if the bytes are sent into PIC in endless stream. I tried to send only 5 bytes, 10 bytes and 15 bytes data in SpO2 module's data format and the PIC can process well using the code I wrote.

Then I tried to let PIC sent out each byte it read in from a endless bytes stream. The output of the PIC is different from the data it read in.

[dhouston]

The SpO2 module I use is of none parity.

That's not the issue. How many STOP bits is the issue. If, as the only reference I could find for the BCI protocol says, it uses 8N2, you cannot handle it - all of the PBP methods require 1 stop bit. Two stop bits could explain the erratic results. You need to provide a link to the datasheet for the specific SP02 device you are using.

[Ioannis]

You need to provide a link to the datasheet for the specific SP02 device you are using.

I double that, although have asked for it earlier.

It seems you loose synchronization. But we are only guessing here as we have no informations about you device.

Ioannis

[unifoxz]

I try my best to translate the manual as the manual is originally written in Chinese language. The manual is too simple and not really provides the details.

serial port setting:

• 1 start bit + 8 data bit + 1 stop bit, no parity
• baud rate: 4800 baud
• 5 bytes format, 60 packages per second, bit 7 is the synchronous bit

byte 1:

• bit 7: synchronous bit, as 1
• bit 6: 1=instruction for pulse sound
• bit 5: 1=SpO2 decrease, 0=OK
• bit 4: 1=searching time too long, 0=OK
• bit 3-0: signal strength (0-8), represent signal strength of pulse

byte 2:

• bit 7: synchronous bit, as 0
• bit 6-0: pulse wave diagram

byte 3:

• bit 7: synchronous bit, as 0
• bit 6: pulse rate bit 7
• bit 5: 1=search pulse, 0=OK
• bit 4: 1=sensor error, 0=OK
• bit 3-0: pulse bar diagram

byte 4:

• bit 7: synchronous bit, as 0
• bit 6-0: pulse rate, (bit 6 to bit 0)

byte 5:

• bit 7: synchronous bit, as 0
• bit 6-0: SpO2

[dhouston]

OK - it has only one stop bit. It appears that various manufacturers use their own protocol.

60 packets per second x 5 bytes per packet x 10 bits per byte = 3000 bits per second which, at 4800bps, means 30 (or fewer - depends on pace) idle bit-times between packets.

I would capture 10 bytes, scan them for the first initial byte (>127) and then look at the following 4 bytes for your data.

EDIT: 9 bytes would guarantee one complete packet.

[Ioannis]

Hmm, seems that is like the pdf I posted. Is it the same module?

Based on the infos and the idea of Dave's:

array    var   byte[9]
temp    var   byte
i          var   byte
j          var   byte

get_new_data:
SerIn2 PORTB.7,16572,[str array\9]
for i=0 to 8
    temp=array[i]
    if temp.7=1 then goto send_at_9600
next i
hserout ["Not found!",13,10]
goto get_new_data

for j=i to i+4
hserout [array[j]]
next j

goto get_new_data

end

Hope this will get you started. Don't forget the defines for the Hserout command to set it at 9600.

Ioannis

[dhouston]

You need only scan the first five bytes to find the start of a packet. And you can save one step on each loop.

for i=7 to 39 STEP 8
    if array[0].i=1 then goto send_at_9600
next i

[unifoxz]

Thanks for the suggestions, but the result I get is still not correct.

Have anyone tried to send an endless stream of bytes to PIC and then resend the stream back to PC? Will the input bytes same as the output bytes?

[Ioannis]

Please post you complete code as is now.

Of course the PIC will send the correct bytes as received, why shouldn't??

Ioannis