Parsing Serial data stream

Hi All,
Wonder if anyone would take a look at this and tell me where I'm missing it.
This started as http://www.picbasic.co.uk/forum/showthread.php?p=64209 and I have made a bit of progress but I could still use some fresh eyes (and more brain cells) I must be searching for the wrong terms as I can not seem to come up with how to handle the data once I get it.

This seems like it would give me the different bytes (sync, address, and command) but it stumbles when it tries to get the command byte. I get an echo for the sync and the address, but it won't grab the command. Seems to lock up too. Any suggestions? At least a link or two where I can learn more. BTW, I have been devouring Jan Axelson's Serial Port Complete book. Some help, but not on the parsing.

Code:

':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 'Based on Hserin with Darryl Taylor's Instant Interrupts ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ' Program to echo incoming serial data. ' RX SYNC byte, if good: ADDRESS byte, if good: COMMAND ' MPASM, LabX-1 , PBP2.47, Using PIC 16F877A @ 4mHz, 9600 Baud ' Using Hardware USART and MAX232 to MCS Serial Communicator on PC DEFINE OSC 4 DEFINE HSER_RCSTA 90h ' enable serial port, define HSER_TXSTA 24h ' enable transmit, define HSER_SPBRG 25 ' set baudrate to 9600 DEFINE HSER_CLOERR 1 ' automatic clear overrun error TRISD = $00000000 'D to outputs for the LEDs TRISC = %10000000 ' PORTC.7 is the RX input, PORTC.6 is the TX output ADCON1 = %00001111 'Set up ADCON1 register no matter what yr doing!!! ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: INCLUDE "MODEDEFS.BAS" ' Include Shiftin/out modes INCLUDE "DT_INTS-14.bas" ' Base Interrupt System INCLUDE "ReEnterPBP.bas" ' Include if using PBP interrupts ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ' Variable definition ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: RCIF VAR PIR1.5 ' Receive interrupt flag (1=full , 0=empty) TXIF VAR PIR1.4 ' Transmit interrupt flag (1=empty, 0=full) led0 var PORTD.0 led1 var PORTD.1 led2 var PORTD.2 led7 var PORTD.7 led0count var byte led2count var byte state var byte 'storage for condition bits holdoff var word CmdBuf var byte 'command for the pwm or acknowledge SerialData var byte ' ID con "A" 'simple address, later 256 led2count = %00000001 state = 0 '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ASM INT_LIST macro ; IntSource, Label, Type, ResetFlag? INT_Handler RX_INT, _Getbytes, PBP, no endm INT_CREATE ; Creates the interrupt processor ENDASM @ INT_ENABLE RX_INT ; enable RX_INT interrupts '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: hserout ["start"] 'splash screen to verify comm HSEROUT [10] ' CR to start new line on terminal program Mainloop: for holdoff = 1 to 500 pause 1 next holdoff led0count = led0count + 1 led0 = led0count.0 'toggle led every loop for heartbeat goto Mainloop '*********** 'ISR for RX_int interrupt *************************************** Getbytes: While RCIF = 1 ' clear the buffer HSERIN 100,error,[Serialdata] ' Get serial led2count = led2count + 1 ' Incr heartbeat Wend led2 = led2count.0 'led to confirm program went to RX ISR if SerialData = "U" then GoodSync if serialdata = ID then GoodAdr if state = %00000011 then GoodCmd 'good sync and address(ID) state.1=0 'If not Sync & ID, retry @ INT_RETURN GoodSync: hserout ["Sync: ",SerialData] HSEROUT [10] 'Carriage Return state.0 = 1 'set sync good flag @ INT_RETURN GoodAdr: hserout ["Adr: ",serialdata] hserout [10] state.1 = 1 'got a good address @ INT_RETURN GoodCmd: CmdBuf = serialdata 'store the command hserout ["Cmd: ",CmdBuf] 'send Command and CR HSEROUT [10] state.1=0 'reset the address verification @ INT_RETURN error: Toggle led7 @ INT_RETURN end

Thanks for any direction.... At least a sound a horn so that I can get out of the fog :-)

Thanks
Mark

How fast are you sending the data to this PIC? Not like baud rate, but are you typing it? Is it another PIC sending the data to this PIC?
What are you using to send the data to this PIC?
You might be overrunning the serial input buffer because you're busy trying to send data out (multiple bytes) at the same time data is coming in (single bytes) at the same baud rate and therefore the incoming data is getting lost.
Try not HSEROUT'ing until you either get a good 'packet' or a 'bad packet' instead of HSEROUT'ing every step of the way and see what happens.

Hi skimask
Thanks for the reply.

The data has been sent using the Serial Communicator window in MCS. Have been testing using single characters and all three at a time. Trying to just get a framework to start from.
The end app is going to be PIC -> PIC in a master/slave config so the scheme has to be a bit more bulletproof than this. There is one controller and needs to be a significant number of slaves (sort of an Egypt/pyramid kind of thing in case Mel happenings by).

I'll be trying your suggestions and exploring it more this morning.
Thanks again

Bo

Code:

if SerialData = "U" then GoodSync if serialdata = ID then GoodAdr if state = %00000011 then GoodCmd

What if
if state
was changed to
if serialdata
?

Hi boroko,

Code:

Getbytes: While RCIF = 1 ' clear the buffer HSERIN 100,error,[Serialdata] ' Get serial led2count = led2count + 1 ' Incr heartbeat Wend

You're throwing away perfectly good data with that.
You need to process everything that comes in, even if it's not for this chips address.

And you don't need a Timeout in the HSERIN, because it's guaranteed to have a byte in the USART if it triggers an interrupt.<hr>
Well, I wrote this last night. I thought I'd get a chance to test/debug it. But today I've got 5 people coming at me from 4 different directions.
Makes me wonder what those 2 are doing together. :eek:

Hopefully, what I was trying to say will show through, if it doesn't work.

Basically it's, make the Handler as fast as possible. Get in, do your thing, and get out.
No PAUSES, no HSEROUTS, nothing that will keep it from leaving the interrupt as fast as possible.

Anything that needs to take time, should be done in the main loop. Where it can be interrupted again..
Flags are used to trigger those "Main Loop" processes.

Hope it gives some direction ...
If nothing else, note the changes to the Getbytes: handler.

Code:

'::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 'Based on Hserin with Darrel Taylor's Instant Interrupts '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ' Program to echo incoming serial data. ' RX SYNC byte, if good: ADDRESS byte, if good: COMMAND ' MPASM, LabX-1 , PBP2.47, Using PIC 16F877A @ 4mHz, 9600 Baud ' Using Hardware USART and MAX232 to MCS Serial Communicator on PC DEFINE OSC 4 DEFINE HSER_RCSTA 90h ' enable serial port, define HSER_TXSTA 24h ' enable transmit, define HSER_BAUD 9600 ' set baudrate to 9600 DEFINE HSER_CLOERR 1 ' automatic clear overrun error TRISD = $00000000 'D to outputs for the LEDs TRISC = %10000000 ' PORTC.7 is the RX input, PORTC.6 is the TX output ADCON1 = %00001111 'Set up ADCON1 register no matter what yr doing!!! '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: INCLUDE "DT_INTS-14.bas" ' Base Interrupt System INCLUDE "ReEnterPBP.bas" ' Include if using PBP interrupts '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ' Variable definition '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: led0 var PORTD.0 led1 var PORTD.1 led2 var PORTD.2 led7 var PORTD.7 ERROR_LED VAR led7 ' rename the LED's LATE_LED VAR led2 HEART_LED VAR led0 LEDonDELAY CON 4 CmdBuf var byte 'command for the pwm or acknowledge SerialData var byte ' ERRORtime var byte LATEtime var byte HEARTtime VAR BYTE state VAR BYTE Sync VAR state.0 ' Sync byte rcvd ForMe VAR state.1 ' Packet is for this device CmdRcvd VAR state.2 ' Command has been rcvd ERROR VAR state.3 ' Sync rcvd out of order LATE VAR state.4 ' command rcvd before last one was processed SyncByte CON "U" ID CON "A" state = 0 '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ASM INT_LIST macro ; IntSource, Label, Type, ResetFlag? INT_Handler RX_INT, _Getbytes, PBP, no INT_Handler TMR0_INT, _TMR0handler, PBP, YES endm INT_CREATE ; Creates the interrupt processor ENDASM @ INT_ENABLE RX_INT ; enable RX_INT interrupts @ INT_ENABLE TMR0_INT ; enable TMR0_INT interrupts 65.5ms '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: hserout ["start"] 'splash screen to verify comm HSEROUT [13,10] ' CR to start new line on terminal program Mainloop: IF CmdRcvd THEN GOSUB ProcessCMD ' Process any incomming data IF ERROR THEN ' if there's been an Error HIGH ERROR_LED ' Turn ON ERROR LED ERROR = 0 ' reset the error flag ERRORtime = LEDonDELAY ' start countdown till LED-OFF HSEROUT ["Error",13,10] ENDIF IF LATE THEN ' if command took too long HIGH LATE_LED ' Turn ON LATE LED LATE = 0 ' reset the LATE flag LATEtime = LEDonDELAY ' start countdown till LED-OFF HSEROUT ["Late",13,10] ENDIF GOTO Mainloop '*********** Process received command ************************************** ProcessCMD: hserout ["Cmd: ",CmdBuf, " [", _ ' send Command and CR/LF DEC CmdBuf,"]",13,10] CmdRcvd = 0 ' indicate CMD has been processed RETURN '*********** 'ISR for TMR0_INT interrupt *********************************** TMR0handler: IF ERRORtime > 0 THEN ' if the Error LED is ON ERRORtime = ERRORtime - 1 ' decrement the count IF ERRORtime = 0 THEN ' when it reaches 0 LOW ERROR_LED ' turn the Error LED OFF ENDIF ENDIF IF LATEtime > 0 THEN ' if the LATE LED is ON LATEtime = LATEtime - 1 ' decrement the count IF LATEtime = 0 THEN ' when it reaches 0 LOW LATE_LED ' turn the LATE LED OFF ENDIF ENDIF HEARTtime = HEARTtime + 1 ' Toggle heartbeat ~.5sec IF HEARTtime = 7 THEN HEARTtime = 0 TOGGLE HEART_LED ENDIF @ INT_RETURN '*********** 'ISR for RX_INT interrupt ************************************* Getbytes: HSERIN [Serialdata] ' Get the serial data IF Serialdata = SyncByte THEN ' if it's a Sync byte, IF Sync THEN ERROR = 1 ' last command was corrupted Sync = 1 ' indicate Sync was rcvd ForMe = 0 ' ID has not been rcvd ELSE IF Sync THEN IF !ForME THEN ' if we haven't rcvd the ID IF Serialdata = ID THEN ' and this byte matches the ID ForMe = 1 ' indicate ID rcvd ELSE Sync = 0 ' Not my packet ForMe = 0 ' reset receive state ENDIF ' and wait for next Sync ELSE CmdBuf = serialdata ' store the command IF CmdRcvd THEN LATE = 1 ' last command not finished CmdRcvd = 1 ' indicate a command was rcvd Sync = 0 ' reset receive state ForMe = 0 ENDIF ENDIF ENDIF @ INT_RETURN

Hi All,

I sure appreciate that you all were willing to look at my question. I got the first response from skimask, and it prompted me to throw myself at the wall again and see if I could find the door. At some point, you feel that you have explored it to the limits of your ability, but you have to try again. I slowed down and tried to remember advise that I had gotten here by reading through anything that seemed relevant. I could send some data, so send registers to see where they were before they locked or corrupted. That started to reveal some interesting things that didn't have obvious answers.

By adding the last part to these lines:
hserout ["Cmd: ",DEC SerialData," ",BIN3 STATE]

I was able to see that there was a "13" being sent (Line Feed). Hmmmm....

Looking at MicroCode Studio Serial Communicatior's transmit window: A right click opens a selection

window. an option is there for Line Terminator where you can select CR, CR+LF, Null,or NO

terminator. Guess I found my "13" and why nothing was showing until I added DEC to see it.

Further testing is showing that I'm throwing a ONERR even though "DEFINE HSER_CLOERR 1" is in there.
Added "RXbyte = RCREG" to read the receive register but I still get the overrun.

Tried 2400 baud, still acts the same.
tried setting DT's Instant Interrupts macro to ResetFlag = yes: still over-runs the buffer.

Why is it that 20 quick "A"s or "U"s don't over-run the buffer and any data does?

Just as a way to check some of the questions, I closed Communicator and tried talking to it with RealTerm. I'll be darned if that didn't clear things right up. Evidently, after cleaning up a bit, it actually works!

I'm going to look carefully at your feedback and incorporate it, but I wanted to show what I have working so far:

Code:

hserout ["start"] 'splash screen to verify comm HSEROUT [10,13] ' CR to start new line on terminal program Mainloop: for holdoff = 1 to 500 pause 1 next holdoff led0count = led0count + 1 led0 = led0count.0 'toggle led every loop for heartbeat goto Mainloop '*********** 'ISR for RX_int interrupt *************************************** Getbytes: While RCIF = 1 ' clear the buffer HSERIN 100,error,[Serialdata] ' Get serial led2count = led2count + 1 ' Incr heartbeat Wend led2 = led2count.0 'led to confirm program went to RX ISR if SerialData = "U" then GoodSync 'must have good sync: 01010101 if SerialData = "=" then GoodaDR 'BROADCAST address to all units if SerialData = ID then GoodAdr 'unique address for each slave if state = %00000011 then GoodCmd 'good sync and address(ID) addr = 0 'If not Sync & ID, clr addr good flag @ INT_RETURN GoodSync: sync = 1 'set sync good flag hserout ["Sync: ",SerialData] HSEROUT [10,13] 'Carriage Return @ INT_RETURN GoodAdr: addr = 1 'got a good address hserout ["Adr: ",SerialData] hserout [10,13] @ INT_RETURN GoodCmd: cmd = 1 CmdBuf = SerialData hserout ["Cmd: ",SerialData," ",DEC CmdBuf]'send Command and CR HSEROUT [10,13] addr = 0 'reset the address verification CMD = 0 'reset the command received flag @ INT_RETURN

Well I am going back into non-internet land for the day, so I'll update after I get back.

Again. I can't thank you and the list enough. I strive not to be a "code-sucker" but there really is a lot of stuff that can trip you up and a lot of tiny things that will frustrate you greatly. Think of it as raising the next generation of question answerers ;-)

Bo