Parsing Serial data stream

[boroko]

Hi All,
Wonder if anyone would take a look at this and tell me where I'm missing it.
This started as http://www.picbasic.co.uk/forum/showthread.php?p=64209 and I have made a bit of progress but I could still use some fresh eyes (and more brain cells) I must be searching for the wrong terms as I can not seem to come up with how to handle the data once I get it.

This seems like it would give me the different bytes (sync, address, and command) but it stumbles when it tries to get the command byte. I get an echo for the sync and the address, but it won't grab the command. Seems to lock up too. Any suggestions? At least a link or two where I can learn more. BTW, I have been devouring Jan Axelson's Serial Port Complete book. Some help, but not on the parsing.



'::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::
'Based on Hserin with Darryl Taylor's Instant Interrupts
'::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::
' Program to echo incoming serial data.
' RX SYNC byte, if good: ADDRESS byte, if good: COMMAND
' MPASM, LabX-1 , PBP2.47, Using PIC 16F877A @ 4mHz, 9600 Baud
' Using Hardware USART and MAX232 to MCS Serial Communicator on PC

DEFINE OSC 4
DEFINE HSER_RCSTA 90h ' enable serial port,
define HSER_TXSTA 24h ' enable transmit,
define HSER_SPBRG 25 ' set baudrate to 9600
DEFINE HSER_CLOERR 1 ' automatic clear overrun error
TRISD = $00000000 'D to outputs for the LEDs
TRISC = %10000000 ' PORTC.7 is the RX input, PORTC.6 is the TX output
ADCON1 = %00001111 'Set up ADCON1 register no matter what yr doing!!!

'::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::
INCLUDE "MODEDEFS.BAS" ' Include Shiftin/out modes
INCLUDE "DT_INTS-14.bas" ' Base Interrupt System
INCLUDE "ReEnterPBP.bas" ' Include if using PBP interrupts
'::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::
' Variable definition
'::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::

RCIF VAR PIR1.5 ' Receive interrupt flag (1=full , 0=empty)
TXIF VAR PIR1.4 ' Transmit interrupt flag (1=empty, 0=full)
led0 var PORTD.0
led1 var PORTD.1
led2 var PORTD.2
led7 var PORTD.7
led0count var byte
led2count var byte
state var byte 'storage for condition bits
holdoff var word
CmdBuf var byte 'command for the pwm or acknowledge
SerialData var byte '

ID con "A" 'simple address, later 256

led2count = %00000001
state = 0
'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::::
ASM
INT_LIST macro ; IntSource, Label, Type, ResetFlag?
INT_Handler RX_INT, _Getbytes, PBP, no
endm
INT_CREATE ; Creates the interrupt processor
ENDASM

@ INT_ENABLE RX_INT ; enable RX_INT interrupts

'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::::

hserout ["start"] 'splash screen to verify comm
HSEROUT [10] ' CR to start new line on terminal program

Mainloop:
for holdoff = 1 to 500
pause 1
next holdoff
led0count = led0count + 1
led0 = led0count.0 'toggle led every loop for heartbeat
goto Mainloop


'*********** 'ISR for RX_int interrupt ***************************************

Getbytes:
While RCIF = 1 ' clear the buffer
HSERIN 100,error,[Serialdata] ' Get serial
led2count = led2count + 1 ' Incr heartbeat
Wend
led2 = led2count.0 'led to confirm program went to RX ISR

if SerialData = "U" then GoodSync
if serialdata = ID then GoodAdr
if state = %00000011 then GoodCmd 'good sync and address(ID)
state.1=0 'If not Sync & ID, retry
@ INT_RETURN

GoodSync:
hserout ["Sync: ",SerialData]
HSEROUT [10] 'Carriage Return
state.0 = 1 'set sync good flag
@ INT_RETURN

GoodAdr:
hserout ["Adr: ",serialdata]
hserout [10]
state.1 = 1 'got a good address
@ INT_RETURN

GoodCmd:
CmdBuf = serialdata 'store the command
hserout ["Cmd: ",CmdBuf] 'send Command and CR
HSEROUT [10]
state.1=0 'reset the address verification
@ INT_RETURN


error:
Toggle led7
@ INT_RETURN
end


Thanks for any direction.... At least a sound a horn so that I can get out of the fog :-)

Thanks
Mark

[skimask]

How fast are you sending the data to this PIC? Not like baud rate, but are you typing it? Is it another PIC sending the data to this PIC?
What are you using to send the data to this PIC?
You might be overrunning the serial input buffer because you're busy trying to send data out (multiple bytes) at the same time data is coming in (single bytes) at the same baud rate and therefore the incoming data is getting lost.
Try not HSEROUT'ing until you either get a good 'packet' or a 'bad packet' instead of HSEROUT'ing every step of the way and see what happens.

[boroko]

Hi skimask
Thanks for the reply.

The data has been sent using the Serial Communicator window in MCS. Have been testing using single characters and all three at a time. Trying to just get a framework to start from.
The end app is going to be PIC -> PIC in a master/slave config so the scheme has to be a bit more bulletproof than this. There is one controller and needs to be a significant number of slaves (sort of an Egypt/pyramid kind of thing in case Mel happenings by).

I'll be trying your suggestions and exploring it more this morning.
Thanks again

Bo

[mackrackit]

if SerialData = "U" then GoodSync
if serialdata = ID then GoodAdr
if state = %00000011 then GoodCmd

What if
if state
was changed to
if serialdata
?

[Darrel Taylor]

Hi boroko,


Getbytes:
While RCIF = 1 ' clear the buffer
HSERIN 100,error,[Serialdata] ' Get serial
led2count = led2count + 1 ' Incr heartbeat
Wend
You're throwing away perfectly good data with that.
You need to process everything that comes in, even if it's not for this chips address.

And you don't need a Timeout in the HSERIN, because it's guaranteed to have a byte in the USART if it triggers an interrupt.<hr>
Well, I wrote this last night. I thought I'd get a chance to test/debug it. But today I've got 5 people coming at me from 4 different directions.
Makes me wonder what those 2 are doing together. :eek:

Hopefully, what I was trying to say will show through, if it doesn't work.

Basically it's, make the Handler as fast as possible. Get in, do your thing, and get out.
No PAUSES, no HSEROUTS, nothing that will keep it from leaving the interrupt as fast as possible.

Anything that needs to take time, should be done in the main loop. Where it can be interrupted again..
Flags are used to trigger those "Main Loop" processes.

Hope it gives some direction ...
If nothing else, note the changes to the Getbytes: handler.


'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
'Based on Hserin with Darrel Taylor's Instant Interrupts
'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
' Program to echo incoming serial data.
' RX SYNC byte, if good: ADDRESS byte, if good: COMMAND
' MPASM, LabX-1 , PBP2.47, Using PIC 16F877A @ 4mHz, 9600 Baud
' Using Hardware USART and MAX232 to MCS Serial Communicator on PC

DEFINE OSC 4
DEFINE HSER_RCSTA 90h ' enable serial port,
define HSER_TXSTA 24h ' enable transmit,
define HSER_BAUD 9600 ' set baudrate to 9600
DEFINE HSER_CLOERR 1 ' automatic clear overrun error
TRISD = $00000000 'D to outputs for the LEDs
TRISC = %10000000 ' PORTC.7 is the RX input, PORTC.6 is the TX output
ADCON1 = %00001111 'Set up ADCON1 register no matter what yr doing!!!

'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
INCLUDE "DT_INTS-14.bas" ' Base Interrupt System
INCLUDE "ReEnterPBP.bas" ' Include if using PBP interrupts
'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
' Variable definition
'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::

led0 var PORTD.0
led1 var PORTD.1
led2 var PORTD.2
led7 var PORTD.7

ERROR_LED VAR led7 ' rename the LED's
LATE_LED VAR led2
HEART_LED VAR led0
LEDonDELAY CON 4

CmdBuf var byte 'command for the pwm or acknowledge
SerialData var byte '
ERRORtime var byte
LATEtime var byte
HEARTtime VAR BYTE

state VAR BYTE
Sync VAR state.0 ' Sync byte rcvd
ForMe VAR state.1 ' Packet is for this device
CmdRcvd VAR state.2 ' Command has been rcvd
ERROR VAR state.3 ' Sync rcvd out of order
LATE VAR state.4 ' command rcvd before last one was processed

SyncByte CON "U"
ID CON "A"


state = 0
'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::
ASM
INT_LIST macro ; IntSource, Label, Type, ResetFlag?
INT_Handler RX_INT, _Getbytes, PBP, no
INT_Handler TMR0_INT, _TMR0handler, PBP, YES
endm
INT_CREATE ; Creates the interrupt processor
ENDASM

@ INT_ENABLE RX_INT ; enable RX_INT interrupts
@ INT_ENABLE TMR0_INT ; enable TMR0_INT interrupts 65.5ms

'::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::::

hserout ["start"] 'splash screen to verify comm
HSEROUT [13,10] ' CR to start new line on terminal program

Mainloop:
IF CmdRcvd THEN GOSUB ProcessCMD ' Process any incomming data
IF ERROR THEN ' if there's been an Error
HIGH ERROR_LED ' Turn ON ERROR LED
ERROR = 0 ' reset the error flag
ERRORtime = LEDonDELAY ' start countdown till LED-OFF
HSEROUT ["Error",13,10]
ENDIF
IF LATE THEN ' if command took too long
HIGH LATE_LED ' Turn ON LATE LED
LATE = 0 ' reset the LATE flag
LATEtime = LEDonDELAY ' start countdown till LED-OFF
HSEROUT ["Late",13,10]
ENDIF
GOTO Mainloop

'*********** Process received command **************************************
ProcessCMD:
hserout ["Cmd: ",CmdBuf, " [", _ ' send Command and CR/LF
DEC CmdBuf,"]",13,10]
CmdRcvd = 0 ' indicate CMD has been processed
RETURN

'*********** 'ISR for TMR0_INT interrupt ***********************************
TMR0handler:
IF ERRORtime > 0 THEN ' if the Error LED is ON
ERRORtime = ERRORtime - 1 ' decrement the count
IF ERRORtime = 0 THEN ' when it reaches 0
LOW ERROR_LED ' turn the Error LED OFF
ENDIF
ENDIF
IF LATEtime > 0 THEN ' if the LATE LED is ON
LATEtime = LATEtime - 1 ' decrement the count
IF LATEtime = 0 THEN ' when it reaches 0
LOW LATE_LED ' turn the LATE LED OFF
ENDIF
ENDIF
HEARTtime = HEARTtime + 1 ' Toggle heartbeat ~.5sec
IF HEARTtime = 7 THEN
HEARTtime = 0
TOGGLE HEART_LED
ENDIF
@ INT_RETURN

'*********** 'ISR for RX_INT interrupt *************************************
Getbytes:
HSERIN [Serialdata] ' Get the serial data
IF Serialdata = SyncByte THEN ' if it's a Sync byte,
IF Sync THEN ERROR = 1 ' last command was corrupted
Sync = 1 ' indicate Sync was rcvd
ForMe = 0 ' ID has not been rcvd
ELSE
IF Sync THEN
IF !ForME THEN ' if we haven't rcvd the ID
IF Serialdata = ID THEN ' and this byte matches the ID
ForMe = 1 ' indicate ID rcvd
ELSE
Sync = 0 ' Not my packet
ForMe = 0 ' reset receive state
ENDIF ' and wait for next Sync
ELSE
CmdBuf = serialdata ' store the command
IF CmdRcvd THEN LATE = 1 ' last command not finished
CmdRcvd = 1 ' indicate a command was rcvd
Sync = 0 ' reset receive state
ForMe = 0
ENDIF
ENDIF
ENDIF
@ INT_RETURN

[boroko]

Hi All,

I sure appreciate that you all were willing to look at my question. I got the first response from skimask, and it prompted me to throw myself at the wall again and see if I could find the door. At some point, you feel that you have explored it to the limits of your ability, but you have to try again. I slowed down and tried to remember advise that I had gotten here by reading through anything that seemed relevant. I could send some data, so send registers to see where they were before they locked or corrupted. That started to reveal some interesting things that didn't have obvious answers.


By adding the last part to these lines:
hserout ["Cmd: ",DEC SerialData," ",BIN3 STATE]

I was able to see that there was a "13" being sent (Line Feed). Hmmmm....

Looking at MicroCode Studio Serial Communicatior's transmit window: A right click opens a selection

window. an option is there for Line Terminator where you can select CR, CR+LF, Null,or NO

terminator. Guess I found my "13" and why nothing was showing until I added DEC to see it.

Further testing is showing that I'm throwing a ONERR even though "DEFINE HSER_CLOERR 1" is in there.
Added "RXbyte = RCREG" to read the receive register but I still get the overrun.

Tried 2400 baud, still acts the same.
tried setting DT's Instant Interrupts macro to ResetFlag = yes: still over-runs the buffer.

Why is it that 20 quick "A"s or "U"s don't over-run the buffer and any data does?

Just as a way to check some of the questions, I closed Communicator and tried talking to it with RealTerm. I'll be darned if that didn't clear things right up. Evidently, after cleaning up a bit, it actually works!

I'm going to look carefully at your feedback and incorporate it, but I wanted to show what I have working so far:

hserout ["start"] 'splash screen to verify comm
HSEROUT [10,13] ' CR to start new line on terminal program

Mainloop:
for holdoff = 1 to 500
pause 1
next holdoff
led0count = led0count + 1
led0 = led0count.0 'toggle led every loop for heartbeat
goto Mainloop


'*********** 'ISR for RX_int interrupt ***************************************

Getbytes:
While RCIF = 1 ' clear the buffer
HSERIN 100,error,[Serialdata] ' Get serial
led2count = led2count + 1 ' Incr heartbeat
Wend
led2 = led2count.0 'led to confirm program went to RX ISR
if SerialData = "U" then GoodSync 'must have good sync: 01010101
if SerialData = "=" then GoodaDR 'BROADCAST address to all units
if SerialData = ID then GoodAdr 'unique address for each slave
if state = %00000011 then GoodCmd 'good sync and address(ID)
addr = 0 'If not Sync & ID, clr addr good flag
@ INT_RETURN

GoodSync:
sync = 1 'set sync good flag
hserout ["Sync: ",SerialData]
HSEROUT [10,13] 'Carriage Return
@ INT_RETURN

GoodAdr:
addr = 1 'got a good address
hserout ["Adr: ",SerialData]
hserout [10,13]
@ INT_RETURN

GoodCmd:
cmd = 1
CmdBuf = SerialData
hserout ["Cmd: ",SerialData," ",DEC CmdBuf]'send Command and CR
HSEROUT [10,13]
addr = 0 'reset the address verification
CMD = 0 'reset the command received flag
@ INT_RETURN


Well I am going back into non-internet land for the day, so I'll update after I get back.

Again. I can't thank you and the list enough. I strive not to be a "code-sucker" but there really is a lot of stuff that can trip you up and a lot of tiny things that will frustrate you greatly. Think of it as raising the next generation of question answerers ;-)

Bo