Parsing Serial data stream

[boroko]

Hi All,
Wonder if anyone would take a look at this and tell me where I'm missing it.
This started as http://www.picbasic.co.uk/forum/showthread.php?p=64209 and I have made a bit of progress but I could still use some fresh eyes (and more brain cells) I must be searching for the wrong terms as I can not seem to come up with how to handle the data once I get it.

This seems like it would give me the different bytes (sync, address, and command) but it stumbles when it tries to get the command byte. I get an echo for the sync and the address, but it won't grab the command. Seems to lock up too. Any suggestions? At least a link or two where I can learn more. BTW, I have been devouring Jan Axelson's Serial Port Complete book. Some help, but not on the parsing.

 '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
            'Based on Hserin with Darryl Taylor's Instant Interrupts
 '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
   '   Program to echo incoming serial data.
   ' RX SYNC byte, if good: ADDRESS byte, if good: COMMAND
    ' MPASM, LabX-1 , PBP2.47, Using PIC 16F877A @ 4mHz, 9600 Baud
    ' Using Hardware USART and MAX232 to MCS Serial Communicator on PC

        DEFINE OSC 4
        DEFINE HSER_RCSTA 90h ' enable serial port, 
        define HSER_TXSTA 24h ' enable transmit, 
        define HSER_SPBRG 25  ' set baudrate to 9600                   
        DEFINE HSER_CLOERR  1 ' automatic clear overrun error  
        TRISD = $00000000     'D to outputs for the LEDs
        TRISC  = %10000000    ' PORTC.7 is the RX input, PORTC.6 is the TX output
        ADCON1 = %00001111    'Set up ADCON1 register no matter what yr doing!!!                      
        
'::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
INCLUDE "MODEDEFS.BAS"       ' Include Shiftin/out modes
INCLUDE "DT_INTS-14.bas"     ' Base Interrupt System
INCLUDE "ReEnterPBP.bas"     ' Include if using PBP interrupts
'::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    '   Variable definition
'::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
        
RCIF       VAR     PIR1.5     ' Receive  interrupt flag (1=full , 0=empty)
TXIF       VAR     PIR1.4     ' Transmit interrupt flag (1=empty, 0=full)
led0       var     PORTD.0
led1       var     PORTD.1
led2       var     PORTD.2
led7       var     PORTD.7
led0count  var     byte
led2count  var     byte
state      var     byte       'storage for condition bits
holdoff    var     word
CmdBuf     var     byte       'command for the pwm or acknowledge
SerialData var     byte       ' 

ID con "A"                    'simple address, later 256

led2count = %00000001
state = 0
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
ASM
INT_LIST  macro    ; IntSource,        Label,  Type, ResetFlag?
        INT_Handler   RX_INT,    _Getbytes,    PBP,  no
    endm
    INT_CREATE               ; Creates the interrupt processor
ENDASM

@   INT_ENABLE  RX_INT       ; enable RX_INT interrupts

':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::             
      
hserout ["start"]               'splash screen to verify comm
HSEROUT [10]                    ' CR to start new line on terminal program

Mainloop:
        for holdoff = 1 to 500
        pause 1
        next holdoff
        led0count = led0count + 1
        led0 = led0count.0      'toggle led every loop for heartbeat  
        goto Mainloop
 

'*********** 'ISR for RX_int interrupt ***************************************
 
Getbytes:        
       While RCIF = 1                    ' clear the buffer
       HSERIN 100,error,[Serialdata]     ' Get serial
       led2count = led2count + 1         ' Incr heartbeat
       Wend  
       led2 = led2count.0                'led to confirm program went to RX ISR
       
        if SerialData = "U" then GoodSync
          if serialdata = ID then GoodAdr  
             if state = %00000011 then GoodCmd 'good sync and address(ID)
       state.1=0                          'If not Sync & ID, retry
@ INT_RETURN
                                             
GoodSync:
       hserout ["Sync: ",SerialData]
       HSEROUT [10]                       'Carriage Return
       state.0 = 1                        'set sync good flag                              
@ INT_RETURN

GoodAdr:
       hserout ["Adr: ",serialdata]
       hserout [10]
       state.1 = 1                         'got a good address
@ INT_RETURN
       
GoodCmd:
        CmdBuf = serialdata                'store the command 
        hserout ["Cmd: ",CmdBuf]           'send Command and CR
        HSEROUT [10]
        state.1=0                          'reset the address verification   
@ INT_RETURN


error:
      Toggle led7
@ INT_RETURN
 end

Thanks for any direction.... At least a sound a horn so that I can get out of the fog :-)

Thanks
Mark

[skimask]

How fast are you sending the data to this PIC? Not like baud rate, but are you typing it? Is it another PIC sending the data to this PIC?
What are you using to send the data to this PIC?
You might be overrunning the serial input buffer because you're busy trying to send data out (multiple bytes) at the same time data is coming in (single bytes) at the same baud rate and therefore the incoming data is getting lost.
Try not HSEROUT'ing until you either get a good 'packet' or a 'bad packet' instead of HSEROUT'ing every step of the way and see what happens.

[boroko]

Hi skimask
Thanks for the reply.

The data has been sent using the Serial Communicator window in MCS. Have been testing using single characters and all three at a time. Trying to just get a framework to start from.
The end app is going to be PIC -> PIC in a master/slave config so the scheme has to be a bit more bulletproof than this. There is one controller and needs to be a significant number of slaves (sort of an Egypt/pyramid kind of thing in case Mel happenings by).

I'll be trying your suggestions and exploring it more this morning.
Thanks again

Bo

[mackrackit]

if SerialData = "U" then GoodSync
          if serialdata = ID then GoodAdr  
             if state = %00000011 then GoodCmd

What if
if state
was changed to
if serialdata
?

[Darrel Taylor]

Hi boroko,

Getbytes:        
       While RCIF = 1                    ' clear the buffer
       HSERIN 100,error,[Serialdata]     ' Get serial
       led2count = led2count + 1         ' Incr heartbeat
       Wend

You're throwing away perfectly good data with that.
You need to process everything that comes in, even if it's not for this chips address.

And you don't need a Timeout in the HSERIN, because it's guaranteed to have a byte in the USART if it triggers an interrupt.

---

Well, I wrote this last night. I thought I'd get a chance to test/debug it. But today I've got 5 people coming at me from 4 different directions.
Makes me wonder what those 2 are doing together.

Hopefully, what I was trying to say will show through, if it doesn't work.

Basically it's, make the Handler as fast as possible. Get in, do your thing, and get out.
No PAUSES, no HSEROUTS, nothing that will keep it from leaving the interrupt as fast as possible.

Anything that needs to take time, should be done in the main loop. Where it can be interrupted again..
Flags are used to trigger those "Main Loop" processes.

Hope it gives some direction ...
If nothing else, note the changes to the Getbytes: handler.

':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
            'Based on Hserin with Darrel Taylor's Instant Interrupts
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
'    Program to echo incoming serial data.
'  RX SYNC byte, if good: ADDRESS byte, if good: COMMAND
'   MPASM, LabX-1 , PBP2.47, Using PIC 16F877A @ 4mHz, 9600 Baud
'   Using Hardware USART and MAX232 to MCS Serial Communicator on PC

DEFINE OSC 4
DEFINE HSER_RCSTA 90h ' enable serial port, 
define HSER_TXSTA 24h ' enable transmit, 
define HSER_BAUD 9600 ' set baudrate to 9600                   
DEFINE HSER_CLOERR  1 ' automatic clear overrun error  
TRISD = $00000000     'D to outputs for the LEDs
TRISC  = %10000000    ' PORTC.7 is the RX input, PORTC.6 is the TX output
ADCON1 = %00001111    'Set up ADCON1 register no matter what yr doing!!!                      
        
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
INCLUDE "DT_INTS-14.bas"     ' Base Interrupt System
INCLUDE "ReEnterPBP.bas"     ' Include if using PBP interrupts
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    '   Variable definition
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
        
led0       var     PORTD.0
led1       var     PORTD.1
led2       var     PORTD.2
led7       var     PORTD.7

ERROR_LED  VAR led7            ' rename the LED's
LATE_LED   VAR led2
HEART_LED  VAR led0
LEDonDELAY CON 4

CmdBuf     var     byte       'command for the pwm or acknowledge
SerialData var     byte       ' 
ERRORtime  var     byte
LATEtime   var     byte
HEARTtime  VAR     BYTE

state      VAR BYTE
  Sync     VAR state.0        ' Sync byte rcvd
  ForMe    VAR state.1        ' Packet is for this device
  CmdRcvd  VAR state.2        ' Command has been rcvd
  ERROR    VAR state.3        ' Sync rcvd out of order
  LATE     VAR state.4        ' command rcvd before last one was processed

SyncByte   CON "U"
ID         CON "A"


state = 0
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
ASM
INT_LIST  macro    ; IntSource,        Label,  Type, ResetFlag?
        INT_Handler     RX_INT,    _Getbytes,    PBP,  no
        INT_Handler   TMR0_INT, _TMR0handler,    PBP,  YES
    endm
    INT_CREATE               ; Creates the interrupt processor
ENDASM

@   INT_ENABLE  RX_INT          ; enable RX_INT interrupts
@   INT_ENABLE  TMR0_INT        ; enable TMR0_INT interrupts 65.5ms

':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::             
      
hserout ["start"]               'splash screen to verify comm
HSEROUT [13,10]                 ' CR to start new line on terminal program

Mainloop:
    IF CmdRcvd THEN GOSUB ProcessCMD       ' Process any incomming data
    IF ERROR THEN                          ' if there's been an Error
        HIGH ERROR_LED                     '   Turn ON ERROR LED
        ERROR = 0                          '   reset the error flag
        ERRORtime = LEDonDELAY             '   start countdown till LED-OFF
        HSEROUT ["Error",13,10]
    ENDIF
    IF LATE THEN                           ' if command took too long
        HIGH LATE_LED                      '   Turn ON LATE LED
        LATE = 0                           '   reset the LATE flag
        LATEtime = LEDonDELAY              '   start countdown till LED-OFF
        HSEROUT ["Late",13,10]
    ENDIF
GOTO Mainloop

'*********** Process received command **************************************
ProcessCMD:
    hserout ["Cmd: ",CmdBuf, " [", _       ' send Command and CR/LF
             DEC CmdBuf,"]",13,10]         
    CmdRcvd = 0                            ' indicate CMD has been processed
RETURN  

'*********** 'ISR for TMR0_INT interrupt ***********************************
TMR0handler:
    IF ERRORtime > 0 THEN                  ' if the Error LED is ON
        ERRORtime = ERRORtime - 1          '   decrement the count
        IF ERRORtime = 0 THEN              '   when it reaches 0
            LOW ERROR_LED                  '   turn the Error LED OFF
        ENDIF
    ENDIF        
    IF LATEtime > 0 THEN                   ' if the LATE LED is ON
        LATEtime = LATEtime - 1            '   decrement the count
        IF LATEtime = 0 THEN               '   when it reaches 0
            LOW LATE_LED                   '   turn the LATE LED OFF
        ENDIF
    ENDIF
    HEARTtime = HEARTtime + 1              ' Toggle heartbeat ~.5sec
    IF HEARTtime = 7 THEN
        HEARTtime = 0
        TOGGLE HEART_LED
    ENDIF       
@ INT_RETURN

'*********** 'ISR for RX_INT interrupt *************************************
Getbytes:        
    HSERIN [Serialdata]                    ' Get the serial data
    IF Serialdata = SyncByte THEN          ' if it's a Sync byte,
        IF Sync THEN ERROR = 1             '   last command was corrupted
        Sync = 1                           '   indicate Sync was rcvd
        ForMe = 0                          '   ID has not been rcvd
    ELSE
        IF Sync THEN
            IF !ForME THEN                 ' if we haven't rcvd the ID
                IF Serialdata = ID THEN    '   and this byte matches the ID
                    ForMe = 1              '     indicate ID rcvd
                ELSE
                    Sync = 0               ' Not my packet
                    ForMe = 0              '   reset receive state    
                ENDIF                      '   and wait for next Sync
            ELSE
                CmdBuf = serialdata        ' store the command
                IF CmdRcvd THEN LATE = 1   '   last command not finished
                CmdRcvd = 1                '   indicate a command was rcvd
                Sync = 0                   '   reset receive state
                ForMe = 0                  
            ENDIF
        ENDIF
    ENDIF
@ INT_RETURN

[boroko]

Hi All,

I sure appreciate that you all were willing to look at my question. I got the first response from skimask, and it prompted me to throw myself at the wall again and see if I could find the door. At some point, you feel that you have explored it to the limits of your ability, but you have to try again. I slowed down and tried to remember advise that I had gotten here by reading through anything that seemed relevant. I could send some data, so send registers to see where they were before they locked or corrupted. That started to reveal some interesting things that didn't have obvious answers.


By adding the last part to these lines:
hserout ["Cmd: ",DEC SerialData," ",BIN3 STATE]

I was able to see that there was a "13" being sent (Line Feed). Hmmmm....

Looking at MicroCode Studio Serial Communicatior's transmit window: A right click opens a selection

window. an option is there for Line Terminator where you can select CR, CR+LF, Null,or NO

terminator. Guess I found my "13" and why nothing was showing until I added DEC to see it.

Further testing is showing that I'm throwing a ONERR even though "DEFINE HSER_CLOERR 1" is in there.
Added "RXbyte = RCREG" to read the receive register but I still get the overrun.

Tried 2400 baud, still acts the same.
tried setting DT's Instant Interrupts macro to ResetFlag = yes: still over-runs the buffer.

Why is it that 20 quick "A"s or "U"s don't over-run the buffer and any data does?

Just as a way to check some of the questions, I closed Communicator and tried talking to it with RealTerm. I'll be darned if that didn't clear things right up. Evidently, after cleaning up a bit, it actually works!

I'm going to look carefully at your feedback and incorporate it, but I wanted to show what I have working so far:

hserout ["start"]               'splash screen to verify comm
HSEROUT [10,13]                 ' CR to start new line on terminal program

Mainloop:
        for holdoff = 1 to 500
        pause 1
        next holdoff
        led0count = led0count + 1
        led0 = led0count.0      'toggle led every loop for heartbeat  
        goto Mainloop
 

'*********** 'ISR for RX_int interrupt ***************************************
 
Getbytes:        
       While RCIF = 1                    ' clear the buffer
       HSERIN 100,error,[Serialdata]     ' Get serial
       led2count = led2count + 1         ' Incr heartbeat
       Wend  
       led2 = led2count.0                'led to confirm program went to RX ISR
       if SerialData = "U" then GoodSync   'must have good sync: 01010101 
          if SerialData = "=" then GoodaDR 'BROADCAST address to all units
          if SerialData = ID then GoodAdr  'unique address for each slave
             if state = %00000011 then GoodCmd 'good sync and address(ID)
       addr = 0                          'If not Sync & ID, clr addr good flag
@ INT_RETURN
                                             
GoodSync:
       sync = 1                          'set sync good flag
       hserout ["Sync: ",SerialData]
       HSEROUT [10,13]                      'Carriage Return                                     
@ INT_RETURN

GoodAdr:
       addr = 1                           'got a good address
       hserout ["Adr: ",SerialData]
       hserout [10,13]
@ INT_RETURN
       
GoodCmd:
        cmd = 1
        CmdBuf = SerialData
        hserout ["Cmd: ",SerialData," ",DEC CmdBuf]'send Command and CR
        HSEROUT [10,13]
        addr = 0                          'reset the address verification 
        CMD = 0                           'reset the command received flag  
@ INT_RETURN

Well I am going back into non-internet land for the day, so I'll update after I get back.

Again. I can't thank you and the list enough. I strive not to be a "code-sucker" but there really is a lot of stuff that can trip you up and a lot of tiny things that will frustrate you greatly. Think of it as raising the next generation of question answerers ;-)

Bo