Parsing Serial data stream


+ Reply to Thread
Results 1 to 6 of 6
  1. #1
    Join Date
    Feb 2008
    Location
    Michigan, USA
    Posts
    231

    Default Parsing Serial data stream

    Hi All,
    Wonder if anyone would take a look at this and tell me where I'm missing it.
    This started as http://www.picbasic.co.uk/forum/showthread.php?p=64209 and I have made a bit of progress but I could still use some fresh eyes (and more brain cells) I must be searching for the wrong terms as I can not seem to come up with how to handle the data once I get it.

    This seems like it would give me the different bytes (sync, address, and command) but it stumbles when it tries to get the command byte. I get an echo for the sync and the address, but it won't grab the command. Seems to lock up too. Any suggestions? At least a link or two where I can learn more. BTW, I have been devouring Jan Axelson's Serial Port Complete book. Some help, but not on the parsing.

    Code:
     '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
                'Based on Hserin with Darryl Taylor's Instant Interrupts
     '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
       '   Program to echo incoming serial data.
       ' RX SYNC byte, if good: ADDRESS byte, if good: COMMAND
        ' MPASM, LabX-1 , PBP2.47, Using PIC 16F877A @ 4mHz, 9600 Baud
        ' Using Hardware USART and MAX232 to MCS Serial Communicator on PC
    
            DEFINE OSC 4
            DEFINE HSER_RCSTA 90h ' enable serial port, 
            define HSER_TXSTA 24h ' enable transmit, 
            define HSER_SPBRG 25  ' set baudrate to 9600                   
            DEFINE HSER_CLOERR  1 ' automatic clear overrun error  
            TRISD = $00000000     'D to outputs for the LEDs
            TRISC  = %10000000    ' PORTC.7 is the RX input, PORTC.6 is the TX output
            ADCON1 = %00001111    'Set up ADCON1 register no matter what yr doing!!!                      
            
    '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    INCLUDE "MODEDEFS.BAS"       ' Include Shiftin/out modes
    INCLUDE "DT_INTS-14.bas"     ' Base Interrupt System
    INCLUDE "ReEnterPBP.bas"     ' Include if using PBP interrupts
    '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
        '   Variable definition
    '::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
            
    RCIF       VAR     PIR1.5     ' Receive  interrupt flag (1=full , 0=empty)
    TXIF       VAR     PIR1.4     ' Transmit interrupt flag (1=empty, 0=full)
    led0       var     PORTD.0
    led1       var     PORTD.1
    led2       var     PORTD.2
    led7       var     PORTD.7
    led0count  var     byte
    led2count  var     byte
    state      var     byte       'storage for condition bits
    holdoff    var     word
    CmdBuf     var     byte       'command for the pwm or acknowledge
    SerialData var     byte       ' 
    
    ID con "A"                    'simple address, later 256
    
    led2count = %00000001
    state = 0
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    ASM
    INT_LIST  macro    ; IntSource,        Label,  Type, ResetFlag?
            INT_Handler   RX_INT,    _Getbytes,    PBP,  no
        endm
        INT_CREATE               ; Creates the interrupt processor
    ENDASM
    
    @   INT_ENABLE  RX_INT       ; enable RX_INT interrupts
    
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::             
          
    hserout ["start"]               'splash screen to verify comm
    HSEROUT [10]                    ' CR to start new line on terminal program
    
    Mainloop:
            for holdoff = 1 to 500
            pause 1
            next holdoff
            led0count = led0count + 1
            led0 = led0count.0      'toggle led every loop for heartbeat  
            goto Mainloop
     
    
    '*********** 'ISR for RX_int interrupt ***************************************
     
    Getbytes:        
           While RCIF = 1                    ' clear the buffer
           HSERIN 100,error,[Serialdata]     ' Get serial
           led2count = led2count + 1         ' Incr heartbeat
           Wend  
           led2 = led2count.0                'led to confirm program went to RX ISR
           
            if SerialData = "U" then GoodSync
              if serialdata = ID then GoodAdr  
                 if state = %00000011 then GoodCmd 'good sync and address(ID)
           state.1=0                          'If not Sync & ID, retry
    @ INT_RETURN
                                                 
    GoodSync:
           hserout ["Sync: ",SerialData]
           HSEROUT [10]                       'Carriage Return
           state.0 = 1                        'set sync good flag                              
    @ INT_RETURN
    
    GoodAdr:
           hserout ["Adr: ",serialdata]
           hserout [10]
           state.1 = 1                         'got a good address
    @ INT_RETURN
           
    GoodCmd:
            CmdBuf = serialdata                'store the command 
            hserout ["Cmd: ",CmdBuf]           'send Command and CR
            HSEROUT [10]
            state.1=0                          'reset the address verification   
    @ INT_RETURN
    
    
    error:
          Toggle led7
    @ INT_RETURN
     end
    Thanks for any direction.... At least a sound a horn so that I can get out of the fog :-)

    Thanks
    Mark

  2. #2
    skimask's Avatar
    skimask Guest

    Default

    How fast are you sending the data to this PIC? Not like baud rate, but are you typing it? Is it another PIC sending the data to this PIC?
    What are you using to send the data to this PIC?
    You might be overrunning the serial input buffer because you're busy trying to send data out (multiple bytes) at the same time data is coming in (single bytes) at the same baud rate and therefore the incoming data is getting lost.
    Try not HSEROUT'ing until you either get a good 'packet' or a 'bad packet' instead of HSEROUT'ing every step of the way and see what happens.

  3. #3
    Join Date
    Feb 2008
    Location
    Michigan, USA
    Posts
    231

    Default

    Hi skimask
    Thanks for the reply.

    The data has been sent using the Serial Communicator window in MCS. Have been testing using single characters and all three at a time. Trying to just get a framework to start from.
    The end app is going to be PIC -> PIC in a master/slave config so the scheme has to be a bit more bulletproof than this. There is one controller and needs to be a significant number of slaves (sort of an Egypt/pyramid kind of thing in case Mel happenings by).

    I'll be trying your suggestions and exploring it more this morning.
    Thanks again

    Bo

  4. #4
    Join Date
    Nov 2003
    Location
    Wellton, U.S.A.
    Posts
    5,924

    Default

    Code:
    if SerialData = "U" then GoodSync
              if serialdata = ID then GoodAdr  
                 if state = %00000011 then GoodCmd
    What if
    if state
    was changed to
    if serialdata
    ?
    Dave
    Always wear safety glasses while programming.

  5. #5
    Join Date
    Jul 2003
    Location
    Colorado Springs
    Posts
    4,959

    Default

    Hi boroko,

    Code:
    Getbytes:        
           While RCIF = 1                    ' clear the buffer
           HSERIN 100,error,[Serialdata]     ' Get serial
           led2count = led2count + 1         ' Incr heartbeat
           Wend
    You're throwing away perfectly good data with that.
    You need to process everything that comes in, even if it's not for this chips address.

    And you don't need a Timeout in the HSERIN, because it's guaranteed to have a byte in the USART if it triggers an interrupt.

    Well, I wrote this last night. I thought I'd get a chance to test/debug it. But today I've got 5 people coming at me from 4 different directions.
    Makes me wonder what those 2 are doing together.

    Hopefully, what I was trying to say will show through, if it doesn't work.

    Basically it's, make the Handler as fast as possible. Get in, do your thing, and get out.
    No PAUSES, no HSEROUTS, nothing that will keep it from leaving the interrupt as fast as possible.

    Anything that needs to take time, should be done in the main loop. Where it can be interrupted again..
    Flags are used to trigger those "Main Loop" processes.

    Hope it gives some direction ...
    If nothing else, note the changes to the Getbytes: handler.
    Code:
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
                'Based on Hserin with Darrel Taylor's Instant Interrupts
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    '    Program to echo incoming serial data.
    '  RX SYNC byte, if good: ADDRESS byte, if good: COMMAND
    '   MPASM, LabX-1 , PBP2.47, Using PIC 16F877A @ 4mHz, 9600 Baud
    '   Using Hardware USART and MAX232 to MCS Serial Communicator on PC
    
    DEFINE OSC 4
    DEFINE HSER_RCSTA 90h ' enable serial port, 
    define HSER_TXSTA 24h ' enable transmit, 
    define HSER_BAUD 9600 ' set baudrate to 9600                   
    DEFINE HSER_CLOERR  1 ' automatic clear overrun error  
    TRISD = $00000000     'D to outputs for the LEDs
    TRISC  = %10000000    ' PORTC.7 is the RX input, PORTC.6 is the TX output
    ADCON1 = %00001111    'Set up ADCON1 register no matter what yr doing!!!                      
            
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    INCLUDE "DT_INTS-14.bas"     ' Base Interrupt System
    INCLUDE "ReEnterPBP.bas"     ' Include if using PBP interrupts
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
        '   Variable definition
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
            
    led0       var     PORTD.0
    led1       var     PORTD.1
    led2       var     PORTD.2
    led7       var     PORTD.7
    
    ERROR_LED  VAR led7            ' rename the LED's
    LATE_LED   VAR led2
    HEART_LED  VAR led0
    LEDonDELAY CON 4
    
    CmdBuf     var     byte       'command for the pwm or acknowledge
    SerialData var     byte       ' 
    ERRORtime  var     byte
    LATEtime   var     byte
    HEARTtime  VAR     BYTE
    
    state      VAR BYTE
      Sync     VAR state.0        ' Sync byte rcvd
      ForMe    VAR state.1        ' Packet is for this device
      CmdRcvd  VAR state.2        ' Command has been rcvd
      ERROR    VAR state.3        ' Sync rcvd out of order
      LATE     VAR state.4        ' command rcvd before last one was processed
    
    SyncByte   CON "U"
    ID         CON "A"
    
    
    state = 0
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
    ASM
    INT_LIST  macro    ; IntSource,        Label,  Type, ResetFlag?
            INT_Handler     RX_INT,    _Getbytes,    PBP,  no
            INT_Handler   TMR0_INT, _TMR0handler,    PBP,  YES
        endm
        INT_CREATE               ; Creates the interrupt processor
    ENDASM
    
    @   INT_ENABLE  RX_INT          ; enable RX_INT interrupts
    @   INT_ENABLE  TMR0_INT        ; enable TMR0_INT interrupts 65.5ms
    
    ':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::             
          
    hserout ["start"]               'splash screen to verify comm
    HSEROUT [13,10]                 ' CR to start new line on terminal program
    
    Mainloop:
        IF CmdRcvd THEN GOSUB ProcessCMD       ' Process any incomming data
        IF ERROR THEN                          ' if there's been an Error
            HIGH ERROR_LED                     '   Turn ON ERROR LED
            ERROR = 0                          '   reset the error flag
            ERRORtime = LEDonDELAY             '   start countdown till LED-OFF
            HSEROUT ["Error",13,10]
        ENDIF
        IF LATE THEN                           ' if command took too long
            HIGH LATE_LED                      '   Turn ON LATE LED
            LATE = 0                           '   reset the LATE flag
            LATEtime = LEDonDELAY              '   start countdown till LED-OFF
            HSEROUT ["Late",13,10]
        ENDIF
    GOTO Mainloop
    
    '*********** Process received command **************************************
    ProcessCMD:
        hserout ["Cmd: ",CmdBuf, " [", _       ' send Command and CR/LF
                 DEC CmdBuf,"]",13,10]         
        CmdRcvd = 0                            ' indicate CMD has been processed
    RETURN  
    
    '*********** 'ISR for TMR0_INT interrupt ***********************************
    TMR0handler:
        IF ERRORtime > 0 THEN                  ' if the Error LED is ON
            ERRORtime = ERRORtime - 1          '   decrement the count
            IF ERRORtime = 0 THEN              '   when it reaches 0
                LOW ERROR_LED                  '   turn the Error LED OFF
            ENDIF
        ENDIF        
        IF LATEtime > 0 THEN                   ' if the LATE LED is ON
            LATEtime = LATEtime - 1            '   decrement the count
            IF LATEtime = 0 THEN               '   when it reaches 0
                LOW LATE_LED                   '   turn the LATE LED OFF
            ENDIF
        ENDIF
        HEARTtime = HEARTtime + 1              ' Toggle heartbeat ~.5sec
        IF HEARTtime = 7 THEN
            HEARTtime = 0
            TOGGLE HEART_LED
        ENDIF       
    @ INT_RETURN
    
    '*********** 'ISR for RX_INT interrupt *************************************
    Getbytes:        
        HSERIN [Serialdata]                    ' Get the serial data
        IF Serialdata = SyncByte THEN          ' if it's a Sync byte,
            IF Sync THEN ERROR = 1             '   last command was corrupted
            Sync = 1                           '   indicate Sync was rcvd
            ForMe = 0                          '   ID has not been rcvd
        ELSE
            IF Sync THEN
                IF !ForME THEN                 ' if we haven't rcvd the ID
                    IF Serialdata = ID THEN    '   and this byte matches the ID
                        ForMe = 1              '     indicate ID rcvd
                    ELSE
                        Sync = 0               ' Not my packet
                        ForMe = 0              '   reset receive state    
                    ENDIF                      '   and wait for next Sync
                ELSE
                    CmdBuf = serialdata        ' store the command
                    IF CmdRcvd THEN LATE = 1   '   last command not finished
                    CmdRcvd = 1                '   indicate a command was rcvd
                    Sync = 0                   '   reset receive state
                    ForMe = 0                  
                ENDIF
            ENDIF
        ENDIF
    @ INT_RETURN
    DT

  6. #6
    Join Date
    Feb 2008
    Location
    Michigan, USA
    Posts
    231

    Default

    Hi All,

    I sure appreciate that you all were willing to look at my question. I got the first response from skimask, and it prompted me to throw myself at the wall again and see if I could find the door. At some point, you feel that you have explored it to the limits of your ability, but you have to try again. I slowed down and tried to remember advise that I had gotten here by reading through anything that seemed relevant. I could send some data, so send registers to see where they were before they locked or corrupted. That started to reveal some interesting things that didn't have obvious answers.


    By adding the last part to these lines:
    hserout ["Cmd: ",DEC SerialData," ",BIN3 STATE]

    I was able to see that there was a "13" being sent (Line Feed). Hmmmm....

    Looking at MicroCode Studio Serial Communicatior's transmit window: A right click opens a selection

    window. an option is there for Line Terminator where you can select CR, CR+LF, Null,or NO

    terminator. Guess I found my "13" and why nothing was showing until I added DEC to see it.

    Further testing is showing that I'm throwing a ONERR even though "DEFINE HSER_CLOERR 1" is in there.
    Added "RXbyte = RCREG" to read the receive register but I still get the overrun.

    Tried 2400 baud, still acts the same.
    tried setting DT's Instant Interrupts macro to ResetFlag = yes: still over-runs the buffer.

    Why is it that 20 quick "A"s or "U"s don't over-run the buffer and any data does?

    Just as a way to check some of the questions, I closed Communicator and tried talking to it with RealTerm. I'll be darned if that didn't clear things right up. Evidently, after cleaning up a bit, it actually works!

    I'm going to look carefully at your feedback and incorporate it, but I wanted to show what I have working so far:
    Code:
    hserout ["start"]               'splash screen to verify comm
    HSEROUT [10,13]                 ' CR to start new line on terminal program
    
    Mainloop:
            for holdoff = 1 to 500
            pause 1
            next holdoff
            led0count = led0count + 1
            led0 = led0count.0      'toggle led every loop for heartbeat  
            goto Mainloop
     
    
    '*********** 'ISR for RX_int interrupt ***************************************
     
    Getbytes:        
           While RCIF = 1                    ' clear the buffer
           HSERIN 100,error,[Serialdata]     ' Get serial
           led2count = led2count + 1         ' Incr heartbeat
           Wend  
           led2 = led2count.0                'led to confirm program went to RX ISR
           if SerialData = "U" then GoodSync   'must have good sync: 01010101 
              if SerialData = "=" then GoodaDR 'BROADCAST address to all units
              if SerialData = ID then GoodAdr  'unique address for each slave
                 if state = %00000011 then GoodCmd 'good sync and address(ID)
           addr = 0                          'If not Sync & ID, clr addr good flag
    @ INT_RETURN
                                                 
    GoodSync:
           sync = 1                          'set sync good flag
           hserout ["Sync: ",SerialData]
           HSEROUT [10,13]                      'Carriage Return                                     
    @ INT_RETURN
    
    GoodAdr:
           addr = 1                           'got a good address
           hserout ["Adr: ",SerialData]
           hserout [10,13]
    @ INT_RETURN
           
    GoodCmd:
            cmd = 1
            CmdBuf = SerialData
            hserout ["Cmd: ",SerialData," ",DEC CmdBuf]'send Command and CR
            HSEROUT [10,13]
            addr = 0                          'reset the address verification 
            CMD = 0                           'reset the command received flag  
    @ INT_RETURN
    Well I am going back into non-internet land for the day, so I'll update after I get back.

    Again. I can't thank you and the list enough. I strive not to be a "code-sucker" but there really is a lot of stuff that can trip you up and a lot of tiny things that will frustrate you greatly. Think of it as raising the next generation of question answerers ;-)

    Bo

Similar Threads

  1. Using Nokia LCD
    By BobP in forum mel PIC BASIC Pro
    Replies: 300
    Last Post: - 3rd May 2018, 04:47
  2. Read/Write Problem
    By Tobias in forum mel PIC BASIC Pro
    Replies: 1
    Last Post: - 10th February 2010, 01:51
  3. Big characters on HD44780 4x20
    By erpalma in forum mel PIC BASIC Pro
    Replies: 23
    Last Post: - 7th January 2007, 02:21
  4. LCD + bar graph
    By DynamoBen in forum mel PIC BASIC Pro
    Replies: 13
    Last Post: - 5th October 2005, 14:50
  5. Sinus calculating !
    By Don Mario in forum mel PIC BASIC Pro
    Replies: 29
    Last Post: - 28th November 2004, 23:56

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts