HSEROUT of array of data with termination on a null

[jimbostlawrence]

Hi all,

Having an issue with outputting data from a pic to PC via a USB-serial COM port.

I have an array of 256 bytes (1048574 of these pages of bytes in fact!)

What I'd like to do is simply:

HSEROUT [STR page_array\256]

Can't do that as limited to 255 char's.

HSEROUT [STR page_array\128] works, but terminates if any of the bytes in the array are 0 (null). Just my luck!

Is there a way of combining HEX2 in this and stopping the null termination?

i.e. HSEROUT [HEX2 STR page_array\128]

Additionally, is there a way to output the second half of the page_array using an offset? i.e. I was wondering if there's a way to access the address of page_array and assign this+128 bytes to a different variable:

page_array2 = {address of}page_array + 128 {bytes}
HSEROUT [HEX2 STR page_array2\128]

?

Essentially, trying to offload up to 256 MB of data from a flash chip to VB on laptop over 115.2k BAUD connection. I know this will take some time. the more efficient the transfer the better. If anyone has experience in this, I'd love some help!

Many thanks in advance.

Jimbo

[jimbostlawrence]

Update:

Currently, as a short term solution, outputting the array as 16x HSEROUTs of 16x pairs of hex digits. Crude, but functional, and VB is able to convert this data back to its integer format at the other end. Not sure how much this slows things down compared to using a single HSEROUT [page_array\256] would be, or similar, were it possible to output a full 256 Byte array.

[tumbleweed]

If you're looking for more efficient then drop the formatting entirely and just send the binary data.
If you format 256MB of data using HEX2 you end up with twice the number of bytes to transfer (512MB), and it's slower because of the conversions.

At 115K each byte takes about 86us to send, so as long as your code doesn't add more than that in overhead it won't matter. The fastest way would be something like this:

i  var     byte
b  var     byte[256]

for i = 0 to 255
    ' wait for TRMT == 1
    while (TXSTA.1 = 0)
    wend
    ' send byte from array
    TXREG = b[i]
next

I think you also need to think about using handshaking (RTS/CTS). On the PC side I doubt that it can keep up with a 256MB stream of data.

[richard]

or setup printing as a background task and dump your buffer into the queue when its empty

'*****************************************************************
'*  Name    : PRINT_18_TX_ASM_INT.pbp                            *
'*  Author  : richard                                            *
'*  Notice  :                                                    *
'*          :                                                    *
'*  Date    :                                                    *
'*  Version :    18f26K22                                        *
'*  Notes   :                                                    *
'*          :                                                    *
'*                                                               *
'*****************************************************************  
  
#CONFIG
  CONFIG  FOSC=INTIO67, PLLCFG=OFF, PRICLKEN=OFF, FCMEN=OFF, IESO=OFF
  CONFIG  PWRTEN=OFF, BOREN=SBORDIS, BORV=190, WDTEN=ON, WDTPS=32768
  CONFIG  CCP2MX=PORTC1, PBADEN=OFF, CCP3MX=PORTB5, HFOFST=ON, T3CMX=PORTC0
  CONFIG  P2BMX=PORTB5, MCLRE=EXTMCLR, STVREN=ON, LVP=OFF, XINST=OFF, DEBUG=OFF
  CONFIG  CP0=OFF, CP1=OFF, CP2=OFF, CP3=OFF, CPB=OFF, CPD=OFF, WRT0=OFF
  CONFIG  WRT1=OFF, WRT2=OFF, WRT3=OFF, WRTC=OFF, WRTB=OFF, WRTD=OFF, EBTR0=OFF
  CONFIG  EBTR1=OFF, EBTR2=OFF, EBTR3=OFF, EBTRB=OFF
#ENDCONFIG

DEFINE OSC 64
DEFINE INTHAND do_tx
 
RCSTA = $90   ' Enable serial port & continuous receive
TXSTA = $24   ' Enable transmit, BRGH = 1
SPBRG = 130   ' 9600 Baud @ 64MHz, -0.02%
SPBRGH = 6
BAUDCON.3 = 1 ' Enable 16 bit baudrate generator

prnb var byte[256]
rpos VAR BYTE
empty VAR BYTE

TRISC=%11111111
OSCCON    = $70   ; 64Mhz
OSCTUNE.6 = 1     ; Enable 4x PLL 
while ! osccon2.7 :WEND ; to make sure the pll has stabilised before you run any other code 
ANSELA=0
ANSELC=0
intcon=$c0
empty=1

pause 1000
hserout ["ready",13,10 ] ;  PROOF OF LIFE
pause 1000

mainloop:
    if empty then
        ;hserout [13,10]
        arraywrite prnb[240],[13,10,0,"RTIOP12345678"]
        gosub print
    ENDIF
    pause 2000
goto   mainloop
 
'  SAMPLE OF RAW BINARY OUTPUT FROM TX PIN IN HEX
'  0D 0A 00 52 54 49 4F 50 31 32 33 34 35 36 37 38
'  0D 0A 00 52 54 49 4F 50 31 32 33 34 35 36 37 38
'  ..............


print: 
    rpos = 240   ;FOR TEST START AT INX 240 IE PRINT CHRS  240-255  IN BUFFER
    empty=0
    PIE1.4=1
return 
 
asm   
do_tx
    LFSR 0, _prnb   ; Store buffer ADDR to FSR0H     
    BANKSEL _rpos 
    MOVF   _rpos,W  ; Add rpos to pointer 
    ADDWF   FSR0L,F ; Store Low byte  in FSR0
    BNC  NOV
    INCF    FSR0H ,F
NOV 
    movf    INDF0,W
    movwf  TXREG   
    INCFSZ  _rpos,F ; Increment  rpos
    BRA    exP      ; if buffer empty turn ofF interrupt
    bcf    PIE1,TXIF
    BANKSEL _empty  ; AND SET FLAG
    BSF   _empty ,0
exP 
    BANKSEL 0
    RETFIE 
endasm

[tumbleweed]

richard's code has some merits, but in almost all cases interrupts just add overhead... they rarely speed things up if that's what you're looking for.

Here's another version of the code I posted with the array access time buried in the byte transmit time:

i  var     byte
b  var     byte[256]
txd var byte

for i = 0 to 255
    ' get byte from the array
    txd = b[i]
    ' wait for TRMT == 1
    while (TXSTA.1 = 0)
    wend
    ' send byte
    TXREG = txd
next

[Dave]

That's great but, at 9600 baud it will be in this routine for 26.5 milliseconds. That's OK if you can afford it...

[amgen]

on second thought, using longs, again requires an 18F device, you could just do one big loop......(up to 4,000,000 at a time) index is the long 32bit.
then using index, get byte, send it, get next byte send it,,,,,, up to about 4 million..... in send loop, check for exit char if needed.
pretty straight forward

[jimbostlawrence]

To all,

MANY thanks for all your help with this guys. I have some data recovery to do on another project but will be back on this either tonight or over the next few days.

I'm running this on an 18LF26J53 hooked up to a 256MB Micron NOR FLASH chip, together with a crystal running at the magical 22.1184 MHz, so 115k BAUD is nice and clean. Will give the binary / TXREG a shot and get back to you. I used to be into assember back in the 80's (6502 / BBC B, and 68000 / Amiga) but not gone that deep in a long time. I think for now, I'll stick high-level and only go that deep if needs must.

Thanks again. Will see what happens.

Much appreciation,

Jimbo

[jimbostlawrence]

Hi all,

using:

          
for cnt = 0 to 255
                
    while (TXSTA.1 = 0)
    wend         
                
    TXREG = page_buffer[cnt]
                
next

Worked very nicely.

Initially, as I was working with VB as well, that I'm not that familiar with, I decided to initially stick with the transfer of double hex chars per byte, working on the multiple 256-page byte capture and export to file. I then brought in the above code (thanks Tumbleweed!) on the PIC side to stream the data as binary, halving the number of transmitted bits. It was then a fun job to adapt the threaded side of things in VB, but finally got it working.

As hex char's, I calculated it to be about 23.5 hrs (!) to transfer the entire 256MB of data from the Micron chip. That's now down to around 14.5 hrs. Still a long time, but transfer is stable; no errors.

I could probably bring this down some more by buffering the multiple pages, rather than "file.Write(byte_array, 0, 256)" each page to file. I expected that with the number of char's halving, that the overall time would halve, so assume this is now the bottleneck.

Perseverance, sweat, copious amounts of coffee, n munch, goes a long way!

Many thanks again all.

Best,

Jimbo

[HenrikOlsson]

I'd try increasing the baudrate even further.
According to Wikipedia 256kbits/second is a predefined serial port speed in Windows so you "should" have that available without doing anything "special". If you put a RS485 tranceiver at each end you get up into the Mbits/seconds without much problem but then something like DMA on the PIC-side would start to make sense.

[tumbleweed]

Most PC serial ports will run at 230400 baud.
The J53 will... just set:
BRG16=1
BRGH=0
SPBRGH=0
SPBRG=5

That might shave it down to under 4 hrs.
Make sure the MAX232 (or whatever you're using) supports 230K baud... some transceivers don't.

If I were sending that much data I'd be inclined to use a transfer protocol like XMODEM or something.
Are you just blasting 256MB out the pic uart and hoping the PC will keep up?