Number Sort Algorithm

[aratti]

... but my concern was that since I will be working with WORD data, the result of the addition could get big enough that I would have to use LONGs...

Using a 10 bits adc, the maximun number you can obtain is 1023. Now summing up 8 readings of 1023 each, you will end up with your word variable at 8184 (overflow will occur at 65K)

Since you are using adc , you have to remove

INCLUDE "AllDigital.pbp"

from your code.

Al.

[boroko]

Hi Al,
Thanks for engaging in this thread. Sometimes when one posts and gets no response, you begin to worry if your question is so off the wall that no one is interested.

The values that I'm working with are actually timer values that are WORD sized. The ultimate result is measuring ultrasonic pulse return to calculate distance. In the real program, I'm using a comparator interrupt to grab of the values. That one uses the analog, but here I'm just trying to sort out the averaging scheme.
The response needs to be smoothed considerably due to the high instance of random response of the ultrasonics. The values go on to determine the PWM output that drive a servo valve that controls some heavy hydraulics that can't tolerate quick movement. The response time needs to be in the 2-3 second range with smooth changes to work well. Since the values are 0-65535, and readings every 30mS, I'm trying to use Melanie's suggestions to dump the extremes, and DT's averaging to get away from LONGs and 32 bit math.

Current testing is with a second PIC generating the return pulse timing to separate the circuit from the program so that I can find the weakness. I'm trying to send a serial in to simulate the echo numbers. The next step is to convert the RX bytes into WORDS and see if the program works correctly.

Back to Burn and Turn
Bo

[aratti]

Hi Bo,
clear now. One workaround for not using long and still average them in the conventional way is the following:

RawData[0] = 0
for DatSort = 4 to 11                               ' pick out middle 8 readings
RawData[0] = RawData[0]  + (RawData[DatSort] - RawData[3])    ' extract the total difference
NEXT DatSort
RawData[0] = RawData[0]  >>3               ' average the total difference
DatAvg =  RawData[0]  + RawData[3]       ' add the average difference to the subtractor
Return

You addup in RawData[0] all the differences , then you average them and finally you add them to the RawData[3], the subtractor.

Al.

[Dave]

Al, I'm confused..... Where does RawData[3] come from and whats in it?

Dave Purola,
N8NTA

[aratti]

Al, I'm confused..... Where does RawData[3] come from and whats in it?

Dave, RawData array will contain the 16 readings you have taken from your system (ADC ; counter; etc.) . Once you apply Miss Melanie's sorting algorithm you will end up with your 16 values ordered in ascending order RawData[0] will contain the smallest reading while RawData[15] will contain the largest one.

The suggested technique is to discard the first 4 readings and the last 4 readings and average the central 8 reading, in such a way you will not enter into your average calculation anomalous reading (spikes for instance).

The Boroko issue was that since he is using the 16 bits counter, he was concerned in averaging the 8 raw data in the coventional way, due to the fact that a word will overflow over 65k and he didn't want to use long type variable.

This method, I have proposed, simply extract the differences from the 8 central values, which can be handled with a single word variable. Once the differences are extracted and summed up into RawData[0] (which is available because its content has been discarded) then you calculate the average shifting the bits with this instruction RawData[0] = RawData[0]>>3

Now if you add this average to the subtractor used (in our case RawData[3]) all you get is the data average.

Naturally the code snippet should be called after a call to the sorting routine.

Hoping to have clear the matter.

Al.

[boroko]

Hi folks,

Well, another learning moment! For the past week or so, I have been working on the previously mentioned sorting algorithm. I have been peeking in on the forum, but saw Al's name as the last one that replied so I didn't open it. Of course, I missed that there were actually 2 replies and that masked that there had been activity.

Thanks for the suggestion. I will study it and see how long it takes me to understand it.

In the mean time, this is where I have been churning. Puzzles me greatly, because it doesn't give anything near what I think it should.

'*          :  18F1330                                          *
'****************************************************************
DEFINE OSC 32
OSCCON  = %11111100          ' INTOSC primary, 8MHz 
OSCTUNE = %11111111          ' ENABLE 4x PLL = 32mHz
DEFINE HSER_RCSTA 90h        ' Enable serial port & continuous receive
DEFINE HSER_TXSTA 24h        ' Enable transmit, BRGH = 1
DEFINE HSER_CLROERR 1        ' Clear overflow automatically
DEFINE HSER_SPBRG 138        ' 57600 Baud @ 32MHz, -0.08%
SPBRGH = 0
BAUDCON.3 = 1                ' Enable 16 bit baudrate generator

DataA   var WORD
RawData var WORD [16]           'full program uses 16 and strips top
DatAvg  var byte                '   and bottom readings
DatSort var byte                ' 
DatCnt  var byte                ' count the 16 data entries
AvgCount con 8                  ' number of readings to average (1/x)
DatAvg   = 2500                 ' seed number to start averaging


RawData[4]= 20000                ' this data represents 8 different
RawData[5]= 21000                ' readings that come from a TMR0
RawData[6]= 22000                ' capture that is working.  Fixed 
RawData[7]= 23000                ' values here for testing
RawData[8]= 24000
RawData[9]= 25000
RawData[10]= 26000
RawData[11]= 27000
pause 2000
'-- Average middle 8 readings ---------------------------------------
Average:
   for DatSort = 0 to 7                    ' get 8 readings
    Hserout["DA=",#DatAvg,", RD",#DatSort,"=",#RawData[DatSort]]
    DatAvg = DatAvg - (DatAvg/AvgCount)             'subtract 1/x from Avg
    hserout[", 1/x=",#(DatAvg/AvgCount)," DA- = ",#DatAvg]
    DatAvg = DatAvg + (RawData[DatSort]/AvgCount)  'add 1/x of new reading 
    Hserout[", DA+ = ",#DatAvg,13,10] 
     next DatSort
  hserout[13,10]
'    hserout[", DA ",#DatAvg,13,10]
   goto Average

Here is a RealTerm screen of the output

DA=219, RD4=20000, 1/x=24 DA- = 192, DA+ = 132                                
DA=132, RD5=21000, 1/x=14 DA- = 116, DA+ = 181                                
DA=181, RD6=22000, 1/x=19 DA- = 159, DA+ = 93                                 
DA=93, RD7=23000, 1/x=10 DA- = 82, DA+ = 141                                  
DA=141, RD8=24000, 1/x=15 DA- = 124, DA+ = 52                                 
DA=52, RD9=25000, 1/x=5 DA- = 46, DA+ = 99                                    
DA=99, RD10=26000, 1/x=10 DA- = 87, DA+ = 9                                   
DA=9, RD11=27000, 1/x=1 DA- = 8, DA+ = 55
                                                                                                                   
DA=55, RD4=20000, 1/x=6 DA- = 49, DA+ = 245                                   
DA=245, RD5=21000, 1/x=26 DA- = 215, DA+ = 24                                 
DA=24, RD6=22000, 1/x=2 DA- = 21, DA+ = 211                                   
DA=211, RD7=23000, 1/x=23 DA- = 185, DA+ = 244                                
DA=244, RD8=24000, 1/x=26 DA- = 214, DA+ = 142                                
DA=142, RD9=25000, 1/x=15 DA- = 125, DA+ = 178                                
DA=178, RD10=26000, 1/x=19 DA- = 156, DA+ = 78                                
DA=78, RD11=27000, 1/x=8 DA- = 69, DA+ = 116

I can't yet find a pattern or a reason. Maybe its time for a bit of sleep. I will work on it more later.
Thanks
Bo

[boroko]

Hi Al
I had the same question as Dave until I mapped it out. Until I put it into Excel, I couldn't understand how that was going to work. Very cool indeed.
The value in RawData[3] didn't matter, since the values were already sorted, it was lower than RD[4]. All it needed to do was act as a reference for the subtraction and then it was added back in canceling itself. I have no idea how you came up with that, but I'm glad folks like you are willing to teach the rest of us.

The .xls is zipped if anyone would like to see it.

Thanks
Bo