Averaging AtoD samples

[HankMcSpank]

So I have a need to average out some AtoD samples & a bit of googling found this (DT's averaging routine)...

http://74.125.77.132/search?q=cache:...&ct=clnk&gl=uk

(Btw, yes I did use the search bt I've just climbed down off the chair having tried to catch all the info going over my head!)

Now that looks link above looks like it might be ideal (perhaps a hammer to crack a nut in my situation), except I don't have a clue how to integrate it into my code! (I'm still massively wet behind the ears & really don't know how I've managed to kludge what I've already got together!)

In a pretty crude manner, up until now I've been averaging by storing the AtoD values in a few variables - then adding them together & then dividing by the number of variables I've used. I removed it from my code because it wasn't very elegant, my (simple) code was getting bloated etc.

Could someone be so kind as to outline how I can elegantly/simply average say 16 samples on a rolling basis?

For any responses, can we sssume my A to D sample is stored in a variable called signal_in

[Darrel Taylor]

If I had a hammer ... I'd hammer in the morning ... Whaaap!
You sure you want that around you nuts?
I like pistachio's, they're cracked already.

So take the code from that page and save it to a file called "average.pbp".

Add a line in your program ...
INCLUDE "average.pbp"

When you get your reading that you want to average ... put it in the variable Value, then GOSUB Average.
The running average will be returned in the same variable (Value).

Now sit back down in your chair before you hurt yourself.
<br>

[HankMcSpank]

Hi Darrel,

Thanks!! (that all sounds too simple...even for someone as clueless as me)

I'll try that soonest ...but I'll be sure to keep the chair nearby to get right back up upon!

(PS I wasn't saying your routine was a hammer to crack a nut per se, but simply a hammer to crack a nut with my simple requirements!)

PPS why do I keep getting that Random Question "missing letter from the word cof_ee" wrong? It's an 'f' damnit.

[HankMcSpank]

This is somewhat related to my ongoing pitch detection quest (in that it's what's driven me to ask - I'm seeing occasional glitches in the input signal, which is causing a hiccup in the 'time measured between comparator interrupts, oncde in a while), but it could relate to any situation...not just pitch detection, so hence coming back to this thread.

My question relates to 'intelligent' averaging (& rejecting spurious values)

Let's say you have a series of data (counts, samples, pulses....whatever) , but within the data occurs the odd 'sample' that's way out of whack. For example.....

2, 2, 2, 3,2,2,3,2,2,2,8,2,2,2,3,3,2

What I'd like to do is average all those samples, but reject any one occurence that's way out of whack (eg if 1 sample per 16 is out of whack, ignore it). so in that example all the 2s & 3s get averaged, but the 8 is ignored....else it'll really dick the average!

Ok, if it were that simple I could probably kludge something together - BUT, lets say that the above series of data is then immediately followed on by by say...

10, 10, 11, 10, 11, 10, 10, 11, 10

therefore giving a stream of values like thus.... 2, 2, 2, 3,2,2,3,2,2,2,8,2,2,2,3,3,2,10, 10, 11, 10, 11, 10, 10, 11, 10

There clearly becomes a problem, because the first couple of 10s of the second series would appear out of whack to any 'moving' simple check that was applied to the first series.

Ideally, I'd like to put in place some form of intelligent averaging routine, where only samples that are close together (say percentage terms) are averaged, any spurious samples out of that range (say 1 in 8 samples) are rejected ..... but if a threshold is breached, then the spurious values aren't treated as spurious anymore, but as new valid data to be then averaged.

Clear as mud?

[mackrackit]

I can not think of a "fast" way of doing it but you could check the difference between each pair and if the difference is more than X toss the value...
maybe...

[HankMcSpank]

I can not think of a "fast" way of doing it but you could check the difference between each pair and if the difference is more than X toss the value...
maybe...

but in that example above, it would mean when the series 10s & 11s arrived, there'd be a problem? If comparing in pairs & throwing the second one a way (ie if it deiviates too much), then the 10s would get rejected cos they're way out of whack wrt to the previous series of 2 & 3s (or have I misunderstood your line of thinking)

[mackrackit]

The first 10 would be tossed but the second 10 would not because now the pairs are 10,10,11...

[ScaleRobotics]

Sounds like you could use this:

http://www.picbasic.co.uk/forum/cont...orting-Numbers

[HankMcSpank]

The first 10 would be tossed but the second 10 would not because now the pairs are 10,10,11...

Aah...ok, gotcha.

Actually my example 'data series' wasn't a good one, as the data values would actually be in the hundreds (vs units which I used), which allows percentages to be used.

So, assuming a series of data coming into a variable called 'sample'...

' compare a pair called value & value2 from the incoming 'sample'

value = sample     '

upper_window = value2+(value2*4/100) 'create a window 4% above previous sample
lower_window = value2-(value2*4/100)  'create a value 4% below previous sample

if (value>= lower_window) AND (value <= upper_window) then incoming 'sample is within 'allowable error window'?
gosub average 'goto averaging routine
endif
value2 = value1

etc

would something like the above work?#

Edit: Just seen your post/link scalerobotics - off to have reader's digestion! (& yes, I did a search but I guess my keywords weren't erhm key)

[HankMcSpank]

ok, so scalerobotic's link to Melanie's 'sorting numbers' article, is exactly what I need, so I'm trying to get the thing to work.

i've basically got a small array that a prepopulate with some random numbers.

@ __CONFIG _FCMEN_OFF & _HS_OSC & _WDT_OFF & _MCLRE_OFF & _CP_OFF & _IESO_OFF & _BOR_OFF & _PWRTE_OFF

DEFINE  OSC 20          ' set Oscillator at 20Mhz.
DEFINE  NO_CLRWDT 1   ' PBP doesn't clear WDT automatically
DEFINE HSER_CLROERR 1
DEFINE HSER_TXSTA 24h ' Enable transmit
DEFINE HSER_SPBRG 129 ' set USART to 9600 baud  @20Mhz

    CounterA var Byte
    DataA var Byte
    RawData var Byte [6]
    Averaged var byte
    hserout [27,91,72]          'home cursor
    hserout [27,91,50,74]       'clear screen
    
    RawData[0]  = 25
    RawData[1]  = 27
    RawData[2]  = 32
    RawData[3]  = 30
    RawData[4]  = 34
    RawData[5]  = 33

    
    hserout ["before....", 13, 10]
    hserout [dec rawdata[0],13,10]
    hserout [dec rawdata[1],13,10]
    hserout [dec rawdata[2],13,10]
    hserout [dec rawdata[3],13,10]
    hserout [dec rawdata[4],13,10]
    hserout [dec rawdata[5],13,10]
    hserout [13,10]
    hserout [13,10]


    CounterA =0
    gosub SortArray
    gosub Average_data
    hserout ["after....", 13, 10]
    hserout [dec rawdata[0],13,10]
    hserout [dec rawdata[1],13,10]
    hserout [dec rawdata[2],13,10]
    hserout [dec rawdata[3],13,10]
    hserout [dec rawdata[4],13,10]
    hserout [dec rawdata[5],13,10]

    hserout ["averaged = ", dec averaged,13,10]
    goto end1
    
    
Average_data:
    hserout [dec rawData[2]," + ", dec rawData[3], 13,10]
    Averaged = (rawData[2]+rawData[3])/2    ' average the middle two of 6 ongoing samples
    return

SortArray:
SortLoop:
    If RawData(CounterA+1) < RawData(CounterA) then
        DataA=RawData(CounterA)
        RawData(CounterA)=RawData(CounterA+1)
        RawData(CounterA+1+0)=DataA
        If CounterA > 0 then CounterA=CounterA-2
        endif
    CounterA=CounterA+1
    If CounterA < 5 then goto SortLoop
    Return

    end1:
    end

here's my screen output (I average the middle two numbers Rawdata[2] & RawData[3] )....

before....
25
27
32
30
34
33

after....
6
25
27
30
32
33
averaged = 31

It all works (ie sorts & averages correctly) - but where on earth has the number 6 in the sorted numbers come from? (there was no number six in the original array - also, whatever number I place in that array position RawData[4] comes out sorted as 6!?!!)

Edit: Ok, think I've stumbled on the issue - it seems to be related to my array declaration ....because when I use RawData var Byte [8] ...everything works.

So why can't I declare an array 5 bytes 'deep'? ie RawData var byte [5] ???

[aratti]

It all works (ie sorts & averages correctly) - but where on earth has the number 6 in the sorted numbers come from?

Check out the sorting routine because it doesn't work. The first number should be 25, while the last number should be 34 not 33.

Also your average is not correct, since 27 + 30 / 2 = 28.5 (28 as integer) not 31

Very likely the error comes from the wrong sorting.

Cheers

Al.

[Ioannis]

I suppose that you corrected the array definition from[6] to [5].

And then the check in the program:

If CounterA < 5 then goto SortLoop

to this:

If CounterA < 4 then goto SortLoop

Ioannis

[HankMcSpank]

Guy - what can I say .....I initially used random 1-6 for my numbers in the array & it sorted them & averaged them fine! (ie 1,2,3,4,5,6 was the screen output!)

I'm puzzled why my screen output posted above doesn't look like it's working (for either the sorting or the averaging!! lol) - it was late ...that's my lame excuse for not picking up on that! The 'sorting' part of the code was a direct lift from Melanie's article.

I'm at work now (don't tell my boss!) , so will revisit this tonight.

[HankMcSpank]

Ok, the thread title really ought to be called stripping out erroneous samples & then averaging the leftovers (vs averaging data which includes erroneous samples), I've revisited my earlier problem

@ __CONFIG _FCMEN_OFF & _HS_OSC & _WDT_OFF & _MCLRE_OFF & _CP_OFF & _IESO_OFF & _BOR_OFF & _PWRTE_OFF

DEFINE  OSC 20          ' set Oscillator at 20Mhz.
DEFINE  NO_CLRWDT 1   ' PBP doesn't clear WDT automatically
DEFINE HSER_CLROERR 1
DEFINE HSER_TXSTA 24h ' Enable transmit
DEFINE HSER_SPBRG 129 ' set USART to 9600 baud  @20Mhz

ANSEL       =  0
ANSELH     =  0

    CounterA            var Byte    'A counter for sorting out the erroneous data
    DataA               var word    'A Variable used in sorting the numbers in the array 
    RawData             var word [6] 'This is the RawData gojng into the array
    Averaged            var word     'This is used to get the average.
  
    top:    
    RawData[0]  = 300
    RawData[1]  = 100
    RawData[2]  = 200
    RawData[3]  = 600
    RawData[4]  = 400
    RawData[5]  = 500

    pause 250
  
    hserout [27,91,72]          'home cursor
    hserout [27,91,50,74]      'clear screen  
    
   next1:
    hserout [13,10]
    hserout [13,10]
    pause 300 
    hserout ["before....", 13, 10]
    hserout [13,10]
    hserout [dec rawdata[0],13,10]
    hserout [dec rawdata[1],13,10]
    hserout [dec rawdata[2],13,10]
    hserout [dec rawdata[3],13,10]
    hserout [dec rawdata[4],13,10]
    hserout [dec rawdata[5],13,10]
    hserout [13,10]
    hserout [13,10]

    pause 2000
    
    CounterA =0
    gosub SortArray
    gosub Average_data
    hserout ["after....", 13, 10]
    hserout [13,10]
    hserout [dec rawdata[0],13,10]
    hserout [dec rawdata[1],13,10]
    hserout [dec rawdata[2],13,10]
    hserout [dec rawdata[3],13,10]
    hserout [dec rawdata[4],13,10]
    hserout [dec rawdata[5],13,10]
    hserout [13,10]
    hserout ["therefore, middle two samples are ", dec rawData[2]," + ", dec rawData[3], 13,10]
    pause 500
    hserout [13,10]
    hserout ["averaged middle two samples = ", dec averaged,13,10]
    pause 500
    pause 3000
    goto top
    
    
Average_data:
    Averaged = (rawData[2]+rawData[3])/2    ' average the middle two of 6 samples
    return

SortArray:
SortLoop:
    If RawData(CounterA+1) < RawData(CounterA) then
        DataA=RawData(CounterA)
        RawData(CounterA)=RawData(CounterA+1)
        RawData(CounterA+1+0)=DataA
        If CounterA > 0 then CounterA=CounterA-2
        endif
    CounterA=CounterA+1
    If CounterA < 5 then goto SortLoop
    Return

    end

And it works (basically, the code throws away all but the middle two samples of six & averages those two - ideal if your input data is all over the place.,...eg comparator counts etc!)...

before....

300
100
200
600
400
500


after....

100
200
300
400
500
600

therefore, middle two samples are 300 + 400

averaged middle two samples = 350

Or a more real world example (eg a comparator interrupt count jittering in or around the 200 count level but with one spurious/erroneous count of 400 in the 'sample of six' to get rid of)...

before....

199
400
200
200
199
200


after....

199
199
200
200
200
400

therefore, middle two samples are 200 + 200

averaged middle two samples = 200

Not entirely sure what the problem was (I changed a few things at once & didn't retrace my steps!)

[aratti]

That's better!

Cheers

Al.

[HankMcSpank]

That's better!

Cheers

Al.

& thanks for offering up some help/support earlier!

[Plcguy]

In a different application I used a "weighted Average" where no one sample could mangle the running average.
This is the regurgitated conversion to PBP:

I initialize the values and add new value to the stack and subtract off the old average, giving the new value 1/weight of effect

I leave the running average in A2D counts to avoid rounding
Must watch (Max value * number of samples) < 65000


Weight var byte
WeightedAve var word
Ave var word
WeightedCounts
NewAve var word
Count var word

AveragingComplete var word

Incoming var word ‘incoming value from a2d
WeightedCounts=0
Weight = 100
AveragingComplete = 0
Ave=0

If Count=0 then WeightedCounts= incoming* Weight
If Count=0 then Ave= incoming

If Count < Weight then Count = Count + 1
NewAve =(incoming+ WeightedCounts-Ave
WeightedCounts= NewAve
Ave= WeightedCounts/ Weight
If Count = Weight then AveragingComplete = 1
If Count = Weight then Display “Weighted Average” ‘lcd routine
If Count < Weight then Display “building average” ‘lcd routine

[HankMcSpank]

In a different application I used a "weighted Average" where no one sample could mangle the running average.
This is the regurgitated conversion to PBP:

I initialize the values and add new value to the stack and subtract off the old average, giving the new value 1/weight of effect

It must be cos I suck at maths, but I've read that a few times & it's not sinking in. Can I use an example here to how that works in practise. Let's say six AtoD Samples are taken....

100
99
100
101
499
100

You said that your weighted average stops a spurious reading skewing the overall average...can you illustrate using those six samples how the 499 would be ignored?

Many thanks!

[ScaleRobotics]

Hey Hank,

You could try Melanie's number sorting routine. She talks about dumping the upper 4 and bottom 4 numbers. You could do something similar. http://www.picbasic.co.uk/forum/cont...orting-Numbers Something like Aratti's example: http://www.picbasic.co.uk/forum/show...1327#post91327

Or, you could use Darrel's oversampling routine. It is not weighted, but still interesting. It worked very nicely for getting 16 bit results from an altimeter sensor on a 10 bit A/D pic. http://www.darreltaylor.com/DT_Analog/

[HankMcSpank]

Hey Hank,

You could try Melanie's number sorting routine. She talks about dumping the upper 4 and bottom 4 numbers. You could do something similar. http://www.picbasic.co.uk/forum/cont...orting-Numbers Something like

Thanks (as ever) for the input - re Melanie's number sorting code......that's what I was doing in post #14 above - it works, but this weighted average has got me curious now!

Whichever averaging/sorting method I eventually use will ultimately go into a frequency detect PIC circuit - I use the PIC's onboard comparators...but the sample 'counts' are a little bit erratic ....eg for Kkhz I get a stream of 'counts' in or around 5000 (which is correct for the clock speed)...but then every 10-12 or so readings, in comes a sample that way out of whack - so it's a case of which method to go with to best eliminate that gracefully!

[Plcguy]

It must be cos I suck at maths, but I've read that a few times & it's not sinking in. Can I use an example here to how that works in practise. Let's say six AtoD Samples are taken....

100
99
100
101
499
100

You said that your weighted average stops a spurious reading skewing the overall average...can you illustrate using those six samples how the 499 would be ignored?

Many thanks!

It will affect it, but only at 1/sample rate.
so, for a 10 sample example, assume the running average = 100
100 *10 (sample rate) = 1000 = sum of the samples

new value 101
101+1000 - 100 = 1001
ave = 1001/sample rate = 100.1 0r 100 for PBP (rounds down)

new value 99
99+1001 - 100 = 1000
ave = 1000/sample rate = 100.0 0r 100 for PBP (rounds down)

new value 499
499+1000 - 100 = 1400
ave = 1400/sample rate = 140

new value 101
101+1400 - 140 = 1361
ave = 1361/sample rate = 136.1 0r 136 for PBP (rounds down)

far less than a (499 +100)/2 = 300

to filter the 1st bad sample add the lines (or similar)
if incoming > ave*10/9 AND if incoming < ave*10/9 then bad_count = 0
if (incoming < ave*10/9 OR if incoming > ave*10/9) AND bad_count = 0 then bad_count = 1
if (incoming < ave*10/9 OR if incoming > ave*10/9) AND bad_count = 1 then bad_count = 0

if bad_count = 0 then gosub average
if bad_count = 1 then Skip average routine

This (or similar) should skip the 1st out of whack signal

[Ioannis]

I played with Darrels 16 bit fast averaging routine and changing the values in HystLevel, AvgCount, FAspread, I got a very slow changing average if the sampling value was close to the average.

But if suddently the input was grounded or connected to Vdd, then the Value of the average was instatly zeroed or maximized (1024 or 255 according).

It is an amazing routine. Great job Darrel, once again.

So, do not be afraid of changing the User Options as Darrel marked them and experiment.

Ioannis

Test code:

include "Fast_Average.pbp"

lcdout com,1

while 1
    lcdout com,$80,#Value,"    "
    adcin channel,var
    Value=var
    gosub average
    pause 100
wend

[HankMcSpank]

I too played with Darrel's averaging & was very impressed, but for this specific need (throwing away a sprurious reading that arrives in every 10 or 12 samples), I think perhaps Melanie's number sorting is more appropriate.

Wrt Melanie's number sorting - I dabbled with a 'rolling window' (which just takes the next sample into an array & then sorts along with the previously stored samples already in the array) or a 'fill an empty array' type (which starts with an empty array....and waits untilt he array is filled before sorting, then empties & repeats). It might have been my cack approach, but I didn't see a whole lot of difference (results wise) between the two methods.

plcguy ...thanks for the extra explanation, but that 140 'spurious' count - though much slimmed down - would still cause chaos in the frequency detect circuit I'm intending this to go in.

I also mulled a "deduce which sample value appeared the most' in a group of samples ....which would be a handy utility to have, but at this stage of my programming life (early!), it's beyond me!

[Plcguy]

Mabe I'm missing part of your challenge.
How many samples will you throw before deciding it's a real change?
If you look at the filter I added you can set the # of samples to skip as well as the % above and % below to determine what level of input is out of range.

I would prob work on the spurious signal issue. good data acquisition is critical if the program is to be robust. That routine I added is really for a failed sensor check, rather than a continuous filter.

[HankMcSpank]

Mabe I'm missing part of your challenge.
How many samples will you throw before deciding it's a real change?
If you look at the filter I added you can set the # of samples to skip as well as the % above and % below to determine what level of input is out of range.

I would prob work on the spurious signal issue. good data acquisition is critical if the program is to be robust. That routine I added is really for a failed sensor check, rather than a continuous filter.

The challenge is that when detecting frequency using the internal PIC comparators......the resulkting count appears to be jittery. By that I mean for say 1Khz, you get 'counts' arriving like thus...

5000, 4999, 5000,4999, 5001, 5000, 4999,5001,3479, 5000,4999,5001


now for the life of me I can't get to the bottom of why the spurious count arrives! (which would be the preferred option), so I'm faced with getting rid of it in software. What Melanie's number sort routine does, is sort an array of samples into numerical order (which will put the spurious count at the beginning), and then averages only the middle of the array. It gets rid of the spurious count completely.

If the frequency changes, then the middle of the array (after sorting) should reflect the count.

So my challenge if you like, is to get rid of spurious counts (ie not 'average' with them at all) ...and also to do this as quickly as possible.

As I say Melanie's routine works....but I'm always open to other methods to rid spurious readings from the equation.

[Ioannis]

Have you tried Darrel's routine with values set high enough?

Ioannis

[HankMcSpank]

Have you tried Darrel's routine with values set high enough?

Ioannis

I tried Darrel's averaging when I wanted to smooth out rapidly changing AtoD values (the output of a peak detect - whose level is proprtional to ac signal amplitude...and moves about a lot) - it worked very well in that situation.

But for this situation - I really don't want the spurious compartor counts to be handled at all - I want them rejected...so no, I haven't used it in this instance....unless I've missed the point, I don't think it's what I need here?

[Ioannis]

As long as the values don't hit the edges your are fine.

Give it a chance.

Ioannis

[Darrel Taylor]

I think the Fast Average section might be modified so that instead of jumping to the spurious noise point, it will reject the sample entirely.

That should only take commenting 1 line from the original code.

[HankMcSpank]

I think the Fast Average section might be modified so that instead of jumping to the spurious noise point, it will reject the sample entirely.

That should only take commenting 1 line from the original code.

I've broke the breadboard circuit up ....so can't test right yet. But this sounds cool.

How does the averging code decide whether what's coming in is spurious...or in fact the beginning of a new stream of samples (eg a new frequency arriving, which will generate a different comparator count - with it's own spurious counts).

I guess what I really need issome form of 'trigger' (or 'change' threshold) so that my program then knows that a new frequency is coming in (& is not just a spurious reading)...

For example, red test below is spurious count, blue is new frequency arriving (therefore count changes and starts coming in at around that new level)

5000,4999,5000,5001, 3956,5000,5001,5000,4999,5000,3789, 2570,2571,2569,2570,2571,18912569,2571....

[cncmachineguy]

Hank, It seems to me you need to set some rules, or rather conditions. something like this:

1) samples must be +- 10 of a target number to be included in the average
2) numbers outside the +- 10 will need to have 2 more samples within the new samples range to be considered a good sample.

That seems like it would weed out the rouge numbers. Now of course I have chosen random numbers. The 10 and 3. I don't really know what the numbers represent (I assume gutiar notes). Nor do I have a feel for how long a note will last. thats what drives the 10 number.

[Darrel Taylor]

We choose to go to the moon. We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard, because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one which we intend to win, and the others, too.
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; John F. Kennedy

Notice in the image that all spurious samples were rejected.
And where the samples changed from 6798 to 3899, the average caught up after 2 samples (adjustable with a constant).

DEFINE OSC 20

DEFINE LCD_DREG PORTB 'LCD data port 
DEFINE LCD_DBIT 0 'LCD data starting bit 0 or 4 
DEFINE LCD_RSREG PORTB 'LCD register select port 
DEFINE LCD_RSBIT 4 'LCD register select bit 
DEFINE LCD_EREG PORTB 'LCD enable port 
DEFINE LCD_EBIT 5 'LCD enable bit 
DEFINE LCD_BITS 4 'LCD bus size 4 or 8 
DEFINE LCD_LINES 2 'Number lines on LCD 
DEFINE LCD_COMMANDUS 2000 'Command delay time in us 
DEFINE LCD_DATAUS 50 'Data delay time in us 

DEFINE HSER_RCSTA 90h ' Enable serial port & continuous receive
DEFINE HSER_TXSTA 24h ' Enable transmit, BRGH = 1
DEFINE HSER_SPBRG 129 ' 9600 Baud @ 20MHz, 0.16%
DEFINE HSER_CLROERR 1 ' Clear overflow automatically

'----[Averaging Options]------------------------------------------------------------------
AvgCount	CON  16		' Number of samples to average
FAspread	CON  50 	' Fast Average threshold +/-
Reject          CON  2          ' Spurious Rejection level

;----[Variables]--------------------------------------------------------------------------
Value		VAR  WORD
RejCount        VAR  BYTE
ADavg		VAR  WORD

;----[Initialize]-------------------------------------------------------------------------
RejCount = 0
ADavg = 0

;----[Main Program Loop]------------------------------------------------------------------
Main:
    HSERIN [wait($FF),DEC Value]                                   ; get a sample
    HSEROUT ["Smpl= ",DEC Value]
    LCDOUT $FE,$80,"Smpl= ",DEC Value,"    "
    GOSUB Average                                                  ; average it
    HSEROUT ["  Avg= ",DEC ADavg] ;   ,"  Avg2= ",DEC ADavg2,13,10]
    IF RejCount > 0 THEN HSEROUT ["  Rejected"]                    ; indicate if rejected
    HSEROUT [13,10]
    LCDOUT $FE,$C0,"Avg = ",DEC Value,"    "
    LCDOUT $FE,$94,"FC  = ",DEC RejCount,"    "
GOTO Main

' -=-=-=-=-=-=  Average Sample values -=-=-=-=-=-=-=-=-=-=
Average:
    IF Value = ADavg Then RejCount = 0 : RETURN  ; NoChange
    IF ABS (Value - ADavg) > FAspread OR Value < AvgCount Then FastAvg
    IF ABS (Value - ADavg) < AvgCount Then RealClose
    ADavg = ADavg - (ADavg/AvgCount)
    ADavg = ADavg + (Value/AvgCount)
    RejCount = 0
    GoTo AVGok
  FastAvg:
    RejCount = RejCount + 1
    IF RejCount = Reject + 1 THEN 
        ADavg = Value
        RejCount = 0
    ENDIF
    GoTo AVGok
  RealClose:
    RejCount = 0
    ADavg = ADavg - (ADavg/(AvgCount/4))
    ADavg = ADavg + (Value/(AvgCount/4))
  AVGok:
    Value = ADavg			' Put Average back into Value
  NoChange:
Return

[Ioannis]

What filters and what rules?

Darrel rules!

A masterpiece, if I may say.

Ioannis

[Darrel Taylor]

Thanks Ioannis.

And I'll say ... cncmachineguy's rules were pretty darn close to how I modified the averaging routine.

[HankMcSpank]

Wow ...what can I say?!!

DT you truly rock...and without a doubt you're my most favouritiest E-personage on the whole of this global network!

I'll give this runout at the weekend...once again, thanks....just brilliant!!

cnc machine guy - great input too ...many thanks.. re the arriving frequencies (or more importantly, what the PIC sees for incoming signal frequency - comparator 'count'), well, on a std tuned guitar, the lowest note is 82.4Hz...which @20Mhz clock gives a comparator interrupt 'count' of about 60700 per 82.4Hz period' ...the highest note on a guitar is about 1.4Khz & gives a comparator count of about 3500 1.4khz 'period'. Each musical note has about 5% difference...but the problem with guitar 'pitch detection' is you can bend a note, which will obviously fall somewhere in between the 5% between notes.

I'd like to be able to eventually get the PIC discern not just which note is being played but also when it is s being bent - but for that I need good solid incoming samples to process...and these spurious samples were really screwing me up - hence I relish having a dabble with what DT has just posted!

[HankMcSpank]

An old thread & just revisiting this issue again ...re the spurious rejection, there's a setting as follows....

Reject          CON  2          ' Spurious Rejection level

Presumably that setting is the amount of samples, before the averaging code accepts/adopts the new incoming sample (vs just throwing away a sample that diverges too much) ....but what is deemed a 'spurious' sample ..in other words what in the Darrel's code a couple of posts above decides that the incoming sample was spurious ...and can this 'sprurious diversion amount' be adjusted?

To help explain, consider the following trail of ADC samples...

99
101
100
91 ------ not spurious, just a low sample
97
101
50 -----------spurious
98
100

How is the spurious rejection set so as to accept the ADC rsample of 91, but to just reject the sample of 50?

[cncmachineguy]

Ok Hank, I'll give this a go. BTW, it only took me 15 mins to remember what I was thinking when I posted almost a year ago!

Anyway, this is what I think is going on. Every sample with a difference greater then 50 in DT's code gets rejected the first and second time through. The 50 can be set with the FAspread constant. the number of times it get rejected is set by Reject constant.

The way it gets rejected is really pretty. Int the avg routine, the value gets tested against FAspread, if it is bigger, the flow jumps to FastAvg, where reject count gets incremented by 1. then reject count gets tested to see if it is now 3

IF RejCount = Reject + 1 THEN

if it is, the new value is accepted as good. It also becomes the new average. and everything moves on from there.
If it is not, then the running average is left as is and the current value is rejected.

BTW, I think using DT's FAspread con 50, 91 would never be considered as spurious since it is within 50

Does this help?

[HankMcSpank]

Thanks Bert....let me see - so it's the FAspread that sets how far the deviation from mean that can be accepted .... with the RejCount counting how many 'deviations from mean' there have been - once the Rejcount number is met, then the new level of incoming sample becomes a valid sample to average against?

[cncmachineguy]

Thats my understanding Hank. It sounds so much better when you say it.
Do you get the code now?

[HankMcSpank]

Yes, I get it - tks (as ever) for the input Bert.