Number Sort Algorithm

[aratti]

Well, the way I achieve that is to take say 16 ADC readings, sort them in sequential order of value, junk the top and bottom four readings as being 'out-of-range', sum and average the middle eight... this makes ADC readings in industrial applications pretty much bomb-proof...

Miss Melanie, in doing so you are potentially junking good data, but you can also potentially retain and use in your mean bad data, since you have no way to know how many bad readings you will take with your sampling.

A statistically correct approch should be to calculate the mean and the standard deviation (sd), of all your 16 readings, than retain all data within +/- 1 or 2 sd depending esclusively on your need.

A simpler approach could be to decide which threshold (in percentage of the mean) to apply to reduce the variance, than take the mean of all the 16 readings and discard all the data beyond the threshold limit.

In both cases you will not need to sort your array.

Al.

[Ioannis]

But the maths to do those calculations I feel are more code hungry than just sorting...

Ioannis

[aratti]

Ioannis, if the number that you will display has no importance then you can fix it with a constant, display will be rock stable (always the same number). On the other hand, if the number should provide you usefull information then you must treat it in such a way to preserve those information, which means you must use the correct statistical approch, regardless how code hungry the algorithm could be.

Al.

[Ioannis]

Of course I agree with you Al.

I just feel that Melanies approach is faster giving almost same result for most cases.

Sure if you need the most precise results, one in the way to follow.

Ioannis

[Melanie]

There are many ways of accomplishing the same task. Give that task to a dozen engineers, you may get a dozen different approaches, each achieving the desired result, and each achieving that result with different methodology. Really, in electronics and programming there is no SINGLE correct way of doing anything and the methodology you employ rests with your imagination which makes our business just as exciting and creative as any Artist painting a canvas.

If I am measuring ambient temperature for example, I don't expect to take sixteen readings 50uS apart and discover twelve of them to measure around +22.7 Celcius, one to measure -20, one measures -40 Celcius, one measures +400 and one measures +700 Celcius. But that is exactly the kind of result you might get is you experience a local lightning strike with spikes being induced into your electronics. Very short bursts of total rubbish.

Now using a conventional method of summation would produce an erroneous result, but the temperature may well drop to -20 or minus 40 in winter, so that could also be technically valid data, but in the case above, we know that the onset of winter doesn't happen in 100uS, so we need to trash the data at the extreemes (even if it might be good data).

Now in the code I published some years back, I put forward an idea of taking multiple samples, sorting them, discarding those results at the fringes and concentrating only on the centre median of our collection of data. This works good in the majority of industrial and environmental applications and even for motion control on an industrial assembly robot (filtering localised welding spikes). The amount of data you discard is not arbitrary, it depends on the number of samples you've taken and the TIME you've sampled them across. I know lightning tends to be short duration spikes, so out of a milliseconds worth of sampling, I know that anything out of the ordinary lasting 50 or 100uS we can happilly junk. But, if I was creating the firmware for a new Digital Oscilloscope or sampling an Audio waveform for an Amplifier, this would obviously not be the correct approach.

[aratti]

Hi Darrel, I tested the sorting snippet you have posted, using a rather interesting swapping technique.

The code worked, but now and than, it was yielding very fanny results.

From a closer code analysis, I came to the conclusion that :

If CounterA > 0 then CounterA=CounterA-2

should be corrected with:

If CounterA > 1 then CounterA=CounterA-2

After the correction, I didn't see any fanny result for the rest of my testing period.

The explanation I have given is the following:

If CounterA = 1 then CounterA - 2 = 255 and this screw up everything.

Can you confirm?

Al.

[Darrel Taylor]

Hi Darrel, I tested the sorting snippet you have posted, using a rather interesting swapping technique.
The code worked, but now and than, it was yielding very fanny results.
From a closer code analysis, I came to the conclusion that :

If CounterA > 0 then CounterA=CounterA-2

should be corrected with:

If CounterA > 1 then CounterA=CounterA-2

After the correction, I didn't see any fanny result for the rest of my testing period.
The explanation I have given is the following:

If CounterA = 1 then CounterA - 2 = 255 and this screw up everything.

Can you confirm?

Al.

Well, I wouldn't think it's a problem.
Right after the IF block that statement is in, Melanie put a ...

CounterA=CounterA+1

So if CounterA = 1, subtracting 2 makes it 255, then adding 1 makes it 0 again.

Had you made any other changes?
<br>

[aratti]

Had you made any other changes?

Yes, a second one, but more for cleaning the code, since it didn't effect functionality:

RawData(CounterA+1+0) = RawData(CounterA) ^ RawData(CounterA+1)

Has been cleaned with:

RawData(CounterA+1) = RawData(CounterA) ^ RawData(CounterA+1)

Well, I wouldn't think it's a problem.
Right after the IF block that statement is in, Melanie put a ...

CounterA=CounterA+1

So if CounterA = 1, subtracting 2 makes it 255, then adding 1 makes it 0 again

But the above make sense, I do better check the whole system again.

Thank you Darrel.

Al.