GPS logger?

hello all,

after some (late nite) investigations on the internet, I ended up here. I was researching for a cheap GPS logger .. and found many of them, but they were not affordable.

what I am looking for is a little device that would take some NMEA data from a GPS receiver mouse (5V, RS-232), eventually filter out some unneeded data (STX, ETX, etc) and save it into a file (or files) on an SD or MMX card. after some days of travel, I would get the SD-card from this device and transfer the data to my PC for postprocessing.

NMEA data looks basically like this (and that's what I would like to see in the files):

$GPRMC,131700.074,A,4628.8544,N,00716.8242,E,22.13 0221,132.60,260705,,*3E
$GPGGA,131701.074,4628.8504,N,00716.8306,E,1,05,1. 7,1080.8,M,48.1,M,9.7,0000*49
$GPGSA,A,3,08,10,29,28,26,,,,,,,,7.9,1.7,7.7*36
$GPRMC,131701.074,A,4628.8504,N,00716.8306,E,21.89 6268,133.45,260705,,*37
$GPGGA,131702.074,4628.8463,N,00716.8367,E,1,05,1. 7,1081.5,M,48.1,M,10.7,0000*79
$GPGSA,A,3,08,10,29,28,26,,,,,,,,7.9,1.7,7.7*36

(the GPS mouse can be programmed to limit the data to the GPGCA or GPRMC strings. optional postprocessing of the strings might turn each line into something like "time, latitude, longitude, altitude", but this is only needed with smaller SD-cards and very long logging. the strings come in every second. with ca. 80bytes per second, this makes about 48MB for a full week of 24h logging.)

I believe that the following HW thingy would be fullfilling my needs: http://www.compsys1.com/workbench/On...c_project.html

is this correct?

the next step is to have the right software to convert the RS-232 data stream into file(s) on the SD-card (from what I read here on the board, the filesystem on the SD/MMC card is the main challenge). then somehow this code needs to be compiled and transferred into the HW thingy.

frankly, I do not have a clou what to do next. I used to program 6809 and 8051, but that is centuries ago. and I had access to dev-environments at the company I was working for. today, I only have a pc at home...

the greatest thing that could happen is when someone of you has already done something like I want, and would share it with me.

thanks

dan

Hmmm, where do I start . . .

ok, let's try to start with the NMEA protocol.

I would focus on one single sentence (like GPRMC) and discard all the others.

Right after reception of that sentence I would extract LAT, LON, etc, and discard all unneeded Bytes.

With some smart code you can squeeze TIM, LAT, LON, and ALT into a 12 Byte (or even smaller) Data Packet.
(with full accuracy!)

Now think about your logging interval:
Does it necessarily have to be one log entry per second?
(wouldn't one every 10 (or 30 or . . .) seconds do?)
And do you really have to log data 24hrs a day?
(even if there is no movement?)

with that kind of data compression and a reasonable logging interval you can minimize Storage requirements.

So some EEPROM or FRAM could probably do.

To save even more memory you could check and
cleanup the log regularly (when the system is idle)

Imagine there is an (almost) linear track from A and B at about constant speed between A to B.
So all you really need is the data for A and B, everything in between can be cleared.

Agreed,
this "cleanup" procedure will require some math that is not that easy to implement having only the Integer brain of a PIC.

However,
the active components for the Logger shoudn't be much more than about $25 (dependant on how much Memory you need)

a PIC and some EEPROM or FRAM

I can tell it is possible,
I have built many systems like that.
(NavMicroSystems)

hi ralph,

thanks a lot for your explanations.

I see that squeezing memory is still in your (engineers) blood ;-) I did a project for a PLD device (A/C & heating controller).. uff.. decades ago, with a 4bit (nibble) processor and 4kB of ROM.. 32bit multiplication with 4bit registers... that was fun..

I would like to use the logged data with some software on the PC. most converter software (like NH-Top50Trans which is also available on PDA's) understand NMEA and will convert it into any other format that is common (e.g. GPX).

the logsize is not important because I want to have it on SD/MMC anyway. and as I mentioned before, even with 80byte/record/s I can let it run a long time until a 64MB SD/MMC would be full. if I would not have the data on SD/MMC, but in a EEPROM/FRAM on the device, I would need to have some sort of download software which also translates the data back to a common format. yet another thing that would have to be created.

I think it's easier and faster to just remove the SD/MMC and put it into the slot each PDA has (or any cardreader). then move the textfile over and use it in the standard PC software...

I still opt for the version with an SD/MMC slot. so the only code that is needed on the PIC is taking the string which comes in on RS232 and write it to a file on the SD/MMC, line by line. perhaps depending on a dip-switch (?), only log very 3 seconds or so (thus discarding all data in between). the GPS mouse can be programmed to send only one kind of string.

if the mouse has no power, there is no data coming in. so, no logging needed.
if the mouse has no reception, the string contains a certain pattern ("V" instead of "A" in the third field, and coordinates of 0, 0).
$GPRMC,104617.997,V,0000.0000,N,00000.0000,E,0.000 000,,260705,,*0E

in that case, repeating logging of that string might be senseless..

thus the code on the PIC does not need to differentiate between data types. just log all that comes in to the SD/MMC (with the exceptions above).


is the SD/MMC filesystem still such a problem?

dan

Quote:

---

Originally Posted by Gonzzo

...I see that squeezing memory is still in your (engineers) blood ;-)

---

Well that's our daily business on those micros

Quote:

---

even with 80byte/record/s I can let it run a long time until a 64MB SD/MMC would be full.

---

Why store data that is to be discarded anyway ?

Quote:

---

...if I would have the data in a EEPROM/FRAM on the device, I would need to have some sort of download software...

---

This is something the PIC could easily do for you.
It would simply send reassembled GPRMC sentences
and since there is a RS232 Interface on board (for the GPS)
you can use it to connect to the PC.
(You wouldn't need any extra hardware)

Of Course you could write a VB application to download the data to a PC, but even something like Hyperterm would do.

Quote:

---

is the SD/MMC filesystem still such a problem?

---

I have never used SD or MMC with a PIC.
(There was actually no need to, as I always "squeeze" code and data.)
(Just to give you an idea: I have NMEA Interfaces running on 8pin PICs with INTRC clock and no external Memory at all)


I have seen PIC Projects with MMC.
So MMC can't be that much of a problem.


This is an interesting project, please keep us posted.

To get started all you need is:

- 1x PIC 16F88 (or other)
- 1x EEPROM 24FC512 (or other)
- some 4k7 resistors
- 1x DB9 connector female (to PC)
- 1x DB9 connector male (to GPS)
- a piece of Veroboard or a Breadboard.

I'm almost sure it is possible to squeeze your application into the hardware mentioned above.

I realy don't see any need to save every part of the sentences unless
you have to play them back into PC mapping software to trick it into
thinking it's connected to a GPS.

Even if the whole sentence needs to be reproduced, the pic program could
put the correct data into a blank sentence itself on playback, so that '$GPRMC' didn't need to be saved a million times.

You need to read the entire sentence in if you want to check the checksum,
but lots of data can be discarded straight after that.

Playing the data from the device you make into the PC as if it was produced
live by a GPS is one way to transfer files to a PC no matter what media the data is stored.

I am using 16 bytes per entry that includes lat,long,time, valid/invalid,
and must soon include at least part of the date.
one 16 byte entry per second gives 34 min and 10 seconds logging on a
24LC256 EEPROM, and it is easy to connect 8 of them to two wires.

It is good if your pic program can set up the GPS antenna from scratch,
since some of them can 'forget' what they are supposed to be doing.
Also, it would make it possible to connect a new GPS straight to the pic
without using a PC to configure it.
Art.

wow.

quite good info here.

I guess I need to give you some more background info. I am a vivid motorbiker. that is, I do make longer trips with the bike. using a Garmin Quest GPS, I'm only able to track (log) 1-2 days worth of travel. unfortunately, the quest is somewhat weak in connectivity. it only has an USB connector, ulike other Garmin equipment that have serial capabilities as well. so, to download Quest track data, I would need my laptop taken with me, and this is just too bulky.

it happens that I have several old GPS cable mice that deliver serial NMEA data. so I took my PDA (iPAQ) with me on trips and used one of the GPS mice and the PDA as logger (there is free software around for this). however, PDA's need to be on power for long trips and they have problems with the connector (mechanically due to vibrations) and with battery temperature.

so I was looking around what other possibilities might exist. and there are plenty, but.. all the devices I found have limitations: too high price (mostly around 500€ and up), only internal and small memory, no connectivity to PDA's, etc.

so, what I am looking for is a device that
- connects to a GPS mouse or a serial Garmin GPS
- logs all NMEA data that comes along (basically no filtering)
- saves the data as plain ASCII files on a SD or MMC
- can be mounted onto my bike (perhaps under the saddle, still providing access to the SD/MMC card)
- does not need any special software to be written on PC or PDA for interfacing and such
- is small and affordable

and what I was imaging to do was:
- let the track logging run as long my bike ignition is on
- on a trip, replace the SD/MMC every now and then (probably every evening) with a new one
- have a PDA with me (which has a SD/MMC slot) and archive/postprocess the NMEA data on it. there are plenty of software available for PDA's for converting data, programming a GPS mouse, showing the track on a map, etc, you name it.

I have asked around with my friends and many of them came over a PDA/mouse combination to a Garmin. thus still have an unneeded mouse at home. on the other hand, cable mice are very cheap in the meantime. other people I have asked have shown interest in a small cheap homebrewed tracking solution for their vehicles that would act as black box and just deliver standard GPS format datafiles on a little SD or MMC. don't ask me for a business case, though. I am far from wanting to make a business out of this. on the other hand, if I find someone here who would be interested to work on this, I would appreciate. frankly, my HW and programming skills were never really good (my quality was debugging).

@ralph, @Art
anyway, I think your points are still valid. it would make sense to think about a version 2 of that device, which would squeeze NMEA data, but still logging it onto the SD/MMC.

taking the 16 bytes proposed by Art, using e.g. a 3 second log interval, assuming a bike/vehicle is in use for 10hours a day, this would end up in 192kB per day and roughly 32MB per full year. you see, I'm not very memory sensitive as long as there are cheap SD's or MMC's around.

now the next phase is more.. let's say.. looking into the future.. therefore
[dream mode ON]
I want to have a little box on the bike that
- logs GPS data on an SD/MMC card (see above)
- shows me the gear I'm in on a display
- logs more data from the bike to the SD/MMC together with the GPS data, for example: gear, outside temperature, oil temperature, drehzahl (RPM), etc.

while browsing the internet, I found this site http://members.aon.at/w.walter/page_4_1.html that describes a datalogger for RC planes. I was intruiged by it, thus all the ideas above came up.
[dream mode OFF]

I hope you now understand why I have those silly ideas above...

dan

Dan,

your ideas so far are not that stupid, it is all feasable.

If you like SD that much why don't you just go and get one of the CompSys SDMM3X Kits with 18F252 ($57.00)

With some programming it would do all you need at a reasonable costs.

Quote:

---

Originally Posted by NavMicroSystems

Dan,

your ideas so far are not that stupid, it is all feasable.

If you like SD that much why don't you just go and get one of the CompSys SDMM3X Kits with 18F252 ($57.00)

With some programming it would do all you need at a reasonable costs.

---

yeah.. programming..

I'll try. perhaps. (my vacation just ends this weekend)

thanks

dan

Quote:

---

Originally Posted by Gonzzo

yeah.. programming..

I'll try. perhaps...

dan

---

Dan, I'm waiting for your response on my latest PM . . .

Hi all,

I'm looking for a solution to capture NMEA data with a PIC micro and retransmit them on another serial interface... a kind of NMEA repeater.
My only problem is that fro example GPGAA sentences are not the same lenght:
$GPGGA,130524,4650.8982,N,01650.2497,E,0,00,,,M,,M ,,*5A
$GPGGA,170834,4124.8963,N,08151.6838,W,1,05,1.5,28 0.2,M,-34.0,M,,,*75

I try to mace something like:
......

dim GPGGA[i don't know the lenght] as byte

SERIN PORTB.0, 16572, [ WAIT( "$GPGGA," ) , STR GPGGA ] 'this would store the string if I would know its lenght
SEROUT PORTB.2, 16572, ["$GPGGA," , str GPGGA, 10, 13]

something like this... :-)
Please help me!!!

this is what i do for GPS data. (simplified, but this will work). NMEA sentences are ended with a CR, LF, so i loop through the data, checking for a CR, once its received i set a flag and pad with zeros. this gives me a constant length output of the complete NMEA sentence without chopping off any data. There are probably more efficient ways which dont add extra bytes, and the loop takes a bit of time, but at least it works ;)

'record in 70 bytes
SERIN2 GPS_TXA, GBAUD, [wait("$GPGGA"),str GPGGA_IN\80]


'CLEAN UP THE GPGGA Read
CR_BYTE = 0
for i = 0 to 77
if GPGGA_IN[i] = 13 then
CR_BYTE = 1 'set flag that CR has been recorded
GPGGA_IN[i] = 160 'replace CR with a space
endif

if CR_BYTE = 1 then
GPGGA_IN[i] = 160 'pad with zeros
endif
next i
GPGGA_IN[78] = 13
GPGGA_IN[79] = 10

'END CLEAN GPRMC READ