PC clocks are usually horrible time keepers.
I had the same problem a while back, thinking the PC would be accurate.
Syncronize the time on the PC with an NTP server before starting the test, and sync it again at the end of the test.
You'll be amazed how far off the PC's clock is.
Also, is your 20Mhz source a crystal, or a resonator?
Resonators are no good for time keeping.
I've tested the elapsed timer to hold within about 4 seconds a week. (with a good crystal)
Bookmarks