 |
 |
 |
 |
 |
 |
 |
 |
|
 | ||||||||
 | ||||||||
|
Our customers frequently ask us for assistance deciding between two or more models. We try to familiarize ourselves with the products that we carry so we can help you learn their strengths and weaknesses. As an authorized dealer for both companies, we sell both of these products and we believe that both are well-built and effective training tools. Some customers will ultimately choose one or the other, depending on their needs. With this review we merely want to shine some light on one factor that might be important in your decision.
In this article, we set out to compare the Garmin Forerunner 305 with SiRFstar III GPS technology (new in Spring of 06) to the Polar S625X with 2nd generation footpod technology in one very important category: distance accuracy in partial tree cover.
For this comparison review, our test subject wore both monitors on two different runs. The out-and-back measured courses - one with moderate tree cover and the other with light tree cover - allowed us to evaluate the accuracy of these two models. Both courses ran under an occasional overpass.
The Forerunner 305 claims on its packaging to contain a “high-sensitivity GPS receiver for improved reception near trees, buildings etc.” On Garmin’s web page, this is marketed as High-Sensitivity SiRFstar III™ architecture. This is an area of concern for a lot of shoppers, and Garmin has taken steps to improve the product relative to the original 301 model. We wanted to learn whether they succeeded.
The Polar S625X has been the gold standard for running speed and distance since its introduction two years ago. Of late, we have been hearing from some customers that it is “just a pedometer”. This, of course, is completely false – an incredible amount of math and physics is packed into the footpod (see sidebar). It sounds complicated, and it is. But our experience with this model over two years has shown that it really is very accurate. The question is, does the new GPS solution fall short, measure up, or does it raise the bar?
This article compares the results of two separate running workouts in which the subject wore both models simultaneously. Both workouts have the same format. After a walking warm-up, we marked a lap at the start of a two mile measured course and began to run. We hit the lap button again at the two-mile point, where we turned around, and again on returning to the starting point. Then walked or jogged a recovery.
The first workout was done on the Boulder Creek Path, starting from the west end of Boulder ( Colorado) and running downhill with the creek for two miles, then back up to the start. For the second workout, we started east of town, again on the Boulder Creek Path, and ran uphill for two miles and then returned to the start.
The first test is moderately tree-covered. At most times large patches of blue sky could be seen, but trees were blocking approximately 50% of the sky. The second test area has a more open sky, but there are still some areas with trees – an estimated 25% of the sky was obscured by tree cover. On both days there was little or no cloud cover; just a clear or partly cloudy day.
Sometimes a picture is worth a thousand words. Compare below the results of the first workout (more trees). I will continue the text below the screen captures to explain what you are seeing.
The Polar S625X results from test workout 1, graphed in PC Coach. Red is heart rate, blue is pace, purple is altitude.
The Garmin Forerunner 305 results from test workout 1, graphed in Garmin’s software. Red is heart rate, blue is pace, green is altitude.
It is immediately obvious that there is a tremendous amount of noise in the Forerunner graph. This is so bad in certain spots, where the tree cover was denser, that it is not really possible to tell much about the workout.
The graph from the S625X is an accurate reflection of the athlete’s experience: a steady pace for much of the run, increasing speed at the end.
I wondered whether, in Garmin's case, the truth was only hidden by the noise and that perhaps it would still produce a good distance measurement. Sadly, this was not the case.
Based on mile markers along the course, lap 2 should be 2 miles, and lap 3 should be 2 miles. We can learn about accuracy even without the markers, however, because laps 2 and 3 should be the same distance; they are the out and back between the same two markers.
Garmin showed lap 2 as 1.87 miles and lap 3 as 1.49. The return route appears as a whopping 19% shorter than the way out. It missed the measure on the 2-mile return trip by about 4/10 (four-tenths) of a mile. Clearly, this is a major failure under these test conditions.
Polar showed lap 2 as 1.89 miles; lap 3 as 1.84. The way back is therefore shown as 2.5% shorter (5/100 mile) than the way out.
On the course, laps 1 and 4 are approximately the same distance, but we were not careful to measure to a precise marker on this portion, so it’s best not to draw conclusions from laps 1 and 4 in this test.
Again, the Garmin screen capture shows a lot of noise in the altitude graph (plus or minus 30 feet, approx.). This was not surprising to us knowing the inherent limitations of GPS for use as an altimeter. The Polar S625X uses a barometric-style altimeter, and its graph shows the true topography of the path.
One result of the extra noise in the Garmin altitude data is that the summed results of total elevation gain or loss for each lap becomes completely meaningless.
Elevation Change |
Polar Gain |
Polar Loss |
Garmin Gain |
Garmin Loss |
Lap 1 |
0' |
60' |
233' |
244' |
Lap 2 |
20' |
125' |
910' |
1008' |
Lap 3 |
145' |
30' |
1493' |
1385' |
Lap 4 |
65' |
10' |
326' |
293' |
The Polar values match the real-world experience. The Garmin values are very far off because fluctuations in the data stream are accumulated as actual elevation gain and loss.
We conclude that the Forerunner 305 is not yet ready for moderately dense tree cover. But let’s see how these products compare in a more open environment. Here are the screen captures of the results of the 2 nd test, with description below.
The Polar S625X results from test workout 2, graphed in PC Coach.
The Garmin Forerunner 305 results from test workout 2, graphed in Garmin’s software.
Even on a more open course, there is a lot of noise in the pace and in the altitude data from the Garmin. In this case, however, it is not such that the story of the workout is lost. Buried in the noise is about the same gradual change in pace and the same topography that is clear from the S625X results. Under this lighter tree cover, the Garmin was able to maintain a lock on the satellites and function properly.
The big question that we started out to answer is simply which monitor is more accurate?
Again in this case, we can’t be sure the trail markers are accurate, but the distance out should equal the distance back. Here are the results:
On this test, we took a lap mark after the first mile out, but did not do so on the way back in. So laps 2 and 3 (one mile each) are on the way out, and lap 4 is coming back two miles to the start of lap 2.
Garmin shows lap 2 as 0.98 miles, lap 3 as 1.00, for a total distance out-bound of 1.98miles, and shows lap 4 (inbound) as 2.04 miles. The way back is 0.06 miles longer than the way out, translating to a 3% difference.
Polar showed lap 2 as 0.84 miles, lap 3 as 0.98 (total outbound of 1.92 miles), and lap 4 as 1.95 miles. The way back is 0.03 miles longer than the way out, translating to a 1.5% difference - twice as accurate as the Garmin.
On this course, laps 1 and 4 (warm-up and cool-down) are the same distance, and we were careful to measure to a precise marker on this portion. This shows one limitation of the Polar footpod: it does not give you full credit for distance walked. However it does under-report this in a consistent way, showing 0.26 miles each way. We speculate that the Garmin measure of warm-up and cool-down, at 0.37 miles each way, is more accurate, since it would not be affected by the difference in stride between walking and running.
Under the conditions of these tests, using Garmin’s best day, Polar is the clear winner for distance accuracy while running. Polar also wins for altitude accuracy: the degree of error in the momentary Garmin data, besides being visually difficult to work with, confounds any effort to glean meaningful data for elevation gained or lost.
One can imagine other tests. We might find a path with no trees at all. We might also, however, test on a day with heavy cloud cover. These are tests for another day and another article.
Accuracy in these two parameters is not the only consideration when selecting a training tool, but it is an important one. One can’t forget how cool it is to see a map of your run (such as the Garmin produces) and to fly over it with Google Earth. But that might become less exciting once the novelty wears off, if you have ten or even twenty courses where you regularly run.
Considering the basic functions of distance accuracy and altitude accuracy, evaluating these products as a training tool, our recommendations are as follows.
- If trees are a part of your training environment, definitely stick with the footpod.
- If there is sometimes partial tree cover and if accuracy is paramount, again, stick with the footpod.
- If there are no trees anywhere, don’t be fooled into thinking that the GPS is inherently more accurate. This should empower you to evaluate your options on some other criteria.
The Polar S625X allows the user to calibrate the monitor for distance measurement to compensate for the runner’s unique stride, style of shoes, etc. This is typically done at a track, by running 800 meters with the monitor in calibrate mode. We believe we did this properly. After calibration, we ran several laps around the track and the monitor reported a solid average of 0.25 miles for each lap. However, the running surface is also a factor, so we correctly calibrated for the track, not the road.
One detail that jumps out from the data in these tests is that the S625X consistently reports distances that are slightly less than the 2-mile measurements marked along the trail. It shows 1.89 and 1.84 in the first test, and 1.92 and 1.95 in the second test. The total distance does not exactly match the markers, but we don’t know how accurate the markers are. (I suspect that the first test is a shorter course than the second test) If we assume that the markers are accurate, one explanation is that the unit is calibrated for the track surface rather than the concrete surface found in these tests.
The Polar is within 2% (on average) in the out and back test of being consistent with itself. Since the difference between the recorded S625X distance and the measured trail markers is consistently in the same direction, using a different calibration would get a tighter correlation to the trail markers, however accurate they are.
One compelling difference between the graphs above is the amount of 'noise' in the Garmin graph - the jiggliness of the lines in other words. Is this a fair comparison, or slight of hand in choosing graphing options? The Garmin software has the option of smoothing on or off, and those screen captures were done with smoothing turned on. PC Coach has smoothing options of none, slight, moderate, and heavy. The S625X graphs were done with 'slight' smoothing. Below, we show one of the above comparisons, test 2 with fewer trees, with smoothing turned off in both cases.
Visit the online
|
