Our many attempts at measurement
Our first camping trip was to the Cusco region of Peru in 2002. We prepared for it by buying our first GPS unit (a Garmin), and we also both took barometric altimeters: Colins was incorporated in his cycle computer (a Cat Eye), Traceys was wrist-worn (a Suunto). We found that the Cat Eye was inaccurate, but the Suunto agreed with the GPS in giving altitudes a few percent lower than claimed by maps. There was no reason to treat the Peruvian survey as gospel, so we accepted the majority view.
The discrepancy between barometric readings and maps wasnt one of calibration: not only did absolute altitudes differ, but also the ascents from valleys to passes.
When we visited the Andes in 2005 we realised that our reasoning had started from a false premise. We made GPS measurements of the height of a campsite in the evening and again on the following morning and found a difference of around 50m, the same effect as is obtained with barometric altimeters due to the diurnal pressure cycle. Our GPS unit incorporated a barometer (as all do which include electronic compases), and we realised after performing some web searches that GPS readings are used to calibrate the barometric readings, but the barometric readings are treated as primary. In the case of spot readings there is no possibility of calibration (which needs about an hour of continuous measurement), so the GPS unit functions as an uncalibrated barometric altimeter.
None of this is documented. GPS units portray themselves as infallible magic boxes, and as differing from barometric altimeters in not needing a host of careful corrections to give trustworthy results. But the real difference is that the limitations of barometric readings are documented whereas GPS units sweep everything under the carpet. Perhaps if you use them in the way the manufacturers expect the results will be accurate, but the manufacturers arent open enough to tell you what assumptions theyre making.
You can test this by measuring your altitude in an aeroplane. A barometric reading will be affected by the pressurisation, and much lower than the true altitude.
So in the end the two forms of altimeter wed been using in Peru were less independent than wed thought; but they were good quality devices, and we couldnt speak for the Peruvian survey.
The pattern of our measurements in these years was consistent. GPS and barometric readings roughly agreed, and they confirmed the heights in maps except in Peru, where they knocked 100m off the passes around Cusco, and in 2006 we found that they knocked 200m off in the Cordillera Blanca. Colin by now had discarded his Cat Eye in favour of a much more accurate VDO.
In 2010 we visited Kumaon, and since the display of our GPS unit packed in during a cold night in Arunachal Pradesh, we treated ourselves to a new one. This time, for the sake of having readings of known provenance, we bought one with no barometric component.
We found that most of the time it gave readings about as accurate as a barometric altimeter. Since it didnt depend on any meteorological assumptions, it therefore seemed a better means of measurement... were it not for the fact that a small number of our GPS readings (about one in 20) were subject to mysterious errors in one case running to thousands of metres. We came to the conclusion that these were caused by the unit being confused by reflected signals.
If a GPS unit is in motion it can distinguish reflected signals from the direct one, but our problem is that we were making spot readings (and perhaps cyclists dont move fast enough to count as being in motion). No doubt the chances of being misled by reflections depend on the location, and our preferred terrain of valleys cutting through high ranges must be the most unfavourable there is.
In 2011 we made our third visit to Peru. As before we found that our barometric altimeters wanted to knock a couple of hundred metres off the official pass heights, but we now found our GPS unit taking sides with the maps instead. We put a fair amount of thought into this, and conducted some web searches; we corresponded with Chris Crocker who has a useful page on the use of barometric altimeters. There was no escaping the fact that we were encountering inconsistencies in Peru which we met nowhere else, and which couldnt be explained from errors in the maps. The difference had to lie in the fact that Peru is tropical whereas our other destinations were temperate.
It is hard to see that the GPS measurements can be the ones at fault: triangulation is the same at all latitudes, whereas barometric altimetry is subject to numerous adjustments for gravitational and meteorological effects. Our suspicions were first directed at centrifugal effects from the earths rotation, but a back-of-the-envelope calculation in Huaraz was enough to show that these were not the cause. In the end we tentatively concluded that our barometric altimeters were failing to make sufficient allowance for temperature (not that Peru is especially hot, but it can be remarkably hot for its altitude: Huaraz at 3000m was warmer than Lima at sea level).