Which is to say, not terribly well.
And here’s a prediction. That will change. I won’t get better but the computers will. Like Deep Blue over Kasparov — or maybe more like Deep Blue over the president of a high school chess club — the machines will triumph because programmers will refine and improve their forecasting models.
Although I might be wrong about that, too.
Because it’s reasonable to ask how good computer models can ever get at forecasting fares. Perhaps not very. Airfare pricing is pretty messy.
Two things triggered this train of thought. First, a friend asked my advice on whether he should buy an airfare that was disappointingly high or wait in the hope that it would drop. Second, I read Nate Silver’s excellent book about predictions, The Signal and the Noise.
I advised my friend to wait. My theory was that the fare seemed higher than the typical autumn fare on that route and he was not yet within 21 days of travel. That was his own inclination and so he waited. It worked out — after some fluctuation he got a better deal.
We weren’t applying a complex forecasting model but simple heuristics. It seems primitive but this method of forecasting — applying some rules of thumb and consulting other people for their opinions — has value. Silver’s book demonstrated that it is often as effective as predictions based on complicated models.
But not always. In some fields, the models are pretty good and some humans have gotten pretty good at using them. Silver says that by applying rich data sets to refined computer models we have vastly improved the ability to forecast such things as the weather and baseball players’ performance. Yet some other things, such as earthquakes, don’t provide enough data and are simply too complex to pin down precisely.
Chaos theory also comes into play. In a complex system, tiny variations in a single factor can trigger large differences in outcomes, making some systems entirely unpredictable.
No doubt, there are a lot of factors to consider in airfare pricing — fuel prices, the fare managers’ moods, weather and a host of competitive considerations. Some of the relevant facts are difficult to know or predict. (The one element I’d really like to nail down is the exact point at which United’s computer system is going to wig out again and start giving away airfares.) It doesn’t necessarily add up to complete chaos but all those factors make it difficult to foresee most outcomes with great certainty.
We should not feel too bad about this. Silver found that stock market predictions are generally worthless, even coming from experts who express a high degree of confidence in their forecasts.
So forecasting fares on a particular route can be devilishly tricky, and even a general prediction of whether fares across the industry will go up or down is difficult. I recently read a prediction that American Airlines and US Airways will hold fares down across the industry while the court challenge to their merger is under way, and that seems like a sound enough theory. But that lawsuit is just one factor among many, and we can’t know for sure whether another factor will overcome it.
Back to forecasts on specific routes. Bing’s Price Predictor (formerly Farecast) and Kayak’s Price Trend function claim accuracy rates around 75 percent, but only when looking seven days ahead. That’s not very far. And the relevant question is not whether that success rate is better than random chance, but whether it’s better than simple heuristics, like those my friend and I applied.
Of course, the heuristics have to be correct. Many people believe, for example, that Tuesday afternoon is the best time to buy airfares but there’s no evidence to back it up.
Kayak puts things in good perspective in its explanation of its fare forecasts, which says: “… our first recommendation is that if you see a good price on a route, book it. Our price trend forecasts provide an additional piece of information, and we also suggest using our general when to book guidelines.”
Those guidelines are pretty much the ones that informed my advice to my friend. They work reasonably well. And, of course, if I’m on the fence, I’ll look at the fare predictor and I’ll ask my friend what he thinks, too. And then I’ll just have to take my chances.