Essay
The Time Series Blind Spot - Why Generative AI Failed at Forecasting
The industry built transformers for language and forgot that most enterprise data moves through time. Now we're realizing that temporal patterns require fundamentally different approaches than next-token prediction.
The industry built transformers for language and forgot that most enterprise data moves through time. Now we’re realizing that temporal patterns require fundamentally different approaches than next-token prediction.
The generative AI wave swept through 2023-2025 with transformer architectures conquering text, images, video, audio. Every modality got its foundation model. Every domain got its specialized variant.
Except time series forecasting. That stayed stubbornly resistant to the transformer revolution.
Not for lack of trying. Research labs threw transformers at temporal data. Papers appeared with promising benchmark results. Startups pivoted to “AI-powered forecasting.” But enterprises still run their production forecasting on decades-old statistical methods—ARIMA, exponential smoothing, Prophet. The methods that were supposed to be obsolete.
There’s a reason for this. The transformer architecture that revolutionized language fundamentally mismatches how temporal patterns actually work. We spent three years trying to force language model thinking onto time series problems, and now the data shows what practitioners knew all along: next-token prediction doesn’t translate to forecasting future values.
Transformers became dominant because they excel at one specific task: predicting the next element in a sequence based on context. Given “The capital of France is ___”, the model learned to output “Paris.” This works because language contains rich contextual patterns where surrounding words constrain possibilities.
Time series data operates differently. Given sales figures for January through November, predicting December isn’t about finding the most probable next token given context—it’s about understanding seasonal patterns, trend components, and noise separation. The patterns exist in different frequencies, different lags, different structural relationships.
A 2024 study by Zeng et al. demonstrated this mismatch empirically. They compared state-of-the-art transformer forecasting models against simple linear models on standard benchmarks. Result: the linear models won on 7 out of 8 datasets. Not close competitions—decisively better performance.