Chasing the Ghost: Why AI on Price Indicators Finds Overfitting, Not Edge

There's a stage every trader goes through. You start with RSI. It works sometimes. So you add MACD — two filters must be better than one. Then Stochastic, Bollinger Bands, maybe a moving average crossover for good measure. Six indicators on one chart, each confirming the others, each giving you permission to take the trade. The backtest looks fantastic. You've built a system.

You've built an illusion. Those six indicators? They're almost the same signal, measured six different ways. Stacking them doesn't add information — it adds overfitting. And when you put AI on top of that stack, you're not discovering edge. You're rediscovering the same factor, faster.

The real edge isn't a better filter on price data. It's grounding your strategy in something with an economic mechanism you can explain — because a system you can't explain is a system you can't trust.

The indicator soup is one signal

Take the six most popular technical indicators — RSI, MACD, Stochastic, Rate of Change (ROC), Commodity Channel Index (CCI), and %b (Bollinger Band position). They look different on a chart. They have different formulas, different names, different inventors. A correlation analysis running on S&P 500 daily price data from March 1957 through December 2023 tells a different story.

The Spearman rank correlation coefficients between them:

Every pair correlates between 0.84 and 0.99. These are not independent signals. They are, almost without exception, measuring the same underlying quantity: price velocity — the first derivative of price with respect to time. RSI asks "how fast has price risen relative to its recent range?" ROC asks "how much has price changed over N days?" MA Distance asks "how far is price from its average?" These are the same question wearing different clothes. The MACD histogram — the one outlier at 0.26–0.49 correlation — measures acceleration, the second derivative. That's it. Six indicators, two degrees of freedom.

The practical implication is uncomfortable. "Diversifying" across RSI and MACD and Stochastic doesn't reduce risk — it concentrates you in a single bet on price momentum. The "confirmation" you see when three indicators agree is not three independent votes. It's one vote, counted three times.

Why AI makes the soup worse, not better

Here's what happens when you add machine learning to this picture. You feed a model 50 indicators — RSI, MACD, Stochastic, ATR, ADX, OBV, Ichimoku, and 43 more — and let it find the optimal combination. The model dutifully searches the parameter space, finds a combination that scored beautifully in the backtest, and returns a strategy with a Sharpe ratio of 1.8.

What did it actually find? Fifty transformations of the same underlying input — price — and a parameter combination that happens to fit the historical sample. The model isn't discovering a new source of return. It's reprocessing one signal fifty ways and selecting the output that most closely conforms to the past. The more indicators you feed it, the more parameter combinations it can search, and the more likely it finds a fit that looks significant but isn't.

Marcos López de Prado and David Bailey demonstrated how pervasive this is. Their research, published in Significance (Journal of the Royal Statistical Society, 2021), shows three structural features that make finance uniquely vulnerable to false discovery: (a) the probability of finding a genuinely profitable strategy is very low due to intense competition; (b) true edges are often short-lived because markets adapt; and (c) it's nearly impossible to debunk a false claim through controlled experiments on new out-of-sample data — you can't rerun 2020.

Their earlier work introduced the Deflated Sharpe Ratio (Journal of Portfolio Management, 2014) and the False Strategy Theorem — statistical tools that correct for the number of trials run during strategy selection. A companion framework, the Probability of Backtest Overfitting (PBO), formalizes the chance that a strategy's in-sample ranking reverses out-of-sample (Bailey, Borwein, López de Prado & Zhu, Journal of Computational Finance, 2017). The point is simple but devastating: if you test 100 parameter combinations and pick the best by Sharpe ratio, the reported Sharpe is systematically overstated. The more you search, the more inflated your best result. Almost nobody in practice applies the correction.

The real-world evidence is bleak. Among 68 market forecasters studied over a 17-year period, the overall accuracy rating was 48% — no different from chance. Among actively managed U.S. large-value funds, only 8.3% beat their comparable passive index over a 10-year window. The backtests these professionals ran before launching their strategies presumably looked excellent. Out-of-sample reality did not cooperate.

You're chasing a ghost

Why doesn't it work? Because price data alone — even processed through the most sophisticated ML pipeline — is a random walk surface. The "patterns" your model finds are not edges. They are artifacts of the specific sample. A V-shaped crash in March 2020, a rate-hike cycle in 2022, a multi-year bull market from 2012–2021 — each leaves footprints in price data that look like exploitable structure. They are not. They are the residue of a single historical path, and the path that follows will be different.

This is the ghost. You can feel it, you can fit it, and it vanishes the moment you need it. The indicator soup makes the ghost feel more solid because six agreeing indicators feel like evidence. But six readings of the same speedometer, all saying you're going 60, are not six confirmations that 60 is the right speed. They're one measurement, six times. The question worth asking is not "what speed am I going?" but "should I be on this road at all?"

As López de Prado puts it: tune many parameters, select the best-performing one, and you are almost guaranteed to overfit — even if you didn't mean to. The problem isn't that AI is bad at finding patterns. The problem is that AI is too good at finding patterns that aren't there.

What grounding looks like instead

Contrast the indicator stack with a different kind of strategy — one built on a risk premium. The equity risk premium is not a pattern in price data. It's an economic mechanism: equity holders bear more risk than bondholders, and markets compensate them for it over long horizons. That compensation exists for a reason you can articulate. It survives across centuries, countries, and market regimes, because the reason doesn't expire.

Or the volatility risk premium: option implied vol systematically exceeds realized vol, because option buyers are paying for insurance, and insurance isn't free. That's a structural fact about how markets price fear, not a parameter setting on an oscillator.

These premia have something the indicator soup never will: a why. A sentence you can write down that explains why this return should exist, why someone is willing to pay you to bear it, and why it should persist after you've found it. When the drawdown comes — and it will — that sentence is what keeps you in the trade. A black box that scored well in-sample gives you nothing to hold onto.

Explainability isn't a nice-to-have. It's the reason you can trust a system through a regime the backtest didn't cover. If your answer to "why should this work?" is "because the model found a pattern," you don't have an answer. If your answer is "because this premium compensates investors for bearing a specific, persistent risk," you have a claim that can be tested, debated, and — most importantly — held when the numbers turn against you.

Where this meets our work

RiskHarvest builds strategies from risk premia — each with an economic rationale, not a price filter re-skinned. Equity, term, credit, volatility: each return stream exists because someone in the market is paying for something — growth exposure, duration, credit risk, insurance. We size each premia to a steady level of risk and strip out the macro and sector exposures that add noise without adding return.

This is why we don't build from indicators. Price-based filters, no matter how many you stack or how cleverly you optimize them, are measuring one thing: how fast price has been moving. That's useful information — but it's one degree of freedom in a system that needs many more. The diversification that comes from combining genuinely different risk premia — each driven by a different economic force — is categorically different from the false diversification of adding RSI to MACD. One combines independent return sources. The other recombines one.

The indicator study's author reached this conclusion from the data side: "why use sophisticated sensors, when the simplest one suffices?" We reach it from the mechanism side: the simplest filter doesn't suffice either, because the edge was never in the filter. It's in the premium. Get the premium right, explain why it exists, and the filters become a detail of implementation — not the foundation of the strategy.

That's also why we publish the research and the numbers. A backtest you can't explain is a backtest you shouldn't trust. The question isn't whether your model fits the past. It's whether you can articulate — in plain language, without referring to the chart — why it should work in the future. If you can't, you're chasing a ghost. Get a ghost out of a dozen indicators or out of a neural network — it's still a ghost.

The takeaway

Most trading indicators are the same signal — price velocity — measured in different formulas. Stacking them doesn't diversify; it concentrates you in a single bet and adds overfitting. AI on price data doesn't find new edges; it finds fifty ways to reprocess one factor and selects the fit that looks best in hindsight.

The strategies that persist aren't the ones with the most indicators or the biggest models. They're the ones grounded in an economic mechanism you can explain: a risk premium, a structural inefficiency, a compensation for bearing something others won't. If your system can't answer "why should this work?" in a sentence — a real sentence, not "because the backtest shows it" — the edge isn't there. The backtest is fitting noise. Match the foundation to the question, or accept that the foundation is a ghost.

Sources

• Link, G. — Spearman's Rank Correlation of Technical Indicators. S&P 500 daily data, March 1957 – December 2023. (Correlation matrix and velocity/acceleration analysis.)
• Bailey, D. H. & López de Prado, M. — How backtest overfitting in finance leads to false discoveries, Significance (Royal Statistical Society), 2021. (False Strategy Theorem; 48% forecaster accuracy; 8.3% fund outperformance.)
• Bailey, D. H. & López de Prado, M. — The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality, Journal of Portfolio Management, 2014. (Multiple-testing correction for backtested Sharpe ratios.)
• Bailey, D. H., Borwein, J., López de Prado, M. & Zhu, Q. J. — The Probability of Backtest Overfitting, Journal of Computational Finance, 20(4), 39–69, 2017. (PBO framework.)

Educational only. Not financial advice. Past performance does not guarantee future results.