Can you rely on chart patterns? Biases, limits and an honest way to test them

Articles on candles, patterns or indicators almost always end with the same question: does it work? The honest answer is more nuanced than a yes or a no, and it requires understanding why our perception of charts is misleading. This article presents the main interpretation biases, the common errors when testing a rule on past data and the criteria for judging a technical signal more seriously. It does not say technical analysis is useless: it says how to read it without fooling yourself.

Perception biases

Hindsight bias. On a chart already drawn, the "obvious" patterns became so because we know what follows. In real time, the same shape looks ambiguous.
Confirmation bias. We notice the cases where the pattern worked and forget those where it failed or never completed.
Selection bias. The examples shown in books or on social networks are chosen because they illustrate the pattern well, not because they are representative.
The subjectivity of drawing. A "neckline" or a "support" is placed differently depending on the analyst.

The hit rate is not enough

A rule that "works 60% of the time" can be a losing one, and a rule that works only 35% of the time can be a winning one. What matters is the expectancy of gain per trade:

Expectancy = (hit rate × average gain) − (failure rate × average loss)

Fictitious example, before costs. A rule wins in 60% of cases, with an average gain of €1 per share, and loses in 40% of cases, with an average loss of €2. Expectancy = 0.60 × 1 − 0.40 × 2 = −€0.20 per share: despite 60% successes, the rule is a losing one. Another rule wins in 35% of cases for an average gain of €3, and loses in 65% of cases for an average loss of €1: expectancy = 0.35 × 3 − 0.65 × 1 = +€0.40. In practice, these parameters are themselves estimated with uncertainty.

The relationship between average gain and average loss is detailed in risk management.

The pitfalls of testing on past data

Testing a rule on historical data (a backtest) is useful, but several errors are common:

Overfitting. By adjusting parameters (an indicator's period, a threshold, a holding period) until a good past result is obtained, you end up describing noise rather than lasting behaviour. The more parameters a rule has and the more it is tuned, the less its past result says about the future.
Forgetting costs and slippage. A rule that makes many trades can see its gain disappear once brokerage fees and execution gaps are counted. See the impact of brokerage fees.
Information from the future. Using, even inadvertently, a price or data point that was not known at the time of the decision (for example deciding from the close of a session, then assuming execution at the open price of that same session) distorts the result.
Survivorship bias in the data. Testing only on stocks that still exist today excludes those that have disappeared.
A sample that is too small. 12 trades say almost nothing: a favourable result can be due to chance.
The absence of an out-of-sample test. A rule is tuned on one period, then checked on another period that was not used to tune it.

How to judge a signal more seriously

Define the rule before looking at the results, with few parameters.
Test on several stocks and several periods, including unfavourable ones.
Count costs, slippage and the execution timing, not just prices.
Look at the expectancy and the worst case (the maximum drawdown suffered, that is, the largest historical decline of the simulated result), not only the total gain.
Keep a verification period that was not used to build the rule.
Be wary of a result that looks too good. A very flattering backtest is often a sign of overfitting or an error.

What a backtest does not say

Even when well run, a backtest describes what could have happened on past data. A different market regime, a change in liquidity or in costs can make a rule ineffective. Past performance, simulated or not, is not a reliable indicator of future performance.

In Earnnest, the backtest applies fees and an execution timing to simulations and presents its results as simulated (simulated equity curve), not as performance actually achieved.

Key takeaways

Our reading of charts is prone to hindsight, confirmation and survivorship biases.
The hit rate is not enough: it is the expectancy of gain that counts.
A backtest can be distorted by overfitting, forgotten costs, a sample that is too small or data from the future.
A good test keeps a verification period, counts costs and distrusts results that look too good.

This article is for information and education only and does not constitute personalised investment advice. The figures in the examples are fictitious and illustrative. Investing involves a risk of capital loss. Past performance, achieved or simulated, is not a reliable indicator of future performance.