blog-cover-image

Backtesting basics in Python (with code)

Backtesting is a powerful and essential technique in quantitative finance and algorithmic trading. It allows traders and analysts to simulate the performance of trading strategies using historical data before risking real capital in live markets. Python, with its robust data manipulation libraries and ease of use, has emerged as a popular language for implementing backtesting frameworks. In this article, we’ll dive deep into the basics of backtesting in Python, covering its utility, real-life applications, illustrative code examples, the mathematics behind it, and tips to help you intuitively understand how to build and evaluate your own trading strategies.

Backtesting Basics in Python (with Code)

What is Backtesting?

Backtesting refers to the process of testing a trading strategy or predictive model using historical market data. The main goal is to estimate how well a strategy would have performed in the past, which can provide insights into its potential future performance.

Why backtest? To validate trading ideas before real-money deployment.
How is it done? By simulating trades and portfolio evolution as if they were executed in the past.

Key Concepts in Backtesting

Strategy: The set of trading rules and logic you want to test.
Historical Data: Market data (prices, volumes, etc.) over a specific period.
Performance Metrics: Metrics such as returns, Sharpe ratio, and maximum drawdown to evaluate strategy performance.

Utility and Real-life Application of Backtesting

Backtesting is used by traders, portfolio managers, quant researchers, and data scientists to:

Evaluate the viability of trading strategies.
Optimize parameters (e.g., moving average lengths).
Compare different approaches objectively.
Identify potential risks or weaknesses in strategies.

Example: A trader designs a simple moving average crossover strategy to buy when a short-term moving average crosses above a long-term moving average. Before risking capital, the trader uses backtesting to simulate historical trades and analyze the resulting performance.

Intuitive Understanding: How Backtesting Works

Imagine you have a rule: "Buy Apple stock when its 20-day moving average crosses above the 50-day moving average." To know if this rule would have made money, you:

Collect Apple’s historical price data.
Apply your rule day by day, as if you were living in the past.
Track every buy and sell signal, updating your hypothetical portfolio along the way.
At the end, analyze the returns, risk, and other statistics.

This process is at the heart of backtesting. It helps you avoid costly mistakes and over-optimistic assumptions before trading in real markets.

Mathematics and Derivation Behind Backtesting

Portfolio Value Calculation

At each point in time, your portfolio value evolves according to your trades and the price movements of your assets. If you start with capital $ C_0 $, and after a series of trades, your holdings and cash position change, your portfolio value at time $ t $ is:

$$ V_t = \sum_{i=1}^{N} h_{i,t} \cdot P_{i,t} + \text{Cash}_t $$

$ V_t $: Portfolio value at time $ t $
$ h_{i,t} $: Number of shares of asset $ i $ held at time $ t $
$ P_{i,t} $: Price of asset $ i $ at time $ t $
$ \text{Cash}_t $: Remaining cash at time $ t $

Return Calculation

Simple returns for each period are calculated as:

$$ r_t = \frac{V_t - V_{t-1}}{V_{t-1}} $$

Cumulative return over $ T $ periods is:

$$ R = \prod_{t=1}^{T} (1 + r_t) - 1 $$

Risk and Performance Metrics

Sharpe Ratio:
$$ \text{Sharpe Ratio} = \frac{E[r_p - r_f]}{\sigma_p} $$ where $ r_p $ is the portfolio return, $ r_f $ is the risk-free rate, and $ \sigma_p $ is the standard deviation of portfolio returns.
Maximum Drawdown:
$$ \text{Max Drawdown} = \max_{t} \left( \frac{\text{Peak}_t - V_t}{\text{Peak}_t} \right) $$ where $ \text{Peak}_t $ is the running maximum of the portfolio value up to time $ t $.

Step-by-Step Backtesting in Python

1. Setting Up Your Environment

For backtesting in Python, popular libraries include Pandas (data manipulation), NumPy (numerical calculations), Matplotlib (visualization), and yfinance (to fetch historical data). Optionally, you can use Backtrader or zipline for more advanced backtesting.


pip install pandas numpy matplotlib yfinance

2. Fetching Historical Data


import yfinance as yf
import pandas as pd

# Download Apple's daily data for the last 5 years
data = yf.download('AAPL', start='2019-01-01', end='2024-01-01')
print(data.head())

3. Implementing a Simple Moving Average Crossover Strategy

We’ll use the classic moving average crossover strategy:

Buy when the 20-day SMA crosses above the 50-day SMA.
Sell when the 20-day SMA crosses below the 50-day SMA.


# Calculate moving averages
data['SMA20'] = data['Close'].rolling(window=20).mean()
data['SMA50'] = data['Close'].rolling(window=50).mean()

# Generate signals
data['Signal'] = 0
data['Signal'][20:] = \
    np.where(data['SMA20'][20:] > data['SMA50'][20:], 1, 0)

# Generate trading orders (1: Buy, 0: Sell/Hold)
data['Position'] = data['Signal'].diff()

print(data[['Close', 'SMA20', 'SMA50', 'Signal', 'Position']].tail(10))

4. Simulating Trades and Portfolio Value


initial_capital = 10000
shares = 0
cash = initial_capital
portfolio_values = []

for i in range(len(data)):
    if data['Position'].iloc[i] == 1:  # Buy
        shares = cash // data['Close'].iloc[i]
        cash -= shares * data['Close'].iloc[i]
    elif data['Position'].iloc[i] == -1:  # Sell
        cash += shares * data['Close'].iloc[i]
        shares = 0
    current_value = cash + shares * data['Close'].iloc[i]
    portfolio_values.append(current_value)

data['Portfolio Value'] = portfolio_values

5. Visualizing Results


import matplotlib.pyplot as plt

plt.figure(figsize=(14, 7))
plt.plot(data.index, data['Portfolio Value'], label='Portfolio Value')
plt.plot(data.index, data['Close'] * (initial_capital / data['Close'][0]), 
         label='Buy & Hold')
plt.legend()
plt.title('Backtest: Moving Average Crossover Strategy')
plt.show()

6. Evaluating Performance Metrics


returns = data['Portfolio Value'].pct_change().fillna(0)
cumulative_return = (data['Portfolio Value'][-1] / initial_capital) - 1
sharpe_ratio = returns.mean() / returns.std() * np.sqrt(252)  # 252 trading days/year

# Maximum Drawdown calculation
rolling_max = data['Portfolio Value'].cummax()
drawdown = (data['Portfolio Value'] - rolling_max) / rolling_max
max_drawdown = drawdown.min()

print(f'Cumulative Return: {cumulative_return:.2%}')
print(f'Sharpe Ratio: {sharpe_ratio:.2f}')
print(f'Maximum Drawdown: {max_drawdown:.2%}')

Real-Life Application Example: Backtesting a Momentum Strategy

Let’s backtest a simple momentum strategy:

Buy if the stock’s return over the past 6 months is positive.
Sell (or stay out) otherwise.


momentum_window = 126  # ~6 months (252 trading days/year)

data['Momentum'] = data['Close'].pct_change(momentum_window)
data['Signal'] = 0
data['Signal'][momentum_window:] = np.where(data['Momentum'][momentum_window:] > 0, 1, 0)
data['Position'] = data['Signal'].diff()

# Reuse trade simulation code from earlier
shares = 0
cash = initial_capital
portfolio_values = []

for i in range(len(data)):
    if data['Position'].iloc[i] == 1:
        shares = cash // data['Close'].iloc[i]
        cash -= shares * data['Close'].iloc[i]
    elif data['Position'].iloc[i] == -1:
        cash += shares * data['Close'].iloc[i]
        shares = 0
    current_value = cash + shares * data['Close'].iloc[i]
    portfolio_values.append(current_value)

data['Portfolio Value'] = portfolio_values

plt.figure(figsize=(14, 7))
plt.plot(data.index, data['Portfolio Value'], label='Momentum Portfolio')
plt.plot(data.index, data['Close'] * (initial_capital / data['Close'][0]), 
         label='Buy & Hold')
plt.legend()
plt.title('Backtest: Momentum Strategy')
plt.show()

Key Points and Common Pitfalls in Backtesting

Look-ahead bias: Never use data from the future when simulating trades for the past.
Survivorship bias: Make sure your dataset includes delisted assets if you want realistic results.
Overfitting: Don’t optimize your strategy to fit past data too closely, or it may fail out-of-sample.
Trading costs: Always account for slippage, commissions, and spreads.
Realistic execution: Trades should occur at realistic prices (e.g., next open or close, not the current tick).

Advanced Backtesting Tools in Python

For more robust, scalable backtesting, consider libraries like Backtrader, zipline, or QuantConnect.

Example: Backtesting with Backtrader


import backtrader as bt

class SmaCrossStrategy(bt.Strategy):
    params = (('fast', 20), ('slow', 50),)

    def __init__(self):
        self.sma_fast = bt.indicators.SimpleMovingAverage(
            self.datas[0], period=self.p.fast)
        self.sma_slow = bt.indicators.SimpleMovingAverage(
            self.datas[0], period=self.p.slow)

    def next(self):
        if not self.position:  # not in the market
            if self.sma_fast > self.sma_slow:
                self.buy()
        else:
            if self.sma_fast < self.sma_slow:
                self.sell()

cerebro = bt.Cerebro()
data = bt.feeds.YahooFinanceData(dataname='AAPL',
                                 fromdate=pd.Timestamp('2019-01-01'),
                                 todate=pd.Timestamp('2024-01-01'))
cerebro.adddata(data)
cerebro.addstrategy(SmaCrossStrategy)
cerebro.broker.set_cash(10000)
cerebro.run()
cerebro.plot()

Summary Table: Comparison of Strategies

Strategy	Cumulative Return	Sharpe Ratio	Max Drawdown
Buy & Hold	Varies (e.g., 120%)	Varies (e.g., 1.1)	Varies (e.g., -35%)
MA Crossover	Varies (e.g., 85%)	Varies (e.g., 0.9)	Varies (e.g., -22%)
Momentum	Varies (e.g., 140%)	Varies (e.g., 1.3)	Varies (e.g., -19%)

Note: Actual performance will vary depending on backtest period and data.

Conclusion: Getting Started with Your Own Backtests

Backtesting is a crucial skill for anyone interested in quantitative finance, trading, or investment analysis. By simulating strategies using Python and real market data, you can gain confidence in your ideas, avoid costly mistakes, and improve your trading edge. Remember to be wary of common pitfalls, always use realistic assumptions, and validate your strategies out-of-sample before risking real capital.

With the code and principles outlined above, you’re now well-equipped to start building and evaluating your own backtesting experiments in Python!

Frequently Asked Questions

Is backtesting 100% reliable?

No. While backtesting can help you identify promising strategies, past performance does not guarantee future results. Always validate results with out-of-ssample and live trading.

What is the difference between backtesting and paper trading?

Backtesting involves simulating strategies on historical data, often run in seconds or minutes. Paper trading (or forward testing) means running your strategy in real-time on live data without risking real money, allowing you to see how it performs under current market conditions and with realistic execution delays and slippage.

How much data do I need for reliable backtesting?

Generally, more data is better. At a minimum, you should backtest over several complete market cycles (bull and bear markets) to capture a range of market conditions. For daily strategies, 5-10 years of data is often a good starting point, but this can vary depending on your goals.

How can I avoid overfitting my strategy?

Test on out-of-sample data (data your strategy hasn’t seen before).
Keep your strategy rules simple and based on sound logic, not just data mining.
Avoid excessive parameter optimization or curve-fitting to the past.
Use walk-forward analysis and cross-validation techniques.

What are some good open-source backtesting frameworks?

Backtrader: Very popular, flexible, and feature-rich.
zipline: Used by Quantopian, good for equities and factor strategies.
PyAlgoTrade: Lightweight and easy to use.
bt: Focused on portfolio optimization and allocation strategies.
QuantConnect (Lean): Professional-grade, supports many asset classes (in C# and Python).

Appendix: Complete Python Backtesting Script


import yfinance as yf
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Download historical data
ticker = 'AAPL'
data = yf.download(ticker, start='2019-01-01', end='2024-01-01')

# Calculate moving averages
data['SMA20'] = data['Close'].rolling(window=20).mean()
data['SMA50'] = data['Close'].rolling(window=50).mean()

# Generate signals
data['Signal'] = 0
data['Signal'][20:] = np.where(data['SMA20'][20:] > data['SMA50'][20:], 1, 0)
data['Position'] = data['Signal'].diff()

# Backtest
initial_capital = 10000
shares = 0
cash = initial_capital
portfolio_values = []

for i in range(len(data)):
    if data['Position'].iloc[i] == 1:  # Buy
        shares = cash // data['Close'].iloc[i]
        cash -= shares * data['Close'].iloc[i]
    elif data['Position'].iloc[i] == -1:  # Sell
        cash += shares * data['Close'].iloc[i]
        shares = 0
    current_value = cash + shares * data['Close'].iloc[i]
    portfolio_values.append(current_value)

data['Portfolio Value'] = portfolio_values

# Performance metrics
returns = data['Portfolio Value'].pct_change().fillna(0)
cumulative_return = (data['Portfolio Value'].iloc[-1] / initial_capital) - 1
sharpe_ratio = returns.mean() / returns.std() * np.sqrt(252)
rolling_max = data['Portfolio Value'].cummax()
drawdown = (data['Portfolio Value'] - rolling_max) / rolling_max
max_drawdown = drawdown.min()

# Output
print(f'Cumulative Return: {cumulative_return:.2%}')
print(f'Sharpe Ratio: {sharpe_ratio:.2f}')
print(f'Max Drawdown: {max_drawdown:.2%}')

plt.figure(figsize=(14, 7))
plt.plot(data.index, data['Portfolio Value'], label='Portfolio Value')
plt.plot(data.index, data['Close'] * (initial_capital / data['Close'][0]), label='Buy & Hold')
plt.legend()
plt.title('Backtest: Moving Average Crossover Strategy')
plt.show()

Conclusion

Backtesting is an indispensable tool in quantitative trading and investing. Python makes it accessible to everyone with its powerful libraries and active community. By understanding the mathematical foundations, practical coding techniques, and common pitfalls, you can confidently test, evaluate, and improve your own trading strategies. Remember, backtesting is not about finding a “perfect” system, but about developing a disciplined, data-driven approach to the markets.

Start simple, iterate, and always validate your results with out-of-sample data and in real-time. Happy coding and successful trading!

Python
Quant