If you like Skforecast , please give us a star on GitHub! ⭐️
Bitcoin price prediction with Python, when the past does not repeat itself
Introduction¶
A time series is a succession of chronologically ordered data spaced at equal or unequal intervals. The forecasting process consists of predicting the future value of a time series, either by modeling the series solely based on its past behavior (autoregressive) or by using other external variables.
When creating a forecaster model, historical data are used to get a mathematical representation capable of predicting future values. This idea is based on a very important assumption: the future behavior of a phenomenon can be explained from its past behavior. However, this rarely happens in reality, or at least not in its entirety. For more on this, see the following definition:
$Forecast = pattern + unexplained\;variance$
The first term of the equation refers to everything that has a repetitive character over time (trend, seasonality, cyclical factors...). The second term represents everything that influences the response variable but is not captured (explained) by the past of the time series.
The greater the importance of the first term relative to the second, the greater the probability of success when attempting to create autoregressive forecasting models. As the second term gains weight, it becomes necessary to incorporate additional variables (if any) into the model to help explain the observed behavior.
A good study of the modeled phenomenon and the ability to recognize to what extent its behavior can be explained by its past can save a lot of unnecessary effort.
This document shows an example of how to identify situations where the autoregressive forecasting process cannot achieve useful results. As an example, an attempt to predict the daily closing price of Bitcoin using machine learning methods is made. For this purpose, Skforecast is used, a simple Python library that allows, among other things, to adapt any Scikit-learn regressor to forecasting problems.
Use Case¶
Bitcoin (₿) is a decentralized cryptocurrency that can be sent from one user to another through the bitcoin peer-to-peer network without intermediaries. Transactions are verified and recorded in a public distributed ledger called Blockchain. Bitcoins are created as a reward for a process known as mining and can be exchanged for other currencies, products, and services.
Although there may be different opinions about Bitcoin, whether as a high-risk speculative asset or, on the other hand, as a store of value, it is undeniable that it has become one of the most valuable financial assets globally. The website Infinite Market Cap shows a list of all financial assets ranked by market capitalization. Bitcoin, at the time of writing, is in the top 10. It is close to world-renowned companies such as Tesla or globally accepted safe-haven assets such as silver. The growing interest in Bitcoin, and the world of cryptocurrencies, makes it an interesting phenomenon to model.
The aim is to generate a forecasting model capable of predicting the price of Bitcoin. A time series is available with the opening (Open), closing (Close), maximum (High), and minimum (Low) prices of Bitcoin in US dollars (USD) from 2013-04-28 to 2022-01-01.
# Data manipulation
# ==============================================================================
import pandas as pd
import numpy as np
import datetime
from cryptocmd import CmcScraper
# Plots
# ==============================================================================
import matplotlib.pyplot as plt
%matplotlib inline
import plotly.graph_objects as go
from plotly.subplots import make_subplots
import plotly.express as px
import seaborn as sns
from statsmodels.graphics.tsaplots import plot_acf
from statsmodels.graphics.tsaplots import plot_pacf
plt.style.use('ggplot')
# Bitcoin colors
# ==============================================================================
palette_btc = {'orange': '#f7931a',
'white' : '#ffffff',
'gray' : '#4d4d4d',
'blue' : '#0d579b',
'green' : '#329239'
}
# Modelling and Forecasting
# ==============================================================================
from skforecast.ForecasterAutoreg import ForecasterAutoreg
from skforecast.model_selection import backtesting_forecaster
from lightgbm import LGBMRegressor
from sklearn.metrics import mean_absolute_error
Data¶
The data download is performed using cryptocmd. This library is useful for downloading historical cryptocurrency data from the Coinmarketcap website. The information in each column is:
Date: date of the record.Open: the opening price, the price at which an asset, in this case, Bitcoin, trades at the beginning of the day. (USD).High: the maximum price of the day, the highest price reached by Bitcoin on that day, (USD).Low: the minimum price of the day, the lowest price reached by the Bitcoin on that day, (USD).Close: the closing price, the price at which Bitcoin trades at the end of the day, (USD).Volume: the sum of actual trades made during the day, (USD).Market Cap: market capitalization, the total value of all shares of a company or, in the case of Bitcoin or another cryptocurrency, of all coins in circulation, (USD).
Note: the cryptocurrency market is uninterrupted. It operates 24 hours a day, 7 days a week. However, it is not strictly necessary that the close price coincides with the open price of the next day because of the fluctuations that the value of Bitcoin, or any cryptocurrency, may undergo during the last second of the day.
# Data download
# ==============================================================================
# Scraper is initialized, symbol, start and end of download are included
scraper = CmcScraper('BTC', '28-04-2013', '01-01-2022')
# Transform collected data into a dataframe
data = scraper.get_dataframe()
data.sort_values(by='Date', ascending=True, inplace=True)
pd.set_option('display.max_columns', None)
display(data)
pd.reset_option('display.max_columns')
# Data preparation
# ==============================================================================
data['date'] = pd.to_datetime(data['Date'], format='%Y-%m-%d %H:%M:%S')
data = data.loc[:, ['date', 'Open', 'Close', 'High', 'Low']]
data = data.rename({'Open': 'open', 'Close': 'close', 'High': 'high', 'Low': 'low'},
axis=1)
data = data.set_index('date')
data = data.asfreq('D')
data = data.sort_index()
When setting a frequency with the asfreq() method, Pandas fills the gaps that may exist in the time series with the value of Null to ensure the indicated frequency. Therefore, it should be checked if missing values have appeared after this transformation.
print(f'Number of rows with missing values: {data.isnull().any(axis=1).mean()}')
Bitcoin halving as an exogenous variable¶
Halving is a programmed event and is part of the design and operation of some cryptocurrencies. The miners validate the transaction blocks of the network, in this case, Bitcoin, and each time they succeed, they receive an amount of that digital currency. This amount varies from time to time.
Every time 210,000 blocks are added, the reward in the Bitcoin blockchain change occurs. This event, called halving, occurs approximately every 4 years and reduces the coins miners receive as a reward by half.
In the history of Bitcoin, there have been 3 halvings. When Bitcoin mining started, miners received 50 BTC for successfully mining a block. In 2012, it reduced this reward to 25 BTC; in 2016, it dropped to 12.5 BTC, and in 2020 to 6.25 BTC, after the third halving. Each halving has affected the price, although it has not been in the short term after it.
It is intended to use the days remaining until the next halving and its mining rewards as exogenous variables to predict the price of Bitcoin. The next halving is estimated to occur approximately in 2024, although its exact date is unknown. The remaining blocks as of 2022-01-14 from the Coinmarketcap website, 121,400, and the average number of Bitcoin network blocks mined per day, 144 (average block time ≈ 10 minutes) are used to determine it.
Note: when incorporating predicted data as an exogenous variable, their error is introduced in the forecasting model since they are predictions.
# Dict with Bitcoin halvings info
# ==============================================================================
btc_halving = {'halving' : [0, 1 , 2, 3, 4],
'date' : ['2009-01-03', '2012-11-28',
'2016-07-09', '2020-05-11', np.nan],
'reward' : [50, 25, 12.5, 6.25, 3.125],
'halving_block_number' : [0, 210000, 420000 ,630000, 840000]
}
# Next halving calculation
# The remaining blocks according to the coinmarketcap.com website for
# the next halving as of 2022-01-14 are taken as a starting point
# ==============================================================================
remaining_blocks = 121400
blocks_per_day = 144
days = remaining_blocks / blocks_per_day
next_halving = pd.to_datetime('2022-01-14', format='%Y-%m-%d') + datetime.timedelta(days=days)
next_halving = next_halving.replace(microsecond=0, second=0, minute=0, hour=0)
next_halving = next_halving.strftime('%Y-%m-%d')
btc_halving['date'][-1] = next_halving
print(f'The next halving will occur on approximately: {next_halving}')
# Include rewards and countdown to next halving in dataset
# ==============================================================================
data['reward'] = np.nan
data['countdown_halving'] = np.nan
for i in range(len(btc_halving['halving'])-1):
# Start and end date of each halving
if btc_halving['date'][i] < data.index.min().strftime('%Y-%m-%d'):
start_date = data.index.min().strftime('%Y-%m-%d')
else:
start_date = btc_halving['date'][i]
end_date = btc_halving['date'][i+1]
mask = (data.index >= start_date) & (data.index < end_date)
# Fill column 'reward' with mining rewards
data.loc[mask, 'reward'] = btc_halving['reward'][i]
# Fill column 'countdown_halving' with remaining days
time_to_next_halving = pd.to_datetime(end_date) - pd.to_datetime(start_date)
data.loc[mask, 'countdown_halving'] = np.arange(time_to_next_halving.days)[::-1][:mask.sum()]
# Check that the data have been created correctly
# ==============================================================================
print('Second halving:', btc_halving['date'][2])
display(data.loc['2016-07-08':'2016-07-09'])
print('')
print('Third halving:', btc_halving['date'][3])
display(data.loc['2020-05-10':'2020-05-11'])
print('')
print('Next halving:', btc_halving['date'][4])
data.tail(2)
Graphic exploration¶
When it is necessary to generate a forecasting model, plotting the time series values could be useful. This allows identifying patterns such as trends and seasonality.
Candlestick chart¶
A candlestick chart is a style of financial chart used to describe price movements of a security, derivative, or currency. The thick body shows the variation between the opening and closing price for a period, while the shadows show the minimum and maximum values reached during that period.
# Interactive candlestick chart with Plotly
# ==============================================================================
candlestick = go.Candlestick(
x = data.index,
open = data.open,
close = data.close,
low = data.low,
high = data.high,
)
fig = go.Figure(data=[candlestick])
fig.update_layout(
width = 900,
height = 450,
title = dict(text='<b>Bitcoin/USD Chart</b>', font=dict(size=30)),
yaxis_title = dict(text='Price (USD)', font=dict(size=15)),
margin = dict(l=10, r=20, t=80, b=20),
shapes = [dict(x0=btc_halving['date'][2], x1=btc_halving['date'][2],
y0=0, y1=1, xref='x', yref='paper', line_width=2),
dict(x0=btc_halving['date'][3], x1=btc_halving['date'][3],
y0=0, y1=1, xref='x', yref='paper', line_width=2),
dict(x0=btc_halving['date'][4], x1=btc_halving['date'][4],
y0=0, y1=1, xref='x', yref='paper', line_width=2)
],
annotations = [dict(x=btc_halving['date'][2], y=1, xref='x', yref='paper',
showarrow=False, xanchor='left', text='Second halving'),
dict(x=btc_halving['date'][3], y=1, xref='x', yref='paper',
showarrow=False, xanchor='left', text='Third halving'),
dict(x=btc_halving['date'][4], y=1, xref='x', yref='paper',
showarrow=False, xanchor='left', text='Fourth halving')
],
xaxis_rangeslider_visible = False,
)
fig.show()