💹 justETF Scraping

Scrape the justETF.

🛠️ Installation

To use justETF scraping package in your project, install the actual version from GitHub:

pip install git+https://github.com/druzsan/justetf-scraping.git

If you are going to play notebooks through, use the following installation:

pip install justetf-scraping[all]@git+https://github.com/druzsan/justetf-scraping.git

🚀 Usage

📋 Scrape the justETF Screener

Load overviews for all available (over 3300 at the moment) ETFs (requires a request for all ETF type: long-only, active, short & leveraged):

import justetf_scraping

df = justetf_scraping.load_overview()
df

	wkn	ticker	valor	name	inception_date	age_in_days	age_in_years	strategy	domicile_country	currency	hedged	securities_lending	dividends	ter	replication	size	is_sustainable	number_of_holdings	yesterday	last_week	last_month	last_three_months	last_six_months	last_year	last_three_years	last_five_years	2023	2022	2021	2020	last_dividends	last_year_dividends	last_year_volatility	last_three_years_volatility	last_five_years_volatility	last_year_return_per_risk	last_three_years_return_per_risk	last_five_years_return_per_risk	max_drawdown	last_year_max_drawdown	last_three_years_max_drawdown	last_five_years_max_drawdown	asset_class	instrument	region	exchange
isin
IE00B5BMR087	A0YEDG	SXR8	10737041	iShares Core S&P 500 UCITS ETF (Acc)	2010-05-19	5198	14.241096	Long-only	Ireland	USD	False	True	Accumulating	0.07	Full replication	78449	False	503	13.67	-1.20	-5.42	0.61	5.04	21.13	34.36	99.91	21.54	-13.30	39.07	8.04	NaN	NaN	12.86	17.99	21.63	1.64	0.57	0.69	-33.71	-9.09	-17.11	-33.71	Equity	ETF		gettex, XETRA, London, Stuttgart, SIX Swiss Exchange, Borsa Italiana, Euronext Amsterdam
IE00B4L5Y983	A0RPWH	EUNL	10608388	iShares Core MSCI World UCITS ETF USD (Acc)	2009-09-25	5434	14.887671	Long-only	Ireland	USD	False	True	Accumulating	0.20	Optimized sampling	67967	False	1429	10.85	-1.06	-4.81	-0.59	4.28	17.87	25.66	79.25	19.55	-12.96	32.10	6.13	NaN	NaN	11.17	15.25	18.27	1.59	0.52	0.68	-33.91	-8.75	-16.88	-33.91	Equity	ETF	World	gettex, XETRA, London, Stuttgart, SIX Swiss Exchange, Borsa Italiana, Euronext Amsterdam
IE00B3XXRP09	A1JX53	VUSA	18575508	Vanguard S&P 500 UCITS ETF	2012-05-22	4464	12.230137	Long-only	Ireland	USD	False	False	Distributing	0.07	Full replication	35981	False	498	13.63	-1.23	-5.45	0.57	5.01	21.08	34.31	99.85	21.54	-13.29	39.08	8.05	1.11	1.33	12.89	18.13	21.70	1.63	0.57	0.68	-33.70	-9.09	-17.28	-33.70	Equity	ETF		gettex, XETRA, London, Euronext Paris, Stuttgart, SIX Swiss Exchange, Borsa Italiana, Euronext Amsterdam

Further enrich the data with additional information (asset class, region, exchanges and instrument, it requires further requests):

df = justetf_scraping.load_overview(enrich=True)

Load long-only ETFs (only requires one single request):

df = justetf_scraping.load_overview(strategy="epg-longOnly")

Load MSCI World ETFs:

df = justetf_scraping.load_overview(strategy="epg-longOnly", index="MSCI World")

📈 Scrape ETF Chart Data from justETF (e.g.)

Load the whole history of a chosen ETF by its ISIN:

df = justetf_scraping.load_chart("IE00B0M62Q58")
df

	quote	relative	dividends	cumulative_dividends	quote_with_dividends	relative_with_dividends	reinvested_dividends	quote_with_reinvested_dividends	relative_with_reinvested_dividends
date
2005-10-28	20.60	0.000000	0.0	0.0	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-29	20.60	0.000000	0.0	0.0	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-30	20.60	0.000000	0.0	0.0	20.60	0.000000	0.000000	20.600000	0.000000
2005-10-31	20.99	1.893204	0.0	0.0	20.99	1.893204	0.000000	20.990000	1.893204
2005-11-01	21.03	2.087379	0.0	0.0	21.03	2.087379	0.000000	21.030000	2.087379
...	...	...	...	...	...	...	...	...	...
2024-08-05	64.90	215.048544	0.0	9.9	74.80	263.106796	20.643909	85.543909	315.261692
2024-08-06	65.95	220.145631	0.0	9.9	75.85	268.203883	20.977901	86.927901	321.980101
2024-08-07	65.74	219.126214	0.0	9.9	75.64	267.184466	20.911102	86.651102	320.636419
2024-08-08	66.80	224.271845	0.0	9.9	76.70	272.330097	21.248276	88.048276	327.418814
2024-08-09	67.05	225.485437	0.0	9.9	76.95	273.543689	21.327798	88.377798	329.018435

6861 rows × 9 columns

Compare

df = justetf_scraping.compare_charts(
    {
        "IE00B0M62Q58": justetf_scraping.load_chart("IE00B0M62Q58"),
        "IE00B0M63177": justetf_scraping.load_chart("IE00B0M63177"),
    },
    input_value="quote_with_dividends"
)
df

	IE00B0M62Q58	IE00B0M63177
date
2005-11-18	0.000000	0.000000
2005-11-19	0.000000	0.000000
2005-11-20	0.000000	0.000000
2005-11-21	-0.539811	-0.887436
2005-11-22	0.629780	-0.934143
...	...	...
2024-08-05	236.482231	120.784680
2024-08-06	241.205578	123.960766
2024-08-07	240.260909	127.136852
2024-08-08	245.029240	127.697338
2024-08-09	246.153846	128.304531

6840 rows × 2 columns

For further exploration examples, see Jupyter Notebooks

⚒️ Development Setup

To setup locally cloned project, first install Poetry:

pip install poetry

In the local project folder, install all dependencies and extras:

poetry install --all-extras

Activate local Poetry environment:

poetry shell

Setup pre-commit hooks:

pre-commit install

Similar Projects

• https://github.com/AshNL/justETF-overview-scraper
• https://github.com/SimonMandlik/etf_filter

Thanks

This project was inspired by this Stack Overflow question.