Yahoo ticker downloader
Produces .csv, .xlsx, .json and .yaml files (All files contain same data but in a different format) for stocks, ETF, futures, indexes, mutual funds, currency, warrants and bonds. The ticker symbol, company name and exchange are saved for all symbols.
It gets its data from https://finance.yahoo.com/lookup/. Please note: it is not possible to get all the symbols due to limitations set by Yahoo.
Python 2.7 or Python 3.5+
From python package manager (preferred):
pip install Yahoo-ticker-downloader
python setup.py install
usage: YahooTickerDownloader.py [-h] [-i] [-e] [-E EXCHANGE] [-s SLEEP] [-p] [type] positional arguments: type The type to download, this can be: generic optional arguments: -h, --help show this help message and exit -i, --insecure use HTTP instead of HTTPS -e, --export export immediately without downloading (Only useful if you already downloaded something to the .pickle file) -E EXCHANGE, --Exchange EXCHANGE Only export ticker symbols from this exchange (the filtering is done during the export phase) -s SLEEP, --sleep SLEEP The time to sleep in seconds between requests -p, --pandantic Stop and warn the user if some rare assertion fails
For example to download all stock symbols you run it like:
The program takes a few weeks before it is finished. The program supports suspending and resuming a download. Press CTRL+C to suspend download. Restart the program in the same working directory to resume downloading. It is possible to export partially downloaded results using the -e flag.
Example of CSV output:
Ticker Name Exchange exchangeDisplay Type TypeDisplay JNUG Direxion Daily Jr Gld Mnrs Bull 3X ETF ASE NYSE MKT E ETF DWDP DowDuPont Inc. NYQ NYSE S Equity E Eni S.p.A. NYQ NYSE S Equity EQH AXA Equitable Holdings, Inc. NYS NYSE S Equity XOM Exxon Mobil Corporation NYQ NYSE S Equity ETP Energy Transfer Partners, L.P. NYQ NYSE S Equity ES=F E-mini S&P 500 Index Futures,Jufuture CME Chicago Mercantile Exchange F Futures NQ=F E-mini Nasdaq 100 Index Futuresfuture CME Chicago Mercantile Exchange F Futures GE=F Eurodollar Futures,Sep-2018future CME Chicago Mercantile Exchange F Futures ...ect
Depending on the type you are downloading, you will get between 3.000 and 100.000+ entries.
Download history for symbols: ystockquote
- Version 3.0.1 (2018-12-01)
- Removed reppy dependency
- Version 3.0.0 (2018-05-27)
- Switched over to different JSON api (searchassist)
- Version 2.2.0 (2018-01-31)
- Continue exporting to different formats if one export fails ( #41 )
- Check robots.txt ( anti-feature )
- Version 2.1.1 (2017-08-02)
- A too old requests dependency was listed ( #35 )
- Version 2.1.0 (2017-05-10)
- Added market parameter ( pull request #33 )
- Version 2.0.1 (2017-05-07)
- Fixed issue where all downloads except stock and currency stopped working.
- Version 2.0.0 (2017-05-05)
- Switched over to JSON api
- Version 1.0.0 (2017-04-04)
- Reverted some changes from 0.10.0. Bond is back. Reverted back to English site instead of German.
- Resolved CSV issue again. Closes #23 and #16.
- Merged #26 Workaround Y! b>2000 limit
- Scraper now scrapes a lot more at the expense of runtime.
- Support for python2 is back. Latest python 2 & 3 are supported.
- Removed xls support
- Added xlsx support (#29)
- Version 0.10.1 (2017-02-04)
- More descriptive help message
- Version 0.10.0 (2017-02-02)
- Removed bond downloading option.
- Uses different yahoo source. Fixes #18
- Removed python2 from classifiers. Related to #16
- Version 0.9.0 (unreleased)
- Added a flag to restrict output to specific stock exchanges.
- Version 0.8.1 (2016-08-17)
- Workaround for #7 : downloading interruption
- Solution for #9 : UnicodeEncodeError
- Version 0.7.0 (2016-03-20)
- Added --export option. It will transcode the .pickle file immediately to the desired output formats.
- Version 0.6.0 (unreleased)
- Add 3 retries with an exponential back-off if HTTPError or ChunkedEncodingError is raised when processing _fetchHtml.
- Version 0.5.0 (2015-08-16)
- Allows downloading using a insecure connection.
- The temporarily download file-names now include the ticker type.
- Version 0.4.0 (2014-10-28)
- Warrant symbols can now be downloaded.
- Bond symbols can now be downloaded.
- Version 0.3.0 (2014-08-14)
- Use HTTPS instead of HTTP
- Retry to fetch a page if it contains no symbols (A "fix" for issue #4)
- Renamed all 'Curreny' to 'Currency'
- Relative imports are used
- Fix: .csv file it outputs is encoded in UTF-8 when using python2
- Performance: Considerable reduced memory consumption
- It now outputs .json, .yaml and .xls files in addition to .csv