The Data Collector is a Python tool that gathers and processes stock or asset data from various providers.
Using Selenium, PyMongo, and Pandas, it stores cleaned and structured data in MongoDB for use by the backend and frontend.
See frontend → | See backend →
🚀 Want to contribute?
We welcome collaborators who wish to contribute and help enhance this trading tool. Feel free to reach out to the maintainers to get involved.
Trader Charts Data Collector is based on Selenium, and use Panda and MongoDB database.
-
Set environment variables on .env.local file
-
Create virtual environemnt
$ python -m venv .venv -
Activate virtual environemnt .venv
$ source .venv/bin/activate -
Install Python modules
$ python -m pip install -e . -
Available main modules
# Historical data $ python -m mains.main_collect_historical_data # RSS feeds $ python -m mains.main_collect_rss_feeds # Train sentumental analysis model for financial RSS feeds $ python -m mains.main_finetune_sentiment_model # Process sentiment analyis RSS feeds $ python -m mains.main_analyze_sentiment_model_rss_feeds # Process topic RSS feeds $ python -m mains.main_analyze_topic_model_rss_feeds -
Deactivate virtual environemnt .venv
$ deactivate
To ensure best practices and maintain code quality, you can run the following scripts:
-
Check for issues
$ ./scripts/check_code.sh -
Fix issues
$ ./scripts/fix-code.sh
Create .env.development.local file, and include all the following:
MONGO_URI=[string]
MONGO_DB_NAME=[string]
MONGO_COLLECTION=[string]