Installation: To install you must have nodejs installed. Project has been created using node v22. Remember to copy the .env.dist file into the .env file and modify the environment variables there to match your setup
npm install
cp .env.dist .env
npx playwright install
npm run build
This repo uses docker for containers and for testing purposes. You must have docker installed. Then
docker compose up
Will launch the development environment you need in order for the framework to work properly
npm start
With all of the services defined in docker compose up and running this dev environment should work for testing. The project only relies on an internet connection (for scrapping) and a mongo database, so if you don't have Docker installed, please install it because it's about time, but you can also run a mongo db in your own host or using mongo db atlas.
Make sure that if you run this for production, you change the .env file to fit your needs. env.dist should connect you to a default setting but this is for testing and dev purposes only
docker compose -f ./full.docker-compose.yml up -d
first call the endpoint via /crawl to crawl the news then /feed to read the news /feed/<elmundo | elpais> to get each of the news from each of the newspapers
Integration tests and e2e tests require development environment.
npm test //Launches all tests. requires development environment
npm test:e2e //e2e tests. launches dev environment automatically
npm test:unit
npm test:int //Integration tests. launches dev environment automatically
You can run tests individually with dev mode so that they restart every time you change a file.
npm dev:test:e2e
npm dev:test:int
Configuration file is loaded from .env file and available via process.env and via app.config for all the modules to reuse. Modules are to be stored in the modules folder, and follow the base.module.ts interface for them to be loaded correctly. Modules (and services) must have an init() async function that is used to connect asynchronusly to any 3rd party data mechanism before the application is spin up.