My solution consists of 5 main models:
- FileWatcher - this module monitors changes in the target folder and adds new files to the processing queue.
- FileProcessQueue - this queue is necessary for synchronizing and distributing files between loaders (you can add multiple file loaders).
- DataLoader - the module receives information about the new file from the queue, verifies it and sends it to reader for processing.
- DataReader - the module reads and parses information from the file, determines the type of a object and sends it to a database.
- DatabaseContext - creates class objects and saves them to the database every 10 seconds (if there are new objects).