Harvest data from online OJS instances at fixed interval

[nuest]

• have a small set of HTML files for testing of harvesting based on HTML tags @nuest
  • so far only one: Kirubamoh/optimetaPortal@e0d1d65 (#3)
• have an example OAI PMH record list as a file @nuest
• list of OJS servers (more precisely: the link to the OAI API) in database with a unique ID, URL, harvesting interval in days
  • e.g., <automatic unique ID>, https://service.tib.eu/optimeta/index.php/optimeta/oai/, 7
• when run, a list of harvesting tasks is scheduled - this is all handled by Django Q plugin!
  • https://mattsegal.dev/simple-scheduled-tasks.html (with Django ORM, https://django-q.readthedocs.io/en/latest/brokers.html)
  • https://stackoverflow.com/questions/62525295/how-to-use-python-to-schedule-tasks-in-a-django-application
  • the harvesting job is stored in the database with timestamps and the user who started the harvesting is stored
  • the harvested records have a field job to connect them with the harvesting
  • the backend has a log of the harvesting events > need a database table for harvesting events
• multiple executions don't create duplicates (test case!)
• have test cases
• use mocking to test the URLs from the XML to the HTML

[nuest]

Use demo server: https://service.tib.eu/optimeta/index.php/optimeta

[nuest]

Next step: Implement a unit test that tests the parsing funktion