New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create API to submit data into #149

ifokkema opened this Issue Nov 11, 2016 · 0 comments


1 participant

ifokkema commented Nov 11, 2016

Creating an API to which data can be sent, allowing automated data import into LOVD3.

  • Design authorization schema
  • Design format (JSON)
  • Implement first step of API
    • Allow users to create and renew authorization token in LOVD3
      • Design token
      • Adapt data model to store token
      • Adapt VE to shown token (hidden, show on request) and provide links
      • When showing token, explain features
      • When creating token while one already exists, warn new token will revoke access of current token
      • Implement creating a token, then forward to showing the token
      • Allow deletion of token, removing all authorization of APIs
      • Let all actions be logged
    • Allow setting data path in config.ini.php file
      • Let API fail when the data path is not configured
    • Implement endpoint to receive data
      • Do we need versioning in the URL?
      • Implement HTTP status codes:
        • HTTP 202 Accepted (accepted for later processing)
        • HTTP 400 Bad Request (malformed syntax)
        • HTTP 401 Unauthorized
        • HTTP 405 Method Not Allowed (when using GET)
        • HTTP 406 Not Acceptable (when response other than plain text or JSON is required)
        • HTTP 413 Payload Too Large (file too large)
        • HTTP 415 Unsupported Media Type (format not supported)
        • HTTP 422 Unprocessable Entity (format OK, syntax OK, semantics wrong)
        • HTTP 500 (other program errors)
        • HTTP 501 Not Implemented (when not using GET or POST)
      • Parse file, generalize it
      • Check authorization
      • Verify minimal data is present
        • Properly check detection_technique field as well, to make sure we don't fail there during actual import.
      • Allow for more transcripts in the file than present in the database; at least one should match, otherwise, throw error
      • Convert the JSON to an LOVD data object
        • We need to calculate all positions for this, and the variant type (#58)
      • Write contents to LOVD import file to data path
      • Write the original JSON file to a file too, for debugging purposes.
  • Implement second step of API
    • Implement manual scheduling of files with LOVD+ implementation, but add:
      • Field where errors go
      • Priority flag
    • Add ajax controls:
      • Unschedule
      • Set priority
      • Reimport file (clear in_progress and process_errors fields)
    • Add menu item for the scheduling page.
    • Implement import of scheduled files
      • Archive files? Archiving folder?
      • Report success? LOVD needs this. Data should still be curated.
      • Link to manual run of automated import from scheduling page.
    • Document how to setup this importing with a cron job
  • Document format and submission method

Depends on #58.

@ifokkema ifokkema self-assigned this Nov 11, 2016

@ifokkema ifokkema added this to the 3.0 Build 18 milestone Nov 11, 2016

@ifokkema ifokkema modified the milestones: 3.0 Build 19, 3.0 Build 18 Dec 23, 2016

@ifokkema ifokkema removed this from In Progress in Milestone 3.0-19 Jun 14, 2017

@ifokkema ifokkema added this to TODO in Milestone 3.0-20 Jun 14, 2017

@ifokkema ifokkema modified the milestones: 3.0 Build 20, 3.0 Build 19 Jun 14, 2017

@ifokkema ifokkema modified the milestones: 3.0 Build 20, 3.0 Build 21 Sep 29, 2017

@ifokkema ifokkema removed this from TODO in Milestone 3.0-20 Oct 6, 2017

@ifokkema ifokkema added this to Todo in Milestone 3.0-21 Oct 6, 2017

@mkroon1 mkroon1 closed this in #320 Dec 13, 2017

Milestone 3.0-21 automation moved this from Todo to Done Dec 13, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment