CCSDS File Uploader

[9876691]

Operators will need some API automated provisioning for transferring CCSDS files in XML to SkyTrace.

One possible solution.

• A unix CLI tool or a Docker container.
• Picks up files from a folder, processes the XML and forward it to SkyTrace.
• Renames the file and adds the extension .processing.
• When the task is complete adds the extension .done.

The operator will need to run some sort of cronjob (daily?) and some how monitor for failure.

Failure would be if the job runs, but files are stuck in .processing

[elprimato]

+1 for the solution of a unix cli tool (python maybe?)
Cronjob is currently in use for these type of uploads. The frequency is determined by the generation of new predictions, i.e. if there are four updates per day they we would try to upload all of them to ensure the latest predictions are distributed.
A CCSDS format and XML tag check will be essential.

With the renaming of the file, I am unsure. Do you propose to have this on client side?
Would you also have a .error state? And a local log?
For monitoring, I can also envision a list view on the webpage. So once the file has been uploaded (cli tool finished with exit code 0), then the user can verify the state via webpage or separate API call.

[9876691]

Another option is to implement the XML upload server side. We lose the digital signatures but we could add that later if needed.

So the API would look something like

curl -F file=@/some/file/on/your/local/disk.xml http://skytrace.space/api/xml_upload

I would just need to take care of any XML security issues.

[elprimato]

Yes, in fact this is how other platforms (i.e., space-track) offer the upload currently, server-sided. And it is by far the best solution. If you login to space-track, they give examples in a couple of languages. The reasoning for server-sided is that it should be as simple as possible for users to participate.
Dealing with the security issues is necessary, yet, for CCSDS, there is a schema which you can use for validation.
Special cases need to be handled of course.

[9876691]

@elprimato I've implemented an OEM upload feature and documented it here https://skytrace.space/docs/api-docs/curl-example/

It doesn't have much in the way of validation, I'll create a separate issue for that.

And I didn't have a lot of test data, so if you have issues please attach any XML.