Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata file names should be configurable #72

Closed
konstjar opened this issue Oct 2, 2024 · 0 comments
Closed

Metadata file names should be configurable #72

konstjar opened this issue Oct 2, 2024 · 0 comments
Assignees
Labels
Milestone

Comments

@konstjar
Copy link
Contributor

konstjar commented Oct 2, 2024

The name of files for parsing should be configurable in DataNode configuration, static value that admin can extend.
It may be an array of file names that should be looked in archive like execution-config.json or analysisMetadata.json

The metadata should be loaded from the first available file in archive.

datanode.metadata.fileNames: execution-config.json,analysisMetadata.json

Implementation Notes

To facilitate the server-side implementation for detection of metadata and entry points, the following endpoints are added:

POST /api/v1/analysis/upload/(zip|files)

Both of these accept multipart/form-data with the file part containing the file(s) being uploaded and store them in the same way currently uploaded analyses are persited, then provide response structure as follows:

{name: string, files: [string], metadata: {...}}

name - unique identifier of the upload.
metadata - the contents of detected metadata file. It is validated to be a valid JSON, however otherwise provided "as is", the responsibility to map specific data items to analysis fields remains with frontend.
In the future, we can consider to have a configurable mapping in addition to name, however this is not in scope of the current task.
files - the list of all files in the archive. For the moment, any filtering requirements should be implemented on frontend, however for the future it can be considered to implement configurable filtering on server side (and we might even skip saving undesirable files)

POST /api/v1/analysis/execute/{name}

{executableFileName, datasourceId, title, study type, environmentId, dockerImage}

Request body specification (as well as response contract) is the same as existing POST /api/v1/analysis/zip endpoint however unlike it, the new endpoint accepts "application/json" and doesn't accept files uploaded.
For execution, it uses preuploaded files stored under name identifier.

@konstjar konstjar added this to the v2.1.0 milestone Oct 2, 2024
@konstjar konstjar added the enhancement New feature or request label Oct 2, 2024
dmitrys-odysseus pushed a commit that referenced this issue Oct 18, 2024
…lysis to server and /api/v1/analysis/execute/* to start analysis with uploaded files
dmitrys-odysseus pushed a commit that referenced this issue Oct 18, 2024
dmitrys-odysseus pushed a commit that referenced this issue Oct 18, 2024
…ysis to server and /api/v1/analysis/execute/* to start analysis with uploaded files
dmitrys-odysseus pushed a commit that referenced this issue Oct 18, 2024
dmitrys-odysseus pushed a commit that referenced this issue Oct 20, 2024
dmitrys-odysseus pushed a commit that referenced this issue Oct 21, 2024
…ysis to server and /api/v1/analysis/execute/* to start analysis with uploaded files
dmitrys-odysseus pushed a commit that referenced this issue Oct 21, 2024
dmitrys-odysseus pushed a commit that referenced this issue Oct 30, 2024
dmitrys-odysseus pushed a commit that referenced this issue Oct 30, 2024
…ysis to server and /api/v1/analysis/execute/* to start analysis with uploaded files
dmitrys-odysseus added a commit that referenced this issue Oct 30, 2024
#72 Endpoints to preupload analysis to server and provide frontend with metadata and entry point list
@konstjar konstjar closed this as completed Nov 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants