Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API sends minimal payload to Engine #750

Closed
donaldgray opened this issue Feb 29, 2024 · 3 comments · Fixed by #756
Closed

API sends minimal payload to Engine #750

donaldgray opened this issue Feb 29, 2024 · 3 comments · Fixed by #756
Assignees
Labels
API https://dlcs-book.readthedocs.io/en/latest/ delivery-channels jira marks issue for crossposting to JIRA

Comments

@donaldgray
Copy link
Member

For both sync and async processing the API currently sends a full Asset model to Engine. This should be updated to send the AssetId only as the Engine will look up all required data (blocked by #622).

Acceptance Criteria

  • API sends AssetId only in HTTP requests (sync)
    • This is for ingest and reingest.
  • API sends AssetId only in SQS messages (async)
  • Engine continues to work
@donaldgray donaldgray added API https://dlcs-book.readthedocs.io/en/latest/ delivery-channels jira marks issue for crossposting to JIRA labels Feb 29, 2024
@griffri griffri self-assigned this Mar 5, 2024
@JackLewis-digirati
Copy link
Contributor

JackLewis-digirati commented Mar 6, 2024

writing this here as it seems relevant - encountered while completing work on #622

might need to have a chat about approach for InitialOrigin - this has entity.Ignore(e => e.InitialOrigin); in the DbContext and a comment stating // TODO - how to handle this? Split model + entity?

The issue is there are some tests which set the InitialOrigin for the API calls and they can't be retrieved from the database, due to the Ignore flag

@tomcrane
Copy link
Contributor

tomcrane commented Mar 6, 2024

Ah... InitialOrigin is "actually ingest from here, but pretend you didn't and record the origin as https://example.org/where-it-lives-long-term"

Example usage is a rapid bulk load from S3 for stuff that long term would live in preservation storage and if ever reprocessed would be pulled from there. We added it for a very early Wellcome migration.

@tomcrane
Copy link
Contributor

tomcrane commented Mar 6, 2024

We could drop this feature, actually, if nobody ever uses it.

We could tell users to simply patch the origin after ingest, but the problem with that is that change of origin is definitely considered a trigger for reprocessing.

Discuss on next call

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API https://dlcs-book.readthedocs.io/en/latest/ delivery-channels jira marks issue for crossposting to JIRA
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

4 participants