Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-25965: Add ButlerURI.abspath() method #489

Merged
merged 5 commits into from
Mar 18, 2021
Merged

DM-25965: Add ButlerURI.abspath() method #489

merged 5 commits into from
Mar 18, 2021

Commits on Mar 17, 2021

  1. Add ButlerURI.abspath() method 

    This is now very lightweight since URIs are immutable.
Forces scheme-less URIs to file URIs.
    @timj
    timj committed Mar 17, 2021
    Configuration menu
    Copy the full SHA
    5c57bb7 View commit details
    Browse the repository at this point in the history

  2. Fix walk for S3 so that it can start at the bucket root

    @timj
    timj committed Mar 17, 2021
    Configuration menu
    Copy the full SHA
    edb2e2f View commit details
    Browse the repository at this point in the history

  3. Add check for unresolved dataset refs in datastore ingest 

    This should never happen but is the last line of defence
against a confusing error message later on when the insert
into the table recording the dataset fails.
    @timj
    timj committed Mar 17, 2021
    Configuration menu
    Copy the full SHA
    4b4ef7f View commit details
    Browse the repository at this point in the history

  4. Check for duplicate dataId/datasetType ingest requests 

    This can happen if by some chance someone asks the same file
to be ingested from two different locations.
    @timj
    timj committed Mar 17, 2021
    Configuration menu
    Copy the full SHA
    3c13ae5 View commit details
    Browse the repository at this point in the history

  5. Allow FileDataset to take a ButlerURI

    @timj
    timj committed Mar 17, 2021
    Configuration menu
    Copy the full SHA
    ae4ddc2 View commit details
    Browse the repository at this point in the history