Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(PXP-6544) Pelican-export allows client to specify the root node #33

Merged
merged 2 commits into from
Sep 8, 2020

Conversation

mpingram
Copy link
Contributor

@mpingram mpingram commented Sep 3, 2020

Link to Jira: PXP-6544
Link to design doc: https://docs.google.com/document/d/12FkAYOpDuSdQScEgYBxXsPUdm8GUuUZgD6xU7EQga5A/edit#
Link to related portal PR: uc-cdis/data-portal#729

New Features

  • Pelican-export job now allows client to specify the root node of a pfb when dispatching a job, allowing clients to export PFBs of entities that are not on the $ROOT_NODE environment variable. This allows for exporting of data file PFBs, so long as the data files are all on the same root node.
  • The pelican-export algorithm's extra_nodes input can now be configured by $EXTRA_NODES environment variable. The old behavior of including reference_file and reference_file_index for BDCat exports is still there for backwards compatibility.

Breaking Changes

Bug Fixes

Improvements

Dependency updates

Deployment changes

- Also make extra_nodes configurable via environment variable
@mpingram mpingram merged commit 74266f4 into master Sep 8, 2020
@mpingram mpingram deleted the feat/limited-file-pfb-export branch September 8, 2020 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants