Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write data set/get/write functions #4

Closed
4 tasks
hammannr opened this issue Jun 15, 2023 · 1 comment
Closed
4 tasks

Write data set/get/write functions #4

hammannr opened this issue Jun 15, 2023 · 1 comment
Assignees
Labels
data handling Code for managing our data enhancement New feature or request

Comments

@hammannr
Copy link
Collaborator

  • Common data format
  • Data write
  • Data read
  • Data set
@hammannr hammannr added enhancement New feature or request data handling Code for managing our data labels Jun 15, 2023
@hammannr
Copy link
Collaborator Author

I think this was solved by #7 and now updated by #19 , right @kdund?

dachengx added a commit that referenced this issue Jul 18, 2024
* Refactor osg submitter (#2)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* modified pegasus config

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Debug path in rc (#3)

* fixed a typo in top_dir

* debug

* x509

* debug

* Shell

* imports

* no string anymore

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove regex

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* renmae jobs and fixed relative path in rc

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* reorganized relative path and more strict type control

* remove unwanted space

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* try json dump as a string format solution

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* str type

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* explicit escaped str

* yaml overwrite for rc

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* sync

* relative dir in template filename

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* put back relative dir

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* resolve conflict

* compact jq

* path from config

* explicit pegasus imports

* first try fixing the ticket generator

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Summary (#4)

* dont cleanup

* first trial on getting summary job

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added cluster_size features

* dont stage out

* revert some Robert commits

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* check on X509

* check on X509

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* setup script

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* annotation

* copy x509

* documentation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* documentation

* removed tarball filename

* debug template path

* no need to pass running_config_filename anymore

* no need to pass running_config_filename anymore

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* var name confusing

* Update setup_pegasus.sh

* handle subdir

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* doc

* doc

* doc

* removed subfolders after untarring templates tarball

* Update README.md

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add input_files attribute

* reload x509

* reload x509

* Debug sensitivity pegasus (#5)

* trial on rewriting threshold path

* correct limit_threshold path to osg format

* debug

* adding limit_threshold into rc

* adding limit_threshold into rc

* debug

* add_inputs

* debug

* debug

* fixed _wf_id

* forbid tc writting

* generated files go to generated folders

* Only add the toydata file if instructed to do so

* not implemented error for disfavored toydata_mode

* dont check specific file since it may not exist

* warning about outputfolder

* recursive combine

* moved out setup_pegasus.sh

* default for max_jobs_to_combine

* support relative path for running config

* support relative template_filename

* check filename uniqueness

* fix typo

* situation after ap23 maintenance

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Put back pegasus path

* You can only find `sources` in `pdf_base_config` (#6)

* Fix pre-commit

* Ignore setup_pegasus.sh

* Remove function related to `input_files`

* Fix code style

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Robert Hammann <robert.hammann@outlook.com>
Co-authored-by: Dacheng Xu <dx2227@columbia.edu>
Co-authored-by: Dacheng Xu <xudctron@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data handling Code for managing our data enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants