Skip to content

Looper should speak pephub #341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
nsheff opened this issue Feb 7, 2023 · 3 comments · Fixed by #369
Closed

Looper should speak pephub #341

nsheff opened this issue Feb 7, 2023 · 3 comments · Fixed by #369
Assignees
@khoroshevskyi
Labels
likely-solved
Milestone
1.5.0

Comments

@nsheff
Copy link
Contributor

nsheff commented Feb 7, 2023

What if you could looper run pephub://databio/project ?

In other words, instead of passing sample metadata via a file, you pass it via registry path. Looper needs to learn to speak the PEPhub API.
@nsheff nsheff added this to the 1.5.0 milestone Feb 15, 2023
@khoroshevskyi
Copy link
Member

Issue #342 blocks this one. Looper config should be decoupled from PEP

@khoroshevskyi khoroshevskyi linked a pull request Apr 20, 2023 that will close this issue
Added pephub support in looper #369
Merged
8 tasks
khoroshevskyi added a commit that referenced this issue Apr 25, 2023
@khoroshevskyi
Fixed #341 #342
25ab2b0
@nsheff
Copy link
Contributor Author

nsheff commented Apr 26, 2023

@khoroshevskyi can you add a section in the docs on how to do this? Actually I think it would be great to:

  1. include the example config in the docs to show how to set it up.
  2. add an example to hello_looper to show how to run it?
  3. update the looper init functionality to be able to also populate it with pifaces

@nsheff
Copy link
Contributor Author

nsheff commented Jul 18, 2023

@khoroshevskyi can you comment on the status of this documentation?

@donaldcampbelljr donaldcampbelljr mentioned this issue Aug 7, 2023
Merged
donaldcampbelljr added a commit that referenced this issue Aug 14, 2023
@donaldcampbelljr @nsheff
@khoroshevskyi @vreuter @ayobi
v1.5.1 (#391)
145c0d6 
* add geofetch tutorial

* Add pephub support for sample-level pipeline interface.

* Fixed tests naming

* added project level for pephub support

* simplify Project constructor implementation

* logger string fix

Co-authored-by: Vince <vince.reuter@gmail.com>

* fix try block

Co-authored-by: Vince <vince.reuter@gmail.com>

* Update looper/utils.py

Co-authored-by: Vince <vince.reuter@gmail.com>

* Fixed #341 #342

* Deprecated write_skipped_sample_scripts. Scripts will now output sequentially (unless toggled). #173

* Remove redundant pooling behavior for skipped samples. #173

* First working version of #344

* divvy reintegration #343

* divvy reintegration #343

* Cleaned up looper with divvy

* removed sample and project pipelines from cli

* fixed old .looper.config specification

* added new config docs

* Remove --toggle key. Add logic during fetch samples for toggle. Remove checking for toggle off during sample submission (redundant). #263

* Remove toggle key constants. Apply formatting. #263

* Remove toggle key property. Simplified logic for fetching samples. #263

* fix pephub failing tests

* cleaned up more to help pass pytests

* distutils.dir_util for shutil for python 3.12

* added divvy entry point inside looper

* black format

* another black reformat

* added docs

* clean up based on feedback

* clean up based on feedback

* removed redundancy

* black fmt

* added divvy inspect

* black fmt

* added sub cmd, docker args

* added line break for inspect output

* added divvy docs #343

* added divvy docs #343

* divvy docs fix

* mkdocs fix

* Fixed mkdocs error

* Update requirements-doc.txt

* updated reqs-doc

* merge mistake fix

* added divvy imgs

* added new looper init

* added to changlog, fix divvy imgs

* divvy readme img fix

* fixed initialization of generic piface

* fixed initialization of generic piface

* added tests

* fixed main setup

* Update how_to_define_looper_config.md

* Update __init__.py

* Update test_other.py

* Ise

* added changelog and minor naming changes

* remove old logging function

* dev version bump

* fix typo in html_report and upgraded pandas requirements for pephubclient

* fixed requirements

* fixed docs requirements

* added versioneer to doc requirements

* added Cython to doc requirements

* added readthedocs config

* added looper to requirements docs

* allow for using pipestat.summarize, align with pipestat 0.4.0

* clean up code, update usage doc

* update doc requirements pephubclient

* downgrade docs to 3.10

* adjust get_status to use proper sample_name if pipestat configured #326

* adjust conductor to retrieve pipestat manager variables with pipestat 0.4.0 refactoring.

* Allows skipping some tests if run offline. Closes #370

* work on using test_args instead of subprocesses

* Finish switching applicable tests away from subprocess

* Lint and update doc string to test_args_expansion

* Change return type.

* lint

* add test for var_templates #357, and clean up tests

* attempt simple check to see if provided pipelines are callable #195

* minor adjustments, polished docstring

* update changelog

* lint

* update version to 1.5.0

* update changelog

* update reqs and changelog to use pipestat v0.5.0

* Refactoring for looper config

* added looper config file argument

* code fix

* Added comment about deprecating for old looper specification

* fixed looper init error

* change logo for docs build tab icon

* fix favicon

* update docs and changelog for 1.5.0 release

* - fix `looper table` failing without sample.protocol , update change log for point release

* fix "--looper-config"

* update version and changelog date

* adjust pipestat requirement to be >=0.5.1

* lint

---------

Co-authored-by: nsheff <nsheff@users.noreply.github.com>
Co-authored-by: Khoroshevskyi <sasha99250@gmail.com>
Co-authored-by: Vince Reuter <vince.reuter@gmail.com>
Co-authored-by: ayobi <17304717+ayobi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@khoroshevskyi khoroshevskyi

Labels
likely-solved
Projects
No open projects
PEP Vision
Status: Done
Milestone
1.5.0
Development

Successfully merging a pull request may close this issue.

Added pephub support in looper pepkit/looper
2 participants
@nsheff @khoroshevskyi