Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sis task 7 and 11 #5

Closed
wants to merge 20 commits into from
Closed

Conversation

uweschmitt
Copy link
Collaborator

Relates to cbg-ethz/V-pipe#116

I created a separate python/ folder with an init.sh script to setup a virtual environment with the needed dependencies.

The script python/upload_to_db.py includes comments on how to use it.

Later usage could be:

  • after completion of V-pipe copy summary.zip to a local folder
  • run .../python/venv/bin/python .../python/upload_to_db.py /path/to/summary.zip to import data

Future work will to to complete the database schema.

uweschmitt and others added 20 commits October 26, 2021 17:44
 - recent versions of PyBIS don't default registrationDate to None anymore
 - switch to kwargs for optional parameters
 - Inform if sync-skipping is in effect
 - Typo in belfry
 - remove NEXUS, add new PhD
 - lofreq resources
 - (also space formatting shenigans)
 - Parametrable timeout for `sync` when dropping cache
 - Parametrable timeout for the automation loop
 - Parameters updated from server.conf on loop
   (with a short timeout in case the storage isn't responding anymore)
   (and fall back to defaults in that case)
 - additional flags in status when loop starts and after loop has returned/crashed
 - when 31 falls between sequencer start and data delivery,
   there is no batch yet named with {thismonth} and
   sort fails, yeilding an empty samples.recent.tsv
 - try reducing parallelism to avoid FS outages
 - New option to suspend submission of jobs while fixing troubles
 - Block-listing broken sequencings at FCGZ
 - replace cvs python reader with more generic
 - AWK-style 'whitespace transition' based
 - initial implementation of logic for non shared plate between
   same-order multiple sequencing
 - temporary use nofuselist until mature code
 - bump up cojac to version with omicron
 - ww.bsub: raise memory limitations
 - ww-new.bsub and ww-new_final.bubs: scripts to rebuild whole cohort
This was referenced Dec 14, 2021
@DrYak
Copy link
Member

DrYak commented Jan 20, 2022

Merged in #6c73b5f9f3e73da90a1021c10b0d7fea22aa5bc0

Next step:

  • raw uploads to be done from Euler / V-pipe (move to Euler)
  • update capsule to be done per-file (python code should walk through samples listed in samples.recent.tsv)

@DrYak
Copy link
Member

DrYak commented Oct 16, 2022

@uweschmitt
Given that this merge is merged, and that the next tasks (actual uploading) are tracked in the git lab's issue, I presume I can close this PR, right ?

@uweschmitt
Copy link
Collaborator Author

uweschmitt commented Oct 17, 2022

yes, please close

@DrYak DrYak closed this Oct 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants