Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Built data should be pushed to depot #10

Closed
hexylena opened this issue Apr 6, 2015 · 8 comments
Closed

Built data should be pushed to depot #10

hexylena opened this issue Apr 6, 2015 · 8 comments
Assignees

Comments

@hexylena
Copy link
Member

hexylena commented Apr 6, 2015

Just an issue for tracking purposes.

@natefoo
Copy link
Member

natefoo commented Jun 4, 2015

@erasche, could you have jenkins place the outputs into subdirectories based on the thing being built? Some output names are not consistent such that i can use fname.split('-')[0].

Actually, what's with the 32 bit linux, mac, and windows builds of tbl2asn (where did they come from)?

xref bgruening/galaxytools#102

@hexylena
Copy link
Member Author

hexylena commented Jun 4, 2015

Okay, can do. Sorry about that.

Tbl2asn is prebuilt for us by NCBI, but without any version information. We
separate it into a set of versioned files.

@hexylena
Copy link
Member Author

hexylena commented Jun 4, 2015

@natefoo okay, looked at the "Collect-for-depot" recipe, and here's what it's running:

find /var/lib/jenkins/workspace/Docker-Build/ -mindepth 2 -maxdepth 3 \
    -not -wholename '*git*' -and -not -name 'build.*' \
    -and -not -name '*.sh' -and -not -name '*.diff' \
    -and -not -name '*.patch' -and -not -name 'Dockerfile' \
    -type f -exec cp '{}' /opt/depot/data/ \;

This may not be a perfectly easy problem to solve unfortunately. Given the file listing, I think the easier option would be to just rename everything properly.

@hexylena
Copy link
Member Author

hexylena commented Jun 4, 2015

Let me resolve the v2 issues first, and I'll name things proiperly as part of that (since most of the issues are in code that I wrote)

@hexylena
Copy link
Member Author

hexylena commented Jun 4, 2015

v2 should be ready to go. #15 (comment)

This includes fixes for versions/names in all of the things I can fix, and I've removed GD-2.56.tar.gz as it also had a bad name, and is now available in https://github.com/bgruening/download_store/tree/master/GD-2.56

@natefoo
Copy link
Member

natefoo commented Jun 4, 2015

Is there a reason not to simply grab */*.tar.gz rather than exclude everything else? How about:

cd /var/lib/jenkins/workspace/Docker-Build && \
    for f in */*.tar.gz; do mkdir -p /opt/depot/data/`dirname $f` && cp $f /opt/depot/data/$f; done

@hexylena
Copy link
Member Author

hexylena commented Jun 4, 2015

@natefoo there was a reason, but it's escaping me. Yeah, we'll switch to that.

Also, fyi, the above would fail with v2 versioned directories. I'll work something out and keep them separated into folders for you.

@natefoo
Copy link
Member

natefoo commented Jun 4, 2015

@erasche awesome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants