New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to upload multiple FTP files within a single request #4513

Merged
merged 40 commits into from Oct 9, 2017

Conversation

Projects
None yet
4 participants
@guerler
Contributor

guerler commented Aug 29, 2017

This PR adds another tab to the uploader view which allows users to submit multiple FTP files at once. In contrast to the regular upload view, this option does not allow the selection of individual datatypes or genome builds per ftp file. All ftp files are submitted using the same datatype/genome specification. However, the advantage is that all FTP files can be submitted to the upload tool at once, which improves the performance for large sets. Fixes #4347 and #1090.

@guerler guerler added this to the 17.09 milestone Aug 29, 2017

@guerler guerler requested a review from jmchilton Aug 29, 2017

@guerler guerler changed the title from Ftp uploader update to FTP uploader update Aug 29, 2017

@guerler guerler changed the title from FTP uploader update to Allow users to upload multiple FTP files within a single request Aug 29, 2017

@guerler guerler requested review from martenson and removed request for jmchilton Aug 29, 2017

@martenson

This comment has been minimized.

Member

martenson commented Sep 6, 2017

Do I need to do something else than checkout guerler/ftp_uploader and start Galaxy to test this? Because I cannot see the ftp tab.

@guerler

This comment has been minimized.

Contributor

guerler commented Sep 6, 2017

@martenson thats the correct branch, we embedded the performance improvements into the regular uploader and removed the ftp tab. You could test the performance differences by selecting many ftp files in the regular view and then by clicking on start with and without this PR. However the multifile submission with differing dbkey/datatypes is still in the works.

@martenson

This comment has been minimized.

Member

martenson commented Sep 6, 2017

That means we are still stuck with the popover-over-modal view into FTP directory. I think we need to improve on this to use actual tree-based folder browser. (And maybe move the whole upload view into mid pane instead of a modal to give it more of a canvas.)

Nevertheless this PR does not seem to be working for me. Nothing gets imported to history.

screenshot 2017-09-06 17 24 30

@guerler

This comment has been minimized.

Contributor

guerler commented Sep 6, 2017

@martenson as I mentioned However the multifile submission with differing dbkey/datatypes is still in the works. All you can test are performance differences in working with large ftp file sets. I let you know when the submission feature for the backend works. See also: #4563.

@martenson

This comment has been minimized.

Member

martenson commented Sep 6, 2017

@guerler I selected 4 FTP files and nothing got imported into history. All have autodetect and ?. I did not even see a request to upload in the devtools.

@guerler

This comment has been minimized.

Contributor

guerler commented Sep 6, 2017

@martenson that is correct the submission to the backend has to be added. I let you know when we added it.

@guerler guerler modified the milestones: 18.01, 17.09 Sep 11, 2017

@guerler

This comment has been minimized.

Contributor

guerler commented Sep 11, 2017

Requires #4563.

@jmchilton

This comment has been minimized.

Member

jmchilton commented Sep 11, 2017

Thanks for moving the milestone @guerler - sorry I wasn't able to get #4563 done in time for the release.

@guerler guerler added status/review and removed status/WIP labels Oct 4, 2017

Merge remote-tracking branch 'jmchilton/dev' into ftp_uploader
Conflicts:
 static/scripts/bundled/libs.bundled.js
 static/scripts/bundled/libs.bundled.js.map
@jmchilton

This comment has been minimized.

Member

jmchilton commented Oct 9, 2017

Very nice - thanks for this!

@jmchilton jmchilton merged commit e03bedf into galaxyproject:dev Oct 9, 2017

0 of 6 checks passed

api test Test started.
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
framework test Test started.
Details
integration test Test started.
Details
lgtm analysis: JavaScript Fetching Git Commits
Details
toolshed test Test started.
Details
@guerler

This comment has been minimized.

Contributor

guerler commented Oct 9, 2017

Thank you. Thanks a lot for #4563 and your overall help with this.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Oct 9, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment