Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable batch correlation of uploaded files #1348

Merged
merged 1 commit into from Jun 20, 2018

Conversation

@dbazile
Copy link
Contributor

commented Jun 19, 2018

This change allows an external consumer of the REST API to submit files in a
batch and ingest them in a single shot as opposed to submitting a single
ingest job per file.

Currently geowave ingest localToGw only accepts a single file or a directory
as the source parameter. Doing "batch" ingests by transferring files via
/restservices/v0/fileupload would require invoking localToGw once for
every file in the batch which seems like it would require extra bookkeeping to
track and merge the operation statuses.

Refer to G3-297

Enable batch correlation of uploaded files
This change allows an external consumer of the REST API to submit files in a
batch and ingest them in a single shot as opposed to submitting a single
ingest job per file.

Currently `geowave ingest localToGw` only accepts a single file or a directory
as the source parameter.  Doing "batch" ingests by transferring files via
`/restservices/v0/fileupload` would require invoking `localToGw` once for
every file in the batch which seems like it would require extra bookkeeping to
track and merge the operation statuses.

Refer to G3-297
@coveralls

This comment has been minimized.

Copy link

commented Jun 19, 2018

Coverage Status

Coverage increased (+1.9%) to 46.346% when pulling ef9164c on dbazile:upload-batching into b58e6fa on locationtech:master.

@rfecher rfecher merged commit adf9059 into locationtech:master Jun 20, 2018

0 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
coverage/coveralls Coverage increased (+1.9%) to 46.346%
Details

@dbazile dbazile deleted the dbazile:upload-batching branch Jun 20, 2018

cjw5db pushed a commit to cjw5db/geowave that referenced this pull request Jul 11, 2018
Conner Ward Conner Ward
Enable batch correlation of uploaded files (locationtech#1348)
This change allows an external consumer of the REST API to submit files in a
batch and ingest them in a single shot as opposed to submitting a single
ingest job per file.

Currently `geowave ingest localToGw` only accepts a single file or a directory
as the source parameter.  Doing "batch" ingests by transferring files via
`/restservices/v0/fileupload` would require invoking `localToGw` once for
every file in the batch which seems like it would require extra bookkeeping to
track and merge the operation statuses.

Refer to G3-297
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.