This repository has been archived by the owner on Apr 27, 2021. It is now read-only.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As a review, could you please checkout the branch and follow the process below. This should uncover any issues/things to improve.
I'll also document this in our playbook.
Fixing stuck offsets
HADOOP_CONF_DIR=<path_to_starscream_repo>/.dev/starscream/spark/current/conf/conf.cloudera.yarn/ hadoop fs -copyToLocal /var/camus/execution/history/2018-01-01-00-00-00 .
java -Xms1G -Xmx2G -cp
/usr/local/Cellar/hadoop/2.8.0/bin/hadoop classpath:/Users/olessia/src/github.com/Shopify/camus/camus-shopify/target/camus-shopify-0.1.0-shopify1.jar org.wikimedia.analytics.refinery.job.CamusOffsetPatcher -e /Users/olessia/Documents/kafka_offsets/2018-02-20-13-30-30 -t nginx.shopify -p 120 -o 1226657431
hadoop fs -libjars /u/apps/camus/current/camus-shopify-0.1.0-shopify1.jar -text file:///<path>/patched-offsets-m-00104