Skip to content
This repository was archived by the owner on Oct 29, 2023. It is now read-only.

Conversation

Careyjmac
Copy link
Contributor

Updating streamers to remove auth issue (jobs that take over an hour should be able to finish now) add aggregators (to be able to view the progress of the job in the dataflow console better) and lower the shard size (for now).

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project, in which case you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
  • If you signed the CLA as a corporation, please let us know the company's name.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Nicole and Dion previously discussed issues with hardcoding it, but maybe having a final constant and just manipulating that might be better.

@Careyjmac
Copy link
Contributor Author

PTAL

@dionloy
Copy link
Contributor

dionloy commented Jul 1, 2015

LGTM once Travis is happy

@pgrosu
Copy link

pgrosu commented Jul 1, 2015

Dion, what do you think about having it be public and not final to play around with it, until we determine an algorithm for the optimal shard size?

@dionloy
Copy link
Contributor

dionloy commented Jul 1, 2015

I think it's fine private now. Public constants are against our coding style, I'd rather not have people depend on it, and hopefully this is just very temporary while we figure something better out.

@pgrosu
Copy link

pgrosu commented Jul 1, 2015

That's understandable - thanks.

@deflaux
Copy link
Contributor

deflaux commented Jul 1, 2015

LGTM

@deflaux
Copy link
Contributor

deflaux commented Jul 10, 2015

@Careyjmac merge this anytime you like

Careyjmac pushed a commit that referenced this pull request Jul 10, 2015
@Careyjmac Careyjmac merged commit 2d69c44 into googlegenomics:master Jul 10, 2015
jiridanek pushed a commit to jiridanek/dataflow-java that referenced this pull request Jan 18, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants