Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify Stream Totaling for Staging Compatibility #2817

Merged
merged 1 commit into from May 7, 2018

Conversation

rajadain
Copy link
Member

@rajadain rajadain commented May 6, 2018

Overview

For reasons currently not understood, the namedtuple result of sum_subbasin_stream_lengths is interpreted as a dict by Celery when run locally, but as a list when run on staging.

By outputting a dict always, we ensure that it is interpreted uniformly across platforms.

I've currently manually applied this patch on staging to get it to work.

Connects #2811

Demo

Staging woking with the patch:

image

Notes

I was unable to find out exactly why this was happening, but I did find some discussion on conflicting behavior of converting namedtuples to dicts here: https://stackoverflow.com/questions/26180528/python-named-tuple-to-dictionary. In the interest of expediency, I settled for a solution sans explanation. 馃殌馃惁

Testing Instructions

  • On develop, run subbasin HUC-10 locally and save the table data as a CSV
  • Check out this branch and run bundle
  • Clear your local Redis cache by running: vagrant ssh services -c 'redis-cli -n 1 --raw KEYS ":1:geop_*" | xargs redis-cli -n 1 DEL'
  • Run subbasin HUC-10 localy and export the table as a CSV again
  • Compare the two CSVs and ensure they are identical

For reasons currently not understood, the namedtuple
result of this method is interpreted as a dict by
Celery when run locally, but as a list when run on
staging.

By outputting a dict always, we ensure that it is
interpreted uniformly across platforms.

Refs #2811
@rajadain rajadain force-pushed the tt/subbasin-staging-timeouts branch from e3404ad to aaf171f Compare May 7, 2018 14:57
@rajadain
Copy link
Member Author

rajadain commented May 7, 2018

Updated commit removes unused import, thus pacifying flake8.

Copy link
Contributor

@mmcfarland mmcfarland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, unusual find. CSV output remains constant and the new code adds some clarity as well.

@mmcfarland mmcfarland assigned rajadain and unassigned mmcfarland May 7, 2018
@rajadain rajadain merged commit 4b5f898 into develop May 7, 2018
@rajadain rajadain deleted the tt/subbasin-staging-timeouts branch May 7, 2018 16:22
@rajadain
Copy link
Member Author

rajadain commented May 7, 2018

Thanks for taking a look and helping with the debugging!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants