Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update deployment process to use conda/pip instead of cloning github repos #826

Merged
merged 23 commits into from Feb 27, 2018

Conversation

hdoupe
Copy link
Collaborator

@hdoupe hdoupe commented Feb 27, 2018

Last night, I was unable to reset the node environments because it was unable to install btax. In this PR, I refactored the install scripts so that all of the OSPC packages are installed with conda instead of some confusing mix of cloned github repos and pip installations.

All packages are installing correctly now. The only problem is that they are in a different location relative to the PUF file. Thus, if you run TaxBrain on the test app, you get:

screen shot 2018-02-27 at 11 35 28 am

taxcalc traceback:

or on CCC you get:

screen shot 2018-02-27 at 11 36 23 am

btax traceback:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda2/envs/aei_dropq/lib/python2.7/site-packages/celery/app/trace.py", line 374, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/ubuntu/miniconda2/envs/aei_dropq/lib/python2.7/site-packages/celery/app/trace.py", line 629, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/ubuntu/deploy/taxbrain_server/celery_tasks.py", line 159, in btax_async
    tables = json.loads(runner_json_tables(**user_mods))
  File "/home/ubuntu/miniconda2/envs/aei_dropq/lib/python2.7/site-packages/btax/front_end_util.py", line 172, in runner_json_tables
    out = runner(test_run, start_year, iit_reform, **user_params)
  File "/home/ubuntu/miniconda2/envs/aei_dropq/lib/python2.7/site-packages/btax/execute.py", line 48, in runner
    run_btax(test_run, True, start_year, {}, **econ_params)
  File "/home/ubuntu/miniconda2/envs/aei_dropq/lib/python2.7/site-packages/btax/run_btax.py", line 110, in run_btax
    iit_reform, **user_params)
  File "/home/ubuntu/miniconda2/envs/aei_dropq/lib/python2.7/site-packages/btax/parameters.py", line 207, in get_params
    indiv_rates = get_rates(baseline, start_year, iit_reform)
  File "/home/ubuntu/miniconda2/envs/aei_dropq/lib/python2.7/site-packages/btax/get_taxcalc_rates.py", line 83, in get_rates
    reform=reform)
  File "/home/ubuntu/miniconda2/envs/aei_dropq/lib/python2.7/site-packages/btax/get_taxcalc_rates.py", line 45, in get_calcu

@hdoupe hdoupe added the WIP label Feb 27, 2018
@hdoupe
Copy link
Collaborator Author

hdoupe commented Feb 27, 2018

Alright, two nodes are up and running on the test app. Progress is slow, but it's happening.

@hdoupe hdoupe changed the title Update deployment process to use conda instead of github/pip/other stuff Update deployment process to use conda/pip instead of cloning github repos Feb 27, 2018
@hdoupe hdoupe merged commit c69168d into ospc-org:master Feb 27, 2018
@hdoupe
Copy link
Collaborator Author

hdoupe commented Feb 28, 2018

Testing strategy:
Detach the worker nodes from the production app one-by-one and update them. If the update was successful, I attached it to the test app which was running on 1.4.0 and checked the log for each each node to make sure that they were all still sending and receiving jobs. I did this for eight of the nine nodes. The ninth is still serving the production app and will be updated when the production app is updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant