Skip to content

Need to make these changes to make it work in the m1 mac. #699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 23, 2022
Merged

Need to make these changes to make it work in the m1 mac. #699

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ jobs:
lint_and_test:
working_directory: ~/cob_datapipeline
docker:
- image: cimg/python:3.7.11
- image: cimg/python:3.9.10
auth:
username: $DOCKERHUB_USER
password: $DOCKERHUB_PASSWORD
Comment on lines +9 to +10
Copy link
Member Author

@dkinzer dkinzer Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle was complaining about username and password not being set so that's why that's here.

environment:
PIPENV_VENV_IN_PROJECT: true
steps:
Expand All @@ -17,7 +20,10 @@ jobs:
make test
qa_deploy:
docker:
- image: cimg/python:3.7.11
- image: cimg/python:3.9.10
auth:
username: $DOCKERHUB_USER
password: $DOCKERHUB_PASSWORD
environment:
PIPENV_VENV_IN_PROJECT: true
steps:
Expand All @@ -38,7 +44,10 @@ jobs:
pipenv run ansible-playbook -i inventory/qa playbook.yml --tags "jumphost,role::airflow::dags" --vault-password-file=~/.vault -e 'ansible_ssh_port=9229'
prod_deploy:
docker:
- image: cimg/python:3.7.11
- image: cimg/python:3.9.10
auth:
username: $DOCKERHUB_USER
password: $DOCKERHUB_PASSWORD
environment:
PIPENV_VENV_IN_PROJECT: true
steps:
Expand Down
2 changes: 1 addition & 1 deletion .python-version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.7.11
3.9.10
9 changes: 7 additions & 2 deletions Pipfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,22 @@ boto3 = "*"
pexpect = "*"
pysftp = "*"
setuptools = "*"
cryptography = "==3.3.2"
cryptography = "==3.4.8"
Copy link
Member Author

@dkinzer dkinzer Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous version would not compile on my local. This is a minor change so it should be backwards compatible with what we were using.

Also since this is --dev only requirement it wont affect production.

Copy link
Contributor

@sensei100 sensei100 Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkinzer This update caused us significant problems in the past. See this issue for more details: https://tulibdev.atlassian.net/browse/DEVO-370

Can you confirm that this problem is resolved?

Copy link
Contributor

@sensei100 sensei100 Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And that is --dev here, but we do have that pinned in the airflow playbook.

Copy link
Member Author

@dkinzer dkinzer Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sensei100 interesting. I wonder why it's working now. That issues claims that it was failing ant that we needed to look into making some changes to "our alpine" images. So was this failing on cirlce? I wonder why it's passing in my PR.

Copy link
Contributor

@sensei100 sensei100 Mar 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkinzer Looking at the issue that is linked in the DEVO ticket, it seems that there is a way to disable the rust integration for 3.4 pyca/cryptography#5771 (comment)

Would you be willing to read the FAQ and ensure that the rust integration is disabled?

Copy link
Contributor

@ebtoner ebtoner Mar 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkinzer @sensei100 I also took a look at #535 for changelog details and it looks like 3.4.8 made some rust updates that may have addressed the issues with 3.4.7? Am I on the something here?

@sensei100 In the final comment on that thread pyca/cryptography#5771 (comment), there are also some embedded notes from later in 2021 that might be relevant.

sqlalchemy = "==1.3.23"
flask-openid = "==1.3.0"
flask-sqlalchemy = "==2.4.4"
jinja2 = "==2.11.3"
werkzeug = "==0.16.1"
marshmallow = "==2.21.0"
wtforms = "==2.3.3"
markupsafe= "==2.0.1"
Copy link
Member Author

@dkinzer dkinzer Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was getting an error where c_types are not defined. This is a known issue that affects all markupsafe after this version. Flaask and Jinja use this.


# required in circleci context.
numpy= "*"


[packages]
pytest = "==6.2.5"

[requires]
python_version = "3.7"
python_version = "3"
Copy link
Member Author

@dkinzer dkinzer Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sets python version to 3 so that we can use 3.9 locally but still be able to use other versions in production. I'm not sure if it matters either since we install production deps via "Requirements.txt in the airflow deployment."

Loading