Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vulnerability: CVE-2015-20107, Python #1406

Closed
Tracked by #1409
lsloan opened this issue Aug 26, 2022 · 8 comments · Fixed by #1426
Closed
Tracked by #1409

Vulnerability: CVE-2015-20107, Python #1406

lsloan opened this issue Aug 26, 2022 · 8 comments · Fixed by #1426
Assignees
@lsloan
Labels
⚠️ priority ☠ vulnerability
Projects
MyLA-2022.01.03

Comments

@lsloan
Copy link
Member

lsloan commented Aug 26, 2022 •
edited

From vulnerabilities spreadsheet, based on Unizin analysis:

CVE ID Effective Severity Severity Impacted Image Vulnerable Package Remediated Package URL
CVE-2015-20107 CRITICAL CRITICAL gcr.io/unizin-core/myla:2022.01.01 python3.9/3.9.2-1 python3.9/MAXIMUM https://security-tracker.debian.org/tracker/CVE-2015-20107
@lsloan lsloan added this to To do in MyLA-2022.02.01 via automation Aug 26, 2022
@jonespm
Copy link
Member

jonespm commented Aug 26, 2022

This issue is still open in Python https://bugs.python.org/issue24778

We aren't using the mailcap module. There is currently some work toward removing this module in 3.13, but it does look like there is a fix in one of the latest versions of 3.10 and there is a backport for the next security release of 3.9.

https://security-tracker.debian.org/tracker/CVE-2015-20107

MyLA is still running Python 3.8 and that should also get this release, but we may want to look to ugprading. I think we'll have to revisit this in the next release?

@jonespm jonespm mentioned this issue Aug 30, 2022
Closed
2 tasks
@jennlove-um
Copy link
Contributor

jennlove-um commented Aug 31, 2022 •
edited by zqian

The fix is in Python 3.10.6-1. Will update docker image to Python 3.10 and verify the OpenShift vulnerability report.

@jennlove-um jennlove-um assigned ssciolla and lsloan and unassigned ssciolla Aug 31, 2022
@ssciolla
Copy link
Contributor

ssciolla commented Aug 31, 2022 •
edited

@lsloan, we discussed this in the meeting. This is how I would approach it.

  1. Look at each of the Python dependencies and see if 3.10 is explicitly supported. Make a list of any that are not.
  2. If the list is empty, or only a couple don't support it, update the Docker base image to 3.10 and do a PRT of the tool, checking logs for errors. If all is well, it's probably okay to update.

If many dependencies don't support 3.10, or there is a problem when trying it, we'll have to wait on a patch to 3.8 or 3.9, and we can inform stakeholders that is our plan.

@jonespm
Copy link
Member

jonespm commented Sep 1, 2022

I think if you switch the base image to 3.10-slim

And then login to the container and run pip check it should tell you if everything is good. You'll also need to update the Openshift Docker file. 3.10-slim is available there too. It hopefully should be noticeable if something isn't right.

@lsloan
Copy link
Member Author

lsloan commented Sep 7, 2022

Building with python:3.10-slim produced a whole raft of errors. It occurred in the pip install step using requirements.txt. The problems began with installing numpy. Looks like it had downloaded the version we specify, 1.22, but some other dependency along the way tries to install 1.19.3. After that, we get a message about RuntimeWarning: NumPy 1.19.3 may not yet support Python 3.10..

So, installing the newer Python and running pip check isn't cutting it at the moment. I'll need to step back to examining each of the dependencies, as @ssciolla suggested.

@lsloan
Copy link
Member Author

lsloan commented Sep 7, 2022 •
edited

Looks like I'll need to update requirements.txt to use a newer version of pandas.

And google-cloud-bigquery[pandas], apparently.

@lsloan lsloan mentioned this issue Sep 7, 2022
Merged
@lsloan lsloan moved this from To do to In progress in MyLA-2022.02.01 Sep 7, 2022
@lsloan lsloan moved this from In progress to Review/QA in MyLA-2022.02.01 Sep 7, 2022
@ssciolla ssciolla moved this from Review/QA to In progress in MyLA-2022.02.01 Sep 7, 2022
@lsloan lsloan moved this from In progress to Review/QA in MyLA-2022.02.01 Sep 7, 2022
@zqian zqian moved this from Review/QA to Review/QA - DEV in MyLA-2022.02.01 Sep 9, 2022
@zqian zqian moved this from Review/QA - DEV to In progress in MyLA-2022.02.01 Sep 12, 2022
ssciolla pushed a commit to ssciolla/my-learning-analytics that referenced this issue Sep 14, 2022
@lsloan
tl-its-umich-edu#1406 - spelling correction & clean-up
4f2126a 
Cleaned up trailing spaces on a few lines.
ssciolla pushed a commit to ssciolla/my-learning-analytics that referenced this issue Sep 14, 2022
@lsloan
tl-its-umich-edu#1406 - upgrade to Python 3.10
36c418b 
Resolves vulnerability CVE-2015-20107.
ssciolla pushed a commit to ssciolla/my-learning-analytics that referenced this issue Sep 14, 2022
@lsloan
tl-its-umich-edu#1406 - upgrade pandas for Python 3.10
ce1c90f 
Upgrade `pandas` and related modules to work with Python 3.10.
ssciolla pushed a commit to ssciolla/my-learning-analytics that referenced this issue Sep 14, 2022
@lsloan
tl-its-umich-edu#1406 - fix all warehouse connections
ce1f4d4 
Derive PostgreSQL connect string for data warehouse DB from Django connections, then create a single engine for all queries that will use that DB.
MyLA-2022.02.01 automation moved this from In progress to Review/QA Sep 14, 2022
ssciolla added a commit that referenced this issue Sep 14, 2022
@lsloan @ssciolla @jonespm
upgrade Python to v3.10 (iss. #1406) (#1426)
aa091fc 
* #1406 - spelling correction & clean-up

Cleaned up trailing spaces on a few lines.

* #1406 - upgrade to Python 3.10

Resolves vulnerability CVE-2015-20107.

* #1406 - upgrade `pandas` for Python 3.10

Upgrade `pandas` and related modules to work with Python 3.10.

* #1406 - fix all warehouse connections

Derive PostgreSQL connect string for data warehouse DB from Django connections, then create a single engine for all queries that will use that DB.

* Create utility function for creating mysql and postgres engines; apply to views'

* Remove other database conn prep

* Reuse create_sqlalchemy_engine in data_validation

* Remove unused variable

* Make a couple minor modifications to db_util

* Make one read_sql call one line

* Remove unused import

* Update numpy, pangres; change mypy version

* Remove type parameter, use Django ENGINE

* Reverting change to validate_udw_vs_udp since it already created an
engine

Co-authored-by: Sam Sciolla <ssciolla@umich.edu>
Co-authored-by: Code Hugger (Matthew Jones) <jonespm@umich.edu>
@jennlove-um jennlove-um moved this from Review/QA to Review/QA - DEV in MyLA-2022.02.01 Sep 19, 2022
@pushyamig
Copy link
Contributor

I will QA this

@jennlove-um jennlove-um removed this from Review/QA - DEV in MyLA-2022.02.01 Sep 20, 2022
@jennlove-um jennlove-um added this to To do in MyLA-2022.01.03 via automation Sep 20, 2022
@pushyamig pushyamig moved this from To do to Review/QA - DEV in MyLA-2022.01.03 Sep 20, 2022
@pushyamig
Copy link
Contributor

pushyamig commented Sep 20, 2022 •
edited

Test passes

  1. Updated to Python version 3.10.7, pandas, numypy, pangres, google-cloud-bigquery[pandas] all updated to correct version
  2. cron ran successfully and did not see this warning UserWarning: pandas only support SQLAlchemy connectable(engine/connection) ordatabase string URI or sqlite3 DBAPI2 connectionother DBAPI2 objects are not tested, please consider using SQLAlchemy
  3. All the views load fine

@pushyamig pushyamig moved this from Review/QA - DEV to Done in MyLA-2022.01.03 Sep 20, 2022
jonespm added a commit to jonespm/student-dashboard-django that referenced this issue Sep 20, 2022
@lsloan @ssciolla @jonespm
upgrade Python to v3.10 (iss. tl-its-umich-edu#1406) (tl-its-umich-ed…
8675f95 
…u#1426)

* tl-its-umich-edu#1406 - spelling correction & clean-up

Cleaned up trailing spaces on a few lines.

* tl-its-umich-edu#1406 - upgrade to Python 3.10

Resolves vulnerability CVE-2015-20107.

* tl-its-umich-edu#1406 - upgrade `pandas` for Python 3.10

Upgrade `pandas` and related modules to work with Python 3.10.

* tl-its-umich-edu#1406 - fix all warehouse connections

Derive PostgreSQL connect string for data warehouse DB from Django connections, then create a single engine for all queries that will use that DB.

* Create utility function for creating mysql and postgres engines; apply to views'

* Remove other database conn prep

* Reuse create_sqlalchemy_engine in data_validation

* Remove unused variable

* Make a couple minor modifications to db_util

* Make one read_sql call one line

* Remove unused import

* Update numpy, pangres; change mypy version

* Remove type parameter, use Django ENGINE

* Reverting change to validate_udw_vs_udp since it already created an
engine

Co-authored-by: Sam Sciolla <ssciolla@umich.edu>
Co-authored-by: Code Hugger (Matthew Jones) <jonespm@umich.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lsloan lsloan

Labels
⚠️ priority ☠ vulnerability
Projects
No open projects
MyLA-2022.01.03
  
Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

1406-upgrade-python lsloan/myla
upgrade Python to v3.10 (iss. #1406) lsloan/myla
5 participants
@lsloan @jonespm @pushyamig @ssciolla @jennlove-um