Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update flask sqlalchemy #158

Merged
merged 10 commits into from
Mar 17, 2023

Conversation

utnapischtim
Copy link
Contributor

@utnapischtim utnapischtim commented Dec 18, 2022

  • setup: increase flask-sqlalchemy version
  • fix: tests LocalProxy
  • change: remove click 3 compatibility
  • change: add proxy file

NOTE: the bugfix has to be discussed before merging!

if we use the change then following two lines have to be updated:
invenio_oauth2server.utils
invenio_oauthclient.utils

@utnapischtim utnapischtim force-pushed the update-flask-sqlalchemy branch 2 times, most recently from 8cee2ed to 8968dc7 Compare December 18, 2022 21:38
* it is not working anymore with localproxy and it seams not necessary
* use the proxy file instead of creating the _db variable in both files

* this removes a DeprecationWarning from flask-sqlalchemy, which
  states that the support of the use '.db' will be removed
* the error indicates that db.init_app is not called, but it is and it
  works in other contexts. It may that the `db = invenio_db = shared.db`
  line in conftest.py is not working as expected anymore. The only found
  solution is this. The drawback is, if that is the only solution it has
  to be done also in the other packages which are using this function.
  With the default value it is backward compatible but as the tests
  indicates it may not work without adding the db parameter to the
  function call
@ntarocco ntarocco self-assigned this Dec 19, 2022
Copy link
Contributor

@ntarocco ntarocco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this! We should carefully test an InvenioRDM app and upgrades to make sure that any of the changes are not affecting an Invenio app.

setup.cfg Show resolved Hide resolved
invenio_db/utils.py Outdated Show resolved Hide resolved
invenio_db/utils.py Outdated Show resolved Hide resolved
invenio_db/utils.py Outdated Show resolved Hide resolved
@ntarocco ntarocco assigned utnapischtim and unassigned ntarocco Dec 20, 2022
* added D202, pydocstyle and black do not habe the same opinion

* remove warning by using StringEncryptedType instead of EncryptedType.
  This removes a warning and there is further a notice in the code that
  the base type of EncryptedType will change in the future and it is
  better to replace it with StringEncryptedType
* the problem is, that assert calls an function which leads to an not
  moving forward test on mysql8. it works on postgresql and mysql5 but not
  mysql8.
@utnapischtim
Copy link
Contributor Author

utnapischtim commented Feb 25, 2023

@ntarocco How should we proceed with this?

The change of the signature of the rebuild_encrypted_properties function is backward compatible but it may be necessary to update the two usages in invenio-oauth2server and invenio-oauthclient

setup.cfg Show resolved Hide resolved
@@ -33,7 +33,8 @@ class Demo(db.Model):
__tablename__ = "demo"
pk = db.Column(sa.Integer, primary_key=True)
et = db.Column(
EncryptedType(type_in=db.Unicode, key=_secret_key), nullable=False
StringEncryptedType(length=255, type_in=db.Unicode, key=_secret_key),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should not be an issue, but I cannot find where the length params comes from in StringEncryptedType. Is it necessary given that it was not there earlier?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is VARCHAR on mysql. Otherwise the tests are not working. Which leads me to a question: how long do we support mysql?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we should support MySQL if it does not create great issues, otherwise we might need an ORM layer at all :) There could be organization that are required to use MySQL for example.

invenio_db/cli.py Outdated Show resolved Hide resolved
Comment on lines 15 to 16
from .proxies import current_db
from .shared import db as _db
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this was changed in the wrong way?

The previous db should be the same, no change. The previous _db, should be the current_db:

Suggested change
from .proxies import current_db
from .shared import db as _db
from .proxies import current_db as _db
from .shared import db

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here i am a little bit confused, because the from .shared import db as _db was your suggestion. The drop_alembic_version_table function uses current_db. If we change current_db to _db we would have to change that too in the drop_alembic_version_table function. Further, as i understand the two usages should not be the same because in rebuild_encrypted_properties the db imported from shared.py was used and not the _db from the LocalProxy

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sorry if I made the wrong suggestion.
What I noticed now is that we have switched the imports, which might lead to issues.
Nothing should be changed, given that the imports will be exactly the same as before the changes. The only needed change in this file is the extra param to rebuild_encrypted_properties

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, but the new parameter has to come from .shared not from .proxies. they are not the same. but i will investigate a little bit more into that problem.

@ntarocco
Copy link
Contributor

ntarocco commented Mar 7, 2023

@ntarocco How should we proceed with this?

The change of the signature of the rebuild_encrypted_properties function is backward compatible but it may be necessary to update the two usages in invenio-oauth2server and invenio-oauthclient

It looks like that there are tests there, so we can run the tests and see if they pass. If there is a change in the future, the tests should not pass.

@fenekku
Copy link
Contributor

fenekku commented Mar 8, 2023

Coordination:

This looks like it can be merged: tests are passing and maintainer has approved. I don't know that another outside reviewer could review this well.

If @ntarocco wants to merge, go ahead.

* the new name better reflects the intended use
@ntarocco
Copy link
Contributor

@fenekku thanks, we are still chatting offline on these changes. I would also test these changes before merging. Almost ready :)

@ntarocco ntarocco merged commit 494bef9 into inveniosoftware:master Mar 17, 2023
@utnapischtim utnapischtim deleted the update-flask-sqlalchemy branch March 24, 2023 07:09
kpsherva added a commit to kpsherva/invenio-db that referenced this pull request May 22, 2023
kpsherva added a commit that referenced this pull request May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Released | Done 🚀
Development

Successfully merging this pull request may close these issues.

None yet

3 participants