-
Notifications
You must be signed in to change notification settings - Fork 100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade to CKAN 2.10 #4209
Labels
Comments
Tenative sketch:
|
This was referenced Apr 10, 2023
This was referenced Apr 17, 2023
Merged
General notes on CKAN 2.10 Upgrade:
|
This was referenced Apr 18, 2023
This was referenced Apr 20, 2023
nickumia-reisys
added a commit
to GSA/catalog.data.gov
that referenced
this issue
May 1, 2023
OWSLib was updates so no need for the ignore. Werkzeug is still being worked on with GSA/data.gov#4209, so give us more time
This was referenced May 1, 2023
Current status:
|
This was referenced May 8, 2023
This was referenced May 18, 2023
Current issue being investigated with the ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/plugin.py", line 199, in after_dataset_show
ckan_1 | source = HarvestSource.get(data_dict["id"])
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/model/__init__.py", line 121, in get
ckan_1 | o = cls.filter(**kwds).first()
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/model/__init__.py", line 130, in filter
ckan_1 | return query.filter_by(**kwds)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 1806, in filter_by
ckan_1 | clauses = [
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 1807, in <listcomp>
ckan_1 | _entity_namespace_key(from_entity, key) == value
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/operators.py", line 366, in __eq__
ckan_1 | return self.operate(eq, other)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 317, in operate
ckan_1 | return op(self.comparator, *other, **kwargs)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/operators.py", line 366, in __eq__
ckan_1 | return self.operate(eq, other)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/properties.py", line 423, in operate
ckan_1 | return op(self.__clause_element__(), *other, **kwargs)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/annotation.py", line 221, in __eq__
ckan_1 | return self.__element.__class__.__eq__(self, other)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/operators.py", line 366, in __eq__
ckan_1 | return self.operate(eq, other)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 869, in operate
ckan_1 | return op(self.comparator, *other, **kwargs)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/operators.py", line 366, in __eq__
ckan_1 | return self.operate(eq, other)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 77, in operate
ckan_1 | return o[0](self.expr, op, *(other + o[1:]), **kwargs)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/default_comparator.py", line 101, in _boolean_compare
ckan_1 | obj = coercions.expect(
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/coercions.py", line 188, in expect
ckan_1 | resolved = impl._literal_coercion(
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/coercions.py", line 523, in _literal_coercion
ckan_1 | return expr._bind_param(operator, element, type_=bindparam_type)
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 4708, in _bind_param
ckan_1 | return BindParameter(
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 1512, in __init__
ckan_1 | self.key = _anonymous_label.safe_construct(
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 5375, in safe_construct
ckan_1 | body = re.sub(r"[%\(\) \$]+", "_", body).strip("_")
ckan_1 | File "/usr/local/lib/python3.8/re.py", line 210, in sub
ckan_1 | return _compile(pattern, flags).sub(repl, string, count)
ckan_1 | File "/usr/local/lib/python3.8/re.py", line 291, in _compile
ckan_1 | if isinstance(flags, RegexFlag):
ckan_1 | RecursionError: maximum recursion depth exceeded while calling a Python object
ckan_1 | Fatal Python error: Cannot recover from stack overflow.
ckan_1 | Python runtime state: initialized
ckan_1 |
ckan_1 | Current thread 0x00007f5be02ec740 (most recent call first):
ckan_1 | File "/usr/local/lib/python3.8/uuid.py", line 782 in uuid4
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/types.py", line 17 in make_uuid
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 2776 in <lambda>
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1735 in _exec_default
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1871 in get_insert_default
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/base.py", line 3226 in get_insert_default
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1927 in _process_executesingle_defaults
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1025 in _init_compiled
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1800 in _execute_context
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572 in _execute_clauseelement
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 333 in _execute_on_connection
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705 in _execute_20
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1238 in _emit_insert_statements
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 245 in save_obj
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630 in execute
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456 in execute
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3486 in _flush
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3386 in flush
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 28 in notify_observers
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 25 in before_commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/meta.py", line 70 in ckan_before_commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 247 in __call__
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 797 in _prepare_impl
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1451 in commit
ckan_1 | File "<string>", line 2 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/base.py", line 152 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/base.py", line 136 in save
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/log.py", line 15 in emit
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 954 in handle
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1661 in callHandlers
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1599 in handle
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1589 in _log
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1475 in error
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/plugin.py", line 201 in after_dataset_show
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/logic/action/get.py", line 1090 in package_show
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/logic/__init__.py", line 551 in wrapped
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/lib/search/__init__.py", line 167 in notify
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 71 in notify
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 39 in notify_observers
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 25 in before_commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/meta.py", line 70 in ckan_before_commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 247 in __call__
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 797 in _prepare_impl
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1451 in commit
ckan_1 | File "<string>", line 2 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/base.py", line 152 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/base.py", line 136 in save
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/log.py", line 15 in emit
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 954 in handle
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1661 in callHandlers
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1599 in handle
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1589 in _log
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1475 in error
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/plugin.py", line 201 in after_dataset_show
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/logic/action/get.py", line 1090 in package_show
db_1 | 2023-06-01 12:34:19.208 UTC [226] LOG: unexpected EOF on client connection with an open transaction
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/logic/__init__.py", line 551 in wrapped
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/lib/search/__init__.py", line 167 in notify
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 71 in notify
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 39 in notify_observers
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 25 in before_commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/meta.py", line 70 in ckan_before_commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 247 in __call__
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 797 in _prepare_impl
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1451 in commit
ckan_1 | File "<string>", line 2 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/base.py", line 152 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/base.py", line 136 in save
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/log.py", line 15 in emit
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 954 in handle
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1661 in callHandlers
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1599 in handle
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1589 in _log
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1475 in error
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/plugin.py", line 201 in after_dataset_show
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/logic/action/get.py", line 1090 in package_show
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/logic/__init__.py", line 551 in wrapped
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/lib/search/__init__.py", line 167 in notify
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 71 in notify
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 65 in notify_observers
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/modification.py", line 25 in before_commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/meta.py", line 70 in ckan_before_commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 247 in __call__
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 797 in _prepare_impl
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1451 in commit
ckan_1 | File "<string>", line 2 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/base.py", line 152 in commit
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/model/base.py", line 136 in save
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/log.py", line 15 in emit
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 954 in handle
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1661 in callHandlers
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1599 in handle
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1589 in _log
ckan_1 | File "/usr/local/lib/python3.8/logging/__init__.py", line 1475 in error
ckan_1 | File "/src/ckanext-harvest/ckanext/harvest/plugin.py", line 201 in after_dataset_show
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/logic/action/get.py", line 1090 in package_show
ckan_1 | File "/usr/local/lib/python3.8/site-packages/ckan/logic/__init__.py", line 551 in wrapped
ckan_1 | ...
ckan_1 | [2023-06-01 12:34:19 +0000] [1] [WARNING] Worker with pid 496 was terminated due to signal 6 |
This was referenced Jun 5, 2023
4 tasks
This was referenced Jun 27, 2023
Catalog and Inventory successfully running CKAN 2.10 now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Turning this into an epic
User Story
In order to utilize latest CKAN patches, data.gov admins want CKAN updated to 2.10.
Acceptance Criteria
[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]
WHEN I go to https://catalog.data.gov/api/action/status_show
THEN ckan version shows 2.10.0 (or equivalent)
Background
https://github.com/GSA/catalog.data.gov/blob/main/ckan/requirements.txt#L11
#4113
See email on CKAN mailing list:
Security Considerations (required)
None
Sketch
This will be a major lift, and will require picking up the work again here: #4113
Ideally every extension should test against 2.10, and verify things work. If they don't work, a new issue may be created to investigate further.
Final step will be upgrading catalog, and may require docker changes (among others).
The text was updated successfully, but these errors were encountered: