Add limits for action functions v2#4484
Merged
Merged
Conversation
added 7 commits
September 28, 2018 12:42
…to be a slip so I fixed.
11 tasks
davidread
commented
Sep 28, 2018
|
|
||
| params['rows'] = min( | ||
| int(params.get('rows', 10)), | ||
| int(config.get('ckan.search.rows_max', 1000))) |
Author
There was a problem hiding this comment.
I needed to add limit here, because it bypasses the logic function and therefore the validator (which now does the limiting when calling package_search)
davidread
force-pushed
the
4480-limits-v2
branch
from
e19c3db to
f7f081e
Compare
davidread
changed the title
11 tasks
davidread
pushed a commit
that referenced
this pull request
Oct 12, 2018
davidread
pushed a commit
that referenced
this pull request
Oct 19, 2018
Author
|
@wardi it would be great to get this merged, if you have a chance to review? |
Contributor
|
Well done! This change was sorely needed. |
jguo144
added a commit
to OpenGov-OpenData/ckan
that referenced
this pull request
Jan 9, 2019
* Add download dropdown * [ckan#4497] connect js, route, form * [ckan#4497] implement download redirect * [ckan#4497] unicode literal * use request.params for 2.7 compatibility * [ckan#4462] _get_types: pass connection instead of context * [ckan#4462] _get_type: pass connection instead of context * [ckan#4462] _get_fields: pass connection, resource_id instead of context, data_dict * [ckan#4462] _get_fields_types: pass connection, resource_id instead of context, data_dict * [ckan#4462] datastore_search: partial result fields fix * [ckan#4462] fix reuse of variable name * [ckan#4462] _textsearch_query: lang, q, plain instead of data_dict * [ckan#4462] datastore_search: exclude rank columns if not in fields * [ckan#4462] _result_fields: remove unused parameter * revert string_types change * revert string_types change * Get visible columns * [ckan#4497] download visible columns * Remove _id column from filtered download * [ckan#4462] _textsearch_query: lang, q, plain instead of data_dict * [ckan#4497] s/const/var for js compatibility * [ckan#4484] manually backported to ckan 2.7.x (#8) Add limits for action functions * Add limits to datastore_search(_sql) (#16) * Backport of 4561-limit-datastore_search 74d8fb2 * CircleCI update from master * Nop * Fix tests for opengov branch The _id column is removed from datastore dumps in this opengov branch because: opengov commit: co0adf16426113394af42bbacc4644f039ebe9e2ec + field_list = [f['id'] for f in rec['fields'] if f['id'] != '_id'] also some minor fixes * Use pop() for efficiency
jguo144
added a commit
to OpenGov-OpenData/ckan
that referenced
this pull request
Jan 9, 2019
* Add download dropdown * [ckan#4497] connect js, route, form * [ckan#4497] implement download redirect * [ckan#4497] unicode literal * use request.params for 2.7 compatibility * [ckan#4462] _get_types: pass connection instead of context * [ckan#4462] _get_type: pass connection instead of context * [ckan#4462] _get_fields: pass connection, resource_id instead of context, data_dict * [ckan#4462] _get_fields_types: pass connection, resource_id instead of context, data_dict * [ckan#4462] datastore_search: partial result fields fix * [ckan#4462] fix reuse of variable name * [ckan#4462] _textsearch_query: lang, q, plain instead of data_dict * [ckan#4462] datastore_search: exclude rank columns if not in fields * [ckan#4462] _result_fields: remove unused parameter * revert string_types change * revert string_types change * Get visible columns * [ckan#4497] download visible columns * Remove _id column from filtered download * [ckan#4462] _textsearch_query: lang, q, plain instead of data_dict * [ckan#4497] s/const/var for js compatibility * [ckan#4484] manually backported to ckan 2.7.x (#8) Add limits for action functions * Add limits to datastore_search(_sql) (#16) * Backport of 4561-limit-datastore_search 74d8fb2 * CircleCI update from master * Nop * Fix tests for opengov branch The _id column is removed from datastore dumps in this opengov branch because: opengov commit: co0adf16426113394af42bbacc4644f039ebe9e2ec + field_list = [f['id'] for f in rec['fields'] if f['id'] != '_id'] also some minor fixes * Use pop() for efficiency
Author
|
@ThrawnCA I have sympathy for this point of view - it does break functionality. However we were keen to add it as a patch for earlier versions, because it was seen as a DOS vector. The default limits are so low, as to try and make it obvious they are limited. |
Contributor
That's a nice thought, but since it still truncates silently, and still allows enough to eg populate a dropdown list enough to need significant scrolling down, it really isn't obvious unless someone was already specifically looking for it. |
Author
|
Agreed, it's problematic for the use case you describe. We had some awareness of this sort of thing when we designed it, balanced against the other concerns mentioned. The questions are whether that balance is wrong and what might be done now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4480
Limits added
This PR adds limits to the following logic functions, configurable with the given configuration option:
ckan.search.rows_maxckan.search.rows_maxckan.group_and_organization_list_max/ckan.group_and_organization_list_all_fields_maxckan.activity_list_limit_maxckan.activity_list_limit_maxckan.activity_list_limit_maxI've also done a backport for CKAN 2.7, should anyone want to apply it to their own ckan fork:
2.7...4484-limits-backport-2.7
Features:
Please [X] all the boxes above that apply