Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datastore API Demo modal window encourages robots to make needless queries #3076

Closed
gjlawran opened this issue May 31, 2016 · 0 comments · Fixed by #3261
Closed

Datastore API Demo modal window encourages robots to make needless queries #3076

gjlawran opened this issue May 31, 2016 · 0 comments · Fixed by #3261
Assignees

Comments

@gjlawran
Copy link

CKAN Version if known (or site URL)

2.1 ---> 2.5.2 http://demo.ckan.org/dataset/gold-prices/resource/b9aae52b-b082-4159-b46f-7bb9c158d013

Please describe the expected behaviour

the CKAN Data API modal window designed to demonstrate the syntax for using the datastore API should not encourage robots to make queries against the database

Please describe the actual behaviour

the modal window includes hyperlinked text for all resources ...

Query example (first 5 results)
http://demo.ckan.org/api/action/datastore_search?resource_id=b9aae52b-b082-4159-b46f-7bb9c158d013&limit=5
Query example (results containing 'jones')
http://demo.ckan.org/api/action/datastore_search?resource_id=b9aae52b-b082-4159-b46f-7bb9c158d013&q=jones
Query example (via SQL statement)
http://demo.ckan.org/api/action/datastore_search_sql?sql=SELECT * from "b9aae52b-b082-4159-b46f-7bb9c158d013" WHERE title LIKE 'jones'

These hyperlinks encourage robots - that do not follow robots.txt instructions - to conduct needless queries that result in db errors against all the data resources.

What steps can be taken to reproduce the issue?

The modal window page template should be redone to remove the hyperlinks.

@amercader amercader self-assigned this Jun 7, 2016
jrods added a commit to HighwayThree/ckan that referenced this issue Aug 29, 2016
Issue ckan#3076 - Help to prevent robots that don't listen to robot.txt from
calling needless querys on api links
jrods added a commit to HighwayThree/ckan that referenced this issue Sep 29, 2016
Issue ckan#3076 - Help to prevent robots that don't listen to robot.txt from
calling needless querys on api links
jrods added a commit to HighwayThree/ckan that referenced this issue Sep 29, 2016
Issue ckan#3076 - Help to prevent robots that don't listen to robot.txt from
calling needless querys on api links
wardi added a commit that referenced this issue Oct 3, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants