Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve resolution speed of version ranges in remotes using the name as pattern #5433

Merged
merged 3 commits into from Jul 3, 2019

Conversation

@memsharded
Copy link
Member

@memsharded memsharded commented Jul 2, 2019

Changelog: Fix: Improve resolution of version ranges for remotes
Docs: omit

Partially address, in the client side #5432

Separate work must be done in Artifactory

@Mark-Hatch-Bose
Copy link

@Mark-Hatch-Bose Mark-Hatch-Bose commented Jul 2, 2019

I installed this from your branch, am not seeing a difference in search times to Artifactory.
Maybe I'm doing something wrong?

{"_action": "REST_API_CALL", "duration": 3.1186702251434326, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093776.1418612, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=<PACKAGE>%2F%2A%40BoseCorp%2Fbose-stable"}
{"_action": "REST_API_CALL", "duration": 3.0984485149383545, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093779.2458003, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=<PACKAGE>%2F%2A%40BoseCorp%2Fbose-master"}
{"_action": "REST_API_CALL", "duration": 3.133497953414917, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093782.3835235, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.0755865573883057, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093785.4605715, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=<PACKAGE>%2F%2A%40BoseCorp%2Fbose-master"}
{"_action": "REST_API_CALL", "duration": 3.1029679775238037, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093788.5661237, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.113391399383545, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093791.682815, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.3839454650878906, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093795.0717094, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.1916117668151855, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093798.2700784, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.2044949531555176, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093801.4781427, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.16910719871521, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093804.6536212, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.162350654602051, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093807.8195848, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.284928798675537, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093811.1089065, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}

@memsharded
Copy link
Member Author

@memsharded memsharded commented Jul 2, 2019

Uhm, yes, I still see the full package reference in your URLs, instead of the shorter one that should result. No idea why, I guess that you have already checked, but just in case, could you add a couple of prints() to the modified code in the PR in your clone, to make sure it is entering there?

@Mark-Hatch-Bose
Copy link

@Mark-Hatch-Bose Mark-Hatch-Bose commented Jul 2, 2019

I'm a fool, I hadn't checked out your branch after cloning... 💯
I see MUCH improved performance now!

{"_action": "REST_API_CALL", "duration": 0.09672117233276367, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100269.735404, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.11811661720275879, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100269.8555326, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.04911184310913086, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100269.910386, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.05893063545227051, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100269.9704657, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.05200672149658203, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.0241113, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.052623748779296875, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.07776, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.059350013732910156, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.138302, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.05178999900817871, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.1919434, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.05158591270446777, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.24533, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.044409751892089844, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.2912815, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.04635310173034668, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.3388534, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.0608220100402832, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.400658, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.06620454788208008, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.469506, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}

@memsharded
Copy link
Member Author

@memsharded memsharded commented Jul 2, 2019

Ok, thanks for the update @Mark-Hatch-Bose, those are great news!

Conan 1.17 is about to be closed, so it seems almost impossible that it will make it, but lets wait for @lasote feedback.

@lasote lasote added this to the 1.17 milestone Jul 3, 2019
@lasote
Copy link
Contributor

@lasote lasote commented Jul 3, 2019

From 3 secs to 0.1! Wow, such an improvement cannot be delayed to 1.18 ;)

@lasote lasote merged commit db0e7c3 into conan-io:develop Jul 3, 2019
2 checks passed
@memsharded memsharded deleted the feature/faster_version_ranges branch Jul 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants