Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve resolution speed of version ranges in remotes using the name as pattern #5433

Merged
merged 3 commits into from Jul 3, 2019

Conversation

Projects
None yet
3 participants
@memsharded
Copy link
Contributor

commented Jul 2, 2019

Changelog: Fix: Improve resolution of version ranges for remotes
Docs: omit

Partially address, in the client side #5432

Separate work must be done in Artifactory

@Mark-Hatch-Bose

This comment has been minimized.

Copy link

commented Jul 2, 2019

I installed this from your branch, am not seeing a difference in search times to Artifactory.
Maybe I'm doing something wrong?

{"_action": "REST_API_CALL", "duration": 3.1186702251434326, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093776.1418612, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=<PACKAGE>%2F%2A%40BoseCorp%2Fbose-stable"}
{"_action": "REST_API_CALL", "duration": 3.0984485149383545, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093779.2458003, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=<PACKAGE>%2F%2A%40BoseCorp%2Fbose-master"}
{"_action": "REST_API_CALL", "duration": 3.133497953414917, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093782.3835235, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.0755865573883057, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093785.4605715, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=<PACKAGE>%2F%2A%40BoseCorp%2Fbose-master"}
{"_action": "REST_API_CALL", "duration": 3.1029679775238037, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093788.5661237, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.113391399383545, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093791.682815, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.3839454650878906, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093795.0717094, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.1916117668151855, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093798.2700784, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.2044949531555176, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093801.4781427, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.16910719871521, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093804.6536212, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.162350654602051, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093807.8195848, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
{"_action": "REST_API_CALL", "duration": 3.284928798675537, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.20.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562093811.1089065, "url": "<REMOTE>/artifactory/api/conan/barbarian/v2/conans/search?ignorecase=False&q=><PACKAGE>%2F%2A%40BoseCorp%2Fmaster"}
@memsharded

This comment has been minimized.

Copy link
Contributor Author

commented Jul 2, 2019

Uhm, yes, I still see the full package reference in your URLs, instead of the shorter one that should result. No idea why, I guess that you have already checked, but just in case, could you add a couple of prints() to the modified code in the PR in your clone, to make sure it is entering there?

@Mark-Hatch-Bose

This comment has been minimized.

Copy link

commented Jul 2, 2019

I'm a fool, I hadn't checked out your branch after cloning... 💯
I see MUCH improved performance now!

{"_action": "REST_API_CALL", "duration": 0.09672117233276367, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100269.735404, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.11811661720275879, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100269.8555326, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.04911184310913086, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100269.910386, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.05893063545227051, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100269.9704657, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.05200672149658203, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.0241113, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.052623748779296875, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.07776, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.059350013732910156, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.138302, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.05178999900817871, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.1919434, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.05158591270446777, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.24533, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.044409751892089844, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.2912815, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>&ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.04635310173034668, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.3388534, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.0608220100402832, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.400658, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
{"_action": "REST_API_CALL", "duration": 0.06620454788208008, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.17.0-dev (Python 3.5.2) python-requests/2.21.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": "<USER>"}, "method": "GET", "time": 1562100270.469506, "url": "<URL_REDACTED>/artifactory/api/conan/barbarian/v2/conans/search?q=<PACKAGE>ignorecase=False"}
@memsharded

This comment has been minimized.

Copy link
Contributor Author

commented Jul 2, 2019

Ok, thanks for the update @Mark-Hatch-Bose, those are great news!

Conan 1.17 is about to be closed, so it seems almost impossible that it will make it, but lets wait for @lasote feedback.

@lasote lasote added this to the 1.17 milestone Jul 3, 2019

@lasote

This comment has been minimized.

Copy link
Contributor

commented Jul 3, 2019

From 3 secs to 0.1! Wow, such an improvement cannot be delayed to 1.18 ;)

@lasote lasote merged commit db0e7c3 into conan-io:develop Jul 3, 2019

2 checks passed

continuous-integration/jenkins/pr-head This commit looks good
Details
license/cla Contributor License Agreement is signed.
Details

@memsharded memsharded deleted the memsharded:feature/faster_version_ranges branch Jul 3, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.