-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Traceback when publishing amazon.aws
#52
Comments
I'm hacking on this a bit locally, manually raised a RuntimeError: {'collection_info': '{"namespace": "amazon", "name": "aws", "version": "5.2.0", "authors": ["Ansible (https://github.com/ansible)"], "readme": "README.md", "tags": ["amazon", "aws", "cloud"], "description": null, "license": [], "license_file": "COPYING", "dependencies": {}, "repository": "https://github.com/ansible-collections/amazon.aws", "documentation": "https://ansible-collections.github.io/amazon.aws/branch/stable-5/collections/amazon/aws/index.html", "homepage": "https://github.com/ansible-collections/amazon.aws", "issues": "https://github.com/ansible-collections/amazon.aws/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc"}', 'namespace': 'amazon', 'name': 'aws', 'version': '5.2.0', 'fqcn': 'amazon.aws'} |
Hi @mamercad ! This happens during publish? The top of the traceback shows a
And the rest of it shows reads (it's a JSON decoding issue that seems to have raised the exception in the first place. So I'm wondering if there's a longer trace that might show more information on the sequence of events. Also could you show me the snippet you added that shows the From this comment, are you seeing that only When looking at it in Artifactory, is it complete/valid JSON? Also wondering if you still have that property fallback option enabled, or set it back to doing uploads in one shot. |
...the "missing properties" might be a red herring; I've been digging all day and what I'm seeing are sporadic Artifactory timeouts. |
Still researching, here's a bug I found upstream which may be related. |
No mention of property constraints here. |
Yup, it's truncated: ❯ curl -s -X GET -H "X-JFrog-Art-Api: ${ARTIFACTORY_API_KEY}" "https://REDACTED/artifactory/api/storage/ansible-dev-local/amazon-aws-5.2.0.tar.gz?properties" | jq -r '.properties.collection_info[0]'
{"namespace": "amazon", "name": "aws", "version": "5.2.0", "authors": ["Ansible (https://github.com/ansible)"], "readme": "README.md", "tags": ["amazon", "aws", "cloud"], "description": null, "license": [], "license_file": "COPYING", "dependencies": {}, "repository": "https://github.com/ansible-collections/amazon.aws", "documentation": "https://ansible-collections.github.io/amazon.aws/branch/stable-5/collections/amazon/aws/index.html", "homepage": "https://github.com/ansible-collections/amazon.aws", "issues": "https://github.com/ansible-collections/amazon.aws/issues Versus: ❯ curl -s -X GET -H "X-JFrog-Art-Api: ${ARTIFACTORY_API_KEY}" "https://REDACTED/artifactory/api/storage/ansible-dev-local/cloud-common-2.1.1.tar.gz?properties" | jq -r '.properties.collection_info[0]' | jq
{
"namespace": "cloud",
"name": "common",
"version": "2.1.1",
"authors": [
"Ansible (https://github.com/ansible)"
],
"readme": "README.md",
"tags": [
"cloud",
"virtualization"
],
"description": "Set of common files for the cloud collections",
"license": [],
"license_file": "LICENSE",
"dependencies": {},
"repository": "https://github.com/ansible-collections/cloud.common",
"documentation": null,
"homepage": "https://github.com/ansible-collections/cloud.common",
"issues": "https://github.com/ansible-collections/cloud.common/issues"
} |
I see! So the truncation is definitely why it fails on JSON decoding, causing the stacktrace. Nice find on this issue, it says:
that part is crazy to me! Are you able to confirm with JFrog if your instance/DB type has a limit and what that limit is? Also since the properties are actually arrays of strings (IIRC) and not just single strings, I am wondering if the limit is for individual strings, or for the combined length, or something else. I did search for limits previously and as you found in the docs, they didn't mention any. But this might lead me to have to change the format, and separate some of the info out into separate properties, maybe as an optional configuration. So maybe we'll turn this into a feature request and I'll look at implementing a different method for storing the info. |
I'll ask the internal team ... |
Yeah, what I'm wondering is (it's in the draft PR comments), what are the implications of not having a complete I suppose that each field of |
Another thing that I'm still pondering is, how the heck did a truncated |
Oh, another thing ... I did check with the internal team regarding my permissions: I only had read and create (they've since granted me update and delete). Also, it turns out there's a "treat 403s as 404s setting to prevent fuzzing" setting (so, the 404s weren't really true, I was actually getting 403s). |
My guess is that it's artifactory that truncates it on write. The load that triggers the failure I think comes from galactory reading it back.
Each field could be separate; those other separate ones were the only ones originally. I opted to go (what I thought was) a simpler route by sticking the whole collection_info into its own property. The way it works right now, only So we could separate out more fields from Knowing what the real limits are in Artifactory (or your configuration at least, for a single data point), will help me determine which direction to go. |
@mamercad @jcox10 a fix for this in the artifactory library has been merged, waiting on a release, but I think we'll finally be able to get some closure on these issues: Once there's a release, I will ensure galactory uses it properly (the behavior is opt-in for now), and probably set the project's minimum version of the library to this new one, stay tuned! |
When trying to publish
amazon.aws:==5.2.0
, I'm seeing:The text was updated successfully, but these errors were encountered: