Skip to content

Add possibility to parameterize S3 service URL#138

Merged
pjbull merged 11 commits into
masterfrom
custom-s3-endpoint
Apr 3, 2021
Merged

Add possibility to parameterize S3 service URL#138
pjbull merged 11 commits into
masterfrom
custom-s3-endpoint

Conversation

@pjbull

@pjbull pjbull commented Mar 23, 2021

Copy link
Copy Markdown
Member

#137 added ability to specify the endpoint url for S3Path

That work is accepted, and we need to now test if the configuration of the live tests for that scenario work from a non-fork branch.

* Add possibility to parameterize S3 service URL

* add usage description into the docs

* add rig for custom S3 servers

* address PR review: add cods to rig, allow mock

* forgot to commit tests/conftest.py :)
@pjbull pjbull changed the title Add possibility to parameterize S3 service URL (#137) Add possibility to parameterize S3 service URL Mar 23, 2021
@github-actions

github-actions Bot commented Mar 23, 2021

Copy link
Copy Markdown
Contributor

@codecov

codecov Bot commented Mar 23, 2021

Copy link
Copy Markdown

Codecov Report

Merging #138 (b1adaca) into master (80b596f) will not change coverage.
The diff coverage is 100.0%.

@@          Coverage Diff           @@
##           master    #138   +/-   ##
======================================
  Coverage    93.5%   93.5%           
======================================
  Files          21      21           
  Lines        1119    1119           
======================================
  Hits         1047    1047           
  Misses         72      72           
Impacted Files Coverage Δ
cloudpathlib/s3/s3client.py 92.8% <100.0%> (ø)
cloudpathlib/anypath.py 100.0% <0.0%> (ø)

@jayqi jayqi linked an issue Mar 26, 2021 that may be closed by this pull request

@jayqi jayqi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests pass and I think this looks good overall, but I think there are some changes we can make. @pjbull do you want to fix this or would you like me to?

Comment thread docs/docs/authentication.md
Comment thread tests/conftest.py Outdated
@pjbull

pjbull commented Mar 26, 2021

Copy link
Copy Markdown
Member Author

@jayqi FYI, the tests don't actually pass. (The rig wasn't added to the list and so they weren't run).

There are some failing tests that may actually be MinIO/S3 incompatibility, but I haven't gotten to the bottom of them yet.

@pjbull

pjbull commented Mar 26, 2021

Copy link
Copy Markdown
Member Author

Also, changes you suggest make sense, feel free to make those and look at the failing tests. I'll check with you next time I have a chance to dig in to see if you uncovered anything

Comment thread .github/workflows/tests-live.yml Outdated
@jayqi jayqi self-requested a review April 2, 2021 20:08
@jayqi

jayqi commented Apr 2, 2021

Copy link
Copy Markdown
Member

@pjbull I think this looks ready now if you have time to take a look.

Comment thread tests/test_client.py
if not getattr(rig, "is_custom_s3", False):
# Skip resetting the default client for custom S3 endpoint, but keep the other tests,
# since they're still useful.
rig.client_class._default_client = None

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏 I hadn't figured out why this one was failing!

@pjbull pjbull merged commit de6b547 into master Apr 3, 2021
@pjbull pjbull deleted the custom-s3-endpoint branch April 3, 2021 01:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

How to use cloudpathlib for accessing custom S3 installations?

3 participants