Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ldap auth method - add missing configure params by vault api names #975

Merged
merged 28 commits into from Aug 14, 2023

Conversation

ceesios
Copy link
Contributor

@ceesios ceesios commented Apr 12, 2023

Deprecation notice from the maintainers

Several parameters on this method have been normalized to match their API names, for example user_dn is now userdn, group_filter is now groupfilter, etc.

For backwards compatibility, the old names still work, however they will be removed in v3.0.0 and if you use them from v1.2.0 you will receive a deprecation warning. Please update your calls to use the new names. Setting these parameters positionally should be unaffected in this case, but we recommend updating to use named parameters.


Would fix #974

Please let me know if this is the way to go or if i'm missing something.

@ceesios ceesios requested a review from a team as a code owner April 12, 2023 08:49
@briantist briantist self-assigned this Apr 12, 2023
@briantist briantist added auth methods generally related to a Vault auth method breaking-change ldap ldap auth method labels Apr 12, 2023
@briantist
Copy link
Contributor

Hi @ceesios ! Thanks for opening the issue and putting up a change!

As written, there are some duplicated dictionary keys.

But the bigger issue is that this would change the order of parameters, which will break anyone passing in unnamed arguments. We could perhaps put them at the end, but I want to be cautious about what we're doing here.

I'd also want to see tests added to ensure coverage for the new parameters.

@briantist briantist changed the title add missing params by vault api names ldap auth method - add missing configure params by vault api names Apr 12, 2023
@codecov
Copy link

codecov bot commented Apr 12, 2023

Codecov Report

Merging #975 (3b9ecdf) into main (b52ed19) will increase coverage by 0.21%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #975      +/-   ##
==========================================
+ Coverage   84.78%   85.00%   +0.21%     
==========================================
  Files          65       65              
  Lines        3083     3127      +44     
==========================================
+ Hits         2614     2658      +44     
  Misses        469      469              
Files Changed Coverage Δ
hvac/api/auth_methods/ldap.py 100.00% <100.00%> (ø)
hvac/utils.py 84.25% <100.00%> (+5.99%) ⬆️

@ceesios
Copy link
Contributor Author

ceesios commented Apr 12, 2023

@briantist thank you for the review. I've removed the duplicates and moved the parameters to the end. I will look into the test coverage tomorrow. I have no experience with it but look forward to learning about it.

@briantist

This comment was marked as outdated.

@ceesios
Copy link
Contributor Author

ceesios commented Apr 20, 2023

@briantist i've added some unit and integration tests. Is this as expected?

@ceesios

This comment was marked as outdated.

@briantist briantist force-pushed the develop branch 3 times, most recently from 085b858 to 9dbfa98 Compare May 10, 2023 06:41
@briantist

This comment was marked as outdated.

@briantist

This comment was marked as outdated.

@briantist

This comment was marked as outdated.

@ceesios ceesios force-pushed the ldap-refactor branch 2 times, most recently from 91de86a to d34abda Compare May 11, 2023 06:33
@briantist

This comment was marked as outdated.

@ceesios

This comment was marked as outdated.

@ceesios
Copy link
Contributor Author

ceesios commented May 11, 2023

I still can't get the test to fail locally, however the error seems to be related to the userfilter trying to parse non-existent variables. Should be fixed by using a raw string.

@briantist
Copy link
Contributor

I still can't get the test to fail locally, however the error seems to be related to the userfilter trying to parse non-existent variables. Should be fixed by using a raw string.

Ok, I still haven't had much of a chance to review the actual content on this PR, I've been so busy with getting the CI and release process fixed, and with external things. It might still be some time before I can properly review it, thanks for your patience.

@ceesios
Copy link
Contributor Author

ceesios commented May 16, 2023

No problem,
I've removed the userfilter from the test. I can't seem to find a way to get it to fail locally, and in the CI the syntax keeps failing.

@briantist
Copy link
Contributor

My guess is that the test failed for a specific (older) Vault version. You may not have been able to reproduce locally because your Vault version is new enough. Removing the test to make it pass means we would have a blind spot in that case.

It should probably be restored and we should figure out why, that way we can figure out what we need to add to the code to account for that in older Vault versions.

Using a container for your local Vault could help you reproduce it easily.

@briantist briantist changed the base branch from develop to main June 17, 2023 20:55
@briantist

This comment was marked as outdated.

@briantist briantist marked this pull request as ready for review July 9, 2023 20:53
@briantist briantist added deprecation Deprecates something, to be removed or changed in a later release and removed breaking-change labels Jul 9, 2023
@briantist
Copy link
Contributor

Ok, so here's what I've come up with.

  • I've added a general utility in the library that works as a decorator around a method, to declare one or more aliases for parameters.
    • It can optionally mark the aliases as deprecated
    • It can raise exceptions when a value is supplied via more than one name
    • It can distinguish between None passed explicitly vs. being assigned as a default on an optional parameter (important for duplicate detection and deprecation warnings)
    • It can handle positional arguments for the canonical name (the name we want to keep), but not for alias names. This one is pretty important since almost all of our existing arguments in the library can be specified positionally.
    • The aliases don't need to be added to the function declaration, but they should be added to the docstring (which was already done here).
    • It has full coverage in its own set of tests, covering all the options.
  • With that in place, we could solve declaration of the aliases, deprecation notices only on use, and value resolution by adding the decorators to the method.
    • Since we only ever need to worry about the canonical name of the parameter within the function, we could also replace the old (now aliased) names with the new (canonical) names, by position, which combined with the decorator's positional support, means all positional uses of the parameters will continue to work smoothly.
  • Updated the docstrings on the aliases to mention that they are aliases and will be removed in v3.0.0.
  • [Minor] I added * before the brand new parameters, so that they will always need to be supplied by keyword. This is something I need to write up more generally as a recommendation but I think we should go through the library and eventually make this the case for everything.
  • [Minor, Internals] Integration tests were set to suppress all deprecation warnings, which took me way longer to figure out than I'd like to admit.
    • I've disabled that completely: if we're intentionally calling something deprecated in tests, then we should also be asserting the warning, which will capture it.
    • Disabling these as revealed some actual internal deprecated stuff that we'll need to address at some point, so, win-win.

@ceesios Since this is something of a substantial change, I will look to get review from someone else on the project for my code, and I'd also appreciate if you can pull down and test the changes and ensure they still work for you as expected.

@briantist briantist added the enhancement a new feature or addition label Jul 9, 2023
@ceesios
Copy link
Contributor Author

ceesios commented Jul 11, 2023

I've been working on another project at work, sorry for the silence the past weeks.

This looks like a significant improvement. I've tested all options against vault 1.11.2 manually and all work as expected. Today and tomorrow i will be in a lot of meetings. I hope to review in more detail this Thursday.

@ceesios
Copy link
Contributor Author

ceesios commented Jul 13, 2023

certificates is defined twice in remove_nones.
I suggest ordering the params in remove_nones alphabetically to make it more readable and prevent this.

There are some more params missing;
Client TLS was always present. I overlooked them becouse they are not returend by a GET just like bindpass.

  • client_tls_cert
  • client_tls_key

added in v1.11.x

  • connection_timeout
  • max_page_size

added in v1.14.x

  • dereference_aliases

I will add them tomorrow and will also look at the tests and deprecation warnings.

@ceesios
Copy link
Contributor Author

ceesios commented Jul 14, 2023

looks good to me. I've also tested on 1.14.0 with the ansible module i am using and after adding all the new params there they all work.

@briantist
Copy link
Contributor

Thanks for these new commits @ceesios ! Could you also add the new parameters to the unit tests?

Copy link
Member

@adammike adammike left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the use of decorators here for the aliased parameters and deprecation warning. Testing for both the aliased and un-aliased paremters is great to see also.

Well done.

@briantist briantist merged commit c398774 into hvac:main Aug 14, 2023
76 checks passed
constantinneagu added a commit to Geries-Ingenieure-GmbH/hvac that referenced this pull request Sep 18, 2023
commit 1f88016
Author: Daniel Kimsey <90741+dekimsey@users.noreply.github.com>
Date:   Mon Aug 14 15:22:10 2023 -0500

    Fix premature read on stream requests in the `sys.take_raft_snapshot` method (hvac#771)

    * Fix premature read on stream requests

    When a caller (such as `sys.take_raft_snapshot`) performs a stream
    request, the act of attempting to parse the response as JSON causes the
    entire response body to be read and the underlying connection to be
    closed. This renders the streaming response moot.

    This change addresses that issue by examining if the caller requested a
    stream response, and if so returning the response as is.

    Without the change, it is impossible to read raft snapshots that are
    larger than memory as the entire response is read into memory to attempt
    to read as JSON.

    * add the Adapter.from_adapter class method

    * add test for adapter class method

    * Modify raft snapshot to always use RawAdapter

    Co-authored-by: Daniel Kimsey <dkimsey@trustwave.com>

    ---------

    Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>

commit c398774
Author: ceesios <cees@virtu-on.nl>
Date:   Mon Aug 14 22:17:12 2023 +0200

    ldap auth method - add missing `configure` params by vault api names (hvac#975)

    * add missing params by vault api names

    * move new parameters to the end

    * remove duplicate keys

    * add ldap tests for new params

    * fix black formatting

    * fix integrtion test with raw string

    * remove userfilter from integration test

    * Revert "remove userfilter from integration test"

    This reverts commit 296e9f2.

    * fix userFilter failure on Vault < 1.9

    * fix capitalization

    * fix conditional for other tests

    * fix typo in user_dn doc

    * add generate_parameter_deprecation_message utility function

    * fixup

    * stop suppressing deprecation errors

    * add aliased_parameter decorator and tests

    * fix asterisks in docstring

    * Revert "fix asterisks in docstring"

    This reverts commit 1a599ec.

    * fix docstring asterisks without side effects

    * add testcases to fill out coverage of alias decorator

    * fix lint

    * update LDAP configure to use alias wrapper for replaced parameter names

    * update test references to use canonical names

    * add client_tls, connection_timeout, max_page_size, dereference_aliases and order remove_nones

    * Revert "add client_tls, connection_timeout, max_page_size, dereference_aliases and order remove_nones"

    This reverts commit 55d28f8.

    * add client_tls, connection_timeout, max_page_size, dereference_aliases and order remove_nones

    * add new params to unit tests

    ---------

    Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auth methods generally related to a Vault auth method deprecation Deprecates something, to be removed or changed in a later release enhancement a new feature or addition ldap ldap auth method
Projects
None yet
Development

Successfully merging this pull request may close these issues.

client.auth.ldap.configure params doesn't match client.auth.ldap.read_configuration
3 participants