Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replicator _session support incompatible with require_valid_user=true setting #1550

Closed
adrienverge opened this issue Aug 9, 2018 · 11 comments
Closed
Labels

Comments

@adrienverge
Copy link
Contributor

@adrienverge adrienverge commented Aug 9, 2018

Hi,

After updating to CouchDB 2.2.0, replication stops working and logs show the following line:

[notice] 2018-08-09T10:46:14.759423Z node1@127.0.0.1 <0.9646.0> 332c814281 localhost:15984 127.0.0.1 undefined POST /_session 401 ok 1

It may be related to #1462.

  • What works:

    • Logging in with Fauxton, listing databases, creating a database, etc., using root:password.
    • curl -X PUT -u root:password http://127.0.0.1:15984/new_db
      
  • What doesn't work:

    • Initiating a local replication with Fauxton, even when providing credentials for both source and target. State stays "retrying", and logs show errors 401.

    • curl -X POST -u root:password http://127.0.0.1:15984/_replicate \
           -H "Content-Type: application/json" \
           -d '{"source":"existing_db", "target":"new_db"}'
      

      fails with:

      {"error":"replication_auth_error","reason":"{session_request_unauthorized,\"http://127.0.0.1:15984/_session\",\"root\"}"}
      
    • Same with the Python client.

For info, I use a 3-node cluster on Fedora. Replication always worked (from 2.0.0-beta to 2.1.2). My local.d/config.ini contains:

[chttpd]
require_valid_user = true
...

[cors]
credentials = true
...

[couch_httpd_auth]
allow_persistent_cookies = true
secret = --

[admins]
root = password

...

I'd be glad to help by providing more details!

@wohali
Copy link
Member

@wohali wohali commented Aug 9, 2018

Hi @adrienverge ,

I suspect this is an incompatibility of the new _session support with the require_valid_user=true setting.

As a temporary workaround, you can disable the new _session support in the replicator by setting:

[replicator]
auth_plugins = couch_replicator_auth_noop

Before making that change, do you have the same issue if you:

  • Post a document to the _replicator DB using curl from the command line?
  • Post to the /_replicate endpoint using curl from the command line?
  • Post a document to the _replicator DB using Fauxton?
@wohali wohali added replication bug security and removed security labels Aug 9, 2018
@wohali wohali changed the title Replication fails with 401 since 2.2.0 Replicator _session support incompatible with require_valid_user=true setting Aug 9, 2018
@adrienverge
Copy link
Contributor Author

@adrienverge adrienverge commented Aug 9, 2018

Hi @wohali, thanks for your answer.

  • You're right, removing require_valid_user = true solves the problem.

  • curl -X POST -u root:password http://127.0.0.1:15984/_replicator \
         -H "Content-Type: application/json" \
         -d '{"_id": "rep_1", "source":"existing_db", "target":"new_db", "create_target": true}'
    

    → The document is created, but the copy DB is not created (hence not replicated).

  • curl -X POST -u root:password http://127.0.0.1:15984/_replicate \
         -H "Content-Type: application/json" \
         -d '{"source":"existing_db", "target":"new_db"}'
    
    {"error":"replication_auth_error","reason":"{session_request_unauthorized,\"http://127.0.0.1:15984/_session\",\"root\"}"}
    
  • Post a document to the _replicator DB using Fauxton:
    → The document is created, but the copy DB is not created (hence not replicated).

Do I understand correctly that this is a temporary problem that happens in our special use-case (require_valid_user = true), and should be fixed in a future release? Or do we have a real forward-incompatibility in our configuration, and we must change it?

Thanks!

@janl
Copy link
Member

@janl janl commented Aug 9, 2018

cc @nickva: do you think we can add a “use basic auth against _session if no session id exists yet“ mode for this?

@wohali
Copy link
Member

@wohali wohali commented Aug 9, 2018

@adrienverge Yes, you understand correctly, this is a bug, we'll work out how to fix it.

Couch should be falling back to the non-session based authentication when the session-based authentication fails, but for some reason, it's not.

The workaround is a perfectly fine thing to do, it's not forward-incompatible. When session-based auth is fixed to work with require_valid_user=true, you'll be able to re-add that functionality back in by removing the 2 lines from your local.ini file.

adrienverge added a commit to adrienverge/copr-couchdb that referenced this issue Aug 9, 2018
Customizing args file is now supported upstream

TODO: Adapt bouzin to use COUCHDB_ARGS_FILE
TODO: Not working well yet, see issue
apache/couchdb#1550 (comment)
Should work when a fix is released.
@adrienverge
Copy link
Contributor Author

@adrienverge adrienverge commented Aug 9, 2018

Thanks, it's clear!

In this case I'll simply wait for the next version. Feel free to ask if you need someone to test the fix!

@nickva
Copy link
Contributor

@nickva nickva commented Aug 9, 2018

@wohali @adrienverge

In the default case, session based auth plugin would fall back to basic auth if when it tries to use /_session endpoint it gets back a 404 (it doesn't exist). It assumes that endpoint doesn't support session and falls back to using basic auth.

In case of require_valid_user=true when it initializes is sends a POST request to _session with the user and name in the form body

POST /_session HTTP/1.1
Content-Type: application/x-www-form-urlencoded

name=adm&password=pass

That fails with a 401. On a 401 failure, session plugin assumes it is a bad username or password.

I made PR to specifically handle the case when 401 response is because of the require_valid_user=true setting and fall back to using basic auth only.

#1551

nickva added a commit to cloudant/couchdb that referenced this issue Aug 10, 2018
…ser set

If _session response is 401 and WWW-Authentication header is set assume
endpoint has require_valid_user set. Remember that in the state and
retry to reinitialize again. If it succeeds, keep sending basic auth
creds with every subsequent _session request.

Since session uses the replicator worker pool, it needs to handle worker
cleanup properly just like couch_replicator_httpc module does. If response
headers indicate the connection will be closed, don't recycle it back to the
pool, otherwise during an immediate retry there will be a connection_closing
error, instead follow what the server indicated and stop the worker then
release it to the pool. The pool already knows how to handle dead worker
processes. This is needed with this commit, because we now have a pattern of an
immediate retry after an auth failure.

Fixes apache#1550
nickva added a commit to cloudant/couchdb that referenced this issue Aug 10, 2018
…ser set

If _session response is 401 and WWW-Authentication header is set assume
endpoint has require_valid_user set. Remember that in the state and
retry to reinitialize again. If it succeeds, keep sending basic auth
creds with every subsequent _session request.

Since session uses the replicator worker pool, it needs to handle worker
cleanup properly just like couch_replicator_httpc module does. If response
headers indicate the connection will be closed, don't recycle it back to the
pool, otherwise during an immediate retry there will be a connection_closing
error, instead follow what the server indicated and stop the worker then
release it to the pool. The pool already knows how to handle dead worker
processes. This is needed with this commit, because we now have a pattern of an
immediate retry after an auth failure.

Fixes apache#1550
nickva added a commit to cloudant/couchdb that referenced this issue Aug 12, 2018
…ser set

If _session response is 401 and WWW-Authentication header is set assume
endpoint has require_valid_user set. Remember that in the state and
retry to reinitialize again. If it succeeds, keep sending basic auth
creds with every subsequent _session request.

Since session uses the replicator worker pool, it needs to handle worker
cleanup properly just like couch_replicator_httpc module does. If response
headers indicate the connection will be closed, don't recycle it back to the
pool, otherwise during an immediate retry there will be a connection_closing
error, instead follow what the server indicated and stop the worker then
release it to the pool. The pool already knows how to handle dead worker
processes. This is needed with this commit, because we now have a pattern of an
immediate retry after an auth failure.

Fixes apache#1550
@janl janl closed this in #1551 Aug 13, 2018
janl added a commit that referenced this issue Aug 13, 2018
…ser set

If _session response is 401 and WWW-Authentication header is set assume
endpoint has require_valid_user set. Remember that in the state and
retry to reinitialize again. If it succeeds, keep sending basic auth
creds with every subsequent _session request.

Since session uses the replicator worker pool, it needs to handle worker
cleanup properly just like couch_replicator_httpc module does. If response
headers indicate the connection will be closed, don't recycle it back to the
pool, otherwise during an immediate retry there will be a connection_closing
error, instead follow what the server indicated and stop the worker then
release it to the pool. The pool already knows how to handle dead worker
processes. This is needed with this commit, because we now have a pattern of an
immediate retry after an auth failure.

Fixes #1550
AlexanderKaraberov added a commit to Spotme/couchdb that referenced this issue Aug 16, 2018
…ser set

If _session response is 401 and WWW-Authentication header is set assume
endpoint has require_valid_user set. Remember that in the state and
retry to reinitialize again. If it succeeds, keep sending basic auth
creds with every subsequent _session request.

Since session uses the replicator worker pool, it needs to handle worker
cleanup properly just like couch_replicator_httpc module does. If response
headers indicate the connection will be closed, don't recycle it back to the
pool, otherwise during an immediate retry there will be a connection_closing
error, instead follow what the server indicated and stop the worker then
release it to the pool. The pool already knows how to handle dead worker
processes. This is needed with this commit, because we now have a pattern of an
immediate retry after an auth failure.

Fixes apache#1550
adrienverge added a commit to adrienverge/copr-couchdb that referenced this issue Sep 24, 2018
Customizing args file is now supported upstream

TODO: Adapt bouzin to use COUCHDB_ARGS_FILE
TODO: Not working well yet, see issue
apache/couchdb#1550 (comment)
Should work when a fix is released.
adrienverge added a commit to adrienverge/copr-couchdb that referenced this issue Sep 25, 2018
Customizing args file is now supported upstream

TODO: Adapt bouzin to use COUCHDB_ARGS_FILE
TODO: Not working well yet, see issue
apache/couchdb#1550 (comment)
Should work when a fix is released.
adrienverge added a commit to adrienverge/copr-couchdb that referenced this issue Sep 25, 2018
Customizing args file is now supported upstream

TODO: Adapt bouzin to use COUCHDB_ARGS_FILE
TODO: Not working well yet, see issue
apache/couchdb#1550 (comment)
Should work when a fix is released.
adrienverge added a commit to adrienverge/copr-couchdb that referenced this issue Dec 11, 2018
- Update to new upstream version (skip 2.2 that has bugs, see for
  instance [1])
- Customizing args file is now supported upstream through
  `COUCHDB_ARGS_FILE`.

[1]: apache/couchdb#1550 (comment)
adrienverge added a commit to adrienverge/copr-couchdb that referenced this issue Dec 12, 2018
- Update to new upstream version (skip 2.2 that has bugs, see for
  instance [1])
- Customizing args file is now supported upstream through
  `COUCHDB_ARGS_FILE`.

[1]: apache/couchdb#1550 (comment)
@SCdF
Copy link

@SCdF SCdF commented Dec 18, 2018

Should we expect this to work in 2.3? It still fails for me:

$ curl -X POST http://admin:pass@localhost:5984/_replicate
    -d '{"source":"my-source-db","target":"my-target-db"}'
    -H "Content-Type: application/json"
{"error":"replication_auth_error","reason":"{session_request_unauthorized,\"http://127.0.0.1:5984/_session\",\"admin\"}"}
$ curl http://admin:pass@localhost:5984/
{"couchdb":"Welcome","version":"2.3.0","git_sha":"07ea0c7","uuid":"8517c5d22122b1e331ec01a0690e9adb","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

I have require_valid_user set to true. MacOS.

@wohali
Copy link
Member

@wohali wohali commented Dec 18, 2018

@SCdF Use full URLs for source and target. In fact, the "short" syntax is deprecated and doesn't work right in any version of 2.x.

This is a sore point that hasn't been fully addressed in documentation/Fauxton.

@SCdF
Copy link

@SCdF SCdF commented Dec 18, 2018

@wohali thanks for the reply. Would moving forward be fixing short syntax in 2.x, or fixing documentation / fauxton to not point toward the short syntax?

@wohali
Copy link
Member

@wohali wohali commented Dec 18, 2018

@SCdF From what I've heard, the latter.

@bgold0
Copy link

@bgold0 bgold0 commented May 31, 2019

Has this been fixed in 2.3? Seem to still have this issue @wohali

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants