New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replicator fails to authenticate to open db after cookie timeout #1607

Closed
ahayes opened this Issue Sep 18, 2018 · 4 comments

Comments

Projects
None yet
3 participants
@ahayes

ahayes commented Sep 18, 2018

Expected Behavior

A replicator document with valid credentials for a target database should continue to present valid credentials when replicating. Even when a target database allows anonymous writes, there may be validate_doc_update scripts that make decisions based on the user performing the replication.

Current Behavior

If the target database does not require authentication, the replicator may function with the user context of the administrator creating the replication document until the authentication cookie expires. At this point, the target database will ignore the expired cookie and accept documents but any validate_doc_update scripts that make decisions based on user context will instead receive an 'undefined' user context.

Possible Solution

  • Reject expired authentication cookies.
  • Use basic auth

Steps to Reproduce (for bugs)

  1. Set up target database that is open for anonymous writing.
  2. Create VDU on target database that rejects documents from non-admin users.
  3. Create a continuous replication document from a source database to the target database and specify admin credentials.
  4. Wait over 10 minutes (cookie session timeout) and add a document to the source database.

Context

Our Nunaliit tool makes a lot of decisions in the VDU about which documents can be updated based on roles and fields in documents. The configuration of Nunaliit permits decisions about whether or not to accept anonymous writes or to require valid users. In addition, valid users may be required to agree to terms (attribute on user doc because terms may be versioned, etc.) of use or take other steps before being granted write or edit permissions.

We have designed Nunaliit to work with couchdb without assuming that we are the only code touching it. Making configuration changes at a database level or customizing plugin settings is something we'd like to avoid. Ideally the cookie auth would have the same behaviour as the basic auth.

Your Environment

  • Version used: 2.2.0
  • Browser Name and version: Chrome/Firefox (latest)
  • Operating System and version (desktop or mobile): Ubuntu 16.04 server. Various clients.
  • Link to your project: http://nunaliit.org

@janl janl added the bug label Sep 18, 2018

@janl janl added this to the 2.3.0 milestone Sep 19, 2018

@nickva

This comment has been minimized.

Contributor

nickva commented Sep 20, 2018

Made a script to replicate the issue:

#!/bin/bash

S=localhost:15984
SA=adm:pass@${S}
JSON="Content-Type: application/json"

curl -s -XDELETE $SA/db > /dev/null
curl -s -XDELETE $SA/srcdb > /dev/null
curl -s -XDELETE $SA/_replicator > /dev/null
curl -s -XPUT $SA/db > /dev/null
curl -s -XPUT $SA/srcdb > /dev/null

echo "Setting [couch_httpd_auth] timeout = 5"
curl -s -XPUT adm:pass@localhost:15986/_config/couch_httpd_auth/timeout -d '"5"' > /dev/null
curl -s -XPUT adm:pass@localhost:25986/_config/couch_httpd_auth/timeout -d '"5"' > /dev/null
curl -s -XPUT adm:pass@localhost:35986/_config/couch_httpd_auth/timeout -d '"5"' > /dev/null

echo "Adding VDU to target"
cat <<EOF > /tmp/vdu.json
  {
    "_id": "_design/vdu",
    "validate_doc_update":"function(newDoc, oldDoc, userCtx) { log(userCtx); if (newDoc.name != userCtx.name) { throw({'forbidden': 'VDU failed'}); } }"
  }
EOF
curl -s -XPUT $SA/db/_design/vdu -H "$JSON" -T /tmp/vdu.json > /dev/null

echo "Replicating from srcdb to db"
curl -s XDELETE $SA/_replicate -H "$JSON" -d '{"source":"http://adm:pass@localhost:15984/srcdb", "target":"http://adm:pass@localhost:15984/db", "worker_proceses":1, "continuous":true}' > /dev/null
curl -s XPOST   $SA/_replicate -H "$JSON" -d '{"source":"http://adm:pass@localhost:15984/srcdb", "target":"http://adm:pass@localhost:15984/db", "worker_proceses":1, "continuous":true}' > /dev/null

sleep 3

DOCID=0

echo "With frequent enough updates, session is getting refreshed"
while [ $DOCID -lt 2 ]; do
    let DOCID++
    echo ""
    echo "DocID: $DOCID"
    curl -s -XPUT $SA/srcdb/$DOCID -H "$JSON" -d '{"name":"adm"}' > /dev/null
    echo "Doc added to source, sleeping 2 seconds"
    sleep 2
    echo "Getting doc from target:"
    curl $SA/db/$DOCID
done

echo "With long sleeps, sesssion will timeout and not get refreshed"
sleep 10

while [ $DOCID -lt 10 ]; do
    let DOCID++
    echo ""
    echo "DocID: $DOCID"
    curl -s -XPUT $SA/srcdb/$DOCID -H "$JSON" -d '{"name":"adm"}' > /dev/null
    echo "Doc added to source, sleeping 7 seconds"
    sleep 7
    echo "Getting doc from target:"
    curl $SA/db/$DOCID
done

It can be run against a default dev cluster:

$. /replicate_session_with_vdu_auth.sh
Setting [couch_httpd_auth] timeout = 5
Adding VDU to target
Replicating from srcdb to db
With frequent enough updates, session is getting refreshed

DocID: 1
Doc added to source, sleeping 2 seconds
Getting doc from target:
{"_id":"1","_rev":"1-66e95cbbe626e928747ecc225341b2d6","name":"adm"}

DocID: 2
Doc added to source, sleeping 2 seconds
Getting doc from target:
{"_id":"2","_rev":"1-66e95cbbe626e928747ecc225341b2d6","name":"adm"}
With long sleeps, sesssion will timeout and not get refreshed

DocID: 3
Doc added to source, sleeping 7 seconds
Getting doc from target:
{"error":"not_found","reason":"missing"}

DocID: 4
Doc added to source, sleeping 7 seconds
Getting doc from target:
{"error":"not_found","reason":"missing"}
@nickva

This comment has been minimized.

Contributor

nickva commented Sep 20, 2018

An interesting observation is that a 403 HTTP response, which a forbidden could generate for individual docs will trigger a cookie refresh.

However replicator, in the normal case would write documents using a _bulk_docs endpoint and the result of that is 201 even though some documents in the batch failed with a forbidden error.

It is possible to trick the replicator to use single requests if attachments are used. Indeed, I had waited till the after the session expired and added a document with an attachment to the source:

 curl -s -XPUT adm:pass@localhost:15984/srcdb/1/att1?rev=1-66e95cbbe626e928747ecc225341b2d6 -H 'Content-Type: application/octet-stream' -T 1mb
{"ok":true,"id":"1","rev":"2-3dec55150ee06ac046419c71f00f5351"}

The script was running in the part with long sleep but as soon as the attachment was added, the session was refreshed and the a few writes succeeded:

DocID: 4
Doc added to source, sleeping 7 seconds
Getting doc from target:
{"error":"not_found","reason":"missing"}

DocID: 5
Doc added to source, sleeping 7 seconds
Getting doc from target:
{"_id":"5","_rev":"1-66e95cbbe626e928747ecc225341b2d6","name":"adm"}

DocID: 6
Doc added to source, sleeping 7 seconds
Getting doc from target:
{"error":"not_found","reason":"missing"}

nickva added a commit to cloudant/couchdb that referenced this issue Sep 21, 2018

Implement replicator forced session refresh
Force couch_replicator_auth_session plugin to refresh the session periodically.
Normally it is not needed as the session would be refreshed when requests start
failing with a 401 (authentication) or 403 (authorization) errors. In some
cases when anonymous writes are allowed to the database and a VDU function is
used to forbid writes based on the authenticated in username, requests with an
expired session cookie will not fail with a 401 and the session will not be
refreshed. When set, the value should be less than the minimum session timeout
for any of the replication endpoints. For example, for CouchDB, the session
timeout is 600 by default so the value would be set to something like 550.

Issue apache#1607
@nickva

This comment has been minimized.

Contributor

nickva commented Sep 21, 2018

Implemented a fix: #1619

nickva added a commit to cloudant/couchdb that referenced this issue Oct 2, 2018

Implement replicator forced session refresh
Force couch_replicator_auth_session plugin to refresh the session periodically.
Normally it is not needed as the session would be refreshed when requests start
failing with a 401 (authentication) or 403 (authorization) errors. In some
cases when anonymous writes are allowed to the database and a VDU function is
used to forbid writes based on the authenticated in username, requests with an
expired session cookie will not fail with a 401 and the session will not be
refreshed. 550 is used as default as it is less than the default 600 used for
CouchDB's session refresh timeout.

Issue apache#1607

nickva added a commit to cloudant/couchdb that referenced this issue Oct 2, 2018

Implement replicator forced session refresh
Force couch_replicator_auth_session plugin to refresh the session periodically.
Normally it is not needed as the session would be refreshed when requests start
failing with a 401 (authentication) or 403 (authorization) errors. In some
cases when anonymous writes are allowed to the database and a VDU function is
used to forbid writes based on the authenticated in username, requests with an
expired session cookie will not fail with a 401 and the session will not be
refreshed. 550 is used as default as it is less than the default 600 used for
CouchDB's session refresh timeout.

Issue apache#1607

nickva added a commit to cloudant/couchdb that referenced this issue Nov 14, 2018

Implement replicator session refresh based on cookie max age
Force couch_replicator_auth_session plugin to refresh the session periodically.
Normally it is not needed as the session would be refreshed when requests start
failing with a 401 (authentication) or 403 (authorization) errors. In some
cases when anonymous writes are allowed to the database and a VDU function is
used to forbid writes based on the authenticated in username, requests with an
expired session cookie will not fail with a 401 and the session will not be
refreshed.

To fix the issue using these two approaches:

 1. Use cookie's max-age expiry time to schedule a refresh. To ensure that time
 is provided in the cookie, switch the the option to enable it by default. This
 handles the issue for endpoints which are updated with this commit.

 2. For endpoints which do not put a max-age time in the cookie, use a value
 that's less than CouchDB's default auth timeout. If users changed their
 auth timeout value, and use VDUs in the pattern described above, and don't
 update their endpoints to version which sends max-age by default, they could
 adjust `[replicator] session_refresh_interval_sec` to their auth timeout minus
 some small delay.

Of course refresh based on auth/authz failures should still works as before.

Fixes apache#1607

nickva added a commit to cloudant/couchdb that referenced this issue Nov 14, 2018

Implement replicator session refresh based on cookie max age
Force couch_replicator_auth_session plugin to refresh the session periodically.
Normally it is not needed as the session would be refreshed when requests start
failing with a 401 (authentication) or 403 (authorization) errors. In some
cases when anonymous writes are allowed to the database and a VDU function is
used to forbid writes based on the authenticated in username, requests with an
expired session cookie will not fail with a 401 and the session will not be
refreshed.

To fix the issue using these two approaches:

 1. Use cookie's max-age expiry time to schedule a refresh. To ensure that time
 is provided in the cookie, switch the the option to enable it by default. This
 handles the issue for endpoints which are updated with this commit.

 2. For endpoints which do not put a max-age time in the cookie, use a value
 that's less than CouchDB's default auth timeout. If users changed their
 auth timeout value, and use VDUs in the pattern described above, and don't
 update their endpoints to version which sends max-age by default, they could
 adjust `[replicator] session_refresh_interval_sec` to their auth timeout minus
 some small delay.

Of course refresh based on auth/authz failures should still works as before.

Fixes apache#1607

nickva added a commit to cloudant/couchdb that referenced this issue Nov 26, 2018

Implement replicator session refresh based on cookie max age
Force couch_replicator_auth_session plugin to refresh the session periodically.
Normally it is not needed as the session would be refreshed when requests start
failing with a 401 (authentication) or 403 (authorization) errors. In some
cases when anonymous writes are allowed to the database and a VDU function is
used to forbid writes based on the authenticated in username, requests with an
expired session cookie will not fail with a 401 and the session will not be
refreshed.

To fix the issue using these two approaches:

 1. Use cookie's max-age expiry time to schedule a refresh. To ensure that time
 is provided in the cookie, switch the the option to enable it by default. This
 handles the issue for endpoints which are updated with this commit.

 2. For endpoints which do not put a max-age time in the cookie, use a value
 that's less than CouchDB's default auth timeout. If users changed their
 auth timeout value, and use VDUs in the pattern described above, and don't
 update their endpoints to version which sends max-age by default, they could
 adjust `[replicator] session_refresh_interval_sec` to their auth timeout minus
 some small delay.

Of course refresh based on auth/authz failures should still works as before.

Fixes apache#1607

nickva added a commit to cloudant/couchdb that referenced this issue Nov 26, 2018

Implement replicator session refresh based on cookie max age
Force couch_replicator_auth_session plugin to refresh the session periodically.
Normally it is not needed as the session would be refreshed when requests start
failing with a 401 (authentication) or 403 (authorization) errors. In some
cases when anonymous writes are allowed to the database and a VDU function is
used to forbid writes based on the authenticated in username, requests with an
expired session cookie will not fail with a 401 and the session will not be
refreshed.

To fix the issue using these two approaches:

 1. Use cookie's max-age expiry time to schedule a refresh. To ensure that time
 is provided in the cookie, switch the the option to enable it by default. This
 handles the issue for endpoints which are updated with this commit.

 2. For endpoints which do not put a max-age time in the cookie, use a value
 that's less than CouchDB's default auth timeout. If users changed their
 auth timeout value, and use VDUs in the pattern described above, and don't
 update their endpoints to version which sends max-age by default, they could
 adjust `[replicator] session_refresh_interval_sec` to their auth timeout minus
 some small delay.

Of course refresh based on auth/authz failures should still works as before.

Fixes apache#1607

@nickva nickva closed this in #1619 Nov 26, 2018

nickva added a commit that referenced this issue Nov 26, 2018

Implement replicator session refresh based on cookie max age
Force couch_replicator_auth_session plugin to refresh the session periodically.
Normally it is not needed as the session would be refreshed when requests start
failing with a 401 (authentication) or 403 (authorization) errors. In some
cases when anonymous writes are allowed to the database and a VDU function is
used to forbid writes based on the authenticated in username, requests with an
expired session cookie will not fail with a 401 and the session will not be
refreshed.

To fix the issue using these two approaches:

 1. Use cookie's max-age expiry time to schedule a refresh. To ensure that time
 is provided in the cookie, switch the the option to enable it by default. This
 handles the issue for endpoints which are updated with this commit.

 2. For endpoints which do not put a max-age time in the cookie, use a value
 that's less than CouchDB's default auth timeout. If users changed their
 auth timeout value, and use VDUs in the pattern described above, and don't
 update their endpoints to version which sends max-age by default, they could
 adjust `[replicator] session_refresh_interval_sec` to their auth timeout minus
 some small delay.

Of course refresh based on auth/authz failures should still works as before.

Fixes #1607
@ahayes

This comment has been minimized.

ahayes commented Nov 27, 2018

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment