Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC-11205for71: XDCR binary filter #3211

Merged
merged 2 commits into from
Aug 4, 2023

Conversation

tonyjhillman
Copy link
Contributor

No description provided.

@@ -101,6 +102,19 @@ A value of `true` means that the replication should be allowed to continue witho
A value of `false` (the default) means that the replication should be restarted.
For more information, see xref:learn:clusters-and-availability/xdcr-filtering.adoc#filter-expression-editing[Filter-Expression Editing].

The `filterBinary` flag is provided only in Couchbase Server Version 7.1.5+, and specifies whether binary documents should be replicated.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filterBinary feature is not in 7.2.0. It will be in 7.2.1.


* If a filter expression is provided, and the expression refers only to either the document's _key_, or its _xattr_, or to both, the expression is applied, and the document is replicated if the expression permits.

* If a filter expression is provided, and the expression refers to the document's body, the document is _not_ replicated.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The statement is incorrect -- the document is replicated. I tested that this is true in 7.1.4 -- the behavior that is exhibited when filterBinary is false is the same behavior in previous versions where filterBinary option does not exist.

  • If a filter expression is provided, and the expression refers to the document's body, the document is replicated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: In previous versions, when the expression referred to the key or xattr of a binary document, was the filter applied (and the binary document either replicated or not replicated based on the results)?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In previous versions, the behavior would be exactly like when filterBinary is false.
So, if a filter expression exists and contains key or xattr only expression, binary documents are checked against the filter expression and replicated or not replicated based on the key or xattr only expression.

Example:
filter expression: REGEXP_CONTAINS(META().id, "cblogo2")

I insert a binary document with doc id "cblogo2.jpg" -- it's replicated.
I insert a binary document with doc id "cblogo3.jpg" -- it is not replicated.

Copy link

@hyunjuV hyunjuV Jul 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's more examples of when filterBinary is false (or no filterBinary as in previous versions):

Example:
filter expression: REGEXP_CONTAINS(META().id, "cblogo") AND REGEXP_CONTAINS(name, "Jane")
I insert a binary document with doc id "cblogo4.jpg" -- it is not replicated

Example:
filter expression: REGEXP_CONTAINS(name, "Jane")
I insert a binary document with doc id "cblogo5.jpg" -- binary document is replicated. (which is problematic ... but that's the current behavior in versions where filterBinary does not exist or set to false. So, with filterBinary, the user can still use this expression and set filterBinary to true, in which case binary documents will not be replicated.)


* If a filter expression is provided, and the expression refers to the document's body, the document is _not_ replicated.

* If a filter expression is provided, and the expression refers to the document's _key_, or its _xattr_, or to both; and also refers to the document's body; then the expression is applied to the _key_ and/or _xattr_, and the document is replicated if the expression permits.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • If a filter expression is provided, and the expression refers to the document's key, or its xattr, or to both; and also refers to the document's body; then the document is not replicated -- regardless of whether or not the key or the xattr expression permits.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize that there is no logic to the behavior when filterBinary is false and the scenarios involving when an expression refers to the document's body -- it's just the behavior. It's the same behavior when filterBinary did not exist -- so, the behavior when filterBinary is false is the same behavior when there was no filterBinary in how binary documents were treated during replication in various scenarios. So, if you just want the same behavior you always had in previous versions, set filterBinary to false.

The documentation should note that the behavior of filterBinary when it is false is the behavior of XDCR in regards to binary documents in versions where filterBinary option did not exist.

@tonyjhillman tonyjhillman merged commit bdebd58 into couchbase:release/7.1 Aug 4, 2023
@tonyjhillman tonyjhillman deleted the DOC-11205for71 branch August 4, 2023 02:25
Copy link

@hyunjuV hyunjuV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the late review. Looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants