Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid re-compiling filter view functions #4473

Merged
merged 1 commit into from Mar 13, 2023

Conversation

nickva
Copy link
Contributor

@nickva nickva commented Mar 13, 2023

Filter view functions feature re-uses view map function for filtering _changes feeds. Instead of accumulating emitted KVs, it uses custom emit() function which just toggles a flag. However, in order to use this optimisation, the function is compiled first with the regular emit function, then the function source is queried with a non-portable toSource() method, and re-compiled again with a new sandbox where emit is overridden.

Instead of reparsing and re-compiling it, pass the sandbox to the compile function and compile filter views with that correct sandbox to start with. Moreover, this helps remove another non-portable function call.

@nickva nickva force-pushed the avoid-recompiling-js-for-view-filters branch 2 times, most recently from d048ed4 to 7495a4e Compare March 13, 2023 18:53
Copy link
Contributor

@jaydoane jaydoane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a JS expert by any means, but this seems like a nice optimization, and I'm assuming there are many tests in CI that exercise this code already.

Filter view functions feature re-uses view map function for filtering _changes
feeds. Instead of accumulating emitted KVs, it uses custom emit() function
which just toggles a flag. However, in order to use this optimisation, the
function is compiled first with the regular emit function, then the function
source is queried with a non-portable toSource() method, and re-compiled again
with a new sandbox where emit is overridden.

Instead of reparsing and re-compiling it, pass the sandbox to the compile
function and compile filter views with that correct sandbox to start with.
Moreover, this helps remove another non-portable function call.
@nickva nickva force-pushed the avoid-recompiling-js-for-view-filters branch from 7495a4e to 6090633 Compare March 13, 2023 19:52
@nickva
Copy link
Contributor Author

nickva commented Mar 13, 2023

Good point @jaydoane. I knew about the new chttpd tests but I double-checked to verify we had a bunch more as well:

t_view_filter_no_match({_, DbUrl}) ->
DDocId = "_design/views",
ViewFun = <<"function(doc) {if (doc._id == 'docX') {emit(1, 1);}}">>,
DDoc = #{<<"views">> => #{<<"v">> => #{<<"map">> => ViewFun}}},
DDocUrl = DbUrl ++ "/" ++ DDocId,
{_, #{<<"rev">> := Rev, <<"ok">> := true}} = req(put, DDocUrl, DDoc),
Params = "?filter=_view&view=views/v",
?assertEqual({8, 0, []}, changes(DbUrl, Params)),
{200, #{}} = req(delete, DDocUrl ++ "?rev=" ++ binary_to_list(Rev)).

should_succeed_with_view({_Ctx, {Source, Target}}) ->
create_docs(Source),
RepObject =
{[
{<<"source">>, db_url(Source)},
{<<"target">>, db_url(Target)},
{<<"filter">>, <<"_view">>},
{<<"query_params">>,
{[
{<<"view">>, <<"filter_ddoc/mammals">>}
]}}
]},
replicate(RepObject),
FilterFun = fun(_DocId, #doc{body = {Props}}) ->
couch_util:get_value(<<"class">>, Props) == <<"mammal">>
end,
{TargetDocCount, AllReplies} = compare_dbs(Source, Target, FilterFun),
% Target DB has proper number of docs
?assertEqual(1, TargetDocCount),
% All the docs filtered as expected
?assert(lists:all(fun(Valid) -> Valid end, AllReplies)).

@tag :with_db
test "map function filtered changes", context do
db_name = context[:db_name]
create_filters_view(db_name)
create_doc(db_name, %{_id: "blah", bop: "plankton"})
resp = Couch.get("/#{db_name}/_changes?filter=_view&view=changes_filter/blah")
assert length(resp.body["results"]) == 1
assert Enum.at(resp.body["results"], 0)["id"] == "blah"
end

@nickva nickva merged commit 6d7cfe8 into main Mar 13, 2023
@nickva nickva deleted the avoid-recompiling-js-for-view-filters branch March 13, 2023 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants