Bad total_rows for /{db}/_design_docs

[nono]

Expected Behavior

http://docs.couchdb.org/en/stable/api/database/bulk-api.html#db-design-docs says that the response for GET /{db}/_design_docs is a JSON object with:

• total_rows (number) – Number of design documents in the database. Note that this is not the number of rows returned in the actual query.
• and some other fields

Current Behavior

The total_rows is the number of documents in the database, not restricted to design documents.

Steps to Reproduce (for bugs)

Here is an example:

$ curl -q 'http://localhost:5984/cozy35ba44d2d1749e6f21646edce51e7190%2Fio-cozy-files/_all_docs'
{"total_rows":34,"offset":0,"rows":[
{"id":"010fd12eacb0f5d72b9a1c0d38015e3b","key":"010fd12eacb0f5d72b9a1c0d38015e3b","value":{"rev":"1-0a71f9fe3dcbd6e43bf2b290215b5864"}},
{"id":"010fd12eacb0f5d72b9a1c0d38016b23","key":"010fd12eacb0f5d72b9a1c0d38016b23","value":{"rev":"2-9a5851637d1a8cced1aec22b0e989272"}},
{"id":"010fd12eacb0f5d72b9a1c0d38018b94","key":"010fd12eacb0f5d72b9a1c0d38018b94","value":{"rev":"2-aeb093ae479719486307fe9477624dff"}},
{"id":"010fd12eacb0f5d72b9a1c0d3801a18f","key":"010fd12eacb0f5d72b9a1c0d3801a18f","value":{"rev":"2-b3956998775125ffa9f583c2ee5ff682"}},
{"id":"498b33c1b8fca8ba30b817980800b852","key":"498b33c1b8fca8ba30b817980800b852","value":{"rev":"1-7777f46df288c2f8b4b476adee32d0d3"}},
{"id":"498b33c1b8fca8ba30b817980800bf61","key":"498b33c1b8fca8ba30b817980800bf61","value":{"rev":"2-c68ed5ee485b3c9b5558eb85b68cdcec"}},
{"id":"498b33c1b8fca8ba30b817980800e139","key":"498b33c1b8fca8ba30b817980800e139","value":{"rev":"2-5d3404520d0ed9c718ef09b55fe3642e"}},
{"id":"498b33c1b8fca8ba30b817980800f294","key":"498b33c1b8fca8ba30b817980800f294","value":{"rev":"2-127effdc723b36fae42f9a2c776180ff"}},
{"id":"4afc12c8362680bfb65db569b9003e7f","key":"4afc12c8362680bfb65db569b9003e7f","value":{"rev":"2-4e01b06becf7119549df048876f7c833"}},
{"id":"4afc12c8362680bfb65db569b9004b9b","key":"4afc12c8362680bfb65db569b9004b9b","value":{"rev":"2-9172d7814a4a71fa622c9557cec84f19"}},
{"id":"4afc12c8362680bfb65db569b90389fd","key":"4afc12c8362680bfb65db569b90389fd","value":{"rev":"2-47b806df0d6c653df85d40ae83f966fb"}},
{"id":"4afc12c8362680bfb65db569b9039844","key":"4afc12c8362680bfb65db569b9039844","value":{"rev":"2-7199e4a7e4c95b066177a52058514045"}},
{"id":"4afc12c8362680bfb65db569b90445ee","key":"4afc12c8362680bfb65db569b90445ee","value":{"rev":"2-571a0058203e85599a1872aa70e4801e"}},
{"id":"4afc12c8362680bfb65db569b904a78e","key":"4afc12c8362680bfb65db569b904a78e","value":{"rev":"2-8b88fdeba760c225e190ed0a27559866"}},
{"id":"4afc12c8362680bfb65db569b9050b45","key":"4afc12c8362680bfb65db569b9050b45","value":{"rev":"2-f1c7c09df31dfbc1ab5015ead01f7ecb"}},
{"id":"4afc12c8362680bfb65db569b905a4da","key":"4afc12c8362680bfb65db569b905a4da","value":{"rev":"2-70e166cb7930b413ed346cd4aaaf3975"}},
{"id":"4afc12c8362680bfb65db569b906f2af","key":"4afc12c8362680bfb65db569b906f2af","value":{"rev":"2-f6107f7425f4810fd644fde5584f2907"}},
{"id":"4afc12c8362680bfb65db569b9073752","key":"4afc12c8362680bfb65db569b9073752","value":{"rev":"2-0bed715806c13119c256f67dcfa430f5"}},
{"id":"4afc12c8362680bfb65db569b907c6f5","key":"4afc12c8362680bfb65db569b907c6f5","value":{"rev":"2-24d8f302ae19a4b5f5b4b04538706941"}},
{"id":"4afc12c8362680bfb65db569b9098d9d","key":"4afc12c8362680bfb65db569b9098d9d","value":{"rev":"2-c2ab60862ba386b18b62f6d989118efb"}},
{"id":"9fa32b4c97f1fd69079b1c40f500106c","key":"9fa32b4c97f1fd69079b1c40f500106c","value":{"rev":"1-61fb1ecd150301495df38f84c4cbd222"}},
{"id":"9fa32b4c97f1fd69079b1c40f500203a","key":"9fa32b4c97f1fd69079b1c40f500203a","value":{"rev":"1-67570fbc92a0c489218e3b4eb45cad09"}},
{"id":"9fa32b4c97f1fd69079b1c40f5002273","key":"9fa32b4c97f1fd69079b1c40f5002273","value":{"rev":"1-a000e62c45d610da6eb366da575029e4"}},
{"id":"9fa32b4c97f1fd69079b1c40f5002b7c","key":"9fa32b4c97f1fd69079b1c40f5002b7c","value":{"rev":"1-fef5d055364fad7d188b80505c30ab26"}},
{"id":"_design/by-parent-type-name","key":"_design/by-parent-type-name","value":{"rev":"1-9ad74947b52a0c6e220c09807ea43fe6"}},
{"id":"_design/dir-by-path","key":"_design/dir-by-path","value":{"rev":"1-b45592638e0ca397ccb82cb9f0303c36"}},
{"id":"_design/dir-children","key":"_design/dir-children","value":{"rev":"1-031e47856210360b44db86669ee83cd1"}},
{"id":"_design/disk-usage","key":"_design/disk-usage","value":{"rev":"1-b598119ae3a61c73c62c8e7571a21f07"}},
{"id":"_design/referenced-by","key":"_design/referenced-by","value":{"rev":"1-1b96707c7208821e56dd39e1d4e90f11"}},
{"id":"_design/referenced-by-sorted-by-datetime","key":"_design/referenced-by-sorted-by-datetime","value":{"rev":"1-bf2dc1d3a977b26729f40bbe31241dcb"}},
{"id":"a9add4dced6d1daca85832538d079cf8","key":"a9add4dced6d1daca85832538d079cf8","value":{"rev":"1-517db6ce7bfd25807f5aaa4ec707de03"}},
{"id":"a9add4dced6d1daca85832538d07acb6","key":"a9add4dced6d1daca85832538d07acb6","value":{"rev":"2-163d1c57bf622338668a4847fa88b1b2"}},
{"id":"io.cozy.files.root-dir","key":"io.cozy.files.root-dir","value":{"rev":"1-117cd4bd67bbfa53b9db6f85ffe49f78"}},
{"id":"io.cozy.files.trash-dir","key":"io.cozy.files.trash-dir","value":{"rev":"1-e96c9e58c6717135d46a0ae07b0a4708"}}
]}

$ curl -q 'http://localhost:5984/cozy35ba44d2d1749e6f21646edce51e7190%2Fio-cozy-files'          
{"db_name":"cozy35ba44d2d1749e6f21646edce51e7190/io-cozy-files","update_seq":"53-g1AAAAEzeJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuRGYeCJAUgmWQPVsOKS40DSE08WA0LLjUJIDX1YDXcONTksQBJhgYgBVQ2PysRl6Mh6hZA1O3PSuTEq-4ARN19QuY9gKgD-TMLAKJoYys","sizes":{"file":218566,"external":18210,"active":25970},"purge_seq":0,"other":{"data_size":18210},"doc_del_count":0,"doc_count":34,"disk_size":218566,"disk_format_version":6,"data_size":25970,"compact_running":false,"cluster":{"q":8,"n":1,"w":1,"r":1},"instance_start_time":"0"}

$ curl -q 'http://localhost:5984/cozy35ba44d2d1749e6f21646edce51e7190%2Fio-cozy-files/_design_docs'
{"total_rows":34,"offset":24,"rows":[
{"id":"_design/by-parent-type-name","key":"_design/by-parent-type-name","value":{"rev":"1-9ad74947b52a0c6e220c09807ea43fe6"}},
{"id":"_design/dir-by-path","key":"_design/dir-by-path","value":{"rev":"1-b45592638e0ca397ccb82cb9f0303c36"}},
{"id":"_design/dir-children","key":"_design/dir-children","value":{"rev":"1-031e47856210360b44db86669ee83cd1"}},
{"id":"_design/disk-usage","key":"_design/disk-usage","value":{"rev":"1-b598119ae3a61c73c62c8e7571a21f07"}},
{"id":"_design/referenced-by","key":"_design/referenced-by","value":{"rev":"1-1b96707c7208821e56dd39e1d4e90f11"}},
{"id":"_design/referenced-by-sorted-by-datetime","key":"_design/referenced-by-sorted-by-datetime","value":{"rev":"1-bf2dc1d3a977b26729f40bbe31241dcb"}}
]}

In this example, the database has 34 documents: 6 design docs and 28 data docs. But the total_rows for _design_docs says 34, and not 6 as the documentation suggests.

Your Environment

• Version used: 2.2.0 ("git_sha": "2a16ec4")

[janl]

I think this is a docs error, since _design_docs with a namespace isn’t handled any special in the _all_docs code path, only _local docs are. See https://github.com/apache/couchdb/blob/master/src/fabric/src/fabric_view_all_docs.erl#L85-L92 and https://github.com/apache/couchdb/blob/master/src/fabric/src/fabric_view_all_docs.erl#L99-L105

I’d like corroboration on this one tho: cc @eiri @nickva.

[eiri]

Hey @janl, sorry for the response delay, I'm just back from vacation (been in Germany, ftw!)

The fabric bit is fine, we don't count rows when walking locals' btree, it's kind of expensive and ~meaningless, so we just pass null there and account for that in fabric.

The _design_docs, on the other hand, is just a filter over _all_docs iirc, so it should work the same. I'll take a look at that, it might be that we don't apply according filter on total_rows or something similar.

[wohali]

Hi @eiri, this is blocking 2.3.0's release. Would you have time to look at a PR for this this week?

[eiri]

@wohali yes, I have a half of the fix for this done and a missing part is POST with keys= on /{db}/_design_docs and the tests. I'll get to it tomorrow or Thursday.