[Bug]: Expand “Others” section for Pages, Event categories, Event actions not working

[rjtbhatnagar]

What happened?

On our Matomo On-premise, the rows under various archived reports (Pages, Events, Downloads etc.) get automatically grouped into “Others” section after displaying the top 500 rows, which is the default behaviour. In order to expand rows from “Others", I followed the configuration mentioned under this article: https://matomo.org/faq/how-to/faq_54/ and performed the following steps:

1.Updated the config.ini.php file as per the article (setting maximum number of rows to 20000)

datatable_archiving_maximum_rows_referrers = 20000
datatable_archiving_maximum_rows_subtable_referrers = 20000
datatable_archiving_maximum_rows_actions = 20000
datatable_archiving_maximum_rows_subtable_actions = 20000
datatable_archiving_maximum_rows_events = 20000
datatable_archiving_maximum_rows_subtable_events = 1000
datatable_archiving_maximum_rows_site_search = 20000
datatable_archiving_maximum_rows_userid_users = 20000
datatable_archiving_maximum_rows_standard = 20000

2.Invalidated the report for a period of one week in February for which I want to see the changes:

console core:invalidate-report-data --sites=1 --dates=2024-02-05,2024-02-11

3.Archived the report again using raw data for that one week period:

console core:archive —url= --force-date-range=2024-02-05,2024-02-11 --force-idsites=1

It seems to have worked as I can confirm seeing 20000 rows under ‘Behaviour → Events → Event Names’ section,

but it doesn’t work for ‘Behaviour → Pages’ section, which we actually want.

Raised this in forums too (https://forum.matomo.org/t/expand-others-section-for-pages/55982), and got a suggestion to create a bug.

What should happen?

Expected behaviour : 'Others' should expand rows to a limit of 20000 rows for Pages section too.

How can this be reproduced?

It can be reproduced by following steps mentioned above in 'What happened' section.

Matomo version

4.13.3

PHP version

No response

Server operating system

No response

What browsers are you seeing the problem on?

No response

Computer operating system

No response

Relevant log output

No response

Validations

• Read our Contributing Guidelines.
• Follow our Security Policy.
• Check that there isn't already an issue that reports the same bug to avoid creating duplicates.
• The provided steps to reproduce is a minimal reproducible of the Bug.

[sgiehl]

Hi @rjtbhatnagar,

It's hard to evaluate if this is a bug or not based on your screenshots.
How many records does the table show if you are viewing it flattened?

Also please consider updating to Matomo 5. Even if this turns out to be a reproducible bug, that still exists, it won't be fixed for Matomo 4.

[rjtbhatnagar]

Hi @sgiehl,

Thanks for your response. This week we updated Matomo to 5.0.3 and tried reproducing the issue. Please find the steps and screenshots below. For faster processing, we decided to set maximum rows to 3000 instead of 20000, and date range applied is 8 April to 14 April :

Changes done:

datatable_archiving_maximum_rows_referrers = 3000
datatable_archiving_maximum_rows_subtable_referrers = 3000
datatable_archiving_maximum_rows_actions = 3000
datatable_archiving_maximum_rows_subtable_actions = 3000
datatable_archiving_maximum_rows_events = 3000
datatable_archiving_maximum_rows_subtable_events = 500
datatable_archiving_maximum_rows_site_search = 3000
datatable_archiving_maximum_rows_userid_users = 3000
datatable_archiving_maximum_rows_standard = 3000

console core:invalidate-report-data --sites=1 --dates=2024-04-08,2024-04-14

console core:archive —url= --force-date-range=2024-04-08,2024-04-14

Details shown BEFORE applying changes:
1.Pages section:
Others: 1,986,914
Hierarchical: 74 rows

Flat: 2260 rows

2.Event names:
Others: 101,074
Hierarchical: 500 rows

Flat: 569 rows

Details shown AFTER applying changes:
1.Pages:
Others: 1,986,914 (should’ve decreased)
Hierarchical: 74 rows (should’ve increased)

Flat: 31017 rows (increased, probably because subtable rows limit 500 has been applied, but 'Others' count remains same)

2.Event names:
Others: 92,716 (decreased as expected)
Hierarchical: 3000 rows (increased as expected)

Flat: 3075 rows (increased as expected)

I hope this gives you more clarification of the issue. Please let me know if any more details are required, or if I'm missing something, thanks.

[rjtbhatnagar]

One more thing, under Behaviour -> Events, the "Others" section doesn't expand for 'Event actions' and 'Event categories' too. It only works for 'Event names'.
The following is the data for 16 May 2024. I didn't have to invalidate and archive the report as the following configuration was already set up last week:

datatable_archiving_maximum_rows_referrers = 3000
datatable_archiving_maximum_rows_subtable_referrers = 3000
datatable_archiving_maximum_rows_actions = 3000
datatable_archiving_maximum_rows_subtable_actions = 3000
datatable_archiving_maximum_rows_events = 3000
datatable_archiving_maximum_rows_subtable_events = 500
datatable_archiving_maximum_rows_site_search = 3000
datatable_archiving_maximum_rows_userid_users = 3000
datatable_archiving_maximum_rows_standard = 3000

Event Actions:
Flat rows count: 1689 (expected >= 3000)
Hierarchical rows count: 75 (expected 3000)
Others: 85,036 (expected less than this)

Event Categories:
Flat rows count: 81 (expected >= 3000)
Hierarchical rows count: 11 (expected 3000)
Others: 84,604 (expected less than this)

Event Names:
Flat rows count: 3036 (correct as expected)
Hierarchical rows count: 3000 (correct as expected)
Others: 133,803