Long generation time

[Daveiano]

The report takes very long to finish. On my Personal RPI 4b (2GB Ram) it takes ~ 50s (Data going back to March 2021, so 16 months of data).

Feedback from https://groups.google.com/g/weewx-user/c/sDK4kzO4vBw/m/RodvkIBoAwAJ

Ran this on my test rpi (simulator mode) with data going back to late Dec 2021. Took about ~70 secs to generate WdcReport. I copied over my up weewx db (data from mid 2020 to present) and it takes about ~180 sec to generate a WdcReport. Any ways to optimize the reports to get the report generation time down? For comparison, takes less than a sec for SeasonsReport.

root@ac95b51595a5:/home/weewx# time ./bin/wee_reports 
Using configuration file /home/weewx/weewx.conf
Generating for all time

real	0m50.115s
user	0m42.035s
sys	0m8.065s

Disabling the tables reduces the time to ~ 1/4:

root@ac95b51595a5:/home/weewx# time ./bin/wee_reports 
Using configuration file /home/weewx/weewx.conf
Generating for all time

real	0m10.339s
user	0m8.465s
sys	0m1.865s

Disabling only the table on statistics.html.tmpl:

root@ac95b51595a5:/home/weewx# time ./bin/wee_reports 
Using configuration file /home/weewx/weewx.conf
Generating for all time

real	0m20.372s
user	0m17.408s
sys	0m2.952s

After feedback from https://groups.google.com/g/weewx-development/c/myXrSc1OVQk/m/UtQiiSDIAwAJ:

root@ac95b51595a5:/home/weewx# time ./bin/wee_reports 
Using configuration file /home/weewx/weewx.conf
Generating for all time

real	0m27.106s
user	0m24.870s
sys	0m2.204s

[Daveiano]

The statistics.html page of your skin is looking for "all-time" values, and if I am not wrong , it is doing several separate queries : tables with an aggregation time of 1 day for all parameters, and diagrams with various aggregation periods, depending on the parameter. The imply that several queries for all time data will be performed for the same parameter. Why not first choosing a given aggregation time, doing one query for each parameter, and using it for both charts and table?

https://groups.google.com/g/weewx-development/c/myXrSc1OVQk/m/bH85goD7AwAJ

[Daveiano]

[...] you really want to make sure that you don't have to search the whole database and, instead, use the daily summaries. To do this:

1. The aggregation time must be a multiple of a day; and
2. The start time must be either on a midnight boundary or the first timestamp in the database; and
3. The end time must either be on a midnight boundary, or the last timestamp in the database.
   If any one of these is not true, then the main archive table must be used, and the query time will be much longer.

https://groups.google.com/g/weewx-development/c/myXrSc1OVQk/m/UtQiiSDIAwAJ

[dogo77]

Hi.
I'm brand new to weewx and wanted to try the weewx-wdc skin.
Installed as described under installation, activated in weewx.conf and restarted weewx.
Then i executed wee_reports. I only have data from yesterday (when i installed the WH1080), but execution takes ~10 MINUTES until now and i still don't have a working web report.

Started all over with deleting everything in /var/www/html/weewx/
then restarted weewx an didn't start the reports generation. After waiting the obligatory 15 minutes is still dont get a webreport showing your skin — well, sort of: i do get a rudimentary navigation menu, but no data.

any hints?
ty

[Daveiano]

@dogo77 Ok that seems very odd.

Any additional output in the log (should be at /var/log/syslog)?

Any chance you could share your weewx.sdb file with me? This way I could generate the reports myself and search for bugs.

[dogo77]

Any additional output in the log (should be at /var/log/syslog)?

systemd running here on dietpi, so it's journalctl, but: no.
no error, no message.
i just see the wee_reports process running with 100% on one core but with no result after 15 minutes

Any chance you could share your weewx.sdb file with me? This way I could generate the reports myself and search for bugs.

shure, where shall i send it to? or should i provide a download link?

thanks for your efforts!

[Daveiano]

shure, where shall i send it to? or should i provide a download link?

You could send it to david.baetge@gmail.com or provide a download link, do whats easier for you.

Thank you!

[dogo77]

follow up:
after emailing forth and back with @Daveiano it turned out to be a permission problem on my side. Starting all over again solved it and everything is working now. thank you!