Repository navigation
Replies: 7 comments 5 replies
|
Hi all Updated the branch with Vega-Light support which seems to fit quite well. As before Vega allows dynamic graphs (select in the first graph) but it means a static file needs to be stored for gc.com Happy to hear pros/cons or other ideas |
|
@nagisml Some thoughts: Splitting: could we land this in 3–4 PRs (write_text + permissions item G / stats + date / SVG charts, maps + html / Vega)? It fits the wave merging and makes review much easier. Agree on removing demo-reports/ and the regenerated docs before merging to beta. |
|
This is all very impressive! You guys are really pushing this forward rapidly. Great work. Unfortunately, and I’m sure you know this already, but the main reason we never added anything like this to GSAK was that the geocaching.com profile page, which is where folks at that time wanted to display stats, is very limited in its support of HTML. It doesn’t allow JavaScript, nor does it allow CSS other than in-line styling. No classes. There are other restrictions as well. That makes it tough! You are pretty much limited to text and images in tables. The only way we could get charts etc to work was by including them as images hosted on the GSAK server. Much of FindStatGen is written to get around those restrictions. With only in-line styling the files end up being enormous! (We did use JavaScript libraries for charts in the early days until Groundspeak banned the use of JavaScript.) |
|
Thanks @GeePa67 and @nagisml. Since I don’t use statistics or reports myself, I’m happy to leave the direction to you, with two ground rules: No image hosting by OpenSAK. Running an upload server means running costs, abuse handling, GDPR responsibility for users’ find data, and broken profiles if it ever goes away. That’s not something the project can take on. Macros are free to write images locally and build the image links from a base URL the user sets, so people can host them wherever they like (own webspace, a GitHub repo, an image host). That should remove the blocker for FSG-style output without OpenSAK storing anything. Within those two, go ahead. @GeePa67, your FSG profile HTML would be a useful reference for what’s possible with inline-only HTML. |
|
Agree 100% about no built-in features for stats as well as not hosting images. Just need support for gathering necessary data from Opensak for macro html creation. I will create and send html and upload tonight or tomorrow. You can also view my profile on geocaching.com. My geocaching name is GeePa. |
|
I haven't looked at the above outline in detail, I just read all the responses in the thread. With regard to the question where to host the created images, I have two ideas:
|
|
Here is my stats html which is all GC.com compliant. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi @AgreeDK @urs-beeli @GeePa67 @lignumaqua and everybody else who has wishes or ideas
Support statistic macros I would propose following branch. target should be that OpenSAK offers python possibilities for statistics as possible and lua takes over the rest.
Why
Macros can read caches, but there is no way to turn them into statistics or a report (GSAK's
SqlToHtml, FindStatGen, Project-GC style pages). This PR adds generic building blocks to theopensakLua table. Which statistics to make stays in the macro: the functions work on any list of rows (fromopensak.caches{},opensak.sql()oropensak.read_csv()) and know nothing about geocaching.Two example macros show what is possible:
find_statistics.lua— statistics, charts and maps of your findsdatabase_statistics.lua— the same for all caches in a databaseBoth could be found here: https://github.com/nagisml/OpenSAK/tree/feature/lua-statistics-report/macros/examples
Here the result of the 2 macros on a dummy DB (download the 2 HTML files and open locally)
https://github.com/nagisml/OpenSAK/tree/feature/lua-statistics-report/demo-reports
btw Italy looks funny but this is based on the boundaries used in Opensak #776
New functions (API version 3 for the moment)
No new additional library had to be used all is based on existing OpenSAK used libaries
Details see: https://github.com/nagisml/OpenSAK/blob/feature/lua-statistics-report/docs/macros/api.md
write_text(path, text [, {append}])stats.group(withperiod= day/week/month/year/weekday,count_distinct,sum,sort,top),stats.fill,stats.running,stats.pivot,stats.every,stats.summary,stats.sort,stats.streaksdate.diff,date.periodchart.bar(also stacked, or from astats.pivotmatrix),chart.line,chart.heatmap,chart.rosechart.map(countries / states / counties coloured by value),chart.points(dot map)geo.regions(layer [, {within}])— e.g. "18 of 26 cantons"html.escape,html.table(withcolumns,link,bars),html.matrix,html.tiles,html.columns,html.page(light/dark theme)Changed:
opensak.caches{ all = true }returns every cache of the database, whatever the active filter shows.How the output works:
currentColor, the series colour from--chart-color, and a<title>tooltip on every bar, point and region.html.matrixand thehtml.tablebars use inline styles only, so they also work when pasted into a geocaching.com profile.Security
write_textis the first macro function that takes a free file name, so it brings item G of the folder-security plan with it:check_access()/safe_write()takeallowed_suffixes.write_textonly writes.txt .csv .tsv .html .htm .svg .md .json, so no.lua,.bat,.ps1, …~/.config/autostart,~/Library/LaunchAgentsand shell rc files.opensak.sql().Dependencies
None new. Maps use
shapely, which is already required (pyproject.toml) and bundled (opensak.spec).Files
src/opensak/stats/(new, Qt-free):tables.py(statistics),charts.py,maps.py,html.pysrc/opensak/macro/runtime.py: bindings andApiFunctionentriessrc/opensak/macro/permissions.py: item Gsrc/opensak/geo/store.py:regions_with_boxes()(read-only)src/opensak/macro/api_docs.py: stub classesopensak.Matrix,opensak.Run; cache tables accept fields added by the macromacros/examples/find_statistics.lua,macros/examples/database_statistics.luatests/unit-tests/test_macro_statistics_report.py(102 tests, with a small fixture boundary dataset)For review only — will be removed before merging to beta
demo-reports/: the HTML output of both example macros, so you can look at the result without running anything. They were generated from a synthetic, anonymous database (5,000 made-up caches across Europe, 2,000 found by "DemoCacher"); there is no real cache, owner or finder data in them.docs/macros/api.mdandmacros/types/*are regenerated here so the new functions can be read in the docs. I'll drop them before merging if the Macro API docs workflow should regenerate them as usual.Testing
stats/,macro/andgeo/.opensak.caches()reader).Open questions
chart.map/chart.pointswithlayer = "county"download county outlines per country on first use, the same as Update location does. Is that acceptable from a macro, or should macros only use outlines that are already on disk?HOMEin the examples defaults to the median position of the finds. So this needs to be fixed.All reactions