contents of the "Statistics" tab

[bansp]

It's time to see what kind of statistics we could derive from the aggregation of the information, and I suggest that we get a proof-of-concept tab going asap (much as the mimeType tab).
The question is, what kind of statistics can we get going now, and what can we work towards?
Our 2019 goal was to see what formats are popular among the centres, relative to the function(s) they are expected to play. That is something that we can get straightforwardly:

• group by domain, display the number of occurrences of each format (by format ID!), as provided in the recommendation files, and sorted alphabetically

Once we get the formal format family tree (or graph) going, we could also:

• group by format family, display the number of occurrences of each format (by format ID!), as provided in the recommendation files, and sorted alphabetically

I guess that, with the information we already have (or can have, easily), we can also

• dynamically compute the relevant KPI, the measure being "percentage of centres offering repository services that have published an overview of formats that can be processed in their repository".

What else? Please share your ideas.

[margaretha]

Some basic statistics about the data (e.g. number of formats, domains, etc) have been added in 1eb2c3f

[bansp]

On the one hand, we can list some internal or technical statistics that basically reflect the work performed on extending the SIS: the number of format descriptions, the number of recorded file extensions or media types (not very useful, these two, but they do signal the data content of the SIS). The number of recommendations and recommendations per domain could also, arguably, fall in this category. Also the number of <centre> elements with some attributes, like, @ri, @status, etc.

But then we perform a kind of an epistemic step and assume that we've built a model of an aspect of CLARIN, and look for some statistics describing CLARIN. This is where we can count "the most popular formats per domain", and where we can compute the relevant KPI for centres where @ri = 'CLARIN', and, in time, maybe draw same cross-RI numbers. I would like to get to that second stage, because apart from its providing interesting information, it also means a gentle incentive for centres to keep their information current and somehow corresponding to that provided by other centres (e.g. in terms of granularity of recommendations), or to provide the information at all (and thus affect the KPI).

[bansp]

This is just to note that, as mentioned in the ticket referenced right above this comment, we may want to consider splitting this page into something like "SIS statistics" and "Data visualization". The latter meant to provide RI-wide (and maybe cross-RI?) visualizations.

[bansp]

We're moving this issue to the next milestone.

[bansp]

Eliza has prepared a list-statistics.xq page where a beginning can be seen, ordered by domains and then by the numbers of recommendations in those domains.
We're going to have a settable threshold; what gets displayed is formats with the number of recommendations equal to the threshold and above.
And there will be another setting (it's an experiment!) for showing either all the formats that make the threshold or just the top three values of occurrences. (By that we mean that if formats are tied for the number of recommendations, we get to see all the formats that have the given value).

[bansp]

A lot from this ticket got implemented, though not everything (for one thing, this is a discussion ticket, so it should actually be a hub for action-oriented children but there's only 24 hours in a... working day...).
I think I will prepare a separate quasi-milestone for discussion tickets (maybe that's what they have the "discussion" facility now at GitHub, but, again, 24hrs...). Crucially, I'm taking this ticket out of milestone 2.3.0. Probably to 2.4.0, while I ponder the sensibility of that extra "timeless" quasi-milestone.

[bansp]

While Pooh keeps pondering, this ticket sneaks out of 2.4.0 and onwards, toward a brighter future.

[bansp]

#180 is a related issue.

[bansp]

Assigning it to myself for careful re-reading, to see if it can be closed.

[bansp]

AH, but remember that the issue is linked from https://clarin.ids-mannheim.de/standards/views/list-popular-formats.xq so when it's closed, the link should either go or get replaced by a link to something more permanent.

[bansp]

Gathering some potentially interesting points from above:

• group by format family, display the number of occurrences of each format (by format ID!), as provided in the recommendation files, and sorted alphabetically
  • note that we have a "hidden page" on format families (and just keep forgetting about it!): https://clarin.ids-mannheim.de/standards/views/format-families.xq (thanks, Eliza!) see also ticket cross-format categorizations (keywords, <formatFamily>) #129
• The number of recommendations and recommendations per domain
• Also the number of <centre> elements with some attributes, like, @ri, @status, etc.

I am going to close this issue, since we need some sort of closure on the goals that have been achieved :-) I'll take the remaining ideas to another ticket.

There is now a new issue dedicated to gathering more ideas on what it can mean for a format to be popular (#201 ) and linked from the corresponding subpage.