Response to proposed changes to WikiData Notability Policy?

[anjackson]

It appears that a new Notability Policy is being considered for WikiData: Wikidata:Requests for comment/Notability policy reform - Wikidata

And here's the link to that page that describes the proposal itself.

My reading of that proposal is that format information in well-defined registries is probably okay, e.g. PRONOM mints identifiers and is much more than a trivial directory. That said, based on my understanding of how WikiThings tend to work, we might have to expect to defend this position at some point.

However, for other less formal sources, or other entities like software, my reading is that we're only likely to get WikiData entries for things that are considered notable enough to be in Wikipedia etc. i.e. it's not clear that legacy software is welcome unless it is very well know.

Having said all that, I'm not really very sure about this. I find the social machinery of Wikipedia and WikiData quite confusing and may have misunderstood. It would be good to have a better idea of the potential impact of this change.

In any case, as for any platform-hosted resource, it probably makes sense to think about how to make a back-up of the data we care about. The Format Aggregator/Format Index code already exports some of this information, so we could build something on that.

[ross-spencer]

"We" is doing some heavy lifting here. If the "we" is the registry dashboard, then it feels like there is little can be done. It can only report on what's there. If the "we" is the community, then it's a little broad, but there may be information there can be used - for example, there is a good chance that there are hyperlinks available in the Wikidata records, e.g. for specifications and source code for specific formats that could be migrated to the Just Solve It Wiki if a crosswalk can be done and there is understanding about what can land there. Overall though, some user stories might benefit this effort and future efforts.

Some specific thoughts:

• There's a decent amount of crud on Wikidata that I've been documenting since my work on the EaaSI/Yale/Siegfried integration that "we" afaik aren't engaging with (at least not that I can easily measure): https://github.com/ffdev-info/wikidp-issues/issues -- it probably does speak to some questions around notability but definitely more so low quality. There are Wikidata records that often just have a name and a TrID signature, some as short as a single byte. Even label/description pairs are also often lacking. I am tackling a few of structured text records in my spare time cross referencing them with Just Solve It in my JSONID work. The more I think we can polish records and engage with Wikidata the less likely there will be a problem with notability -- the corollary here is that whatever we backup is still going to contain low-quality records.
• Investment in Wikidata -- specifically:
  • Building on the work of Katherine's Wikidata for Digital Preservaitonists https://www.dpconline.org/docs/technology-watch-reports/2551-thorntonwikidatadpc-revsionthornton/file the DPC might do more to surface aspects of the "social machinery" of Wikidata so that yourself and that slither of the community are more able to engage with what is happening there.
  • Tackling Wikidata at the contributor's cafe -- unfortunately I can't easily attend these while my DP work remains spare-time work so please don't take my non-attendance as not willingness -- but for the collective who do attend it strikes me there's a resource there in my WikiDP-Issues repo that can work together to make robust records as well as begin to work through the social machinery -- even if that is ignored, it's easy enough to look at records through the lens of Siegfried's identification results with Wikidata turned on and other methods.
• Wikibase as a platform -- the change need not be too scary if we embrace the decentralization of efforts and understand that we can still connect back to the Wikidata graph via SPARQL and other interfaces. if the higher quality format data turns out to be really at risk then investment in standing up a Wikibase instance (suggested in the proposal you linked to) but also something that I think is broadly of interest to a few user groups. @samalloing has suggested its use for format policy purposes associated with a repository -- I also think it's required simply because we do have some more bespoke needs, e.g. modelling container format signatures with Wikidata's current model is almost impossible without doing something clever with the syntax to encode more information; we also have other community needs that don't sit easily in Wikidata-prime. My previous work also stood up a mirror of the Wikidata data model in a Wikibase to demonstrate the ease with which we could connect Siegfried to a custom Wikibase https://exponentialdecay.co.uk/blog/talk-using-a-custom-wikibase-with-siegfried/ and I am sure your tooling would connect just as easily. First we would need investment for a Wikibase host, this could provide mezzanine storage for a mirror of the Wikidata records. Second, that could then be used to build a community resource that could maybe bring in other use cases like the format policy registers and so on. (Wikibase cloud could be an invaluable resource here https://www.wikibase.cloud/). Your organization and remit might put you in a position to do this..
• Connecting communities -- maybe this is something else that can be exposed in the rest of this discussion -- but if records are to be deleted from Wikidata then a Just Solve it crosswalk should be inevitable. The registry uses Wikidata entries for its info boxes. LoC references Wikidata as well, but crucially they add records in Wikidata that probably adds to notability. Still it might be worth bringing these communities together in in some way so as to tackle the problem holistically (if it turns out we will see deletions) -- you may be in a position to do this as well.

In short, if there is only capacity to take a backup then some of the information I am sure could be useful in time. If, however, this is a moment in time to embrace change, then there's are more pro-active approaches that can be taken which personally I'd love to see and I can do whatever I can with whatever resources I have.

[ross-spencer]

@anjackson IIRC there was an update from Kat at Yale on the impact of these changes, and a good chance for a number of reasons they are unlikely to have much impact, at least on existing Wikidata entries.

I can't find the text and think it might have been via Mastodon, or the mailing list. Are you able to re-share the snippet here for the record? thanks 🙏

[ross-spencer]

Ah, got it.

I guess caveats are we have to pay attention and there are no guarantees on a community/shared resource. But useful food for thought.

"I do not anticipate that changes to the notability policy will lead to deletions of existing items. The discussions seem to be geared more toward new items that get created, and the nominations for deletion that could happen in the future. I am not seeing language in the discussions around this RFC that make me concerned about the data related to software and file formats in Wikidata that the digital preservation community members are currently contributing.

Connecting new items to existing items meets the notability requirement option that accommodates data that fulfills a structural need. If an editor contributes a new file format item, if they then connect it to existing software items, then this demonstrates that the new item fulfills a structural need to make the information about those software items more complete. If an editor contributes a new item with no external identifiers, no references, and does not connect it to any other items then that could be nominated for deletion.

I feel hopeful that the impact of this change could be that more people connect newly created items to other items in the knowledge graph, and are more diligent about adding references which I think will be positive improvements that will increase data quality."

[anjackson]

Thanks @ross-spencer - much appreciated!

[anjackson]

We talked about this a little at the last PR-SIG call, and two things came up that are worth noting here.

In terms of taking backups, I realised I'm not sure how that would work. I've struggled to understand the scope to operate at. i.e. how to decide which subset of triples/assertions to download as a backup. We'd also have to find somewhere to keep them. If they are not too big, we can probably find a way using Git or Git + LFS. Not sure what to do if they are very chunky.

Kai suggested an interesting complimentary tactic: Can/should we monitor WikiData somehow so if records are being dropped, we'd notice? That seems like something that might be possible to build on top of the current format information aggregator.

[ross-spencer]

You have a way of monitoring deleted via the Wikibase API:

https://www.wikidata.org/wiki/Special:Log/delete

API:

curl \
  --get 'https://www.wikidata.org/w/api.php' \
  --data 'action=query' \
  --data 'list=logevents' \
  --data 'format=json' \
  --data 'letype=delete'

But I'm not sure how to get what I think is a full list of 50 results at a time, and the events might need to be expanded, say, for merges, which I think are important for some of the records as well.