How do you distinguish curated quality from reproducible capability evidence?

[yuehua1213]

I noticed the list explicitly prioritizes code quality, security, and originality, and is currently migrating maintained legacy entries into a new format. That makes the collection much more useful than a raw directory, but it also raises a decision boundary for developers: a resource can be well curated while its actual behavior still depends on version, environment, and installation context.

One useful companion signal might be source-linked evidence such as the tested version, task, environment, observed output, and known limitations—kept separate from the editorial inclusion decision. That would let readers distinguish “worth discovering” from “reproduced successfully for this use case.”

Have you considered accepting or linking structured verification evidence for individual entries, or would that be outside the scope of this collection?

[hesreallyhim]

@yuehua1213 can you elaborate? do you mean claude code version? i assume that most visitors are using relatively up-to-date CC versions but I can't really know, and I can't really track compatibility drift anyway - if i list a resource, i can't pin the resource to a SHA or something, so compatibility depends on the source repo, and i can't track that kind of thing with high regularity - so I presume that if there are compatibility issues they will be surfaced in Issues and so forth. Environment, installation context - also would need some clarification. I don't and can't test out resources on multiple environments - i try to find resources for not just Mac users, and I use community adoption and cross-platform support as positive indicators. If you mean something else, please feel free to let me know.

Have you considered accepting or linking structured verification evidence for individual entries, or would that be outside the scope of this collection?

I tried to require people to provide this evidence themselves - frankly, it seems like a smart thing for a project to offer evidence - but many people didn't comply. I think I do mention it somewhere that providing this kind of evidence or even a demo video is basically a fast-track to getting reviewed, but I can't force people to do things.

Does that answer your question? Let me know if you have other thoughts. It's not outside the scope at all, although expecting the maintainer to produce and provide this sort of evidence definitely is.

Thanks.

[yuehua1213]

Thanks — that answers it clearly.

I wasn’t asking you to pin every listed resource to a SHA or to regression-test environments yourself. The useful distinction for me is:

1. Collection scope — discovery + signals (adoption, cross-platform mentions, Issues) is enough for a curated list.
2. Entry-level evidence — if an author wants a fast-track, they bring their own verification notes/demo; the maintainer doesn’t have to manufacture that evidence.

That matches what you described. I’ll treat compatibility drift as “surfaced by Issues / upstream,” not as something the list can honestly claim to track continuously.

If useful, I can suggest one concise CONTRIBUTING sentence explaining that reproducible evidence or a demo can accelerate review. I wouldn’t expect the list maintainers to generate that evidence themselves.