Hi and welcome to the cOHMunity :)
We’re working with developers, researchers, and diverse communities to build WisdOHM, an open source system for quantifying the relative value of diverse contributions to any given collective mission, so that we can empower and connect communities in pursuit of the common good. WisdOHM is our particular implementation of the Metavaluation framework, which we're developing in parallel for public use and collaboration.
Contributions can be recorded and peer-reviewed by any member of the community. Reviews are completed as pairwise comparisons: voting between pairs of contributions on one or more dimensions of interest to the community. Open source algorithms then convert these data into standardised metrics, which serve to (a) recognise the unique qualities of each contribution, (b) reward contributors automatically, and (c) respect contributors for the value they provide.
The key innovation of WisdOHM is the way in which pairwise comparisons are used to peg the value of all other contributions. WisdOHM treats pairwise comparisons as valuable contributions in their own right, assigns them a standard unit of value, and includes them in the contribution set to be reviewed. This gives rise to meta-reviews (reviews of reviews), where one of the two items being compared is itself some number of reviews. Meta-review votes are then used to ‘weight’ the value of regular contributions, producing relative valuations where each unit is equivalent to the value of a single pairwise comparison. (-> read more).
WisdOHM began life as an open evaluation (or peer-review) model and evolved through experiments at Open Heart + Mind (OHM) to become a flexible framework for diverse contexts and communities. We prototyped in spreadsheets, built a proof-of-concept web-app (deprecated), a working prototype web-app, and are in the process of building an MVP (see below). (-> read more)
WisdOHM is designed to be simple yet flexible, so that it can be applied to diverse use cases. At OHM, we’ve explored WisdOHM in the context of festivals (recognising art, food, donationsm, etc.), conferences (recognising talks, workshops, registration fees, etc.), and decentralised projects (recognising code, invidible labour, etc.) -- and demonstrated it can be used both prospectively (e.g., voting on potential offerings) and retrospectively (e.g., valuing both planned and spontaneous contributions). The core mechanism is highly generalisable, however, and so could potentially be applied to a wide range of use cases beyond these that we've tested at OHM. (-> read more).
In addition to fixed-duration events, WisdOHM could also be used to evaluate dynamic contributions to an ongoing project. We intend to prove this concept via the WisdOHM project itself, using WisdOHM to recognise the full range of project contributions, including but not limited to documents, code, donations, and administration tasks. We also intend to explore the concept of a decentralised autonomous funding system, using WisdOHM metrics to distribute any financial contributions that we receive (e.g., donations or grants).
We've released a working prototype of the WisdOHM system, which allows any community to record, review, and recognise a set of contributions with respect to their mission and values. This version was based on our collective designs and discussions, and spearheaded by Claymost to enable rapid data collection. At OHM, we're using the app to value past contributions, and to process applications to our OHM Gathering 2025 (November, Gold Coast Hinterland). The process and app are designed to be generalisable, however, and so we'd love to hear from any other communities who find it useful.
Our main focus is on developing a minimal viable product (MVP), which will focus on the first three stages of the broader framework (Record, Review, Recognise) and will allow any community to create their own instance and populate it with relevant contributions. It's designed to serve our most proven use case: recognising contributions to in-person gatherings, whether they be community arts festivals or research conferences —— but could also be extended to many other use cases. The MVP will be similar in design to the prototype, but open source, more secure and reliable, and built collaboratively through the generous contributions of our international developer team. It will feature the following components:
- Well established data model we can use as a basis for any future data analysis.
- Front-end facing application that will engage users with a pairwise comparison voting system.
- A ‘pairing algorithm’ for assigning contribution pairs to reviewers, and a ‘scoring algorithm’ for assigning values to contributions. These algorithms will be relatively simple for the MVP, but can evolve into more complex algorithms over time.
- Back-end management server that will manage all collected data and participating users.
- Security best practices to protect users against cyber security breaches, alongside consideration of potential user fraud and manipulation of the algorithms.