This is a side project and an evolving research prototype. It represents an idea and is not intended as a definitive production system. Anyone is welcome to participate, comment, and modify repository contents, provided that license terms are respected and contributions remain constructive.
- Petrarca Project
Petrarca is an experimental, modular project building a pipeline for digitizing, encoding, publishing, and eventually networking scholarly editions and their source materials — from a scan of a physical page to a searchable, citable, distributed digital library. The objective is to support scholarly workflows, enable reproducible text-based research, and provide datasets and tooling that can be reused for digital humanities work and machine learning experiments.
This repository is the project's documentation hub: the charter, the architecture, and the planning notes for the whole system. It does not itself contain the components' source code — each component is developed in its own repository, linked below.
- Host and serve both source (raw/facsimile) and edited (scholarly) materials.
- Provide desktop tools covering the full editorial pipeline: digitization, TEI encoding, and static-site publication.
- Enable programmatic access to corpora and metadata for research and training tasks, once a networking layer exists to make that possible.
The components below are built and actively maintained, each in its own repository.
Repository: DBA991/Scriptorium
A modular Electron desktop application covering the editorial side of the pipeline, from a scanned page to a finished TEI edition:
- TEI-XML encoding with schema validation and poetic autotagging (Scriptor).
- OCR and image preprocessing for scanned pages (Copyist).
- A reading view for the encoded text, with search and annotations (Librarius).
- Multi-document assembly and collation (Compilator).
- Automated
teiHeadergeneration from extracted named entities (Exemplator). - Vocabulary extraction and management (Glossographus).
- Philological/stylometric statistics and word clouds (Speculum) — see the planned networked Speculum below for how this local analysis module is expected to grow into a cross-repository tool.
- Export to PDF, XML, and HTML, including a structured batch mode (Pulpitum export) feeding directly into Praelum/Pulpitum below.
Repositories: DBA991/Praelum · DBA991/Pulpitum
The publication stage of the pipeline, split across two repositories:
- Praelum is the Electron desktop app: it validates a set of document folders exported from Scriptorium (checking required fields and cross-folder UUID collisions) and runs the actual site build.
- Pulpitum is the Astro site template Praelum builds: a static, synchronized side-by-side reading room, presenting a document's HTML transcription and PDF facsimile with bidirectional page sync, plus pop-out single-panel windows that stay in sync with each other via
BroadcastChannel. Pulpitum is also published as its own directly-editable, standalone project, for anyone who wants to build or host a Pulpitum library without going through Praelum.
A Pulpitum demo is here
Repositories: DBA991/Oculus · DBA991/Oculus-Mobile
The digitization front-end shared with the Boccaccio Project: a document-scanning tool that turns photos of physical pages into clean, straightened, export-ready page images, feeding into Scriptorium's Copyist for OCR and encoding.
- Oculus (desktop) offers full automatic page detection and perspective correction via OpenCV.js, double-page-spread splitting, PDF import, and PDF/ZIP export.
- Oculus Mobile (Android, via Capacitor) covers the same manual filter/crop/straighten workflow with the device's native camera and gallery, without the OpenCV auto-processing pipeline or double-page-spread support.
The two components below are not yet built. They depend on real architectural decisions — outlined here — that haven't been made yet, and each will get its own repository once work on them starts.
A server-side coordination service intended to network multiple independent Pulpitum libraries into one searchable, federated system: receiving queries, resolving them to collections and individual resources across libraries, and assembling results.
Building Dispatcher isn't only a matter of writing a server — it requires solving problems that don't exist yet in a single-library setup:
- A metadata architecture covering how resources, collections, and provenance are described and indexed across independent libraries (candidates include a flexible document store or an RDF/OWL-backed triplestore — see Architecture), designed so it can be queried federatively rather than per-library.
- Changes to Scriptorium's export step, since a document bound for a networked library will likely need to carry additional metadata (stable identifiers, provenance, licensing, trust/attribution information) beyond what a standalone Pulpitum export needs today.
- A trust/curation layer — some entity or process needs to vet which sources and libraries are trustworthy enough to federate, since Dispatcher has no way of knowing this on its own. What that looks like (a maintained allowlist, a review process, a web-of-trust model, or something else) is an open design question, not yet decided.
Scriptorium already ships a Speculum module that computes some philological and stylometric statistics (tokens, types, TTR, hapax legomena, frequent tags, named entities, distributions) on a single document open in the editor. The planned, currently-unbuilt evolution of Speculum takes that same local analysis logic and points it outward: searching and selecting texts across the network of libraries that Dispatcher would expose, and running comparative analysis (edition comparison, corpus-wide statistics, export of analysis reports) across many documents and libraries at once, instead of just the one currently open in Scriptorium. Since it depends entirely on Dispatcher existing first, its own architecture will follow from whatever metadata/federation design Dispatcher ends up with.
- Facilitate collaborative editorial workflows across desktop tools, from digitization through publication.
- Provide an open network for sharing edition-ready and raw documentary materials, once Dispatcher exists.
- Enable reproducible pipelines that transform scanned or transcribed sources into structured, citable corpora.
- Supply curated datasets and tooling for training and evaluating machine-learning models on historical/documentary texts.
- Offer classroom demonstrations and teaching resources.
- Deliver pilot datasets for NLP research in historical languages and orthographies.
This repository holds only documentation — no component source code. The structure below reflects that:
/
├─ README.md
├─ LICENSE.md # AGPL v3 — main code license (keep as primary license file)
├─ LICENSE-DOC.md # CC BY-SA 4.0 — documentation/media license
├─ LICENSES.md # short overview mapping components to licenses
Each built component lives in its own repository (linked in Components above); this repo does not mirror or vendor their code.
High-level data flow, current and planned:
[Oculus / Oculus Mobile] --(scans)--> [Scriptorium] --(export)--> [Praelum] --(build)--> [Pulpitum site]
|
(planned) |
v
[Dispatcher]
|
v
[Speculum, networked]
- Oculus/Oculus Mobile produce scanned page images, digitized and encoded into TEI editions in Scriptorium.
- Scriptorium's export step (Pulpitum export mode) produces the document folders that Praelum validates and builds into a Pulpitum static site.
- (Planned) Dispatcher would index metadata across multiple independently hosted Pulpitum sites and federate them into virtual corpora, once the metadata architecture and export changes described above are actually designed and implemented.
- (Planned) A networked Speculum would consume Dispatcher's federated results for cross-library search and comparative analysis.
Persistence considerations (Dispatcher, planned):
- Metadata: flexible document store (e.g., MongoDB, CouchDB) or RDF triplestore (OWL/graph DB) for semantic queries — an open decision, see Dispatcher.
- Binary assets (images, PDFs): object storage with a CDN layer for distribution.
Security & privacy (Dispatcher, planned):
- Define access policies per repository/collection.
- Implement authenticated API endpoints for write operations.
- Maintain audit logs for provenance and reproducibility.
- Establish the trust/curation process described above before federating any external library.
- Code: GNU Affero General Public License v3.0 — see
LICENSE.md. Applies individually to each component repository as well. - Documentation, texts, and media: Creative Commons Attribution-ShareAlike 4.0 International — see
LICENSE-DOC.md.
See LICENSES.md for a concise map of components and their licences.
Contributions are welcome, on this hub repository and on each component's own repository. Suggested process:
-
Open an issue (here for architecture/planning topics, or on the relevant component repository for anything specific to that tool) describing the feature, bug, or improvement.
-
Fork the relevant repository and create a branch for your change:
git checkout -b feat/<short-description>
-
Include tests where applicable and update documentation.
-
Open a pull request referencing the issue.
For major changes to architecture or license usage, open an issue first to discuss.
- Design the metadata architecture and trust/curation model that Dispatcher depends on (see Dispatcher).
- Define the Scriptorium export changes needed to carry networked-library metadata.
- Build Dispatcher and, once it exists, the networked evolution of Speculum.
If you use Petrarca or its components in research or publications, please cite this repository and the specific component repositories you used.
For questions, collaborations, or contributions, please open an issue on this repository or on the relevant component repository, or reach out via the GitHub Discussions section.