Skip to content
WIP implementation of CSL and CSL-M in Rust.
Rust TypeScript Lua Haskell JavaScript Shell Other
Branch: master
Clone or download
cormacrelf Merge pull request #8 from cormacrelf/imperative-tests
conform to citeproc-js for the CITATIONS imperative tests
Latest commit 4ac9453 Sep 12, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.cargo `cargo pull-locales` command Aug 25, 2019
citeproc-js-runner test-suite: show YAML supporting CITATIONS compactly Sep 12, 2019
crates demo: update to use the simplified locator field Sep 12, 2019
docs Set theme jekyll-theme-minimal Jul 1, 2019
pandoc-ffi-bridge MPL 2.0 license notices on all Rust source code & toml. Jun 28, 2019
pandoc-lua-preproc MPL 2.0 license notices on Haskell/Lua source Jun 28, 2019
pandoc-preproc MPL 2.0 license notices on Haskell/Lua source Jun 28, 2019
script wasm js-demo: init, parse & display one cluster Jul 2, 2019
.appveyor.yml get it working in WebAssembly with wasm-pack Nov 28, 2018
.dockerignore dockerfile and flame_span Dec 5, 2018
.gitignore more restructuring - citeproc-proc, citeproc-io Jul 30, 2019
.gitmodules make test-suite a git submodule -> cormacrelf/test-suite Aug 25, 2019
.ignore add the test suite to ripgrep/fd's ignore file Sep 3, 2019
.projections.json add a runner to check how citeproc-js would execute a YAML test Aug 26, 2019
.travis.yml travis: fix missing yarn install Aug 13, 2019
Cargo.lock cargo upgrade --all Sep 12, 2019
Cargo.toml include citeproc-wasm in default-members to test/document Sep 12, 2019
LICENSE.md Change the overall license to MPL 2.0 Jun 28, 2019
README.md `cargo pull-locales` command Aug 25, 2019
bors.toml MPL 2.0 license notices on all Rust source code & toml. Jun 28, 2019
example.csl feature gate the conditions syntax Feb 18, 2019
link.sh move design out of readme, add more docs Dec 11, 2018
rustfmt.toml MPL 2.0 license notices on all Rust source code & toml. Jun 28, 2019

README.md

citeproc-rs

An early-stage work-in-progress implementation of CSL and CSL-M in Rust. It is geared at:

  • replacing citeproc-js by providing WebAssembly bindings such that it could be embedded in Zotero or fulfil any other JavaScript-based use;
  • replacing much of pandoc-citeproc, by running as a Pandoc Filter;
  • providing a compiled static library to replace other divergent or incomplete implementations and make it easy to integrate citeproc into document processing pipelines and web services; and
  • correctness and high performance.

Nearly every programming language in existence can link against a static C library; this effort is therefore also aimed at reducing the number of implementations to 1, thereby reducing the burden on implementers and making CSL evolution more nimble.

Currently, the codebase is evolving rapidly and if you submit a PR, chances are I've either already done it or made a change that breaks the subsystem you were working on. Or force-pushed on master. If you really want to contribute, let me know and we can sort something out.

Status tracker issue here.

Technology overview

Compiling requires Rust 2018 Edition with Cargo, i.e. stable version 1.31 or later, or a nightly compiler. You should install it with rustup.

  • XML parsing with roxmltree
  • Error reporting with codespan
  • Little utility parsers written with nom
  • Incremental computation using salsa
  • Parallel processing using rayon's work-stealing queues
  • JSON IO using serde_json
  • Pandoc-JSON interop using (currently) an internal fork of pandoc_types.

Try it out!

Currently it can:

  • parse a CSL style (ignoring <info>) with built-in validation, type-checking, error reporting, and semantic versioning,
  • parse a CSL-M style (ignoring <info>, probably still missing many undocumented citeproc-js extensions to the spec),
  • parse locale files and perform locale fallback and merging for terms/dates/etc inside it
  • parse a CSL-JSON file into references
  • pluck out a particular reference, and execute the style against only that one
  • read and write cites for an entire Pandoc JSON document

Parse a style

git clone https://github.com/cormacrelf/citeproc-rs
cd citeproc-rs/crates/citeproc-cli
cargo run -- --csl ../example.csl # runs on a predefined single ref
cargo run -- --csl ../example.csl --library path/to/csl-json/file.json

To test it across the entire styles repo:

cd citeproc-rs/crates/citeproc-cli
cargo install --path . --force
cd ../..
git clone https://github.com/citation-style-language/styles
for style in styles/*.csl; do citeproc-rs --csl $style | pandoc -f json -t html; done
  • Some styles in the repo are possibly invalid (mostly for using terms that don't exist).
  • Some will successfully output HTML!

Parse a locale

You can also parse a locale to check for errors. It can find a locale in a locales directory assuming it is structured like the official CSL locales repo, found via directories (the cache directory). Shortcut:

# clones the locales repo into place for you
cargo pull-locales

Then:

# currently broken
cd crates/citeproc-cli
cargo run -- parse-locale --lang en-GB

The big end-to-end Pandoc filter (currently broken)

Step 1: export a CSL-JSON library somewhere, with Zotero for example

Step 2: create a markdown file

It must contain inline csl/bibliography metadata. Currently, and contrary to its documentation, Pandoc will automatically add -F pandoc-citeproc whenever you add command line --metadata csl=XXX or --metadata bibliography=XXX flags. (That is, as far as I know, only supposed to happen if you use shorthand --csl XXX or --bibliography XXX.)

---
csl: path-to-my-csl.csl
bibliography: path-to-my-csl-json-library.json
---

First paragraph.[@knownCitekey]

Second paragraph.[@knownCitekey; @anotherOne]

Step 4: Run as a filter!

# much quicker than `build --release` or `install --path .`
cargo build

pandoc -F ../target/debug/citeproc-rs input.md -s -o out.html

open out.html

Running the CSL test suite

# setup once
cargo pull-locales # if not done already
cargo pull-test-suite

cd crates/citeproc

# the whole suite in parallel
cargo test

# for a particular test, paste the file name
cargo test name_ParsedDroppingParticleWithApostrophe.txt

# for a subset of tests with some commonality in the name (this runs 8 of them)
cargo test name_Initials

Run cargo test -- --test-threads 1 to have the tests run in a deterministic order (i.e. alphabetically); this helps show related tests alongside one another in the terminal output. Run cargo test -- --help for more options.

You can’t perform that action at this time.