Documentation has been rendered here.
This is an implementation of the algorithm described in our paper at POPL '23 (with appendices), extended with basic support for task parallelism as described in an extended version in JFP. The sequential algorithm is what you get if you take standard dual-numbers reverse AD as described e.g. by Brunel et al. (2019) and Huot et al. (2020, §6), as well as in Fig. 6 of our paper, and optimise it to be efficient. For details on how exactly these code transformations look and what the reasoning behind them is, we refer to our paper.
The non-parallel version of the implementation submitted as artifact for the POPL '23 paper can be found at commit 88854d0f2.
This is a reference implementation. While it correctly implements the algorithms, better performance can probably be obtained using basic taping as done in ad, although implementing the transformation at compile time (as this library does) does yield better performance in some cases, as GHC can optimise some intermediate computations. Any library with first-class support for arrays will be orders of magnitude faster than this library for array-heavy code.
This is a normal Haskell library, to be used in the standard fashion with Cabal or Stack.
The user API is exported from the Language.Haskell.ReverseAD.TH module.
There is a test suite (test) and a Criterion benchmark suite (bench) included in the package.
The script generate_table_1.py runs the benchmarks and generates something like Table 1 in the paper.
The code in this repository is available under the MIT licence from the authors of the paper.
@article{smeding-vakar-2025-parallel-dualrev,
author={Smeding, Tom J. and V{\'a}k{\'a}r, Matthijs I. L.},
title={Parallel dual-numbers reverse {AD}},
volume={35},
doi={10.1017/S0956796825100051},
journal={Journal of Functional Programming},
year={2025},
pages={e16}
}To cite this implementation specifically, you may also use:
@misc{2025-ad-dualrev-th,
author = {Smeding, Tom J. and V{\'{a}}k{\'{a}}r, Matthijs I. L.},
title = {Implementation of parallel dual-numbers reverse AD in Haskell},
month = {8},
year = {2025},
url = {https://github.com/tomsmeding/ad-dualrev-th}
}In case of questions or interest, feel free to reach out to me (tomsmeding.com).
This project received funding from NWO Veni grant number VI.Veni.202.124 and the ERC project FoRECAST.