Skip to content

Commit

Permalink
Add libcudf and spark-rapids to Implementations
Browse files Browse the repository at this point in the history
Add links to libcudf and spark-rapids implementations of t-digest
1. libcudf provides a generic CUDA kernel
2.  spark-rapids uses 1 to accelerate Spark SQL expressions approx_percentile and percentile_approx https://spark.apache.org/docs/latest/api/sql/#approx_percentile
  • Loading branch information
gerashegalov authored and tdunning committed Aug 26, 2022
1 parent 1af8a08 commit 721f82c
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Expand Up @@ -239,6 +239,7 @@ The t-digest algorithm has been ported to other languages:
- C++: [CPP TDigest](https://github.com/gpichot/cpp-tdigest), [FB's Folly Implementation (high performance)](https://github.com/facebook/folly/blob/master/folly/stats/TDigest.h)
- C++: [TDigest](https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/tdigest.h
) as part of [Apache Arrow](https://arrow.apache.org/)
- CUDA C++: [tdigest.cu](https://github.com/rapidsai/cudf/blob/branch-22.10/cpp/src/quantiles/tdigest/tdigest.cu) as part of `libcudf` in [RAPIDS](https://rapids.ai/) powering the [`approx_percentile` and `percentile_approx`](https://github.com/NVIDIA/spark-rapids/blob/b35311f7c6950fd5d8f7f6ed66aeffa87c480850/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuApproximatePercentile.scala#L123-L130) expressions in Spark SQL with [RAPIDS Accelerator for Apache Spark](https://nvidia.github.io/spark-rapids/)
- Rust: [t-digest](https://github.com/MnO2/t-digest) and its modified version in [Apache Arrow Datafusion](https://github.com/apache/arrow-datafusion/blob/ca952bd33402816dbb1550debb9b8cac3b13e8f2/datafusion-physical-expr/src/tdigest/mod.rs#L19-L28)
- Scala: [TDigest.scala](https://github.com/stripe-archive/brushfire/blob/master/brushfire-training/src/main/scala/com/stripe/brushfire/TDigest.scala)
- C: [tdigestc (w/ bindings to Go, Java, Python, JS via wasm)](https://github.com/ajwerner/tdigestc)
Expand Down

0 comments on commit 721f82c

Please sign in to comment.