Skip to content

CategoricalData/hydra

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4,052 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Hydra

Welcome to the Hydra project! Hydra is a unique functional programming language based on the LambdaGraph data model. It explores an isomorphism between labeled hypergraphs and typed lambda calculus: in Hydra, programs are graphs, and graphs are programs. Hydra has the ability to translate its own kernel into several other languages, including Haskell (Hydra-Haskell), Java (Hydra-Java), Python (Hydra-Python), Scala (Hydra-Scala), and Lisp (Hydra-Lisp, with four dialects: Clojure, Scheme, Common Lisp, and Emacs Lisp). All five implementations are complete and pass the common test suite. Hydra is used for data modeling, validation, and transforms at Microsoft, while its closed-source predecessor Dragon was used for data integration and graph construction at Uber. The language is now being developed for its own sake, with the intention of becoming an Apache Incubator project, integrating more directly with Apache TinkerPop, and branching out into additional concrete programming languages. Typical use cases include:

  • Graph construction. Hydra supports TinkerPop-style property graphs as well as RDF and SHACL, and has been used in combination with the ISO/IEC GQL standard. Hydra provides DSLs for defining schemas and mappings, as well as tools for validating schemas and data, and moving them seamlessly into and out of the graph formats.

  • Data integration. Hydra includes "coders" (encoders+decoders) for many data and schema languages which you can easily compose together to build data transform pipelines. Some of the currently supported languages and formats include Protobuf, Avro, JSON and YAML, RDF formats including N-Triples, GraphQL, LinkedIn's PDL Schema language, as well as simple tabular data (CSV/TSV). Hydra has been used extensively with (Delta) Parquet, although this support is not currently open source. There is also limited support for C Sharp.

  • Computational graphs. Unusually among graph data models and query languages, Hydra has deep support for polymorphism, as well as embedding computational elements with a graph (sometimes called computational knowledge graphs). As we mentioned, programs are graphs and vice versa. See the KGC 2024 presentation Graphs, logics, and lambda calculus for examples.

One of the less exciting, but very important aspects of Hydra is its test suite, which guarantees parity across the supported programming languages. This ensures that each Hydra implementation behaves the same, and we think it will be very useful in heterogeneous environments like TinkerPop where we need exactly the same logic -- validation, query steps, user-defined functions, etc. -- to be manifested identically in more than one programming language.

If any of the above sounds interesting, feel free to ask questions or get involved via the LambdaGraph Discord server. We are preparing for the 1.0 release, with all five core implementations (Haskell, Java, Python, Scala, Lisp) now complete. Near-term goals include integrating with Apache TinkerPop, expanding language support, and growing the community. There is also an introductory blog post by G.V (Amber Lennox), a bootstrapping demo video showing mutual self-hosting across three languages, and a translingual programming demo from Data Day Texas. See also the earlier presentations here and here, and the original design document.

Documentation

Releases

All Hydra implementations share a single version number. The current release is 0.14.0; see the CHANGELOG for details.

Implementation Package Install
Haskell hydra on Hackage cabal install hydra or add to package.yaml
Java hydra-java on Maven Central net.fortytwo.hydra:hydra-java:0.14.0
Java (extensions) hydra-ext on Maven Central net.fortytwo.hydra:hydra-ext:0.14.0
Python conda-forge (in progress) Coming soon
Scala Not yet published Build locally with sbt compile in hydra-scala/
Lisp Not yet published See dialect-specific instructions in hydra-lisp/

See the release process for how releases are built and published.

About

Graph programming language

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors