Comprehensive guide to generative AI projects and resources in/for/associated with Julia.
Julia is a high-level, high-performance dynamic language for technical computing.
Generative AI encompasses algorithms and models that leverage large-scale machine learning to generate new content (across many modalities), automate, understand, parse, extract and much more, adapting to a wide range of applications beyond mere content creation.
- Generative AI Projects and Julia
- Models
- API SDKs
- Packages
- JLL Packages
- Benchmarks/Comparisons
- Applications/Products
- Tutorials/learning
- Noteworthy Mentions
- Must-Know Python Projects
- Other Awesome Lists
- JuliaGenAI Organization - A GitHub organization and a community of Julia developers and researchers working on generative AI.
Important
Google Summer of Code 2024 with JuliaGenAI
Leap into JuliaGenAI's Google Summer of Code: Unleash your potential, learn about practical applications, and make pivotal contributions in the exciting world of generative AI with Julia—a chance to grow, impact, and thrive in an expanding ecosystem. Learn more
Build, train, and deploy Large language models in Julia.
- Flux.jl - Flux is a machine learning library for Julia that is flexible and allows building complex models. However, at the time of writing, I'm not aware of any Large Language Models (LLMs) that have been implemented and trained in Flux.
- Transformers.jl - Transformers.jl is a Julia package that provides a high-level API for using pre-trained transformer models. It also allows to download any models from Hugging Face hub with
@hgf_strmacro string. - Pickle.jl - Great package for loading Pytorch weights into Julia (if you want to implement models yourself).
- BytePairEncoding.jl - Pure Julia implementation of Byte Pair Encoding (BPE) algorithm. It's used by Transformers.jl to tokenize text.
- Llama2.jl - Llama2.jl provides simple code for inference and training of llama2-based language models based on llama2.c. It supports loading quantized weights in GGUF format (
q4_K_Svariant). Other similar projects: LanguageModels.jl - Llama.jl - Julia interface to llama.cpp, a C/C++ library for running language models locally. Supports a wide range of models.
Access Generative AI models via official APIs.
- OpenAI.jl - A community-maintained Julia wrapper to the OpenAI API.
Access Generative AI models via SDKs of popular cloud service providers.
- GoogleCloud.jl - SDK for Google Cloud. There is an open PR to enable Vertex AI endpoints.
- Pinecone.jl - SDK for Pinecone.io vector database.
| Name | Julia Client | Usage Examples |
|---|---|---|
| Elasticsearch | ElasticsearchClient.jl | GptSeachPlugin with Elasticsearch |
| OpenSearch | ElasticsearchClient.jl | GptSeachPlugin with Opensearch |
| PostgreSQL + pgvector | LibPQ.jl | pgvector examples for Julia |
- ReplGPT.jl - Brings ChatGPT interface as a Julia REPL mode.
- HelpGPT.jl - Calls ChatGPT to explain any errors in Julia code.
- GenGPT3.jl - A Gen.jl generative function that wraps the OpenAI API.
- GPTCodingTools.jl - Code generation tool for Julia language with useful prompt templates and self-healing features (ala OpenAI Code Interpreter). It does work, but development has been abandoned. (Disclaimer: I'm the author of this package.)
- PromptingTools.jl - Helps with everyday applications of Large Language Models in Julia by wrapping coming APIs, re-using prompts via templates, and enabling easy transition between different model providers (eg, OpenAI, Ollama). (Disclaimer: I'm the author of this package.)
- LLMTextAnalysis.jl - Leverage Large Language Models to uncover, evaluate, and label themes/concepts/spectra in large document collections. (Disclaimer: I'm the author of this package.)
JLLs are prebuilt libraries and executables to easily install and call non-Julia projects (eg, C/C++). Often they are the first step towards a Julia package with an idiomatic interface.
- llama_cpp_jll.jl - JLL package for llama.cpp, the best interface for quantized llama2-style models.
- Julia LLM Leaderboard - Comparison of Julia language generation capabilities of various Large Language Models across a range of tasks. Visit if you want help choosing the right model for your application.
- HumanEval.jl - The Julia version of openai/human-eval. It rewrites the original Python problems into the Julia version and provides evaluation results with several latest LLMs.
Applications and products that "work" with Julia language.
- GitHub Copilot - Excellent inline suggestions with the help of OpenAI models. It works extremely well with Julia language for repetitive tasks one line at a time, but larger code chunks are rarely correct.
- Codium.ai - Great IDE or VSCode plugin for code analysis, suggestion and generation of test suites. Although the tests are written more in the style of Pytest rather than idiomatic Julia. It has a free tier.
- Replit - Replit's REPL is powered by an in-house model that supports Julia language.
- Codeium - Free alternative to GitHub Copilot with extensions for most editors.
Julia-affiliated applications and products using LLMs
- JuliaHub AskAI - AskAI is a JuliaHub's RAG (Retrieval Augmented Generation) application that allows users to ask questions about the Julia language and its ecosystem. It is free, but you need to be logged in to JuliaHub to use it.
- Genie UI Assistant - Genie UI Assistant is a GPT-4 powered UI builder helping Genie.jl's users create UIs faster using natural language.
- Comind - A social network, messaging, and LLM interface built in Julia.
- Tutorial for using LLMs with Transformers.jl - A brief tutorial on how to use Transformers.jl to access LLMs from HuggingFace Hub.
- Building a RAG Chatbot over DataFrames.jl Documentation - Hands-on Guide - A hands-on guide on how to build a RAG chatbot over DataFrames.jl documentation using only minimal dependencies.
- GenAI Mini-Tasks: Extracting Data from (.*)? Look No Further! - A tutorial on structured data extraction. A part of a larger series of tutorials on small tasks that can be done with GenAI.
Some of the below projects are not necessarily Julia-specific, but noteworthy mentions in the generative AI space and interesting for Julia developers.
- Ollama - The best option for those looking to host a Large Language Model locally. Simply start the server and send the requests with HTTP.jl.
- LM Studio - A desktop app for hosting and interacting with LLMs locally. It's a great option for those who want to use LLMs without coding. It's free for personal use.
- GenerativeModels.jl - Useful library to train more traditional generative models like VAEs. It's built on top of Flux.jl.
Python is on the leading edge of the generative AI revolution. Fortunately, we have PythonCall.jl allowing us to easily call all the below Python packages.
- HuggingFace Transformers - The most popular library for accessing LLMs and other models. It can be mostly used via Transformers.jl (see above).
- LangChain - The best option for building applications on top of LLMs (eg, Chains, Agents). It has a lot of adapters for common models, databases, and other services.
- Llama Index - Similar to LangChain but with a focus on data-centered applications like RAG.
- Instructor - Simple yet powerful structured extraction framework on top of OpenAI API. Excellent to understand the power of function calling API together with Pydantic.
- Marvin - Powerful building blocks to quickly build AI applications and expose them via a production-ready API.
- Open Interpreter - Let LLMs run code on your computer (eg, Python, JavaScript, Shell, and more). An open-source local alternative to OpenAI Code Interpreter.
- Awesome Generative AI - Great list for all things generative AI. An inspiration for this list!