Embabel Benchmark

Embabel (Em-BAY-bel) is a framework for authoring agentic flows on the JVM that seamlessly mix LLM-prompted interactions with code and domain models. Supports intelligent path finding towards goals. Written in Kotlin but offers a natural usage model from Java. From the creator of Spring, Rod Johnson.

Rod has started publishing a series of blog posts converting examples written in CrewAI to Embabel.

It is well known that Java will generally outperform Python in performance, sometimes by a significant margin. While agentic workflows the majority time will be spent on LLM interactions, the question remains how much of the time is spent with the program execution itself?

This repository contains a series of benchmarks that compare the performance of Java and Python implementations of the same agentic workflow.

This is a work in progress. We will attempt to keep the comparisions apples to apples as much as possible. CrewAI has a number of examples published that we can use as reference examples.

Project Pages

Embabel

Home Page

CrewAI

Home Page

Java vs Python Benchmarks

All data is from The Computer Language Benchmarks Game.

fannkuch-redux

Source	secs	mem
Java	10.48	62,661
Python	331.18	28,701

n-body

Source	secs	mem
Java	24.54	60,711
Python	641.51	11,625

spectral-norm

Source	secs	mem
Java	4.16	101,253
Python	172.95	30,195

mandelbrot

Source	secs	mem
Java	1.65	61,784
Python	153.11	30,832

pidigits

Source	secs	mem
Java	1.91	60,671
Python	2.16	14,260

regex-redux

Source	secs	mem
Java	9.38	679,531
Python	7.21	194,561

fasta

Source	secs	mem
Java	2.36	60,948
Python	39.03	12,218

k-nucleotide

Source	secs	mem
Java	17.26	366,419
Python	147.52	181,992

reverse-complement

Source	secs	mem
Java	3.19	713,016
Python	8.44	499,396

binary-trees

Source	secs	mem
Java	8.35	1,012,333
Python	142.37	450,490

Project Structure

embabel-benchmark/
├── README.md
├── crewai/                # Python project (CrewAI examples/benchmarks)
│   ├── pyproject.toml
│   └── src/
│       └── meeting_prep_app/
│           ├── cli.py
│           └── agents.py
├── embabel/               # Java project (Embabel on the JVM)
│   ├── HELP.md
│   ├── src/
│   │   └── test/
│   │       └── java/
│   │           └── guru/
│   │               └── springframework/
│   │                   └── embabel/
│   │                       └── EmbabelApplicationTests.java
└── embabel-benchmark.iml

Agentic Flows

Meeting Prep App

• Overview blog post by Rod Johnson: Build Better Agents in Java than Python: Improving on CrewAI example
• Embabel Implementation: Meeting Prep App
• CrewAI Implementation: Meeting Prep App

Write a Book Flow

• Overview blog post by Rod Johnson: You Can Build Better AI Agents in Java than Python
• Embabel Implementation: Write a Book Flow
• CrewAI Implementation: Write a Book Flow