Provide an asynchronous LLM caller that can integrate with various LLM services, including Ollama.
- OpenAI
- Gemini
- Anthropic
- Deepseek
- Ollama
To use this library in your Rust project, add it as a dependency in your Cargo.toml:
[dependencies]
easy_llm = "0.1.2"
# although it is not necessary, recommend use tokio because llm call function is async funtion.
tokio = { version = "1.47.*", features = ["rt", "rt-multi-thread"] }- For
OpenAi,Gemini,Anthropic, andDeepseekproviders, you must provide a valid API key. - For
Ollama, an API key is not required, as it typically interacts with a local or self-hosted instance.
let user_input: String = input(); // input function...
let selected_provider: Provider = Provider::parse_str(user_input).unwrap();Ensure you have an Ollama server running locally (default: http://localhost:11434) and a model downloaded (e.g., gpt-oss or llama2).
use easy_llm::{Error, Llm, LlmControl, Provider};
#[tokio::main]
async fn main() -> Result<(), Error> {
let ollama_client = Llm::new(
Provider::Ollama, // or use `Provider::parse_str()`
"gpt-oss".to_string(), // Replace with your downloaded Ollama model
None, // No API key needed for Ollama
None, // Optional: temperature
None, // Optional: max_tokens
None, // Optional: base_url for Ollama (defaults to http://localhost:11434, 11434)
// Type: `Option<(String, u16)>`
);
println!("Calling Ollama...");
match ollama_client
.call("Tell me a short story about a brave knight.")
.await
{
Ok(response) => println!("Ollama Response: {}", response),
Err(e) => eprintln!("Error calling Ollama: {:?}", e),
}
Ok(())
}e.g. response by gpt-oss
Calling Ollama... Ollama Response: In the mist‑shrouded valleys of Eldrath, a young squire named Rowan dreamt of becoming a knight. One dawn, while the sun painted the cliffs gold, a desperate plea echoed through the stone walls of the castle: the dragon of Mount Varth had returned, breathing fire upon the nearby villages. The king, weary and fearful, called upon the bravest among his knights to confront the beast.Rowan, though still unknighted, stepped forward. His heart thudded louder than any drum, but his resolve was iron‑clad. He wore a plain leather tunic, but he carried the gleam of his mentor’s sword, a relic of the old wars. The king, humbled, granted him a simple steel helm and the knight’s oath: “May the bravest heart guide your blade.”
The dragon’s roar tore the air as Rowan approached the mountain’s cliff. The beast, scales shimmering like storm clouds, spat flame, but Rowan’s shield, forged from the valley’s own iron, reflected the fire, turning it back upon the dragon’s nostrils. He slipped past the flame, and with a swift motion, struck the dragon’s eye, a glinting speck of death in the beast’s mighty gaze. The dragon roared in pain, but the wound was fatal; it collapsed, its fire dying like a dying star.
When Rowan returned, the people cheered, the king’s eyes glistening with gratitude. The young squire had proven that bravery is not measured by titles, but by the courage to act when the world calls. Rowan was crowned the bravest knight of Eldrath, but he kept his simple helm, a reminder that the true armor of a knight is the heart that never yields. Shared connection to archlinux closed.
You will need a Google Gemini API key.
use easy_llm::{Error, Llm, LlmControl, Provider};
fn get_test_gemini_api() -> String {
std::fs::read_to_string(std::env::current_dir().unwrap().join("_test_api_key")).unwrap()
}
#[tokio::main]
async fn main() -> Result<(), Error> {
let gemini_client = Llm::new(
Provider::Gemini, // or use `Provider::parse_str()`
"gemini-2.5-pro".to_string(), // Replace with your downloaded Ollama model
Some(get_test_gemini_api()), // No API key needed for Ollama
None, // Optional: temperature
None, // Optional: max_tokens
None, // Optional: base_url for Ollama (defaults to http://localhost:11434, 11434)
// Type: `Option<(String, u16)>`
);
println!("Calling Gemini...");
match gemini_client
.call("Tell me a short story about a brave knight.")
.await
{
Ok(response) => println!("Gemini Response: {}", response),
Err(e) => eprintln!("Error calling Ollama: {:?}", e),
}
Ok(())
}e.g. response by gemini-2.5-pro
Calling Gemini... Gemini Response: Sir Kaelan was a knight forged in the fires of war. His shield was dented from a dozen battles, and a fine white scar cut across his left eyebrow, a memento from an orc’s axe. He was known throughout the kingdom as Kaelan the Fearless, a man whose courage was as reliable as the rising sun.But the challenge that came to him was not one of steel and fury. It arrived with a desperate farmer from the quiet valley of Stillwater, a place known for its golden wheat and peaceful days.
"It's the Silence, Sir Knight," the farmer pleaded, his hands trembling. "A creeping dread that settles in the fields. The birds have stopped singing. The river makes no sound. Our children no longer laugh. It drains the life from everything, and we are afraid."
Kaelan, expecting a beast to slay, was puzzled. A knight cannot fight a feeling with a sword. Still, he rode to Stillwater.
The farmer had not exaggerated. The moment he crossed the old stone bridge into the valley, the world went unnervingly quiet. The wind rustled the wheat, but made no sound. The air was heavy, thick with an unseen sorrow. He saw villagers moving like ghosts, their faces etched with a deep, weary despair.
He patrolled the valley, his sword at the ready, but found no monster. He spent a night in the silent fields, but no ghouls rose from the earth. The enemy was invisible, intangible. It wasn't a physical threat, but a spiritual one. He felt it himself—a cold tendril of hopelessness wrapping around his own heart. For the first time, Kaelan felt a flicker of true fear. This was a foe his strength was useless against.
Defeated, he sat by the motionless river as dusk settled. He watched a little girl sitting on her porch, clutching a wooden doll, her eyes wide and empty. He remembered the songs his mother used to sing, songs of heroes and sunlight.
An idea, small and uncertain, sparked within him. He was a knight. His duty was to protect the people. If he couldn't protect their bodies, perhaps he could protect their spirits.
He unstrapped his great war horn from his saddle, an instrument used to signal charges in battle. His hands felt clumsy. He had never used it for anything but war. He took a deep breath, filling his lungs with the valley's oppressive air, and blew.
A single, clear note shattered the Silence. It wasn’t a call to arms, but a long, steady, and hopeful sound that echoed from the silent hills. It was startlingly loud in the dead quiet. Heads turned. Windows opened. The little girl on the porch looked up, a flicker of surprise in her eyes.
Kaelan didn't stop. He began to walk through the village, playing the simplest tune he could remember, a lullaby from his childhood. His music was not masterful, but it was real. It was a sound of life in a place that had forgotten it.
An old woman began to hum along, her voice cracked and weak at first, then growing stronger. A blacksmith, hearing the horn, tentatively picked up his hammer and struck his anvil—Clang! The sound was a thunderclap. The little girl on the porch let out a small, hesitant giggle.
The sounds grew. A dog barked. A baker began to sing as he kneaded dough. The villagers, roused by Kaelan’s simple act, began to make noise, not out of duty, but out of a rediscovery of joy. They talked, they laughed, they worked, filling their valley with the music of their own lives.
As the sun rose the next morning, Kaelan heard it. A faint chirping. Then another. The birds had returned. The river seemed to whisper over its stones. The Silence had been broken, not by a sword, but by a song.
Sir Kaelan rode away from Stillwater not with a monster's head on his saddle, but with a lighter heart. He had learned that true bravery wasn't just about facing down a dragon. Sometimes, it was about being the first voice to sing in the dark.
MIT or Apache-2.0