Anyone have quick guides or clearer instructions on how to build an Android Studio app with Gemma 3n Model

[htxzx20]

"Getting Started with Gradle
While LiteRT-LM is developed with Bazel, we provide the Maven packages for Gradle/Maven users.

1. Add the Gradle dependency
   dependencies {
   // For Android
   implementation("com.google.ai.edge.litertlm:litertlm-android:latest.release")

   // For JVM (Linux, MacOS, Windows)
   implementation("com.google.ai.edge.litertlm:litertlm-jvm:latest.release")
   }
   You can find the available versions on Google Maven in litertlm-android and litertlm-jvm.

latest.release could be used to get the latest release.

2. Initialize the Engine
   The Engine is the entry point to the API. Initialize it with the model path and configuration. Remember to close the engine to release resources.

Note: The engine.initialize() method can take a significant amount of time (e.g., up to 10 seconds) to load the model. It is strongly recommended to call this on a background thread or coroutine to avoid blocking the UI thread.

import com.google.ai.edge.litertlm.Backend
import com.google.ai.edge.litertlm.Engine
import com.google.ai.edge.litertlm.EngineConfig

val engineConfig = EngineConfig(
modelPath = "/path/to/your/model.litertlm", // Replace with your model path
backend = Backend.CPU, // Or Backend.GPU
// optional: Pick a writable dir. This can improve 2nd load time.
// cacheDir = "/tmp/" or context.cacheDir.path (for Android)
)

val engine = Engine(engineConfig)
engine.initialize()
// ... Use the engine to create conversation ...

// Close the engine when done
engine.close()
On Android, to use the GPU backend, the app needs to request explicitly by adding the following to your AndroidManifest.xml inside the tag:

3. Create a Conversation Once the engine is initialized, create a Conversation instance. You can provide a ConversationConfig to customize its behavior.

import com.google.ai.edge.litertlm.ConversationConfig
import com.google.ai.edge.litertlm.Message
import com.google.ai.edge.litertlm.SamplerConfig

// Optional: Configure system message and sampling parameters
val conversationConfig = ConversationConfig(
systemMessage = Message.of("You are a helpful assistant."),
samplerConfig = SamplerConfig(topK = 10, topP = 0.95, temperature = 0.8),
)

val conversation = engine.createConversation(conversationConfig)
// Or with default config:
// val conversation = engine.createConversation()

// ... Use the conversation ...

// Close the conversation when done
conversation.close()
Conversation implements AutoCloseable, so you can use the use block for automatic resource management for one-shot or short-lived conversation:

engine.createConversation(conversationConfig).use { conversation ->
// Interact with the conversation
}
4. Sending Messages
There are three ways to send messages:

sendMessage(message: Message): Message: Synchronous call that blocks until the model returns a complete response. This is simpler for basic request/response interactions.
sendMessageAsync(message: Message, callback: MessageCallback): Asynchronous call for streaming responses. This is better for long-running requests or when you want to display the response as it's being generated.
sendMessageAsync(message: Message): Flow: Asynchronous call that returns a Kotlin Flow for streaming responses. This is the recommended approach for Coroutine users.
Synchronous Example:

import com.google.ai.edge.litertlm.Content
import com.google.ai.edge.litertlm.Message

val userMessage = Message.of("What is the capital of France?")
print(conversation.sendMessage(userMessage))
Asynchronous Example with callback:

Use sendMessageAsync to send a message to the model and receive responses through callback.

import com.google.ai.edge.litertlm.Content
import com.google.ai.edge.litertlm.Message
import com.google.ai.edge.litertlm.MessageCallback
import java.util.concurrent.CountDownLatch
import java.util.concurrent.TimeUnit

val callback = object : MessageCallback {
override fun onMessage(message: Message) {
print(message)
}

override fun onDone() {
    // Streaming completed
}

override fun onError(throwable: Throwable) {
    // Error during streaming
}

}

val userMessage = Message.of("What is the capital of France?")
conversation.sendMessageAsync(userMessage, callback)
Asynchronous Example with Flow:

Use sendMessageAsync (without the callback arg) to send a message to the model and receive responses through a Kotlin Flow.

import com.google.ai.edge.litertlm.Content
import com.google.ai.edge.litertlm.Message
import kotlinx.coroutines.launch

// Within a coroutine scope
val userMessage = Message.of("What is the capital of France?")
conversation.sendMessageAsync(userMessage)
.catch { ... } // error during streaming
.collect{ print(it.toString()) }
5. Multi-Modality
Note: this only works with models with multi-modality support. e.g., the Gemma3n.

Message objects can contain different types of Content, including Text, ImageBytes, ImageFile, and AudioBytes, AudioFile.

// Initialize the visionBackend and/or the audioBackend
val engineConfig = EngineConfig(
modelPath = "/path/to/your/model.litertlm", // Replace with your model path
backend = Backend.CPU, // Or Backend.GPU
visionBackend = Backend.GPU,
audioBackend = Backend.CPU,
)

// See the Content class for other variants.
val multiModalMessage = Message.of(
Content.ImageFile("/path/to/image"),
Content.AudioBytes(audioBytes), // ByteArray of the audio
Content.Text("Describe this image and audio."),
)
6. Defining and Using Tools
Note: this only works with models with tool support.

You can define custom Kotlin functions as tools that the model can call to perform actions or fetch information.

Defining a ToolSet
Create a class and annotate methods with @tool and parameters with @ToolParam.

import com.google.ai.edge.litertlm.Tool
import com.google.ai.edge.litertlm.ToolParam

class SampleToolSet {
@tool(description = "Get the current weather for a city")
fun getCurrentWeather(
@ToolParam(description = "The city name, e.g., San Francisco") city: String,
@ToolParam(description = "Optional country code, e.g., US") country: String? = null,
@ToolParam(description = "Temperature unit (celsius or fahrenheit). Default: celsius") unit: String = "celsius"
): Map<String, Any> {
// In a real application, you would call a weather API here
return mapOf("temperature" to 25, "unit" to unit, "condition" to "Sunny")
}

@Tool(description = "Get the sum of a list of numbers.")
fun sum(
    @ToolParam(description = "The numbers, could be floating point.") numbers: List<Double>,
): Double {
    return numbers.sum()
}

}
Behind the scenes, the API inspects these annotations and the function signature to generate an OpenAPI-style schema. This schema describes the tool's functionality, parameters (including their types and descriptions from @ToolParam), and return type to the language model.

Parameter Types
The types for parameters annotated with @ToolParam can be String, Int, Boolean, Float, Double, or a List of these types (e.g., List). Use nullable types (e.g., String?) to indicate nullable parameters. Set a default value to indicate that the parameter is optional, and mention the default value in the description in @ToolParam.

Return Type
The return type of your tool function can be any Kotlin type. The result will be converted to a JSON element before being sent back to the model.

List types are converted to JSON array.
Map types are converted to JSON object.
Primitive types (String, Number, Boolean) are converted to the corresponding JSON primitive.
Other types are converted to string with the toString() method.
For structured data, returning Map or a data class that will be converted to a JSON object is recommended.

Registering Tools
Include instances of your tool sets in the ConversationConfig.

val conversation = engine.createConversation(
ConversationConfig(
tools = listOf(SampleToolSet())
// ... other configs
)
)

// Send messages that might trigger the tool
val userMessage = Message.of("What's the weather like in London?")
conversation.sendMessageAsync(userMessage, callback)
The model will decide when to call the tool based on the conversation. The results from the tool execution are automatically sent back to the model to generate the final response.

Error Handling
API methods can throw LiteRtLmJniException for errors from the native layer or standard Kotlin exceptions like IllegalStateException for lifecycle
issues. Always wrap API calls in try-catch blocks. The onError callback in MessageCallback will also report errors during asynchronous operations."

^I found this under the kotlin folder, but for some steps, there's no clear instructions like which file to add the codes to. Will appreciate if someone has a more detailed guide!

[whhone]

For a quick start on Android, you can adb push the model to a phone location.

# Push the model file from computer to your Android device
adb push model.litertlm /data/local/tmp/

val engineConfig = EngineConfig(
  modelPath = "/data/local/tmp/model.litertlm", // Replace with your model path
  backend = Backend.CPU, // Or Backend.GPU
  cacheDir = context.cacheDir.path, // the app might not have "write" permission on /data/local/tmp/
)

For production, you might need to implement the logic to download the model.

Thanks for the feedback. I will update the guide with some tips like this.

Also check out our sample app https://github.com/google-ai-edge/gallery.

[htxzx20]

Hi @whhone , thank you for the reply! I'm trying to look at the sample app and trying to download and launch from Android Studio, but when I open in Android Studio, there's no function to run the app (it only shows Add Configuration). Am I supposed to launch only the Android folder or is there something else I'm missing?

[scsonic]

it looks like you open the wrong folder
open the android folder and press gradle sync