Document AI/LLM, ChatView, speech, and ML Kit cn1libs from PR #5035#5057
Merged
Conversation
PR #5035 introduced com.codename1.ai (LlmClient, ChatRequest, streaming SSE, tool calls, embeddings, image generation, conversation persistence, retries, simulator Ollama redirect), the com.codename1.components.ChatView component, com.codename1.media.SpeechRecognizer / TextToSpeech, the non-prompting SecureStorage overloads, and the cn1-ai-mlkit-{barcode, docscan,face} cn1libs. The PR shipped the framework code but left both the developer guide and the initializr authoring skill silent on every new API. Adds a new "AI, Chat UI, and Speech" chapter to the developer guide: provider factories, streaming, tool calling, structured output, multi- modal messages, embeddings, image generation, conversation store, prompt templates, retry policy, simulator redirect, SecureStorage key handling, ChatView wiring and theming UIIDs, SpeechRecognizer / TextToSpeech, the three ML Kit cn1libs, and a capture-photo -> describe -> speak example. Includes an SVG mockup of the ChatView surface. Adds references/ai-and-speech.md to the bundled initializr authoring skill, indexed from SKILL.md and exercised from GeneratorModelMatrixTest, so Claude Code (and any agent that respects the skills convention) inside a generated project picks up the new APIs automatically. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Cloudflare Preview
|
Contributor
|
Developer Guide build artifacts are available for download from this workflow run:
Developer Guide quality checks: |
Local reproduction with language-tool-python==2.9.4 (the CI-pinned
version) flagged three matches in Ai-And-Speech.asciidoc:
- PCT_SINGULAR_NOUN_PLURAL_VERB_AGREEMENT on "the simulator, the cloud
build, and your CI pipeline route everything" -- the rule wanted
the singular-vs-plural agreement reworded. Rephrased to ":
the simulator, the cloud builder, and your CI pipeline all run
the same code."
- MORFOLOGIK_RULE_EN_US on "thread-safe mutators" -- the en_US
dictionary doesn't recognise "mutators". Spelled the methods out
("addMessage, appendToLastMessage, and setTypingIndicatorVisible").
- MORFOLOGIK_RULE_EN_US on "SAPI on Windows" -- the flagged span was
the two-word phrase, so the accept-list entry for "SAPI" alone did
not match it. Wrapped SAPI in a code span so the HTML extractor
replaces it with the inline placeholder before LanguageTool sees
the surrounding prose.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The first revision shipped a hand-authored SVG sketch of the ChatView component. Swap it for an actual PNG produced by the JavaSE simulator running the component under the iOS Modern theme so the developer guide shows the real rendering, not a wireframe. - Adds ChatViewDevGuideScreenshotTest in scripts/hellocodenameone (under common/src/test/java, where cn1:test discovers AbstractTest implementers). The test installs iOSModernTheme.res, builds a Form with a populated ChatView, paints it to a PNG, and writes it to Storage as chat-view.png. Running it via mvn -pl javase -am test (with the test profile activator) drops ~/.cn1/chat-view.png. - Adds docs/developer-guide/img/chat-view.png, captured from a local simulator run via the test above. - Drops docs/developer-guide/img/chat-view-mockup.svg. - Points the chapter's image:: at the new PNG with caption "ChatView rendered in the JavaSE simulator under the iOS Modern theme". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Compared 47 screenshots: 47 matched. |
Collaborator
Author
|
Compared 116 screenshots: 116 matched. Native Android coverage
✅ Native Android screenshot tests passed. Native Android coverage
Benchmark ResultsDetailed Performance Metrics
|
Collaborator
Author
|
Compared 115 screenshots: 115 matched. Benchmark Results
Build and Run Timing
Detailed Performance Metrics
|
Collaborator
Author
|
Compared 116 screenshots: 116 matched. Benchmark Results
Build and Run Timing
Detailed Performance Metrics
|
The original AnthropicClient and GeminiClient were scaffolds that threw UnsupportedOperationException on every call, with documentation telling callers to route through LlmClient.localOpenAiCompatible manually. Both providers now publish first-party OpenAI-compatible endpoints that speak the canonical /chat/completions wire format: - https://api.anthropic.com/v1/chat/completions - https://generativelanguage.googleapis.com/v1beta/openai/chat/completions That is the same shape OpenAiClient already implements end-to-end -- streaming, tool calls, multi-modal image parts, structured JSON output, the entire StreamingChatRequest plumbing. So both subclasses now extend OpenAiClient and only override the provider name and the default model: - LlmClient.anthropic(key) -> claude-sonnet-4-5 - LlmClient.gemini(key) -> gemini-2.0-flash LlmClient.DEFAULT_GEMINI_URL is bumped to the OpenAI-compat path (`/v1beta/openai`). AnthropicClient still overrides embed() with the helpful "Anthropic doesn't publish embeddings; use Voyage AI" error because that part remains true. Documentation updated: the AI chapter and the initializr authoring skill no longer hedge with "throws an LlmException until native clients land"; both providers work out of the box, with a default- model table per provider added to the skill. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
✅ Continuous Quality ReportTest & Coverage
Static Analysis
Generated automatically by the PR CI workflow. |
The previous wording explained two SecureStorage code paths (biometric-gated vs non-prompting) and ended with "remain the right choice for credentials a human user authenticates against," which read more like an API tour than safety guidance. The actual risk that needs surfacing is that hard-coded keys in a mobile binary are extractable. Replace the prose block with an asciidoc WARNING admonition that spells out the rule (no key in source / resource / git, fetch from a server the user authenticates against, then cache locally with the non-prompting overloads) and keeps just the short SecureStorage code sample. Mirror the same simplification in the initializr authoring skill. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 29, 2026
shai-almog
added a commit
that referenced
this pull request
May 29, 2026
Consolidated follow-up to the May 29 weekly index. Covers the four surfaces that moved from "you need a cn1lib for this" to "it is in the framework" this release: - Connectivity (PR #5021): com.codename1.io.{wifi,bonjour,usb} + NetworkManager.addNetworkTypeListener with the per-platform implementations and the three new CN1_INCLUDE_* defines that keep unused entitlements out of Apple's API-usage scan. - Identity (PRs #5018, #5039): OIDC client backed by ASWebAuthenticationSession / Custom Tabs with PKCE; Sign in with Apple in core; refreshed Google/Facebook/Microsoft/Auth0/Firebase wrappers; WebAuthn / passkey client in W3C JSON wire format with iOS 16 and Android API 28 native bindings; Auth0/Firebase passkey helpers; legacy Oauth2 deprecated. - Sharing (PR #5036): ShareResult callbacks (SHARED_TO/DISMISSED/FAILED) on iOS and Android via UIActivityViewController.completionWithItemsHandler and Intent.createChooser with IntentSender; IOSShareExtensionBuilder in the Maven plugin generates a complete .ios.appext bundle. - AI (PRs #5035, #5057): com.codename1.ai with LlmClient for OpenAI/Anthropic/Gemini/Ollama, streaming SSE, ChatView, SpeechRecognizer/TextToSpeech, SecureStorage non-prompting overloads, simulator Ollama redirect, AiDependencyTable build-time injection, and the cn1-ai-mlkit-{barcode,docscan,face} cn1libs. Closes with the structural element common to all four (scanner-driven gating that mirrors the NFC/biometrics pattern from two weeks ago).
shai-almog
added a commit
that referenced
this pull request
May 29, 2026
Consolidated follow-up to the May 29 weekly index. Covers the four surfaces that moved from "you need a cn1lib for this" to "it is in the framework" this release: - Connectivity (PR #5021): com.codename1.io.{wifi,bonjour,usb} + NetworkManager.addNetworkTypeListener with the per-platform implementations and the three new CN1_INCLUDE_* defines that keep unused entitlements out of Apple's API-usage scan. - Identity (PRs #5018, #5039): OIDC client backed by ASWebAuthenticationSession / Custom Tabs with PKCE; Sign in with Apple in core; refreshed Google/Facebook/Microsoft/Auth0/Firebase wrappers; WebAuthn / passkey client in W3C JSON wire format with iOS 16 and Android API 28 native bindings; Auth0/Firebase passkey helpers; legacy Oauth2 deprecated. - Sharing (PR #5036): ShareResult callbacks (SHARED_TO/DISMISSED/FAILED) on iOS and Android via UIActivityViewController.completionWithItemsHandler and Intent.createChooser with IntentSender; IOSShareExtensionBuilder in the Maven plugin generates a complete .ios.appext bundle. - AI (PRs #5035, #5057): com.codename1.ai with LlmClient for OpenAI/Anthropic/Gemini/Ollama, streaming SSE, ChatView, SpeechRecognizer/TextToSpeech, SecureStorage non-prompting overloads, simulator Ollama redirect, AiDependencyTable build-time injection, and the cn1-ai-mlkit-{barcode,docscan,face} cn1libs. Closes with the structural element common to all four (scanner-driven gating that mirrors the NFC/biometrics pattern from two weeks ago).
shai-almog
added a commit
that referenced
this pull request
May 29, 2026
Consolidated follow-up to the May 29 weekly index. Covers the four surfaces that moved from "you need a cn1lib for this" to "it is in the framework" this release: - Connectivity (PR #5021): com.codename1.io.{wifi,bonjour,usb} + NetworkManager.addNetworkTypeListener with the per-platform implementations and the three new CN1_INCLUDE_* defines that keep unused entitlements out of Apple's API-usage scan. - Identity (PRs #5018, #5039): OIDC client backed by ASWebAuthenticationSession / Custom Tabs with PKCE; Sign in with Apple in core; refreshed Google/Facebook/Microsoft/Auth0/Firebase wrappers; WebAuthn / passkey client in W3C JSON wire format with iOS 16 and Android API 28 native bindings; Auth0/Firebase passkey helpers; legacy Oauth2 deprecated. - Sharing (PR #5036): ShareResult callbacks (SHARED_TO/DISMISSED/FAILED) on iOS and Android via UIActivityViewController.completionWithItemsHandler and Intent.createChooser with IntentSender; IOSShareExtensionBuilder in the Maven plugin generates a complete .ios.appext bundle. - AI (PRs #5035, #5057): com.codename1.ai with LlmClient for OpenAI/Anthropic/Gemini/Ollama, streaming SSE, ChatView, SpeechRecognizer/TextToSpeech, SecureStorage non-prompting overloads, simulator Ollama redirect, AiDependencyTable build-time injection, and the cn1-ai-mlkit-{barcode,docscan,face} cn1libs. Closes with the structural element common to all four (scanner-driven gating that mirrors the NFC/biometrics pattern from two weeks ago).
shai-almog
added a commit
that referenced
this pull request
May 29, 2026
Consolidated follow-up to the May 29 weekly index. Covers the four surfaces that moved from "you need a cn1lib for this" to "it is in the framework" this release: - Connectivity (PR #5021): com.codename1.io.{wifi,bonjour,usb} + NetworkManager.addNetworkTypeListener with the per-platform implementations and the three new CN1_INCLUDE_* defines that keep unused entitlements out of Apple's API-usage scan. - Identity (PRs #5018, #5039): OIDC client backed by ASWebAuthenticationSession / Custom Tabs with PKCE; Sign in with Apple in core; refreshed Google/Facebook/Microsoft/Auth0/Firebase wrappers; WebAuthn / passkey client in W3C JSON wire format with iOS 16 and Android API 28 native bindings; Auth0/Firebase passkey helpers; legacy Oauth2 deprecated. - Sharing (PR #5036): ShareResult callbacks (SHARED_TO/DISMISSED/FAILED) on iOS and Android via UIActivityViewController.completionWithItemsHandler and Intent.createChooser with IntentSender; IOSShareExtensionBuilder in the Maven plugin generates a complete .ios.appext bundle. - AI (PRs #5035, #5057): com.codename1.ai with LlmClient for OpenAI/Anthropic/Gemini/Ollama, streaming SSE, ChatView, SpeechRecognizer/TextToSpeech, SecureStorage non-prompting overloads, simulator Ollama redirect, AiDependencyTable build-time injection, and the cn1-ai-mlkit-{barcode,docscan,face} cn1libs. Closes with the structural element common to all four (scanner-driven gating that mirrors the NFC/biometrics pattern from two weeks ago).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
com.codename1.ai(LlmClient, ChatRequest, streaming SSE, tool calls, embeddings, image generation, conversation persistence, retries, simulator Ollama redirect),com.codename1.components.ChatView,SpeechRecognizer/TextToSpeech, the non-promptingSecureStorageoverloads, and thecn1-ai-mlkit-{barcode,docscan,face}cn1libs. Includes an SVG mockup of theChatViewsurface.references/ai-and-speech.mdto the bundled initializr authoring skill (so Claude Code inside any generated project inherits the new APIs), index it fromSKILL.md, and add it toGeneratorModelMatrixTestfor regression.docs/developer-guide/languagetool-accept.txtwith the AI proper nouns (Ollama,[Ll]lama,[Vv]LLM,[Aa]nthropic,SAPI,espeak) that the en_US dictionary doesn't recognise.Test plan
asciidoctor --failure-level WARN docs/developer-guide/developer-guide.asciidocis clean (only the pre-existingINFO: possible invalid reference:lines remain)vale --config docs/developer-guide/.vale.ini --minAlertLevel=suggestion docs/developer-guide/Ai-And-Speech.asciidoc-> 0 errors / 0 warnings / 0 suggestionsruby scripts/developer-guide/check_paragraph_capitalization.rb docs/developer-guide/developer-guide.asciidocpassespython3 scripts/developer-guide/find_unused_images.py docs/developer-guide-> "No unused images found." (the newimg/chat-view-mockup.svgis referenced from the new chapter)🤖 Generated with Claude Code