Skip to content

ADFA-4589 | Add suite of AI-powered developer plugins for Code on the Go#41

Open
jatezzz wants to merge 17 commits into
mainfrom
refactor/ADFA-4589-ai-hackathon-plugin
Open

ADFA-4589 | Add suite of AI-powered developer plugins for Code on the Go#41
jatezzz wants to merge 17 commits into
mainfrom
refactor/ADFA-4589-ai-hackathon-plugin

Conversation

@jatezzz

@jatezzz jatezzz commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Description

Adds three new AI plugins and hardens the existing ai-core / ai-assistant (both already on main):

  • Code Suggestions - inline ghost-text completions as you type.
  • Speech-to-Text - voice input transcribed into the editor.
  • Vector Search - embeddings-based semantic search over the project (uses ai-core's getEmbeddings(), with a lexical fallback).

All three share ai-core's LlmInferenceService (on-device llama.cpp and/or Gemini) instead of bundling their own models.

Details

Stability & fixes to the existing plugins:

  • ai-core: reject embedding models on the chat path before they SIGABRT the IDE (ADFA-4388); load the user-selected model instead of the newest .gguf in Downloads; shut down the Llm-RunLoop executor on dispose to stop the classloader leak; move Gemini API key to the x-goog-api-key header.
  • ai-assistant: fetch the live Gemini model catalog (default gemini-2.5-flash, retire gemini-1.5-flash); fix settings-dialog UX and backend selection/routing.
  • Cross-cutting: cancel coroutine scopes on disposal; version ranges, READMEs, in-IDE help HTML, and day/night PNG icons across the AI plugins.
document_5073463760878832078.mp4

Ticket

ADFA-4589

Observation

  • llama.cpp is a submodule at ai-core/subprojects/llama.cpp (private fork, branch androidide-custom, needs SSH): git submodule update --init ai-core/subprojects/llama.cpp. A prebuilt ai-core/libs/v8/llama-v8-release.aar is committed, so building llama.cpp from source isn't required just to compile.
  • libs/*.jar are CI-managed and intentionally not committed here, but vector-search needs LlmInferenceService.getEmbeddings(), so the refreshed plugin-api.jar must be in place (CI / scripts/update-libs.sh) before building.

Notes:

  • Dropped the original .cgp / internal-docs observation: I verified none are tracked in this diff.
  • Corrected the submodule path (ai-core/…, not ai-assistant/…) and the "adds four plugins" claim (ai-assistant/ai-core already exist on main; this adds three).

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@jatezzz
jatezzz requested review from a team, Daniel-ADFA and itsaky-adfa July 7, 2026 23:09
@jatezzz
jatezzz force-pushed the refactor/ADFA-4589-ai-hackathon-plugin branch from e1a6598 to 4986735 Compare July 7, 2026 23:37
@hal-eisen-adfa

Copy link
Copy Markdown
Contributor

Might be too early to properly review this. They all need version ranges, html documentation, tier 1/2/3 help, and README.md.

CoroutineScope(Dispatchers.IO) and never gets cancel() in dispose().

@volatile static pluginContext never cleared, native model never unloaded, llama executor never shut down.

ai-assistant has security problems

  1. Gemini API key in plaintext SharedPreferences with a misleading "encrypted" comment (AgentSettingsStore.kt:39-40, AiAssistantPlugin.kt:176)
  2. API key in URL query string (GeminiBackend.kt:336-341) — should be x-goog-api-key header
  3. ai-core makes HTTP calls but doesn't declare network.access/INTERNET.

We also need the new plugin api jar otherwise these won't build.

@jatezzz
jatezzz force-pushed the refactor/ADFA-4589-ai-hackathon-plugin branch 2 times, most recently from c9ce9b8 to 9f7c87e Compare July 15, 2026 21:33
@jatezzz

jatezzz commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

@hal-eisen-adfa Thanks for the feedback, I've addressed all the concers, attached the proof that the new plugins build with the new Code On the Go plugin api.

Screenshot 2026-07-16 at 1 17 22 PM Screenshot 2026-07-16 at 1 47 27 PM

https://github.com/appdevforall/plugin-examples/actions/runs/29523333464

@hal-eisen-adfa

Copy link
Copy Markdown
Contributor

Almost everything is now fixed.

One big gap that wasn't addressed yet: LLamaAndroid needs to expose a shutdown that calls (runLoop as ExecutorCoroutineDispatcher).close() / executor.shutdown(), invoked from AiCorePlugin.dispose().

@jatezzz

jatezzz commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Added day and night icons for AI plugins

photo_5102620275676744858_y photo_5102620275676744859_y

@jatezzz

jatezzz commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Update theme and model validation on fresh installs

document_5114081575834749100.mp4

@hal-eisen-adfa hal-eisen-adfa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. Findings from an xhigh code review of the AI-plugin suite that live in this PR's diff, ranked most-severe first. (Two additional items scoped to the stacked API-key PR #49 are posted there.)

Also worth a cleanup pass (non-blocking): CodeChunker.simpleChunk has an unreachable "single line too large" else branch (the first line is always added, so currentChunk is never empty), and reconcileOverlaps has an empty if body / unused nextChunk and just returns its input unchanged.

jatezzz added 15 commits July 23, 2026 13:27
Inline ghost-text AI code completions; restored onto main's split layout.
Voice-to-code speech input; restored onto main's split layout.
Semantic code search via embeddings; restored onto main's split layout.
… Downloads

Stream the selected content:// URI into a private cache file; drop the Downloads fallback that silently loaded the wrong model (ADFA-4388).
…shing

Add GgufModelInspector + UserFeedback; classify the .gguf and throw IncompatibleModelException before native decode SIGABRTs the IDE (ADFA-4388).
….5-flash

listModels now queries the live v1beta ListModels catalog (generateContent-only); default bumped to gemini-2.5-flash and retired models migrated on load.
Readable settings title; real model name shown and persisted across reopen; and the chat backend indicator + routing now follow the selected backend, refreshing when the settings dialog closes.
…ggestion-plugin and vector-search-plugin names
- Add version ranges, README/docs, and tiered help coverage for AI plugins
- Cancel plugin coroutine scopes during disposal
- Move Gemini listModels API key from URL query to x-goog-api-key header
- Document Gemini API key storage tradeoff and remove misleading encryption note
- Update AI plugin review status with verification results
LLamaAndroid's single-thread run-loop executor was never shut down, so the non-daemon Llm-RunLoop thread outlived the plugin and pinned its classloader. Expose LLamaAndroid.shutdown() (closes the ExecutorCoroutineDispatcher) and call it from LocalLlmBackend.close() after the model unload completes. Regenerate the prebuilt llama AAR so the new API is visible to the plugin build.
… light/dark theme

- Agent tab now shows an error bubble + Snackbar ("Open Settings...") when no local model or Gemini key is configured, instead of silently doing nothing.
- Theme UI to the IDE day/night setting: route inflation through PluginFragmentHelper.getPluginInflater, make PluginTheme a Material3 DayNight theme, add values-night/ colors, and replace hardcoded hex with @color refs.
- Fix AI Settings toolbar contrast (colorOnPrimary title/icon on the primary bar).
llama run-loop now recreates after per-plugin dispose, embedding-dim guard, synced suggestion cache, capped file walk, fallback-safe Gemini model migration and vector search indexes in the background (bounded wait)
@jatezzz
jatezzz force-pushed the refactor/ADFA-4589-ai-hackathon-plugin branch from 3d33b7a to b939ff5 Compare July 23, 2026 18:27
@jatezzz
jatezzz requested a review from hal-eisen-adfa July 23, 2026 19:41

@hal-eisen-adfa hal-eisen-adfa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: 4 correctness issues in the new AI plugin suite. Each is inlined below. Happy to pair on fixes — the two hardcoded-"local" ones (#2, #3) and the cache-key one (#1) are the highest impact.

- Resolve backend via "auto" in ai-core
- Migrate Gemini to REST instead of the google-genai SDK to fix OkHttp 3.x conflict
- Implement context-aware suggestion cache key
- Add STT raw-transcript fallback and vector-search roots
- Refactor code: remove unused code and permissions
- Fix version numbers and update plugins documentation
@jatezzz
jatezzz requested a review from hal-eisen-adfa July 24, 2026 16:54
… suite

Refresh manifests, README/HTML docs, and day/night icons for ai-core, ai-assistant, code-suggestions, speech-to-text, and vector-search; update PathGuard and search handler.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants