Skip to content

Fix/adfa 3803 -- ADFA-3803 -- Add Pebble support to the documentation system#1318

Merged
hal-eisen-adfa merged 12 commits into
stagefrom
fix/ADFA-3803
May 20, 2026
Merged

Fix/adfa 3803 -- ADFA-3803 -- Add Pebble support to the documentation system#1318
hal-eisen-adfa merged 12 commits into
stagefrom
fix/ADFA-3803

Conversation

@alexmmiller
Copy link
Copy Markdown
Collaborator

This branch has an updated local web server that can instantiate templates with data stored on disk in JSON format.

https://appdevforall.atlassian.net/browse/ADFA-3803

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 18, 2026

Review Change Stack

Caution

Review failed

Failed to post review comments

📝 Walkthrough
  • Features & Enhancements

    • Add Pebble template engine support (io.pebbletemplates:pebble:4.1.1) and Jackson databind (com.fasterxml.jackson.core:jackson-databind:2.15.2).
    • Enable JSON-driven template instantiation: DB-stored content bytes are parsed as UTF-8 JSON and used as the Pebble template context.
    • Shared PebbleEngine and ConcurrentHashMap-backed templateCache to avoid recompiling templates on each request.
    • Consolidated DB GET query to retrieve content blob, MIME type, compression, and templateId in one query.
    • Introduced content chunk-size constant and logic to assemble multi-part content when a chunk-sized blob indicates additional fragments (path-1, path-2, …).
    • Brotli handling: if DB compression is brotli, the server retains Content-Encoding: br only when the client advertises Brotli support and templateId == 0; otherwise the server locally decompresses and sets compression to none so templates are applied to decoded bytes.
    • Added helper instantiatePebbleTemplate(...) to load/compile/cache templates, parse JSON into a Map, evaluate the template with a StringWriter, and return rendered UTF-8 bytes.
    • Improved resource handling: SQL cursor closed in a finally block; explicit checks for zero or multiple matching DB rows resulting in 404/500 responses.
  • Dependencies Added

    • io.pebbletemplates:pebble:4.1.1
    • com.fasterxml.jackson.core:jackson-databind:2.15.2
  • Risks & Best-Practice Violations

    • Security
      • Unvalidated JSON parsing: DB JSON is parsed directly with Jackson without schema validation — malformed or malicious JSON may cause failures or unexpected behavior.
      • Arbitrary template context injection: parsed JSON is passed wholly into Pebble, increasing risk of data leakage or template injection if templates or DB content are untrusted.
      • No explicit sanitization/escaping of template outputs — potential XSS or injection vulnerabilities when rendering HTML or other contexts.
      • Template lookup by templateId lacks explicit authorization/scope checks, risking cross-content template access.
    • Memory & Performance
      • Unbounded template cache (ConcurrentHashMap) with no eviction policy — potential unbounded memory growth with many distinct templates.
      • Brotli decompression when templates must be applied (unless client supports br and templateId == 0) increases CPU usage.
      • Fragment assembly concatenates byte arrays and may cause GC pressure for large/many fragments.
    • Reliability & Error Handling
      • Risk of double-response or partial-response corruption if an error is attempted after headers/body have been flushed.
      • Broad exception handling for template/JSON/DB errors reduces observability and complicates targeted recovery.
      • Minimal validation of DB column types/shape beyond row-count checks.
    • Maintainability
      • Added complexity and LOC increases require regression testing across compression and templating scenarios.
  • Recommended Actions Before Production

    1. Implement a bounded template cache (LRU/time-based eviction) and add cache metrics (hits/misses) and monitoring.
    2. Validate DB JSON against schemas or perform structural checks before passing data to the template engine.
    3. Add context-aware escaping/sanitization for rendered outputs to mitigate XSS/injection risks.
    4. Ensure response state tracking to prevent attempting to send error responses after partial writes (avoid double-response).
    5. Replace broad exception catches with specific error handling for template rendering, JSON parsing, and DB errors; add structured logging and metrics for failures.
    6. Add authorization/scope checks for template lookups to prevent cross-content/template access.
    7. Optimize fragment assembly (streaming, buffer pooling, or pre-sized buffers) to reduce allocations and GC pressure.
    8. Add unit and integration tests covering:
      • Template rendering with varied/malformed JSON inputs
      • Brotli permutations (client supports br vs not)
      • Multi-fragment content assembly
      • Error paths for malformed JSON/templates and partial writes
    9. Perform load and performance testing for template rendering at expected scale and monitor CPU/memory impact.

Walkthrough

Adds Pebble and Jackson dependencies and integrates Pebble template rendering into the local WebServer: compiles/caches templates, parses DB-stored JSON content as template context, conditionally decompresses Brotli for templating, and refactors the GET handler to fetch templateId/compression, assemble fragments, render templates, and send responses.

Changes

Pebble Template Rendering for Database-Backed Web Content

Layer / File(s) Summary
Pebble and Jackson Dependencies
app/build.gradle.kts
Adds Pebble io.pebbletemplates:pebble:4.1.1 and Jackson com.fasterxml.jackson.core:jackson-databind:2.15.2 implementation dependencies.
Imports, Engine, and Cache Setup
app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
Imports StringWriter, Pebble, and Jackson classes; defines shared pebbleEngine, concurrent templateCache, contentChunkSize, and adds a startup comment.
Request Handler Refactor with Template and Compression Support
app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
Refactors GET handler to join compression and templateId in the DB query, validate single-row semantics, assemble 1MiB content fragments, decompress Brotli when templating is required or client lacks Brotli, call instantiatePebbleTemplate(...) when templateId > 0, write HTTP/1.1 200 headers with conditional Content-Encoding, flush before body, and use sendError on processing exceptions; ensures cursor closed in finally.
Template Instantiation Helper
app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
Adds instantiatePebbleTemplate(...) which loads template source from the Templates table, compiles/caches the PebbleTemplate per templateId, parses DB-provided content bytes as UTF-8 JSON into a Map<String,Any> using Jackson, evaluates the template into a StringWriter, and returns rendered UTF-8 bytes.

Sequence Diagram

sequenceDiagram
  participant Client
  participant RequestHandler
  participant Database
  participant instantiatePebbleTemplate
  participant PebbleEngine
  participant Jackson
  Client->>RequestHandler: GET /db path (Accept-Encoding header)
  RequestHandler->>Database: query Content JOIN ContentTypes (fetch content, compression, templateId)
  Database-->>RequestHandler: content bytes, compression, templateId
  RequestHandler->>instantiatePebbleTemplate: call with content bytes, templateId (if templateId>0)
  instantiatePebbleTemplate->>Database: query Templates table for template source (templateId)
  Database-->>instantiatePebbleTemplate: template source
  instantiatePebbleTemplate->>PebbleEngine: compile (if missing) and cache template
  instantiatePebbleTemplate->>Jackson: parse content bytes (UTF-8 JSON) -> Map<String,Any>
  Jackson-->>instantiatePebbleTemplate: context Map
  instantiatePebbleTemplate->>PebbleEngine: evaluate template with context -> rendered String
  PebbleEngine-->>instantiatePebbleTemplate: rendered String
  instantiatePebbleTemplate-->>RequestHandler: rendered UTF-8 bytes
  RequestHandler-->>Client: HTTP/1.1 200 + headers (maybe Content-Encoding) then body
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • appdevforall/CodeOnTheGo#976: Modifies WebServer.kt's DB-driven request handling—compression logic and cursor/error-response paths related to this PR's refactored handler.

Suggested reviewers

  • davidschachterADFA
  • jimturner-adfa

Poem

🐰 I nibbled bytes beneath the server's glow,
Pebble seeds and JSON maps in a row,
Fragments stitched, Brotli loosened with care,
Templates bloom and pixels fill the air,
A little hop — a rendered page to show.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: adding Pebble template support to the documentation system, which matches the core implementation of Pebble templating in WebServer.kt and new Pebble dependency in build.gradle.
Description check ✅ Passed The description accurately relates to the changeset by explaining that the local web server now instantiates templates with JSON data from disk, which aligns with the Pebble templating implementation and Jackson JSON databind additions.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ADFA-3803

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt (1)

358-374: ⚡ Quick win

Avoid quadratic copying when reassembling fragmented content.

dbContent += dbContent2 reallocates and copies the full accumulated payload on every fragment. Multi-fragment documents will do O(n²) copying on the request path.

Suggested fix
             if (dbContent.size == 1024 * 1024) {
                 val query2 = "SELECT content FROM Content WHERE path = ? AND languageId = 1"
                 var fragmentNumber = 1
+                val combined = ByteArrayOutputStream().apply { write(dbContent) }
                 var dbContent2 = dbContent
                 while (dbContent2.size == 1024 * 1024) {
                     val path2 = "$path-$fragmentNumber"
                     val cursor2 = database.rawQuery(query2, arrayOf(path2))
                     try {
                         if (cursor2.moveToFirst()) {
                             dbContent2 = cursor2.getBlob(0)
-                            dbContent += dbContent2
+                            combined.write(dbContent2)
                             fragmentNumber++
                         } else break
                     } finally { cursor2.close() }
                 }
+                dbContent = combined.toByteArray()
             }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 358 - 374, The current fragment reassembly in WebServer.kt uses repeated
concatenation (dbContent += dbContent2) which causes quadratic copying; change
the logic to stream-append fragments instead: after reading the initial
dbContent blob, create a ByteArrayOutputStream (or a MutableList<ByteArray>) and
write the initial dbContent and each dbContent2 into it inside the while loop
(using fragmentNumber, query2, cursor2 as-is), then after the loop call
toByteArray() once to produce the final byte[]; ensure cursor2 is still closed
in the finally and replace in-place concatenation with the single-buffer
assembly approach.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt`:
- Around line 415-460: The cached templates (templateCache keyed by templateId
in the templateCache.getOrPut block) are not invalidated when the backing
database is hot-swapped by handleClient(), so cached entries serve stale
template sources; fix by making cache entries sensitive to the current database
(either clear templateCache whenever handleClient() swaps the database or
include a unique database identifier (e.g., DB file path, timestamp, or a
databaseVersion token exposed where the swap happens) in the cache key used by
templateCache.getOrPut), and update the swap code path in handleClient() to call
templateCache.clear() or bump the DB identifier so pebbleEngine.getTemplate(...)
always uses templates from the active database.
- Around line 424-458: The tCursor returned by database.rawQuery(...) is only
closed in the success branch; wrap the rawQuery call in a Kotlin use block so
the cursor is closed on every path (including the error branches). Replace the
current val tCursor = database.rawQuery(...) and later tCursor.use { ... } with
a single database.rawQuery(tQuery, arrayOf(templateId.toString())).use { tCursor
-> ... } and move the count checks and throw calls inside that use block so
tCursor is always closed; update references to tCursor inside the block
accordingly.

---

Nitpick comments:
In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt`:
- Around line 358-374: The current fragment reassembly in WebServer.kt uses
repeated concatenation (dbContent += dbContent2) which causes quadratic copying;
change the logic to stream-append fragments instead: after reading the initial
dbContent blob, create a ByteArrayOutputStream (or a MutableList<ByteArray>) and
write the initial dbContent and each dbContent2 into it inside the while loop
(using fragmentNumber, query2, cursor2 as-is), then after the loop call
toByteArray() once to produce the final byte[]; ensure cursor2 is still closed
in the finally and replace in-place concatenation with the single-buffer
assembly approach.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b51ecb60-5cbd-4606-9e16-c72e98d64ced

📥 Commits

Reviewing files that changed from the base of the PR and between de44892 and 6de701c.

📒 Files selected for processing (2)
  • app/build.gradle.kts
  • app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt

Comment thread app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
Comment thread app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
Comment thread app/build.gradle.kts Outdated
Comment thread app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
Comment thread app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt Outdated
Comment thread app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt Outdated
Comment thread app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt Outdated
Comment thread app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt Outdated
…me case logic in database response handler for standard GET requests
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt (1)

469-470: ⚡ Quick win

Reuse ObjectMapper instance instead of creating one per call.

ObjectMapper is thread-safe and expensive to instantiate. Creating a new instance on every template render adds unnecessary overhead.

Proposed fix

Add a class-level field:

private val objectMapper = ObjectMapper()

Then update the usage:

-        val mapper = ObjectMapper()
-        val context: Map<String, Any> = mapper.readValue(dbContent.toString(Charsets.UTF_8), object : TypeReference<Map<String, Any>>() {})
+        val context: Map<String, Any> = objectMapper.readValue(dbContent.toString(Charsets.UTF_8), object : TypeReference<Map<String, Any>>() {})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt` around
lines 469 - 470, The code creates a new ObjectMapper inside the render path (the
local variable mapper in WebServer.kt), which is expensive; make a single
reusable instance by adding a class-level property (e.g., private val
objectMapper = ObjectMapper()) and replace the local mapper usage with that
objectMapper when calling readValue to build context (the readValue call that
currently uses mapper). Ensure the new field is used across methods instead of
instantiating ObjectMapper per call.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt`:
- Line 67: Rename the misspelled constant comtentChunkSize to contentChunkSize
in WebServer.kt (keep the same value 1024 * 1024) and update all usages to the
new name (replace references to comtentChunkSize with contentChunkSize where the
chunk size is read/used). Ensure the identifier is updated consistently
(declaration and every reference) so compilation and behavior remain unchanged.

---

Nitpick comments:
In `@app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt`:
- Around line 469-470: The code creates a new ObjectMapper inside the render
path (the local variable mapper in WebServer.kt), which is expensive; make a
single reusable instance by adding a class-level property (e.g., private val
objectMapper = ObjectMapper()) and replace the local mapper usage with that
objectMapper when calling readValue to build context (the readValue call that
currently uses mapper). Ensure the new field is used across methods instead of
instantiating ObjectMapper per call.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6644b417-d3af-4ea0-8158-b2c1b5fa8c4a

📥 Commits

Reviewing files that changed from the base of the PR and between 6de701c and 42ebd9f.

📒 Files selected for processing (2)
  • app/build.gradle.kts
  • app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/build.gradle.kts

Comment thread app/src/main/java/com/itsaky/androidide/localWebServer/WebServer.kt Outdated
@alexmmiller alexmmiller requested a review from jatezzz May 19, 2026 20:18
@hal-eisen-adfa hal-eisen-adfa dismissed jatezzz’s stale review May 20, 2026 15:18

Github appears to be glitching

@hal-eisen-adfa hal-eisen-adfa merged commit 9c0fc5a into stage May 20, 2026
2 checks passed
@hal-eisen-adfa hal-eisen-adfa deleted the fix/ADFA-3803 branch May 20, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants