Skip to content

feat(formulus-formplayer): Add dynamic choice lists with WHERE clause and age filtering support#277

Merged
Mishael-2584 merged 15 commits intodevfrom
feature/dynamic-choice-lists
Feb 5, 2026
Merged

feat(formulus-formplayer): Add dynamic choice lists with WHERE clause and age filtering support#277
Mishael-2584 merged 15 commits intodevfrom
feature/dynamic-choice-lists

Conversation

@Mishael-2584
Copy link
Contributor

Description

This PR introduces a comprehensive dynamic choice list feature that enables form fields to populate dropdown options dynamically from the local database at runtime. This eliminates the need for static enum values that can become outdated, allowing forms to always display current data.

Key Features

  • Dynamic Enum Control: Custom JSON Forms renderer (DynamicEnumControl) that supports x-dynamicEnum schema extension
  • Built-in Query Function: getDynamicChoiceList function that queries local observations via the Formulus WebView bridge
  • WHERE Clause Support: Advanced filtering using SQL-like WHERE clauses (e.g., data.sex = 'male', age_from_dob(data.dob) >= 18)
  • Age Filtering: Special support for age-based queries using age_from_dob(data.dob) function that calculates age from date of birth
  • Combined Filters: Support for combining static filter parameters with WHERE clauses
  • Form Evaluation Context: New React context (FormEvaluationContext) that provides extension functions to form components
  • Comprehensive Documentation: Complete reference guide with examples and troubleshooting

Technical Implementation

  • formulus-formplayer:

    • Added DynamicEnumControl.tsx - Custom renderer for dynamic enums
    • Added builtinExtensions.ts - Built-in getDynamicChoiceList function with age calculation logic
    • Added FormEvaluationContext.tsx - Context provider for extension functions
    • Updated App.tsx - Integrated built-in extensions and removed error message display
    • Updated ExtensionsLoader.ts - Support for built-in extension functions
  • formulus:

    • Updated FormplayerModal.tsx - Fixed extension base path to /forms directory
    • Updated ExtensionService.ts - Enhanced extension loading logic
    • Updated react-native.config.js - Added webview assets to React Native config
    • Updated android/app/build.gradle - Added JSX and HTML file copying for webview assets
  • Documentation:

    • Added DYNAMIC_CHOICE_LISTS.md - Comprehensive 780-line reference guide with examples

What This Enables

Forms can now:

  • Query any form type (e.g., hh_person, household) for dropdown options
  • Filter by any field using WHERE clauses
  • Filter by calculated age from date of birth
  • Combine multiple filters (e.g., sex: 'male' + age_from_dob(data.dob) >= 18)
  • Display distinct values only
  • Use custom value and label fields

Example Usage

{
  "select_person": {
    "type": "string",
    "x-dynamicEnum": {
      "function": "getDynamicChoiceList",
      "query": "hh_person",
      "params": {
        "whereClause": "age_from_dob(data.dob) >= 18 AND data.sex = 'male'"
      },
      "valueField": "observationId",
      "labelField": "data.names"
    }
  }
}

Type of Change

  • Bug Fix
  • New Feature / Enhancement
  • Refactor / Code Cleanup
  • Documentation Update
  • Maintenance / Chore
  • Other (please specify):

Component(s) Affected

  • formulus (React Native mobile app)
  • formulus-formplayer (React web app)
  • synkronus (Go backend server)
  • synkronus-cli (Command-line utility)
  • Documentation
  • DevOps / CI/CD
  • Other:

Related Issue(s)

Closes/Fixes/Resolves:


Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • Manually tested
  • Tested on multiple platforms (if applicable)
    • Tested on Android via Formulus app
    • Tested with various WHERE clause combinations
    • Tested age filtering with age_from_dob() function
    • Tested combined filters (static params + WHERE clauses)
  • Not applicable

Testing Notes

  • All dynamic choice list queries tested with test_dynamic form
  • Age-based filtering verified with multiple age conditions
  • Combined filters (e.g., sex + age) tested and working
  • Error handling verified (graceful degradation when API unavailable)

Breaking Changes

  • This PR introduces breaking changes
  • This PR does NOT introduce breaking changes

If breaking changes, please describe migration steps:

N/A - This is a purely additive feature. Existing forms continue to work unchanged. The x-dynamicEnum extension is optional and only used when specified in the schema.


Documentation Updates

  • Documentation has been updated
  • Documentation update is not required

Documentation Added

  • DYNAMIC_CHOICE_LISTS.md: Comprehensive 780-line reference guide including:
    • Quick start guide
    • Basic and advanced usage examples
    • WHERE clause syntax reference
    • Age filtering with age_from_dob() function
    • Combined filter examples
    • Troubleshooting guide
    • Production checklist

Checklist

  • Code follows project style guidelines (formatted with Prettier)
  • All existing tests pass
  • New tests added for new functionality (manual testing completed)
  • PR title follows Conventional Commits format

Implementation Details

Files Changed

Core Feature Files:

  • formulus-formplayer/src/DynamicEnumControl.tsx (359 lines) - Custom renderer
  • formulus-formplayer/src/builtinExtensions.ts (583 lines) - Query function with age logic
  • formulus-formplayer/src/FormEvaluationContext.tsx (74 lines) - Context provider
  • formulus-formplayer/src/App.tsx - Integration and error handling updates
  • formulus/src/components/FormplayerModal.tsx - Extension path fix
  • formulus/src/services/ExtensionService.ts - Extension loading enhancements
  • DYNAMIC_CHOICE_LISTS.md (779 lines) - Complete documentation

Implements dynamic dropdown population from local observations with:
- Custom DynamicEnumControl renderer for x-dynamicEnum fields
- Template parameter resolution ({{data.field}} syntax)
- WHERE clause filtering with operators (=, !=, <, >, <=, >=, AND, OR)
- Cascading dropdown support with real-time dependency tracking
- Distinct value filtering for unique choices
- FormEvaluationContext for extension function management

Core Components:
- DynamicEnumControl.tsx: React renderer with MUI Autocomplete
- FormEvaluationContext.tsx: React context provider
- builtinExtensions.ts: getDynamicChoiceList function with WHERE clause generation
- FormulusInjectionScript.js: Native bridge with WHERE clause parser
- ExtensionService.ts: Extension loading with ext.json normalization

Replaces ODK-X linked tables and 'select person' functionality.

Features:
- Query any form type (household, hh_person, etc.)
- Filter by parameters or complex WHERE clauses
- Cascading village → subvillage → household → person
- Age-based filtering, sex-based filtering
- Real-time updates when dependency values change

Documentation:
- Comprehensive guide with 8 real-world examples
- Query syntax reference with common patterns
- Troubleshooting guide with debug procedures
- Migration guide from static enums and ODK-X
- Production readiness checklist

Breaking Changes: None - fully backward compatible

Testing: Tested with cascading village/subvillage, multiple filters,
WHERE clauses, and production error handling.
@Mishael-2584 Mishael-2584 changed the base branch from main to dev February 3, 2026 06:32
@Mishael-2584 Mishael-2584 marked this pull request as draft February 3, 2026 06:44
@Mishael-2584 Mishael-2584 changed the title Feature/dynamic-choice-lists feat(formulus-formplayer): Add dynamic choice lists with WHERE clause and age filtering support Feb 3, 2026
@Mishael-2584 Mishael-2584 marked this pull request as ready for review February 3, 2026 07:46
@Mishael-2584 Mishael-2584 changed the title feat(formulus-formplayer): Add dynamic choice lists with WHERE clause and age filtering support feat(formulus-formplayer): Add dynamic choice lists with WHERE clause and age filtering support Feb 3, 2026
@Jexsie
Copy link
Collaborator

Jexsie commented Feb 3, 2026

the reason you have all these changes is that you rebased wrongly @Mishael-2584. Let me see how I can help. You can look through again to see if all your intended changes are still available

@Mishael-2584
Copy link
Contributor Author

the reason you have all these changes is that you rebased wrongly @Mishael-2584. Let me see how I can help. You can look through again to see if all your intended changes are still available

Hi @Jexsie. Thanks for the assistance.. I have noticed the large diff isn’t actually from the rebase. If you try track down the commits, only 4 files changed. (e568f61)) were from that. I think the main issue comes in from the Prettier commit.
NB
The Prettier commit (1ea4d81)
215 files changed
1,481,366 insertions), 3,252 deletions

My actual feature commit (71edc35)
7 files changed
501 insertions, 210 deletions
This is the actual feature work
The rebase merge has just 4 files changed. (e568f61)
58 insertions, 23 deletions

Let us see how we can still solve these conflicts and clean it up. I will set as draft for now

@Mishael-2584 Mishael-2584 marked this pull request as draft February 3, 2026 23:24
@Jexsie
Copy link
Collaborator

Jexsie commented Feb 4, 2026

Your prettier changes are against the prettier configuration. You have form player build assets pushed. These are not tracked by git. But yeah, let's have this fixed

@Mishael-2584
Copy link
Contributor Author

Mishael-2584 commented Feb 4, 2026

Yes precisely this was the issue

Mishael-2584 and others added 9 commits February 4, 2026 10:21
Formplayer assets are built by the formulus-android workflow and provided
as artifacts during the APK build. They should not be committed.
…filtering, age_from_dob

- Add getObservationsByQuery with whereClause support in FormService
- Support data.field and json_extract formats in filterObservationsByWhereClause
- builtinExtensions: getDynamicChoiceList with age_from_dob JS filtering
- FormulusMessageHandlers: fix payload extraction for getObservationsByQuery
- formulus-load.js: polyfill ensures correct message routing to native
- Synkronus: app/forms/ bundle structure for AnthroCollect
- Log cleanup, documentation updates (DYNAMIC_CHOICE_LISTS.md)
- getObservationsByQuery with whereClause, filterObservationsByWhereClause
- builtinExtensions: getDynamicChoiceList, age_from_dob JS filtering
- FormulusMessageHandlers: getObservationsByQuery payload handling
- formulus-load.js polyfill for correct message routing
- WebView assets: FormulusInjectionScript, formulus-api, formulus-load
- Synkronus: app/forms/ bundle structure for AnthroCollect
- ExtensionService, FormplayerModal, log cleanup
- DYNAMIC_CHOICE_LISTS.md documentation
Brings in complete feature from dev:
- getObservationsByQuery, filterObservationsByWhereClause
- builtinExtensions, FormulusMessageHandlers, formulus-load.js
- WebView assets (FormulusInjectionScript, formulus-api, formulus-load)
- formplayer_dist formulus-load.js, index.html
- ExtensionService, Synkronus app/forms/, documentation
…anner logic

- Introduced VisionCameraErrorBoundary to handle errors from the VisionCamera module.
- Refactored QRScannerModal to conditionally load QRScannerModalImpl based on the availability of the VisionCamera native module.
- Updated gradle.properties to optimize memory settings for lower-end devices, reducing JVM args and worker count for Gradle builds.
@Mishael-2584 Mishael-2584 marked this pull request as ready for review February 5, 2026 11:56
@najuna-brian
Copy link
Contributor

Thanks @Mishael-2584
Approved

@najuna-brian najuna-brian requested review from najuna-brian and removed request for najuna-brian February 5, 2026 14:46
@Mishael-2584 Mishael-2584 merged commit 7633d4e into dev Feb 5, 2026
6 checks passed
@najuna-brian najuna-brian deleted the feature/dynamic-choice-lists branch February 9, 2026 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants