Skip to content

feat(sync): observation metadata, tags, exports, and Formulus LWW tagging#567

Merged
r0ssing merged 13 commits intoOpenDataEnsemble:devfrom
HelloSapiens:feature/portal-things
Mar 22, 2026
Merged

feat(sync): observation metadata, tags, exports, and Formulus LWW tagging#567
r0ssing merged 13 commits intoOpenDataEnsemble:devfrom
HelloSapiens:feature/portal-things

Conversation

@r0ssing
Copy link
Contributor

@r0ssing r0ssing commented Mar 22, 2026

feat(sync): observation metadata, tags, exports, and Formulus LWW tagging

Description

This PR adds optional root-level observation fields (author, device_id, tags) end-to-end: Postgres migration, Synkronus sync (pull/push), OpenAPI and protocol docs, Parquet + raw JSON export, and Formulus local storage + UI. Sync remains backward-compatible when these fields are absent.
Synkronus

  • New migration adds nullable author, device_id, and tags (TEXT[]) on observations (with comments).
  • Sync service reads/writes geolocation, author, device_id, and tags on pull and push; fields are optional and omitted when unset.
  • Data export: rows and queries include the new fields; Parquet adds nullable author, device_id, and tags (tags as a JSON string column, consistent with geolocation); raw JSON ZIP includes the same fields.
  • OpenAPI Observation schema: author / device_id / tags optional; spec version bump.
  • documentation/sync-protocol.md: documents optional root metadata including tags.
  • Test DB helper and export mocks updated so integration/unit tests match the new columns.
    Formulus
  • WatermelonDB schema v5: tags column (JSON array as string); migration from v4.
  • Domain model, mapper, repo, and generated API types support author, device_id, and tags; push omits empty optional fields when appropriate.
  • Observation detail screen shows tags as small pill/chip UI.
  • Last-write-wins on pull: when a server row is skipped because the local row is dirty, append last_write_won once (deduped), exported as LAST_WRITE_WON_TAG in syncConstants.ts; tag syncs to the server on the next push unless removed.

Type of Change

  • Bug Fix
  • New Feature / Enhancement
  • Refactor / Code Cleanup
  • Documentation Update
  • Maintenance / Chore
  • Other (please specify):

Component(s) Affected

  • formulus (React Native mobile app)
  • formulus-formplayer (React web app)
  • synkronus (Go backend server)
  • synkronus-cli (Command-line utility)
  • Documentation
  • DevOps / CI/CD
  • Other:

Related Issue(s)

Closes/Fixes/Resolves:

Testing

  • Unit tests added/updated
  • Integration tests added/updated (Synkronus DB test schema / export mocks)
  • Manually tested
  • Tested on multiple platforms (if applicable)
  • Not applicable

Breaking Changes

  • This PR introduces breaking changes
  • This PR does NOT introduce breaking changes
    If breaking changes, please describe migration steps:
  • N/A — new columns and API fields are optional; existing clients continue to work. Deploy: run DB migrations on Synkronus; Formulus installs pick up WatermelonDB migration to v5 for tags.

Documentation Updates

  • Documentation has been updated
  • Documentation update is not required

Checklist

  • Code follows project style guidelines
  • All existing tests pass
  • New tests added for new functionality
  • PR title follows Conventional Commits format

Thank you for contributing to Open Data Ensemble (ODE)!

r0ssing added 13 commits March 22, 2026 12:03
…rvice functionality

- Added RawJSONExportHandler to handle requests for exporting observations as a ZIP of JSON files.
- Implemented ExportRawJSONZip method in the data export service to generate the ZIP archive.
- Enhanced mock service to support raw JSON export functionality.
- Updated API routes to include the new endpoint for raw JSON export.
- Added unit tests for both the handler and service to ensure correct behavior and error handling.
- Added support for multiple export formats (parquet, json, attachments) in the data export command.
- Updated the client to handle downloading of raw JSON and attachments ZIP exports.
- Improved error handling for API responses during export operations.
- Updated command examples to reflect new usage patterns.
- Enhanced the observation model to include optional fields: author, device ID, and tags.
- Updated database schema and migration to accommodate new fields.
- Modified data export and sync functionalities to handle the new fields.
- Adjusted tests and documentation to reflect the changes in the observation structure.
@r0ssing r0ssing requested a review from najuna-brian March 22, 2026 12:49
@r0ssing r0ssing merged commit da88b01 into OpenDataEnsemble:dev Mar 22, 2026
18 checks passed
@r0ssing r0ssing deleted the feature/portal-things branch March 22, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants