diff --git a/README.md b/README.md index 4b7f82e7d..f087d8267 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,9 @@ The server component written in Go. This handles data synchronization, storage, ### **synkronus-cli** A command-line utility to interact with the Synkronus server. Use it to manage custom app data, handle user administration, export data to Parquet format, and perform various administrative tasks. +### **synkronus-portal** +A web-based version of the the **synkronus-cli** + ## We're Young & Fresh! 🌱🌱🌱 ODE is a **young and vibrant open-source project**, and we're incredibly welcoming to contributors of all experience levels and interests! Whether you're passionate about: @@ -47,43 +50,11 @@ We believe that diverse perspectives and varied skill sets make our project stro - Share how you're using ODE in your work ## CI/CD Pipeline +This monorepo uses GitHub Actions for CI/CD. For details (trigger conditions, tags, and workflows), see `.github/CICD.md`. -This monorepo uses GitHub Actions for continuous integration and deployment: - -### Current Pipelines - -#### Synkronus Docker Build & Publish -- **Trigger**: Push to `main` branch or pull requests affecting `synkronus/` directory -- **Registry**: GitHub Container Registry (ghcr.io) -- **Image**: `ghcr.io/opendataensemble/synkronus` -- **Tagging Strategy**: - - `main` branch β†’ `latest` + `v{version}` (release versions) - - Other branches β†’ `{branch-name}` (pre-release versions) -- **Workflow**: `.github/workflows/synkronus-docker.yml` - -### Image Versioning - -Images follow semantic versioning: -- **Release**: `ghcr.io/opendataensemble/synkronus:latest` or `ghcr.io/opendataensemble/synkronus:v1.0.0` -- **Pre-release**: `ghcr.io/opendataensemble/synkronus:develop` or `ghcr.io/opendataensemble/synkronus:feature-xyz` - -### Using Published Images - -Pull and run the latest Synkronus image: - -```bash -docker pull ghcr.io/opendataensemble/synkronus:latest -docker run -d -p 8080:8080 \ - -e DB_CONNECTION="postgres://user:pass@host:5432/synkronus" \ - -e JWT_SECRET="your-secret-key" \ - -v synkronus-bundles:/app/data/app-bundles \ - ghcr.io/opendataensemble/synkronus:latest -``` - -**Documentation:** -- [CI/CD Pipeline Details](.github/CICD.md) - Comprehensive CI/CD documentation -- [Synkronus Docker Guide](synkronus/DOCKER.md) - Quick start guide -- [Synkronus Deployment Guide](synkronus/DEPLOYMENT.md) - Production deployment +- Synkronus Docker build & publish: `.github/workflows/synkronus-docker.yml` +- Formulus Android build (includes Formplayer asset build): `.github/workflows/formulus-android.yml` +- Synkronus deployment docs: `synkronus/DOCKER.md`, `synkronus/DEPLOYMENT.md` ## Code Quality: Linting & Formatting diff --git a/packages/components/README.md b/packages/components/README.md index 5ed39861b..d4bb25a78 100644 --- a/packages/components/README.md +++ b/packages/components/README.md @@ -9,7 +9,7 @@ A comprehensive component library for the Open Data Ensemble (ODE) ecosystem. Th - **Modern Minimalist Design** - Clean, simple, and aesthetically beautiful components - **Platform-Specific** - Optimized implementations for React Native and React Web - **Token-Based** - Built on `@ode/tokens` for consistent design -- **Accessible** - WCAG compliant with proper ARIA attributes +- **Accessible** - supports `accessibilityLabel` (web: `aria-label`, React Native: native accessibility props where supported) - **Responsive** - Works seamlessly across all device sizes - **Dark Mode Ready** - Supports light and dark themes @@ -91,13 +91,12 @@ function App() { ### Button -Modern minimalist button with a unique fading border effect. When two buttons are placed together in a `ButtonGroup`, they automatically have opposite styles. +Modern minimalist button with transparent/default styling and token-based colors. When used inside a `ButtonGroup`, paired buttons automatically use contrasting variants for visual emphasis. **Features:** -- Fading border effect on one end (left button fades on right, right button fades on left) - Transparent background with border-colored text by default -- On hover: background fills with border color, text changes to contrast color -- Automatic opposite styling when paired +- On hover/active: background fills with the border color, and text switches to a contrasting color +- Automatic contrasting styling when paired in a `ButtonGroup` ```tsx // Single button @@ -113,7 +112,7 @@ Modern minimalist button with a unique fading border effect. When two buttons ar ``` **Props:** -- `variant`: `'primary' | 'secondary' | 'neutral'` (default: `'primary'`) +- `variant`: `'primary' | 'secondary' | 'neutral' | 'danger'` (default: `'primary'`) - `size`: `'small' | 'medium' | 'large'` (default: `'medium'`) - `disabled`: `boolean` (default: `false`) - `loading`: `boolean` (default: `false`) @@ -185,7 +184,7 @@ All components follow these design principles: 1. **Minimalism** - Clean, uncluttered interfaces that enhance usability 2. **Consistency** - Unified design language using ODE tokens -3. **Accessibility** - WCAG compliant with proper contrast ratios and touch targets +3. **Accessibility** - supports `accessibilityLabel`/ARIA where applicable 4. **Responsiveness** - Adapts seamlessly to all screen sizes 5. **Performance** - Lightweight and optimized for fast rendering @@ -194,9 +193,8 @@ All components follow these design principles: The button component features a unique design inspired by modern minimalist aesthetics: - **Default State**: Transparent background with colored border and matching text -- **Fading Border**: Border fades on one end (left button fades right, right button fades left) -- **Hover State**: Background fills with border color, text changes to high-contrast color -- **Paired Buttons**: When two buttons are together, they have opposite color schemes +- **Hover State**: Background fills with border color, text changes to a high-contrast color +- **Paired Buttons**: When two buttons are together, they use contrasting color variants - **Smooth Transitions**: All state changes use smooth cubic-bezier animations ## Platform Differences diff --git a/synkronus/README.md b/synkronus/README.md index 635da33dc..9b4b4275a 100644 --- a/synkronus/README.md +++ b/synkronus/README.md @@ -38,30 +38,7 @@ synkronus/ **For production deployment**, we recommend using Docker Compose with nginx and cloudflared tunnel. -See [DOCKER.md](./DOCKER.md) for quick start or [DEPLOYMENT.md](./DEPLOYMENT.md) for comprehensive deployment guide. - -```bash -# Quick start with docker-compose -cp docker-compose.example.yml docker-compose.yml -# Edit docker-compose.yml with your secrets -docker compose up -d - -# Or run directly with Docker -docker pull ghcr.io/opendataensemble/synkronus:latest -docker run -d -p 8080:8080 \ - -e DB_CONNECTION="postgres://user:pass@host:5432/synkronus" \ - -e JWT_SECRET="your-secret-key" \ - -v synkronus-bundles:/app/data/app-bundles \ - ghcr.io/opendataensemble/synkronus:latest -``` - -When using the provided `docker-compose` setup, the recommended way to initialize databases is: - -- Run a single PostgreSQL container (service `postgres`). -- Use `docker compose exec postgres psql -U postgres` to create one or more databases and users for Synkronus. -- Point each Synkronus instance at its own database via the `DB_CONNECTION` string. - -See [DEPLOYMENT.md](./DEPLOYMENT.md) for the full step-by-step database initialization procedure. +See [DOCKER.md](./DOCKER.md) for running pre-built images and local Docker usage, and [DEPLOYMENT.md](./DEPLOYMENT.md) for full production setup (including database initialization). ### Development Setup @@ -122,40 +99,7 @@ go run cmd/synkronus/main.go ## Deployment Architecture -Synkronus is designed to be deployed as a single Docker container with multiple processes managed by a supervisor: - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ Docker Container β”‚ -β”‚ β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Nginx │───────▢│ Synkronus β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ Go Server β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ -β”‚ β–Ό β–Ό β”‚ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Static β”‚ β”‚ PostgreSQL β”‚ β”‚ -β”‚ β”‚ UI β”‚ β”‚ Database β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β”‚ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -### Benefits - -- **Simple deployment**: Single container to manage -- **Proper separation of concerns**: Nginx handles TLS, static files, and proxying -- **Easy scaling**: Can be deployed to any container orchestration system - -### Implementation Details - -- **TLS/Let's Encrypt**: Automatic certificate management via Certbot with Nginx -- **UI Integration**: Static SPA (Single Page Application) served by Nginx -- **API Proxying**: Nginx proxies API requests to the Go server -- **Data Persistence**: PostgreSQL database for robust data storage -- **Configuration**: Environment variables for flexible deployment options +For Docker-based deployment details (pre-built images, docker-compose configuration, and production setup), see `DOCKER.md` and `DEPLOYMENT.md`. ## API Documentation @@ -163,104 +107,7 @@ API documentation is generated from the OpenAPI specification in `openapi/synkro ## Sync protocol -Attachments (e.g. photos, audio recordings) are **binary blobs** referenced by observations. They are stored and transferred separately from the observation metadata to simplify synchronization, improve offline support, and reduce conflicts. - -### Core design principles - -- **Immutable attachments** - - Once an attachment is uploaded, it cannot be modified in place. - - Any "update" requires creating a new attachment with a new unique ID (typically a GUID). - -- **Separation of concerns** - - Observation records sync via `/sync/pull` and `/sync/push`. - - Attachments are uploaded and downloaded via dedicated endpoints. - - This design ensures simpler, smaller payloads and clearer transaction boundaries. - -- **Stateless server** - - The server does not maintain per-client state about which attachments have been uploaded or downloaded. - - Clients manage their own attachment sync state. - -### Conflict avoidance - -- Because attachment IDs are generated as GUIDs client-side, filename clashes are extremely unlikely. -- The server can include a simple existence check to reject accidental overwrites. - -### Clean-up and maintenance - -- Server can run periodic jobs to: -- Identify orphaned attachments (no observation references). -- Prune old or unused files. -- Enforce retention policies. - -### Security considerations - -- Require authentication (e.g. bearer tokens) for all attachment endpoints. -- Validate file types and sizes on upload to prevent abuse. -- Optionally scan files for malware. - -### Example implementation notes - -- Filesystem storage for MVP: -- Simple to implement and inspect. -- Suitable for single-server deployments. - -- Cloud object storage (e.g. S3, GCS) for future: -- Supports presigned URLs for direct client upload/download. -- Handles scalability and redundancy. -- Reduces server load. - -### Advantages of this server design - -- Keeps observation data sync API simple and JSON-only. -- Supports large binary payloads without bloating /sync/push or /sync/pull requests. -- Encourages incremental, resumable, offline-friendly sync strategies. -- Leaves attachment sync policy (e.g. lazy, bulk, partial) up to the client. -- Scales from MVP file-system storage to cloud-native solutions. - -## Minimal Sync Protocol Design - -This project uses a minimal sync approach for the `/sync/pull` and `/sync/push` endpoints that remains compatible with WatermelonDB's client-side helpers without requiring a full server-side change-tracking table. - -### Approach - -- Server stores a monotonic version number (via PostgreSQL trigger) for every observation change. -- Each Observation record includes `created_at`, `updated_at`, and `deleted` fields. -- Server simply returns all observations changed since the client's last known version. - -### Client-side adaptation - -- The client *infers* WatermelonDB's `_status`: - - `deleted` β†’ `_status: "deleted"` - - `created_at == updated_at` β†’ `_status: "created"` - - Else β†’ `_status: "updated"` -- `_changed` is hardcoded to `"data,deleted"` since most content is stored in the `data` JSON column. -- The client wraps the server's flat `records` array into the expected WatermelonDB format: - - ```json - { - "changes": { - "observations": [ ... ] - }, - "timestamp": current_version - } - ``` - -### Advantages - -- Requires no server-side schema changes or change-tracking tables. -- Enables immediate shipping. -- Compatible with WatermelonDB's `experimentalApplyRemoteChanges`. -- Easy to side-load data or perform manual database edits on the server. - -### Trade-offs - -- Client carries responsibility for inferring `_status` and `_changed`. -- Always sends/receives full `data` blobs. -- Less granular change tracking. - -### Future Consideration - -This minimal approach is ideal for the current single-table use case. As requirements grow (e.g. multiple tables or better conflict resolution), the server can evolve to maintain an explicit changes log and deliver a richer sync protocol with exact `_status` and `_changed` values. See [WatermelonDB/Sync](https://watermelondb.dev/docs/Sync/Intro) for more details on how to align more closely with the WatermelonDB approach to sync. +For the sync protocol design details (record model, attachment handling, pagination, and conflict strategy), see [`documentation/sync-protocol.md`](./documentation/sync-protocol.md). ## License diff --git a/synkronus/documentation/README.md b/synkronus/documentation/README.md index 0ceb4892b..5f4346a0a 100644 --- a/synkronus/documentation/README.md +++ b/synkronus/documentation/README.md @@ -25,7 +25,7 @@ - Optional API versioning via `x-api-version` header - Optional ETag support for caching and efficiency -Full OpenAPI spec lives in [`Synkronus Openapi`](Synkronus/Openapi.yaml) +Full OpenAPI spec lives in [`synkronus/openapi/synkronus.yaml`](../openapi/synkronus.yaml) ---