diff --git a/.github/workflows/trigger-embed.yml b/.github/workflows/trigger-embed.yml
new file mode 100644
index 000000000000..0b4c23e7a17c
--- /dev/null
+++ b/.github/workflows/trigger-embed.yml
@@ -0,0 +1,22 @@
+name: Trigger Embed Service
+
+on:
+ push:
+ branches:
+ - main
+
+jobs:
+ trigger-embed:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Trigger Embed Service
+ env: # The API keys are the same as those used by the AI Assistant. There is only one (production) database.
+ PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
+ OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+ run: |
+ curl -X POST https://apollo.staging.openfn.org/services/embed_docsite \
+ -H "Content-Type: application/json" \
+ -d '{
+ "pinecone_api_key": "${{ secrets.PINECONE_API_KEY }}",
+ "openai_api_key": "${{ secrets.OPENAI_API_KEY }}"
+ }'
\ No newline at end of file
diff --git a/README.md b/README.md
index 1248efd38979..341169868e8b 100644
--- a/README.md
+++ b/README.md
@@ -5,7 +5,7 @@ site._**
## Contributing to this documentation site
-Visit https://docs.openfn.org/documentation/writing-docs/.
+Visit: https://docs.openfn.org/documentation/writing-docs/
This website is built using [Docusaurus](https://docusaurus.io/), a static
website generator that uses Markdown and Javascript.
diff --git a/adaptors/asana.md b/adaptors/asana.md
new file mode 100644
index 000000000000..a888dc4a6b19
--- /dev/null
+++ b/adaptors/asana.md
@@ -0,0 +1,44 @@
+---
+title: Asana Adaptor
+---
+
+## About Asana
+
+[Asana](https://app.asana.com/) is a web-based project management tool that helps teams organize, plan, collaborate, and execute tasks.
+
+## Integration Options
+
+Asana supports 2 primary integration options:
+
+1. Rest API: Asana has an available REST API that enables external services like OpenFn to pull data from Asana, or push data from external apps to Asana. This option is suited for scheduled, bulk syncs or workflows that must update data in Asana with external information. See [functions](/adaptors/packages/asana-docs) for more on how to use this adaptor to work with the API.
+
+2. Webhook: Asana also has a [Webhook or Data Forwarding](https://developers.asana.com/docs/webhooks-guide) to push data from Asana to external systems. This option is suited for real-time, event-based data integration. Check out the Asana [developer documentation](/adaptors/packages/asana-docs) to learn how to set up a webhook to push data to OpenFn.
+
+## Authentication
+
+See [Asana docs](https://developers.asana.com/docs/authentication) for the latest on supported authentication methods.
+
+When integrating with Asana via OpenFn, there is one primary authentication method that is supported: **Personal Access Token (PAT)**. You can generate a personal access token from the Asana [developer console](https://developers.asana.com/docs/personal-access-token).
+
+See this adaptor's [Configuration docs](/adaptors/packages/asana-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "apiVersion": "1.0",
+ "token": "sample-tokenyWSJdXBACMLLWMNGgADFA"
+}
+```
+
+### Helpful Links
+
+1. [API documentation](https://developers.asana.com/docs/overview)
+
+### Implementation Examples
+
+1. The Wildlife Conservation Society (WCS) - KoboToolBox -> GoogleSheets -> Asana sync: [https://openfn.github.io/ConSoSci/asana/](https://openfn.github.io/ConSoSci/asana/)
+
+
+
+
diff --git a/adaptors/azure-storage.md b/adaptors/azure-storage.md
new file mode 100644
index 000000000000..432aa0b17823
--- /dev/null
+++ b/adaptors/azure-storage.md
@@ -0,0 +1,40 @@
+---
+title: Azure Storage
+---
+
+## About Azure Storage
+
+[Azure Storage](https://azure.microsoft.com/en-us/products/category/storage) is a cloud-based storage solution provided by Microsoft Azure. It offers scalable and secure storage options for a wide range of data, including unstructured, semi-structured, and structured data.
+## Integration Options
+
+Azure Storage supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** Azure Storage has a REST API that enables external services like OpenFn to pull data from Azure Storage, or push data from external apps to Azure Storage. This option is suited for scheduled, bulk syncs or workflows that must update data in Azure Storage with external information. See [functions](/adaptors/packages/azure-storage-docs) for more on how to use this adaptor to work with the API.
+2. **Webhook:** Webhook or Data Forwarding to push data from Azure Storage to external systems (see [docs](https://learn.microsoft.com/en-us/azure/storage/)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+1. See [Azure Storage docs](https://learn.microsoft.com/en-us/azure/storage/) for the latest on supported authentication methods.
+2. When integrating with Azure Storage via OpenFn, there is one primary authentication method that is supported: **API Key**. See this adaptor's [Configuration docs](/adaptors/packages/azure-storage-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "accountName": "examplestorageaccount01",
+ "accountKey": "zzUx5rA4BZkjPet8ztLTEll7azF65Sez7WxGKsJv5B+rp/dby8ARgYpEVe93xFCQDqGBi3FUlxjP+ASt4uepZg=="
+}
+```
+
+### Helpful Links
+
+1. [Azure Storage Documentation](https://learn.microsoft.com/en-us/azure/storage/)
+2. [Azure Blob Storage REST API](https://learn.microsoft.com/en-us/rest/api/storageservices/)
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
+
diff --git a/adaptors/beyonic.md b/adaptors/beyonic.md
new file mode 100644
index 000000000000..f9941164d3e8
--- /dev/null
+++ b/adaptors/beyonic.md
@@ -0,0 +1,42 @@
+---
+title: Beyonic
+---
+
+## About Beyonic
+
+Beyonic is a digital payments platform that supports mobile money payments, collections, and bulk disbursements, offering integration with major mobile network operators and banks.
+
+## Integration Options
+
+Beyonic supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** Beyonic has a REST API that enables external services like OpenFn to pull data from Beyonic, or push data from external apps to Beyonic. This option is suited for scheduled, bulk syncs or workflows that must update data in Beyonic with external information. See [functions](/adaptors/packages/beyonic-docs) for more on how to use this adaptor to work with the API.
+2. **Webhook:** Webhook or Data Forwarding to push data from Beyonic to external systems (see [docs](https://apidocs.beyonic.com/webhooks)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+1. See [Beyonic docs](https://apidocs.beyonic.com) for the latest on supported authentication methods.
+2. When integrating with Beyonic via OpenFn, there is one primary authentication method that is supported: **API Key**. See this adaptor's [Configuration docs](/adaptors/packages/beyonic-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "apiUrl": "https://beyonic.com/api",
+ "apiToken": "cdjhiooiSIUJHDYSYUIHASIOHKAANOAJJjsiujhaka"
+}
+```
+
+### Helpful Links
+
+1. [Beyonic Documentation](https://apidocs.beyonic.com)
+2. [Beyonic API Reference](https://apidocs.beyonic.com/reference)
+3. [Webhook Guide](https://apidocs.beyonic.com/webhooks)
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
+
diff --git a/adaptors/bigquery.md b/adaptors/bigquery.md
new file mode 100644
index 000000000000..7c4a05846eb7
--- /dev/null
+++ b/adaptors/bigquery.md
@@ -0,0 +1,50 @@
+---
+title: Big Query
+---
+
+## About Big Query
+
+BigQuery is Google Cloud’s fully managed, serverless, and highly scalable data warehouse solution. It enables organizations to store, analyze, and visualize large datasets using SQL queries, machine learning, and real-time analytics.
+
+## Integration Options
+
+BigQuery supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** BigQuery has a REST API that enables external services like OpenFn to pull data from BigQuery, or push data from external apps to BigQuery. This option is suited for scheduled, bulk syncs or workflows that must update data in BigQuery with external information. See [functions](/adaptors/packages/bigquery-docs) for more on how to use this adaptor to work with the API.
+2. **Webhook:** Webhook or Data Forwarding to push data from BigQuery to external systems (see [docs](https://cloud.google.com/run/docs/triggering/webhooks)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+1. See [BigQuery docs](https://cloud.google.com/bigquery/docs) for the latest on supported authentication methods.
+2. When integrating with BigQuery via OpenFn, there is one primary authentication method that is supported: **API Key**. See this adaptor's [Configuration docs](/adaptors/packages/bigquery-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "type": "service_acconut",
+ "project_id": "some-project-id",
+ "private_key_id": "670b9e3c8c366e83aa569dd57cbfc5c575b72e42",
+ "private_key": "-----BEGIN PRIVATE KEY-----\nblah\nmoreblah=\n-----END PRIVATE KEY-----\n",
+ "client_email": "bigquery@some-project.iam.gserviceaccount.com",
+ "client_id": "someid",
+ "auth_uri": "https://accounts.google.com/o/oauth2/auth",
+ "token_uri": "https://oauth2.googleapis.com/token",
+ "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
+ "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/bigquery%40some-project.iam.gserviceaccount.com"
+}
+```
+
+### Helpful Links
+
+1. [BigQuery Documentation](https://cloud.google.com/bigquery/docs)
+2. [BigQuery REST API](https://cloud.google.com/bigquery/docs/reference/rest)
+
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
+
diff --git a/adaptors/chatgpt.md b/adaptors/chatgpt.md
new file mode 100644
index 000000000000..cf880686f7f8
--- /dev/null
+++ b/adaptors/chatgpt.md
@@ -0,0 +1,47 @@
+---
+title: ChatGPT AI Adaptor
+---
+
+## About ChatGPT
+
+[ChatGPT](https://chatgpt.com/) is an AI assistant built by OpenAI. It provides
+a family of large language models (LLMs) developed for natural language
+processing tasks such as text generation, summarization, and question answering.
+
+ChatGPT models process text input to generate responses but do not have built-in
+memory or autonomous decision-making. They can be integrated with external tools
+or APIs to execute actions beyond text-based outputs.
+
+## Use OpenFn + LLMs for AI-Driven Workflows
+
+While LLMs (like Claude or ChatGPT) can generate insights, recommendations, or
+structured data, they often **lack direct execution capabilities**—meaning they
+need an automation system to act on their outputs.
+
+**OpenFn workflows fill this gap by orchestrating actions based on LLM-generated
+responses.**
+
+Remember that LLMs are imperfect. Learn more about your recommended approach to
+AI in our [Responsible AI Policy](https://www.openfn.org/ai).
+
+## Integration Options
+
+Use this adpator to integrate with the
+[OpenAI Platform API](https://platform.openai.com/docs/overview), to integrate
+ChatGPT AI models into your workflows to leverage their natural language
+processing (NLP) capabilities.
+
+Example AI-driven workflow on OpenFn:
+
+
+## Authentication
+
+See this adaptor's
+[Configuration docs](/adaptors/packages/chatgpt-configuration-schema) for
+technical docs on the authentication parameters required.
+
+## Helpful Links
+
+- About ChatGPT: https://chatgpt.com/
+- API docs: https://platform.openai.com/docs/overview
+- API Intro: https://www.chatbase.co/blog/chatgpt-api
diff --git a/adaptors/claude.md b/adaptors/claude.md
new file mode 100644
index 000000000000..dcbf59b2a4b6
--- /dev/null
+++ b/adaptors/claude.md
@@ -0,0 +1,45 @@
+---
+title: Claude AI Adaptor
+---
+
+## About Claude
+
+[Claude](https://claude.ai/) is an AI assistant built by Anthropic. It provides
+a family of large language models (LLMs) developed for natural language
+processing tasks such as text generation, summarization, and question answering.
+
+Claude models process text input to generate responses but do not have built-in
+memory or autonomous decision-making. They can be integrated with external tools
+or APIs to execute actions beyond text-based outputs.
+
+## Use OpenFn + LLMs for AI-Driven Workflows
+
+While LLMs (like Claude or ChatGPT) can generate insights, recommendations, or
+structured data, they often **lack direct execution capabilities**—meaning they
+need an automation system to act on their outputs.
+
+**OpenFn workflows fill this gap by orchestrating actions based on LLM-generated
+responses.**
+
+Remember that LLMs are imperfect. Learn more about your recommended approach to
+AI in our [Responsible AI Policy](https://www.openfn.org/ai).
+
+## Integration Options
+
+Use this adpator to integrate with the
+[Claude API](https://www.anthropic.com/api), to integrate Claude AI models into
+your workflows to leverage their natural language processing (NLP) capabilities.
+
+Example AI-driven workflow on OpenFn:
+
+
+## Authentication
+
+See this adaptor's
+[Configuration docs](/adaptors/packages/claude-configuration-schema) for
+technical docs on the authentication parameters required.
+
+## Helpful Links
+
+- About Claude: https://claude.ai/
+- API docs: https://www.anthropic.com/api
diff --git a/adaptors/collections.md b/adaptors/collections.md
index 2ef3de8b7450..af9595e3f491 100644
--- a/adaptors/collections.md
+++ b/adaptors/collections.md
@@ -4,38 +4,60 @@ title: Collections Adaptor
## Collections Overview
-The Collections API provides access to a secure key/value store on the OpenFn
-Platform. It is designed for high performance over a large volume of data.
+The Collections API provides access to a secure key/value store that allows
+users to store, update and reuse data across workflows in their OpenFn projects.
+It is designed for high performance over a large volume of data.
-Collections are secure, private datastores which are visible only to Workflows
-within a particular OpenFn Project. They can be created, managed and destroyed
-from the OpenFn Admin page.
+Collections are secure, private datastores which are only accessible to
+Workflows within a particular OpenFn Project. They can be created, managed and
+destroyed from the OpenFn Admin page.
+
+The Collections Adaptor provides an interface to workflows to use the
+Collections API.
When running in the CLI, a Personal Access Token can be used to get access to
the collection (generated from the app at /profile/tokens).
-See the [Collections](/documentation/build/collections) Platform Docs to learn
-more about Collections.
-
-:::caution
-
-Collections must be created in the platform Admin page before they can be used.
-
-Refer to the [Collections Docs](/documentation/build/collections) for details
+Learn more about Collections and common use cases in the
+[Collections Docs](/documentation/build/collections).
+
+:::caution Collections is _not_ a long-term data storage solution!
+
+The OpenFn Collections feature is designed to be a _temporary_ data store or
+integration “buffer” during data transfer and processing between digital
+systems.
+
+- This feature is optimized for throughput and performance (rather then
+ long-term retention and making the data accessible to end users for
+ querying/analysis)
+- Technically, collections is a key value store, so performance for complex
+ aggregations or queries is either unsupported or inefficient
+- Collections doesn't support advance archiving, backup, and recovery features
+ that you’d want in a long-term storage solution
+- Most buffer implementations have retention mechanisms that automatically
+ expire/purge data after a certain period of time to clear up space and memory
+ to ensure resources aren’t exhausted for active processing
+
+Therefore, if you're interested in storing data collected longer term–we
+recommend you set up a dedicated database and configure OpenFn to route data to
+there as a long-term storage and/or backup option. Ask on
+[Community](https://community.openfn.org) or get in touch with
+[our support team](mailto://support@openfn.org) to learn more about related
+OpenFn services.
:::
## The Collections Adaptor
-The Collections API is inserted into all Steps through a special kind of
-adaptor.
+The Collections API is inserted into all each Step of a Workflow through a
+special kind of adaptor.
Uniquely, the Collections adaptor it is designed to be run _alongside_ other
-adaptors, not by itself. It is injected into the runtime environment for you for
-you by OpenFn. This makes the Collections API available to every Step in a
-Workflow, regardless of which adaptor it is using.
+adaptors, not by itself. It is automatically injected into the runtime
+environment making the Collections API available to every Step in a Workflow,
+regardless of which adaptor it is using.
-If using the CLI run a workflow with Collections, refer to the
+If using the CLI the use Collections locally, refer to the
[CLI Usage](#cli-usage) guide below.
## Usage Guide
@@ -56,17 +78,17 @@ fetch all keys which relate to Mr Benn in 2024).
The Collections API gives you four functions to read, write and remove data from
a collection.
-- Use [`collections.get()`](adaptors/packages/collections-docs#collections_get)
+- Use [`collections.get()`](/adaptors/packages/collections-docs#collections_get)
to fetch a single value, or batch-download a range of values.
- Use
- [`collections.each()`](adaptors/packages/collections-docs#collections_each) to
- efficiently iterate over a range of items in a collection. Recommended for
+ [`collections.each()`](/adaptors/packages/collections-docs#collections_each)
+ to efficiently iterate over a range of items in a collection. Recommended for
large data sets.
-- Use [`collections.set()`](adaptors/packages/collections-docs#collections_set)
+- Use [`collections.set()`](/adaptors/packages/collections-docs#collections_set)
to upload one or more values to a collection. `set()` is always an "upsert":
if a key already exists, it's value will be replaced by the new value
- Use
- [`collections.remove()`](adaptors/packages/collections-docs#collections_remove)
+ [`collections.remove()`](/adaptors/packages/collections-docs#collections_remove)
to remove one or more values.
Detailed usage examples are provided below.
@@ -180,7 +202,7 @@ collections.get('my-collection', 'commcare-fhir-value-mappings').then(state => {
state.mappings = state.data;
return state;
});
-collecions.each($.inputs, state => {
+collections.each($.inputs, state => {
const mappedString = state.mappings[state.data.diagnosis];
state.resources ??= {};
state.resources[state.data.id] = mappedString;
@@ -234,7 +256,7 @@ collections.each(
You can use `createdBefore` and `createdAfter` dates, which must be ISO 1806
formatted strings. The `createdBefore` timestamp will match all dates less than
-or equal to (<=) the _start_ of the provided date. Conversely, `createdAfter`
+or equal to (<=) the _start_ of the provided date. Conversely, `createdAfter`
will match dates greater than or equal to the _end_ of the provided date.
By default, all matching values will be returned to you, but you can limit how
@@ -256,21 +278,22 @@ fn(state => {
## CLI usage
-:::info
+Workflows which use Collections can be run through the CLI. You will need to:
-Improved Collections support is coming to the CLI soon.
+- Get a Personal Access Token (PAT)
+- Update the `workflow.json` with your PAT and the OpenFn endpoint
+- Set the step to use the Collections adaptor
+
+:::tip
+
+You can also call the Collections API directly from the CLI. See the
+[CLI Collections Guide](/documentation/collections-cli)
:::
Collections are designed for close integration with the platform app, but can be
used from the CLI too.
-You will need to:
-
-- Set the job to use two adaptors
-- Pass a Personal Access Token
-- Set the Collections endpoint
-
You can get a Personal Access Token from any v2 deployment.
Remember that a Collection must be created from the Admin page before it can be
@@ -290,7 +313,7 @@ You'll need to set configuration on the state.json:
{
"configuration": {
"collections_endpoint": "http://localhost:4000/collections",
- "collections_token": "...paste the token from the app..."
+ "collections_token": "...paste your Personal Access Token..."
}
}
```
@@ -306,7 +329,7 @@ If you're using `workflow.json`, set the token and endpoint on
"steps": [ ... ],
"credentials": {
"collections_endpoint": "http://localhost:4000/collections",
- "collections_token": "...paste the token from the app..."
+ "collections_token": "...paste your Personal Access Token..."
}
}
}
diff --git a/adaptors/commcare.md b/adaptors/commcare.md
index a555275cc674..f4e4045135aa 100644
--- a/adaptors/commcare.md
+++ b/adaptors/commcare.md
@@ -41,11 +41,12 @@ CommCare supports 2 primary integration options:
to _push_ `cases` and `forms` data from CommCare to external systems. This
option is suited for _real-time_, event-based data integration.
-2. **[REST APIs](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143958022/API+Access)**
+2. **[REST APIs](https://commcare-hq.readthedocs.io/api/index.html#commcare-apis)**
that enable external services like OpenFn to _pull_ data from CommCare, or
push data from external apps to CommCare. This option is suited for
_scheduled, bulk syncs_ or workflows that must update data in CommCare with
- external information.
+ external information. Also [see here](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2279637003/CommCare+API+Overview)
+ for more on the API Explorer.
This OpenFn adaptor is designed for option #2
[CommCare's APIs](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2279637003/CommCare+API+Overview).
@@ -77,7 +78,7 @@ To set up a connection, in CommCareHQ you'll need to:
set up on OpenFn, add the authentication type, the username and password here
7. You can test the connection, then save it
-
+
### Data Forwarding Options
@@ -107,7 +108,7 @@ Here's how you can configure CommCare to forward a form to OpenFn.
to find the XMLNS of any form.
9. Click "Start Forwarding" to save and activate
-
+
#### Forwarding Specific Case Types
@@ -123,7 +124,7 @@ forms. Here's how you can configure CommCare to forward a case type to OpenFn.
7. Select which case type(s) you want to forward, for example `patient`
8. Exclude any (eg. test) users
-
+
:::tip Data Forwarding and OpenFn Workflow Design
@@ -182,7 +183,7 @@ See platform docs [on managing credentials](/documentation/manage-projects/manag
how to configure a credential in OpenFn and see the below CommCare credential
example.
-
+
If you're using the `Raw JSON` credential type, your configuration may look like this:
@@ -234,7 +235,7 @@ for a detailed description of the types of data.
> time through a CommCare form. Ultimately form data is the source of all case
> data, but not all form data is case data.
-
+
### Mapping CommCare Metadata to External Systems
Use the [CommCare App Summary](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143956371/App+Summary) to view and export case or form metadata to XLS. This will help you discover what data is available to be mapped to an external system.
@@ -253,21 +254,91 @@ If integrating with CommCare `forms`, you may need to make sure that any unique
:::
+### Lookup Tables in CommCare
+Lookup tables in CommCare store reference data that can be used across multiple forms and workflows. They are often used for predefined lists such as as health facility names, geographic locations, product catalogs, or standardized response options.
+
+#### Querying Lookup Tables
+When fetching lookup table data in using CommCare APIs, there are two main approaches:
+
+**1. Using the Fixture API**
+[See here](https://commcare-hq.readthedocs.io/api/fixture.html) for the CommCare docs on this API. FYI `fixture` is a more technical term that the CommCare docs sometimes use to refer to a `lookup table`.
+```
+//sample openfn job to get a specific 'diagnosis' lookup table
+get("fixture/?fixture_type=diagnosis")
+```
+
+**Pros:**
+- Simple and direct API for querying a specific lookup table; response include lookup table metadata and data.
+- Works well when items from only a couple of tables (e.g., 1-3) need to be queried.
+
+**Cons:**
+- Requires multiple API calls if several tables are needed, which can be inefficient at scale.
+- See `lookup_table_item` API if querying data across multiple lookup tables.
+
+**2. Using the lookup_table_item API**
+[See here](https://commcare-hq.readthedocs.io/api/fixture.html#list-lookup-table-row) for the CommCare docs on this API.
+You can use this API to query _and_ update lookup table items or rows.
+
+```
+get('lookup_table_item', //to list all lookup table items across multiple tables -> bulk query
+ { limit: 100000 }); //will return 100k items at a time, which can be paged through if more are expected
+
+fn(state => {
+ //custom function to then assign & group lookup_table_items to new variables
+ const findLookupById = (id) => state.data.filter((i) => i.data_type_id === id);
+
+ //assign to facility, product, medications variables to use later in WF
+ state.facility = findLookupById("facility_table_id");
+ state.product = findLookupById("product_table_id");
+ state.medications = findLookupById("medications_table_id");
+ return state;
+})
+```
+
+**Pros:**
+- Good for bulk querying lookup table rows in a single request, reducing API calls.
+- Useful for OpenFn workflows requiring data from multiple lookup tables.
+- Support for create & update of lookup table items.
+
+**Cons:**
+- Retrieves all lookup tables and filters them in-memory, which can be inefficient if only a few tables are needed.
+
+#### Updating Lookup Tables
+You can bulk update rows in lookup tables using the [`bulk()` function](/adaptors/packages/commcare-docs#bulk) in the CommCare adaptor that will utilize this [CommCare bulk upload API](https://commcare-hq.readthedocs.io/api/fixture.html#bulk-upload-lookup-tables). **Tip:** Set the `replace` option as `false` if you want to _update_ (and not overwrite) tables.
+
+Or, you can edit or delete an individual lookup table row via the [lookup_table_item API](https://commcare-hq.readthedocs.io/api/fixture.html#edit-or-delete-lookup-table-row).
+```
+request('PUT', `/a/${$.configuration.domain}/api/v1/lookup_table_item/${item-id}`} //to update 1 row
+
+request('DELETE', `/a/${$.configuration.domain}/api/v1/lookup_table_item/${item-id}`} //to delete 1 row
+```
+
+#### Best Practices
+- Use the `Fixture` API when fetching data for only a couple of (1-3) lookup tables.
+- Use the `lookup_table` API for scenarios where data from multiple lookup tables needs to be queried in bulk.
+- Consider performance trade-offs when selecting which API to use, balancing API efficiency with data processing overhead. [See CommCare docs](https://commcare-hq.readthedocs.io/api/index.html#data-apis) for all available data APIs.
+
+#### Troubleshooting tips
+If some tables are throwing errors when being fetched using the fixtures API, the lookup table might be corrupted. Consider exporting the table and re-importing it.
+
## Helpful Links
### About Forms, case and data management
+- [CommCare API Overview](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2279637003/CommCare+API+Overview)
- [Case management overview](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955170/Case+Management+Overview)
- [Form and case data in CommCare](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143954460/Metadata+Glossary)
+- [CommCare Lookup tables](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955074/Lookup+Tables)
+- [Lookup table APIs](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143957320/Lookup+Table+API)
### CommCare API Docs
CommCare has different APIs for reading vs. modifying data. Some helpful links:
-- [Data APIs](https://confluence.dimagi.com/display/commcarepublic/Data+APIs)
+- [Data APIs](https://commcare-hq.readthedocs.io/api/index.html#data-apis)
- [API Explorer](https://commcare-api-explorer.dimagi.com/)
-- [Bulk Case Upload API to mass update case records](https://confluence.dimagi.com/display/commcarepublic/Bulk+Upload+Case+Data)
+- [CommCare API Overview](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2279637003/CommCare+API+Overview)
### Implementation Examples
@@ -275,4 +346,4 @@ CommCare has different APIs for reading vs. modifying data. Some helpful links:
- MiracleFeet (CommCare-to-Salesforce sync):
https://github.com/OpenFn/miracle-feet
- Lwala (CommCare-Salesforce 2-way sync): https://github.com/OpenFn/lwala
-- myAgrto (CommCare-Salesforce): https://github.com/OpenFn/myagro-commcare-sf
\ No newline at end of file
+- myAgro (CommCare-Salesforce): https://github.com/OpenFn/myagro-commcare-sf
diff --git a/adaptors/dynamics.md b/adaptors/dynamics.md
new file mode 100644
index 000000000000..1c13598f34ef
--- /dev/null
+++ b/adaptors/dynamics.md
@@ -0,0 +1,41 @@
+---
+title: Dynamics Adaptor
+---
+
+## About Microsoft Dynamics
+
+[Microsoft Dynamics](https://www.microsoft.com/en-us/dynamics-365) is a suite of enterprise resource planning (ERP) and customer relationship management (CRM) applications. It helps organizations manage sales, customer service, marketing, operations, and more.
+
+## Integration Options
+
+**1. Rest API:** Dynamics offers a REST API that enables external applications to interact with its database and perform bulk operations. This option is ideal for applications requiring scheduled or bulk synchronization with Dynamics. Refer to the Dynamics REST API [documentation](https://learn.microsoft.com/en-us/powerapps/developer/data-platform/webapi/overview) for detailed guidelines on endpoints and payload formats.
+
+**2. Webhook:** Webhook or Data Forwarding to push data from Dynamics to external systems (see [docs](https://learn.microsoft.com/en-us/dynamics365/customer-engagement/developer/webhooks)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+See [Microsoft docs](https://learn.microsoft.com/en-us/azure/active-directory/) for the latest on supported authentication methods. When integrating with Dynamics via OpenFn, **Access Token** is supported. See this adaptor's [Configuration docs](/adaptors/packages/dynamics-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "resource": "https://openfn.crm2.dynamics.com",
+ "apiVersion": "8.2.0",
+ "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjlGWERwYmZNRlQyU3ZRdVhoODQ2WVR3RUlCdyIsI"
+}
+```
+
+### Helpful Links
+
+1. [Microsoft Dynamics Documentation](https://learn.microsoft.com/en-us/dynamics365/)
+2. [Dynamics REST API Guide](https://learn.microsoft.com/en-us/powerapps/developer/data-platform/webapi/overview)
+3. [Webhooks Setup](https://learn.microsoft.com/en-us/dynamics365/customer-engagement/developer/webhooks)
+
+### Implementation Examples
+
+_Comming soon!_
+
+
+
+
diff --git a/adaptors/fhir-4.md b/adaptors/fhir-4.md
new file mode 100644
index 000000000000..fc156a2f8d4e
--- /dev/null
+++ b/adaptors/fhir-4.md
@@ -0,0 +1,721 @@
+---
+title: FHIR r4 Adaptor
+---
+
+Language support for FHIR version 4. This adaptor is compatible with any FHIR R4
+compliant server.
+
+The FHIR 4 adaptor is implemented in TypeScript and comes complete with full
+TypeScript definitions.
+
+:::tip
+
+This adaptor is strictly only compatible with
+[FHIR R4](https://www.hl7.org/fhir/R4). But all versions are supported by our
+[generic fhir](/adaptors/fhir) adaptor!
+
+:::
+
+## About FHIR
+
+[FHIR](https://www.hl7.org/fhir/overview.html) stands for Fast Healthcare
+Interoperability Resources. It is a standard for representing and exchanging
+healthcare data electronically.
+
+Learn more about FHIR and
+[FHIR for heath data exchange](/documentation/get-started/standards#fhir-for-health-data-exchange).
+
+## Authentication
+
+The FHIR standard does not directly prescribe authentication and authorization
+methods. Instead, it provides security guidelines and leaves the choice of
+implementation to the developers of FHIR servers and clients. See the FHIR
+[docs](https://www.hl7.org/fhir/security.html) for the latest security-related
+recommendations. Depending on the FHIR systems being integrated via OpenFn, you
+might employ a Basic Auth, API key, or OAuth authentication scheme.
+
+See this adaptor's
+[Configuration docs](/adaptors/packages/fhir-4-configuration-schema) for more on
+the required authentication parameters.
+
+See platform docs on
+[managing credentials](/documentation/manage-projects/manage-credentials) for
+how to configure a credential in OpenFn. If working locally or if using a Raw
+JSON credential type, then your configuration will look something like this to
+define your target endpoint and FHIR version:
+
+```
+{
+ "baseUrl": "https://hapi.fhir.org", //fhir endpoint
+ "apiPath": "baseR4" //fhir version
+}
+```
+
+## Code Assist
+
+The OpenFn Inspector provides full code-assist and code-complete for all FHIR
+resource types.
+
+Pressing `ctrl + space` will open up the Suggestions list, which provides
+contextual auto-completion. Use this to more easily create FHIR resources.
+
+Note that code assist will only provide suggestions for commonly used
+properties, but custom properties can also be passed directly and will be
+written to the new resource
+
+## Bundles
+
+The FHIR-4 adaptor provides support for bundles.
+
+You can create resources and add them to a bundle:
+
+```js
+addToBundle($.resource);
+```
+
+Where `$.resource` is a FHIR resource, or an array of resources, on the state
+object.
+
+This will add the resources to a bundle resource, under the `entry` key, on
+state under a key called `bundle`. To save the bundle under a different key,
+pass a string as the second argument:
+
+```js
+addToBundle($.resource, 'patients-bundle');
+```
+
+Each item in the bundle will be given a `request` object with the `PUT` method
+and a URL. For example:
+
+```js
+addToBundle({
+ id: 'x',
+ resourceType: 'Patient',
+});
+```
+
+Will create a state object like this:
+
+```json
+{
+ "bundle": {
+ "resourceType": "bundle",
+ "entry": [
+ {
+ "resource": {
+ "id": "x",
+ "resourceType": "Patient"
+ },
+ "request": {
+ "method": "PUT",
+ "url": "Patient/x"
+ }
+ }
+ ]
+ }
+}
+```
+
+To send the Bundle to the FHIR server defined in `state.configuration.baseURL`,
+call `uploadBundle()`.
+
+You can pass the name of a key on state to upload a specific bundle:
+
+```js
+uploadBundle('patients-bundle');
+```
+
+Or you can pass your own bundle object in directly:
+
+```js
+uploadBundle($.bundle);
+```
+
+## Builder Functions
+
+The FHIR-4 adaptor provides builder functions which make it easier to define
+FHIR resources.
+
+Builders provide short-hand properties and default values, reducing the
+boiler-plate needed to create FHIR resources. They also enable code-assist to be
+provided by code editors.
+
+The basic pattern is something like this:
+
+```js
+create(
+ builders.patient({
+ id: 'example',
+ identifier: [
+ {
+ value: '12345',
+ type: ['MR', 'http://terminology.hl7.org/CodeSystem/v2-0203'],
+ system: 'urn:oid:1.2.36.146.595.217.0.1',
+ period: { start: '2001-05-06' },
+ },
+ ],
+ active: true,
+ name: [
+ {
+ family: 'Chalmers',
+ given: ['Peter', 'James'],
+ },
+ ],
+ managingOrganization: 'Organization/1',
+ })
+);
+```
+
+### Basic Usage
+
+All builders are available on the `b` (short for `builders`) namespace within
+your OpenFn expression.
+
+Every resource in FHIR should have a corresponding builder function on the `b.`
+namespace (check the reference for details): for example, `b.patient()`,
+`b.observation()`, `b.encounter()` and `b.invoice()`.
+
+The first argument for the builder is an object with properties corresponding to
+FHIR json:
+
+```js
+b.patient({
+ identifier: $.data.patientId,
+});
+```
+
+Which will generate the following data structure:
+
+```json
+{
+ "resourceType": "Patient"
+ "id": "123",
+}
+```
+
+You don't have to use builder functions - you can just write our your FHIR
+structures in-line. But the builders come with some key benefits:
+
+1. They backed by full TypeScript definitions, so you'll get validation and
+ code-assist as you type
+2. They simplify some FHIR data structures
+
+The builders are designed to take a fuzzy, relaxed definition of a FHIR resource
+and convert it to correct JSON. For example, a Reference type can be passed as a
+fully FHIR reference object, a string id, or an existing FHIR resource. The
+builder will figure out how to parse and map it to a Reference with reasonable
+defaults.
+
+Builders are not operations, so they don't sit at the top of your job code. You
+can either pass them as an argument to an operation:
+
+```js
+create(b.patient($.data));
+```
+
+Or within a callback function (like `fn()`):
+
+```js
+fn(state => {
+ state.newPatient = b.patient(state.data);
+ return state;
+});
+```
+
+## Cardinality
+
+Resource builders will automatically embed values into arrays where appropriate,
+simplifying cardinality handling.
+
+For example, a Patient's `identifier` property contains an array of identifiers.
+But when using a builder function, you can pass a single identifier:
+
+```js
+b.patient({
+ identifier: {
+ value: 'abc',
+ },
+});
+```
+
+Which will be embedded into an array for you:
+
+```json
+{
+ "resourceType": "Patient",
+ "identifier": [
+ {
+ "value": "abc"
+ }
+ ]
+}
+```
+
+## DataTypes Builders
+
+The builders have special handling and helpers for FHIR datatypes, like
+`identifier` and `coding`.
+
+Like resource builders, they exist on the `b.` or `builders.` namespace. Unlike
+the other builders, they are hand-written and not auto-generated from the spec.
+This means they have a slightly different API.
+
+The most important datatype builders are detailed below.
+
+:::tip
+
+When you pass an object into a builder, internally that object will be passed to
+a datatype builder.
+
+So you can either pass a reference directly:
+
+```js
+b.patient({
+ identifier: {
+ value: 'abc',
+ },
+});
+```
+
+Or use a builder explicitly:
+
+```js
+b.patient({
+ identifier: b.id('abc'),
+});
+```
+
+You may want to use a builder to pass an extension.
+
+```js
+b.patient({
+ identifier: b.identifier('abc', {
+ url: 'http://hospital.example.org/fhir/StructureDefinition/identifier-issuingDepartment',
+ value: 'Emergency Department',
+ }),
+});
+```
+
+:::
+
+### Identifiers
+
+Use `b.identifier(id, ...extensions)` or its shorthand,
+`b.id(id, ...extensions)`, to create an Identifier object.
+
+You can pass a single string value to create an identifier without a system:
+
+```js
+b.identifier('ip_request1.1');
+```
+
+More commonly, you'll pass a `{ value, system }` pair:
+
+```js
+b.identifier({
+ system: 'http://moh.gov.et/fhir/hiv/identifier/MRN',
+ value: 'MRN12345671',
+});
+```
+
+Any additional arguments will be treated as extensions:
+
+```js
+b.identifier(
+ {
+ system: 'http://moh.gov.et/fhir/hiv/identifier/MRN',
+ value: 'MRN12345671',
+ },
+ {
+ url: 'http://hospital.example.org/fhir/StructureDefinition/identifier-issuingDepartment',
+ valueString: 'Emergency Department',
+ },
+ {
+ url: 'http://hospital.example.org/fhir/StructureDefinition/identifier-issuedDate',
+ valueDateTime: '2023-06-15T14:30:00Z',
+ }
+);
+```
+
+Resulting in the following JSON structure:
+
+```json
+{
+ "resourceType": "Patient",
+ "identifier": [
+ {
+ "system": "http://moh.gov.et/fhir/hiv/identifier/MRN",
+ "value": "MRN12345671",
+ "extension": [
+ {
+ "url": "http://hospital.example.org/fhir/StructureDefinition/identifier-issuingDepartment",
+ "valueString": "Emergency Department"
+ },
+ {
+ "url": "http://hospital.example.org/fhir/StructureDefinition/identifier-issuedDate",
+ "valueDateTime": "2023-06-15T14:30:00Z"
+ }
+ ]
+ }
+ ]
+}
+```
+
+### Codings
+
+Use `b.coding(code, system, extra)` or its shorthand,
+`b.c(code, system, extra)`, to create a Coding object
+
+You can create a coding directly with the `coding` builder:
+
+```js
+b.coding('MR', 'http://terminology.hl7.org/CodeSystem/v2-0203');
+// returns { code: 'MR', system: 'http://terminology.hl7.org/CodeSystem/v2-0203' }
+```
+
+You can also use the shorthand form, `c`:
+
+```js
+b.c('MR', 'http://terminology.hl7.org/CodeSystem/v2-0203', {
+ display: 'Medical record number',
+});
+// returns { code: 'MR', system: 'http://terminology.hl7.org/CodeSystem/v2-0203' }
+```
+
+You can pass any extra properties to the coding to add them to the object.
+Standard FHIR properties are type-supported and code-assisted.
+
+```js
+b.c('MR', 'http://terminology.hl7.org/CodeSystem/v2-0203', {
+ display: 'Medical record number',
+});
+// returns {
+// code: 'MR',
+// system: 'http://terminology.hl7.org/CodeSystem/v2-0203',
+// display: 'Medical record number' ,
+// }
+```
+
+### Codeable Concepts
+
+Use `b.concept(codings, extra)` or its shorthand, `b.cc(codings, extra)`, to
+create a Codeable Concept with one or more codings.
+
+Pass one or more codings, as tuples or objects, and optionally extra keys to add
+to the concept:
+
+```js
+b.concept(codings, props);
+```
+
+For example, to create a concept with a single coding:
+
+```js
+b.concept(['MR', 'http://terminology.hl7.org/CodeSystem/v2-0203']);
+// outputs { coding: [{ type: 'MR', system: 'http://terminology.hl7.org/CodeSystem/v2-0203' }]}
+```
+
+Or for multiple codings (note the extra array!)
+
+```js
+b.concept([
+ ['M', 'http://terminology.hl7.org/CodeSystem/v3-MaritalStatus'],
+ ['02', 'http://national-registry.example.org/marital-status'],
+]);
+```
+
+You can add extra props to the concept, like text:
+
+```js
+b.concept($.codings, { text: 'Married' });
+// outputs { text: 'Married', coding: [...]} }
+```
+
+Where a CodeableConcept is expected, it can be represented as a "tuple", or an
+array of two strings where the first value is the code, and the second is the
+system. The tuple will be expanded to a `{ code, system }` Coding object.
+
+For example, an Identifier's `type` accepts a CodeableConcept. We can create a
+patient like this:
+
+```js
+builders.patient({
+ identifier: [
+ {
+ type: ['MR', 'http://terminology.hl7.org/CodeSystem/v2-0203'],
+ },
+ ],
+});
+```
+
+Which will generate a resource like:
+
+```json
+{
+ "resourceType": "Patient",
+ "identifier": [
+ {
+ "type": {
+ "code": "MR",
+ "system": "http://terminology.hl7.org/CodeSystem/v2-0203"
+ }
+ }
+ ]
+}
+```
+
+You can also pass a concept object directly, with full type support.
+
+### References
+
+Some FHIR resources reference others. Reference types in the builders provide a
+number of short-hands.
+
+For example, a FHIR reference looks like this:
+
+```js
+"managingOrganization": {
+ "reference": "Organization/1"
+}
+```
+
+In the patient builder, we can provide this reference in a few ways.
+
+You can pass the reference string directly, which will be "expanded" into a
+reference.
+
+```js
+b.patient({
+ managingOrganization: 'Organization/1',
+});
+```
+
+You can pass an existing reference object (useful when mapping FHIR to FHIR)
+
+```js
+b.patient({
+ managingOrganization: $.patient.managingOrganization,
+});
+```
+
+Or you can pass an existing FHIR resource, if you happen to have access to the
+one you want to reference:
+
+```js
+b.patient({
+ managingOrganization: $.myOrg,
+});
+```
+
+Where supported, you can also pass an array of references.
+
+You can use `b.reference()` to create your own reference with the above rules
+(particularly useful when dealing with extensions which aren't typed).
+
+```js
+b.patient({
+ managingOrganization: b.ref($.org), // $.org can be a string, resource or reference
+});
+```
+
+### Multiple Types
+
+Some FHIR properties are polymorphic and support multiple data types. For
+example, a Patient's `deceased` property can be of type `boolean` or `datetime`.
+
+In FHIR json, this results in a type suffix being appended to the key. For
+example:
+
+```json
+{
+ "resourceType": "Patient",
+ "deceasedBoolean": false,
+ "deceasedDateTime": "2025-01-01"
+}
+```
+
+Builder functions allow you to specify the base property name and not worry
+about typing the key.
+
+For example, these both work in a FHIR-compliant way:
+
+```js
+b.patient({
+ deceased: true,
+});
+b.patient({
+ deceased: '2025-01-01',
+});
+```
+
+Internally, the builder will determine the type of the incoming value and
+generate the correct key.
+
+Note that passing suffixed keys still works just fine.
+
+```js
+b.patient({
+ deceasedDateTime: '2025-01-01',
+});
+```
+
+### Extensions
+
+FHIR is highly extensible, and the adaptor builder APIs have many features to
+support this.
+
+Any property can be prefixed with `_` to provide an extension, e.g,,:
+
+```js
+b.identifier({ _use: 'custom use' });
+```
+
+You can add an extension to any resource using the
+`addExtension(resource, extensionUrl, extensionValue)` builder.
+
+For example, the following code:
+
+```js
+fn(state => {
+ const patient = b.patient({});
+
+ b.addExtension(
+ patient,
+ 'http://hl7.org/fhir/StructureDefinition/patient-religion',
+ // declare a codeableConcept
+ b.cc(
+ ['1023', 'http://terminology.hl7.org/CodeSystem/v3-ReligiousAffiliation'],
+ { text: 'Muslim' }
+ )
+ );
+
+ return state;
+});
+```
+
+Creates a patient with an `religious affiliation` extension:
+
+```json
+{
+ "resourceType": "Patient",
+ "extension": [
+ {
+ "url": "http://hl7.org/fhir/StructureDefinition/patient-religion",
+ "valueCodeableConcept": {
+ "coding": [
+ {
+ "code": "1023",
+ "system": "http://terminology.hl7.org/CodeSystem/v3-ReligiousAffiliation"
+ }
+ ],
+ "text": "Muslim"
+ }
+ }
+ ]
+}
+```
+
+You can call `addExtension` Multiple times for a given resource. It will
+intelligently manage the `extension` array on the target resource for you.
+
+In addition, some datatype builders, like, `identifier`, allow extensions to be
+passed as extra arguments. For example:
+
+```js
+b.identifier('12345', {
+ url: 'http://hospital.example.org/fhir/StructureDefinition/identifier-issuingDepartment',
+ value: 'Emergency Department',
+});
+```
+
+Produces an identifier like:
+
+```json
+{
+ "extension": [
+ {
+ "url": "http://hospital.example.org/fhir/StructureDefinition/identifier-issuingDepartment",
+ "valueString": "Emergency Department"
+ }
+ ],
+ "value": "12345"
+}
+```
+
+### System Mappings
+
+FHIR builders allow `system` strings in identifiers and codings to be mapped.
+This allows you to specify short-form system strings, or more easily map
+resources between FHIR systems.
+
+This is done using the `util.setSystemMap(map)` function. This takes an object
+which maps source strings to destinations. Keys in the object are values you
+want to use in your job code - the values you pass to the builders. The values
+of the object are the target values - the systems you want to appear in your
+generated resources.
+
+Set the system map at the top of every step that needs a system map (note that
+mappings are NOT remembered between steps)
+
+```js
+util.setSystemMap({ icd10: 'http://hl7.org/fhir/sid/icd-10' });
+```
+
+Because systems tend to be long strings, it can be convenient to define a
+short-hand syntax, which simplifies mapping code.
+
+For example:
+
+```js
+b.setSystemMap({
+ smartcare: 'http://moh.gov.et/fhir/hiv/identifier/SmartCareID',
+});
+
+create(
+ b.patient({
+ identifier: { value: 'xyz', system: 'smartcare' },
+ })
+);
+```
+
+This will generate a resource with a full system:
+
+```js
+{
+ "resourceType": "Patient"
+ "identifier": [
+ {
+ "system": "http://moh.gov.et/fhir/hiv/identifier/SmartCareID",
+ "value": "xyz"
+ }
+ ],
+}
+```
+
+You can also use the datatype helpers directly:
+
+```js
+util.setSystemMap({ icd10: 'http://hl7.org/fhir/sid/icd-10' });
+
+fn(state => {
+ const headache = b.coding('G44.1', 'icd10');
+ // ...
+ return state;
+});
+```
+
+This is also useful when mapping resources from one FHIR system to another.
+Simply configure a system map and source systems will auto-map to the
+destinations:
+
+```js
+// map incoming CDR systems to MOH systems
+util.setSystemMap({
+ 'http://cdr.aacahb.gov.et/Encounter':
+ 'http://moh.gov.et/fhir/hiv/identifier/encounter',
+});
+
+b.patient({
+ identifier: $.patient.
+});
+```
diff --git a/adaptors/fhir-fr.md b/adaptors/fhir-fr.md
new file mode 100644
index 000000000000..38ac51106d25
--- /dev/null
+++ b/adaptors/fhir-fr.md
@@ -0,0 +1,15 @@
+---
+title: FHIR-FR IG Adaptor
+---
+
+## Custom FHIR Adaptor: fhir-fr
+Note❗: This is a custom adaptor generated from this France FHIR Implementation Guide: https://hl7.fr/ig/fhir/core/2.0.0/index.html
+
+Custom FHIR adaptors generate a suite of helper functions specific to their source Implementation Guides.
+
+See the generic [fhir adaptor](/adaptors/fhir) and our [docs on standards](/documentation/get-started/standards) for more general guidance on OpenFn + FHIR.
+
+## Build your own FHIR Adaptor
+See the [Adaptors Wiki](https://github.com/OpenFn/adaptors/wiki/Generating-Fhir-Adaptors) to build your own adaptor for _your_ implementation guide by trying out our fhir-adaptor-generator (which is a new tool still in testing).
+
+Please share any questions or feedback on [community.openfn.org](https://community.openfn.org).
\ No newline at end of file
diff --git a/adaptors/fhir-ndr-et.md b/adaptors/fhir-ndr-et.md
new file mode 100644
index 000000000000..2a42c6a3ab2e
--- /dev/null
+++ b/adaptors/fhir-ndr-et.md
@@ -0,0 +1,15 @@
+---
+title: FHIR-NDR-ET IG Adaptor
+---
+
+## Custom FHIR Adaptor: fhir-ndr-et
+Note❗: This is a custom adaptor generated from this Implementation Guide `Ethiopia FHIR Implementation Guide - HIV Treatment & Care Services` authored by Jembi Health Systems: https://build.fhir.org/ig/jembi/ethiopia-hiv/branches/master/index.html
+
+Custom FHIR adaptors generate a suite of helper functions specific to their source Implementation Guides.
+
+See the generic [fhir adaptor](/adaptors/fhir) and our [docs on standards](/documentation/get-started/standards) for more general guidance on OpenFn + FHIR.
+
+## Build your own FHIR Adaptor
+See the [Adaptors Wiki](https://github.com/OpenFn/adaptors/wiki/Generating-Fhir-Adaptors) to build your own adaptor for _your_ implementation guide by trying out our fhir-adaptor-generator (which is a new tool still in testing).
+
+Please share any questions or feedback on [community.openfn.org](https://community.openfn.org).
\ No newline at end of file
diff --git a/adaptors/fhir.md b/adaptors/fhir.md
new file mode 100644
index 000000000000..91d22834dcb9
--- /dev/null
+++ b/adaptors/fhir.md
@@ -0,0 +1,48 @@
+---
+title: FHIR Adaptor
+---
+
+## About FHIR
+
+[FHIR](https://www.hl7.org/fhir/overview.html) stands for Fast Healthcare Interoperability Resources. It is a standard for representing and exchanging healthcare data electronically.
+
+
+:::tip About this adaptor and features coming soon!
+
+This adaptor is very basic and generic, used mostly to integrate demo FHIR servers. It's a work-in-progress, so share questions and feedback on [community.openfn.org](https://community.openfn.org).
+
+**FHIR version-specific adaptors (e.g., `fhir-r4`) with enhanced functionality are coming soon** to fast-track integration setup with more helper functions, templates, and docs than this simple adaptor. See the [Adaptors Wiki](https://github.com/OpenFn/adaptors/wiki/Generating-Fhir-Adaptors) for how to build an adaptor specific to your FHIR Implementation Guide.
+
+:::
+
+## Integration Options
+
+**1. Rest API:** The FHIR specification includes a REST API that enables external services like OpenFn to pull data from the FHIR server, or push data from external apps to FHIR servers. This option is suited for scheduled, bulk syncs or workflows that must update data with external information. See [functions](/adaptors/packages/fhir-docs) for more on how to use this adaptor to work with the API.
+
+**2. Webhook:** The FHIR specification does not inherently define a webhook or data-forwarding mechanism. However, many FHIR implementations and platforms offer extensions or configurations that support similar functionality. This option is suited for real-time, event-based data integration. Check out the FHIR `Subscription` resource [documentation](https://build.fhir.org/subscription-definitions.html) to learn more about one way this might be implemented.
+
+## Authentication
+
+The FHIR standard does not directly prescribe authentication and authorization methods. Instead, it provides security guidelines and leaves the choice of implementation to the developers of FHIR servers and clients. See the FHIR [docs](https://www.hl7.org/fhir/security.html) for the latest security-related recommendations. Depending on the FHIR systems being integrated via OpenFn, you might employ a Basic Auth, API key, or OAuth authentication scheme.
+
+See this adaptor's [Configuration docs](/adaptors/packages/fhir-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this to define your target endpoint and FHIR version:
+
+```
+{
+ "baseUrl": "https://hapi.fhir.org", //fhir endpoint
+ "apiPath": "baseR4" //fhir version
+}
+```
+
+### Helpful Links
+
+1. [API documentation](https://www.hl7.org/fhir/http.html)
+2. [Digital Square on FHIR](https://digitalsquare.org/resourcesrepository/digital-square-on-fhir-4c78p)
+3. [Basic guide to interacting with FHIR Server](https://smilecdr.com/docs/fhir_standard/fhir_introduction.html)
+4. [Creating your first FHIR resource](https://medblocks.com/blog/fhir-101-creating-your-first-patient-resource-like-a-pro)
+5. Google's [Open Health Stack](https://developers.google.com/open-health-stack) tooling for working with FHIR
+
+Have resources or links to share? Submit a PR to edit this page or post on [community.openfn.org](https://community.openfn.org).
+
diff --git a/adaptors/gmail.md b/adaptors/gmail.md
new file mode 100644
index 000000000000..185d900ae5d6
--- /dev/null
+++ b/adaptors/gmail.md
@@ -0,0 +1,41 @@
+---
+title: Gmail Adaptor
+---
+
+## About Gmail
+
+Gmail is a free email service developed by Google that supports email communication, collaboration, file sharing and integration with other Google services.
+
+## Integration Options
+
+Gmail supports 2 primary integration options with OpenFn:
+
+1. Rest API: Gmail has a REST API that enables external services like OpenFn to pull data from Gmail, or push data from external apps to Gmail. This option is suited for scheduled, bulk syncs or workflows that must update data in Gmail with external information. See [functions](/adaptors/packages/gmail-docs) for more on how to use this adaptor to work with the API.
+
+2. Webhook: While Gmail does not natively support webhooks, you can implement webhook-like functionality to push data from Gmail to external systems. This option is suited for real-time, event-based data integration. Check out the Gmail [developer documentation](https://developers.google.com/gmail/api/guides/push) to learn how to set up push notifications on Gmail.
+
+## Authentication
+
+1. See [Gmail docs](https://developers.google.com/gmail/api/auth/scopes) for the latest on supported authentication methods.
+2. When integrating with Gmail via OpenFn, there is one primary authentication method that is supported: **Access Token**. You can generate an access token from Gmail using these [instructions](https://developers.google.com/identity/protocols/oauth2).
+3. See this adaptor's [Configuration docs](/adaptors/packages/gmail-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjlGWERwYmhax0rZNRlQyU3ZRdVhoODQ2WVR3RUlCdyIsI"
+}
+```
+
+### Helpful Links
+
+1. [API documentation](https://developers.google.com/gmail/api/guides)
+2. [Community Support](https://support.google.com/mail/)
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
diff --git a/adaptors/googlehealthcare.md b/adaptors/googlehealthcare.md
new file mode 100644
index 000000000000..8fc8fe0ea027
--- /dev/null
+++ b/adaptors/googlehealthcare.md
@@ -0,0 +1,42 @@
+---
+title: Google Cloud Healthcare Adaptor
+---
+
+## About Google Cloud Healthcare
+
+The [Google Cloud Healthcare API](https://cloud.google.com/healthcare-api) is a solution that enables the secure storage, management, and exchange of healthcare data on Google Cloud. It bridges existing care systems and applications hosted on Google Cloud, facilitating interoperability and integration with advanced analytics and machine learning tools.
+
+## Integration Options
+
+**1. Rest API:** Google Cloud Healthcare offers a REST API that enables external applications to interact with its database and perform bulk operations. This option is ideal for applications requiring scheduled or bulk synchronization with Google Cloud Healthcare. The API supports the [HL7® FHIR®](https://www.hl7.org/fhir/overview.html) (Fast Healthcare Interoperability Resources) standard. See [functions](/adaptors/packages/googlehealthcare-docs) for more on how to use this adaptor to work with the API.
+
+**2. Webhook:** The Google Cloud Healthcare API does not natively support webhooks as part of its core functionality. However, you can implement webhook-like behavior using other Google Cloud services and tools in combination with the Healthcare API.
+
+## Authentication
+
+1. See [Google Cloud Healthcare API docs](https://cloud.google.com/healthcare-api/docs/authentication#:~:text=You%20can%20authenticate%20to%20the,CLI%20credentials%20and%20ADC%20credentials.) for the latest on supported authentication methods.
+2. When integrating with Google Cloud Healthcare via OpenFn, there is one primary authentication method that is supported: **Access Token**. See this adaptor's [Configuration docs](/adaptors/packages/googlehealthcare-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjlGWERwYmZNRlQyU3ZRdVhoODQ2WVR3RUlCdyIsI"
+}
+```
+
+### Helpful Links
+
+1. [Google Cloud Healthcare API Documentation](https://cloud.google.com/healthcare-api)
+2. [Cloud Healthcare API Documentation](https://cloud.google.com/healthcare-api/docs)
+3. [FHIR Standard](https://www.hl7.org/fhir/)
+4. [Google Cloud Authentication Overview](https://cloud.google.com/docs/authentication)
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
+
+
diff --git a/adaptors/googlesheets.md b/adaptors/googlesheets.md
index fb303ceab652..37db9ee2507a 100644
--- a/adaptors/googlesheets.md
+++ b/adaptors/googlesheets.md
@@ -2,12 +2,10 @@
title: Google Forms/Google Sheets
---
-## Google Sheets Adaptor Overview
+The Google Sheets adaptor provides seamless integration between Google Forms,
+Google Sheets, and the OpenFn platform, enabling robust data flow management.
-Google Sheets adaptor provides seamless integration between Google Forms, Google
-Sheets, and the OpenFn platform, enabling robust data flow management.
-
-### Authentication and Authorization
+## Authentication and Authorization
This adaptor requires OAuth authorization to connect with Google Sheets. This
authorization can be achieved by a user or organization admin consenting to an
@@ -31,7 +29,7 @@ for how to complete the OAuth Client Setup form on OpenFn.
:::
-#### Permissions (Scopes)
+### Permissions (Scopes)
Permissions and access in an OAuth instance are defined by scopes which are
named differently by providers based on their functions within their platform.
@@ -42,17 +40,20 @@ said, please refer to
[Google's documentation on Oauth scopes](https://developers.google.com/identity/protocols/oauth2/scopes)
for the latest information.
-- `openid`
-- `email`
-- `profile`
-- `https://www.googleapis.com/auth/spreadsheets`
+- **The required endpoints** for the Google Oauth Client are available at
+ https://accounts.google.com/.well-known/openid-configuration.
+- **The mandatory scopes** are `openid email profile`
+- **The optional scopes** are available here:
+ https://developers.google.com/identity/protocols/oauth2/scopes (note that you
+ need to use the full URL, e.g.,
+ `https://www.googleapis.com/auth/spreadsheets`)
-### Integration Options
+## Integration Options
There are a couple of primary ways to integrate with this app, each catering to
different use cases.
-#### 1. Pushing Data to OpenFn via Google App Scripts
+### 1. Pushing Data to OpenFn via Google App Scripts
With this method, data from Google Forms or Google Sheets is automatically
pushed to an OpenFn webhook trigger workflow whenever new entries are made. This
@@ -110,7 +111,7 @@ function onFormSubmit(e) {
}
```
-#### 2. Pulling Data from Google Sheets using OpenFn
+### 2. Pulling Data from Google Sheets using OpenFn
Alternatively, you can pull data from Google Sheets at specific intervals or
on-demand using a `cron` workflow in OpenFn, allowing for more controlled data
@@ -124,7 +125,7 @@ learn how to configure a workflow step to use this OpenFn adaptor to
**Use Cases:** - Aggregating data for periodic reporting or analysis. -
Implementing batch processing for efficiency and resource optimization.
-#### 3. Pushing Data to Google Sheets via OpenFn
+### 3. Pushing Data to Google Sheets via OpenFn
The Google Sheets adaptor can also be used to push data to Google Sheets from
other systems via OpenFn. This allows for seamless integration between external
diff --git a/adaptors/hive.md b/adaptors/hive.md
new file mode 100644
index 000000000000..8f7f82827dfc
--- /dev/null
+++ b/adaptors/hive.md
@@ -0,0 +1,40 @@
+---
+title: Hive Adaptor
+---
+
+## About Apache Hive
+
+[Apache Hive](https://hive.apache.org/) is a data warehouse software that facilitates reading, writing, and managing large datasets stored in distributed storage systems.
+
+## Integration Options
+
+The `hive` adaptor provides direct database connections for accessing data and executing SQL and standard database operations. See [functions](/adaptors/packages/hive-docs) for more on how to use this adaptor.
+
+
+## Authentication
+
+See [Hive docs](https://hive.apache.org/docs/) for the latest on supported authentication methods. When integrating with a Hive database via OpenFn, you authenticate via SSH using authorized database credentials. See this adaptor's [Configuration docs](/adaptors/packages/hive-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "host": "some-host-url.compute-1.amazonaws.com",
+ "database": "demo-db",
+ "username": "admin-demo",
+ "password": "@super(!)Secretpass"
+}
+```
+
+### Helpful Links
+
+1. [Hive documentation](https://hive.apache.org/)
+
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
+
diff --git a/adaptors/http.md b/adaptors/http.md
new file mode 100644
index 000000000000..295cb231281d
--- /dev/null
+++ b/adaptors/http.md
@@ -0,0 +1,39 @@
+---
+title: HTTP Adaptor
+---
+
+## About the HTTP "universal" adaptor
+
+Communicate with web apps using [HTTP (HyperText Transfer Protocol)](https://www.cloudflare.com/learning/ddos/glossary/hypertext-transfer-protocol-http/).
+This adaptor enables out-of-box integration with any REST API!
+
+## Integration Options
+
+Use this adaptor to communicate with **any REST API** or any other app that can communicate via HTTP.
+
+**Note that OpenFn also supports Webhooks, but that is a workflow trigger type ([see docs](/documentation/build/triggers#webhook-event-triggers)), not an adaptor.**
+
+## Authentication
+
+HTTP itself does not enforce authentication, but many applications that use HTTP implement security mechanisms to control access. Common methods that can be used when integrating with OpenFn include Basic Authentication, API Keys and OAuth. See this adaptor's [Configuration docs](/adaptors/packages/http-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "username": "test@openfn.org",
+ "password": "@some(!)Password",
+ "access_token": "00QCjAl4MlV-WPX",
+ "baseUrl": "https://instance_name.surveycto.com"
+}
+```
+
+
+### Implementation Examples
+
+1. UNICEF Primero - UNHCR Progres Interoperability: [https://github.com/OpenFn/primero-progres](https://github.com/OpenFn/primero-progres)
+2. UNICEF Thailand Primero Interoperability: [https://openfn.github.io/primero-thailand/](https://openfn.github.io/primero-thailand/)
+
+
+
+
diff --git a/adaptors/hubtel.md b/adaptors/hubtel.md
new file mode 100644
index 000000000000..fcea9b3a7fe8
--- /dev/null
+++ b/adaptors/hubtel.md
@@ -0,0 +1,45 @@
+---
+title: Hubtel Adaptor
+---
+
+## About Hubtel
+
+[Hubtel](https://www.hubtel.com/) is a payment, messaging, and e-commerce platform that enables businesses to send SMS messages, process mobile money transactions, and manage digital commerce operations. Hubtel supports API integrations to facilitate seamless business automation.
+
+## Integration Options
+
+Hubtel supports two primary integration options with OpenFn:
+
+**1. Rest API:** Hubtel offers a REST API that enables external applications to interact with its services. This option is ideal for applications requiring scheduled or bulk synchronization with Hubtel. Refer to the Hubtel REST API [documentation](https://api.hubtel.com/) for detailed guidelines on endpoints and payload formats.
+
+**2. Webhook:** Webhook or Data Forwarding to push data from Hubtel to external systems ([see docs](https://developers.hubtel.com/docs/webhooks)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+1. See [Hubtel docs](https://developers.hubtel.com/docs/authentication) for the latest on supported authentication methods.
+2. When integrating with Hubtel via OpenFn, authentication via **API Key** is supported.
+3. See this adaptor's [Configuration docs](/adaptors/packages/hubtel-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "baseUrl": "https://example.com",
+ "clientId": "abc123def",
+ "clientSecret": "ghi456jkl"
+}
+```
+
+### Helpful Links
+
+1. [Hubtel Developer Portal](https://developers.hubtel.com/)
+2. [API Reference](https://developers.hubtel.com/docs/)
+3. [Webhook Setup Guide](https://developers.hubtel.com/docs/webhooks)
+
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
+
diff --git a/adaptors/intro.mdx b/adaptors/intro.mdx
index ef2a86add724..4dfe9bfdc9ac 100644
--- a/adaptors/intro.mdx
+++ b/adaptors/intro.mdx
@@ -44,6 +44,12 @@ adaptors is as follows:
In short, _most_ adaptors follow the naming convention `@openfn/language-xyz`.
+## Learn More About Adaptors
+
+Explore this YouTube playlist to gain a deeper understanding of OpenFn adaptors.
+
+
+
## Adaptors vs. Workflows
Adaptors are reusable components that make connecting with a specific app
@@ -53,7 +59,7 @@ always leverage Adaptors to better understand how to work with specific APIs, to
access Adaptor helper functions (or "shortcuts") to more quickly build
Workflows, and to handle authenticatio. See the below diagram.
-
+
## Where to find adaptors
@@ -148,7 +154,7 @@ adaptors, and click on one.
Most of our adaptors are also available on
[npmjs.com](https://www.npmjs.com/search?q=%40openfn).
-
+
## Don't see an adaptor for your app?
diff --git a/adaptors/intuit.md b/adaptors/intuit.md
new file mode 100644
index 000000000000..09d4890261e9
--- /dev/null
+++ b/adaptors/intuit.md
@@ -0,0 +1,35 @@
+---
+title: Intuit Adaptor (QuickBooks)
+---
+
+## About Intuit (QuickBooks)
+
+[Intuit](https://www.intuit.com/) provides financial, accounting, and tax management solutions for businesses and individuals, including **QuickBooks**, **TurboTax**, and **Mint**.
+
+## Integration Options
+
+Intuit supports 2 primary integration options:
+
+1. **Rest API**: Intuit has an available [REST API](https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/account#the-account-object) that enables external services like OpenFn to pull data from Intuit, or push data from external apps to Intuit. This option is suited for scheduled, bulk syncs or workflows that must update data in Intuit with external information. See [functions](/adaptors/packages/intuit-docs) for more on how to use this adaptor to work with the API.
+
+2. **Webhook**: Intuit also has a [Webhook or Data Forwarding](https://developer.intuit.com/app/developer/qbo/docs/api/webhooks) to push data from Intuit to external systems. This option is suited for real-time, event-based data integration. Check out the Intuit [developer documentation](https://developer.intuit.com/app/developer/qbo/docs/api/webhooks) to learn how to set up a webhook to push data to OpenFn.
+
+## Authentication
+
+See [Intuit docs](https://developer.intuit.com/app/developer/qbo/docs/develop/authentication-and-authorization) for the latest on supported authentication methods.
+
+When integrating with Intuit via OpenFn, there is one primary authentication method supported: **Access Token**. See this adaptor's [Configuration docs](/adaptors/packages/intuit-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "access_token": "your-qpi-token-JSUzI1NiIsIng1dCI6IjlGW",
+ "baseUrl": "https://sandbox-quickbooks.api.intuit.com"
+}
+```
+
+### Helpful Links
+
+1. [API Documentation](https://developer.intuit.com/)
+2. [Webhook Configuration Documentation](https://developer.intuit.com/app/developer/qbo/docs/api/webhooks)
diff --git a/adaptors/kobotoolbox.md b/adaptors/kobotoolbox.md
index 8a1ee88d6fa2..ccbfe51b2994 100644
--- a/adaptors/kobotoolbox.md
+++ b/adaptors/kobotoolbox.md
@@ -113,43 +113,48 @@ With this OpenFn job snippet we fetch submission data from a list of surveys,
indicated by their IDs.
```js
+// set the cursor to use for this run
+cursor($.lastEnd || $.manualCursor || '2020-11-20T14:32:43.325+01:00');
+
+// set the cursor for the next run
+cursor('now', {
+ key: 'lastEnd',
+ format: c => dateFns(c, 'YYYY-MM-DD:HH:mm:ss'),
+});
+
fn(state => {
- console.log('Current cursor value:', state.lastEnd);
- // Set a manual cursor if you'd like to only fetch data after this date.
- const manualCursor = '2020-11-20T14:32:43.325+01:00';
- state.data = {
- surveys: [
- //** Specify new forms to fetch here **//
- {
- id: 'aVdh90L9979L945lb02',
- name: 'Initial Data Collection',
- },
- {
- id: 'bkgIF96fK7v9n7Hfj2',
- name: 'Follow-up',
- },
- ].map(survey => ({
- formId: survey.id,
- name: survey.name,
- url: `https://kf.kobotoolbox.org/api/v2/assets/${survey.id}/data/?format=json`,
- query: `&query={"end":{"$gte":"${state.lastEnd || manualCursor}"}}`,
- })),
- };
+ console.log('Current cursor value:', state.cursor);
+
+ state.surveys = [
+ //** Specify new forms to fetch here **//
+ {
+ formId: 'aVdh90L9979L945lb02',
+ name: 'Initial Data Collection',
+ },
+ {
+ formId: 'bkgIF96fK7v9n7Hfj2',
+ name: 'Follow-up',
+ },
+ ];
return state;
});
-each(dataPath('surveys[*]'), state => {
- const { url, query, formId, name } = state.data;
- return get(`${url}${query}`, {}, state => {
- state.data.submissions = state.data.results.map((submission, i) => {
- return {
- i,
- // Here we append the names defined above to the Kobo form submission data
- formName: name,
- };
- });
- });
-});
+each(
+ $.surveys,
+ getSubmissions($.data.formId, {
+ query: { end: { $gte: `${$.cursor}` } },
+ }).then(state => {
+ // Lookup each form's id from the previous state
+ const { name, formId } = state.references.at(-1);
+
+ state.submissions[formId] = {
+ name,
+ submissions: state.data,
+ };
+
+ return state;
+ })
+);
```
Check out some of our
diff --git a/adaptors/library/jobs/DHIS2-DataValues-API.js b/adaptors/library/jobs/DHIS2-DataValues-API.js
index b305cd977439..ef582f0a7c03 100644
--- a/adaptors/library/jobs/DHIS2-DataValues-API.js
+++ b/adaptors/library/jobs/DHIS2-DataValues-API.js
@@ -1,20 +1,25 @@
// ----
// Add data to data value sets in DHIS2 using a generic JSON message, submitted
-// by Taylor Downs @ OpenFn.
+// by Taylor Downs @ OpenFn. Co-authored by @mtuchi
// ---
-dataValueSet(
- fields(
- field('dataSet', 'pBOMPrpg1QX'),
- field('orgUnit', 'DiszpKrYNg8'),
- field('period', '201401'),
- field('completeData', dataValue('form.date')),
- field('dataValues', function (state) {
- return [
- dataElement('qrur9Dvnyt5', dataValue('form.prop_a')(state)),
- dataElement('oZg33kd9taw', dataValue('form.prop_b')(state)),
- dataElement('msodh3rEMJa', dataValue('form.prop_c')(state)),
- ];
- })
- )
-);
+create('dataValueSets', {
+ dataSet: 'pBOMPrpg1QX',
+ completeDate: $.form.date,
+ period: '201401',
+ orgUnit: 'DiszpKrYNg8',
+ dataValues: [
+ {
+ dataElement: 'f7n9E0hX8qk',
+ value: $.form.prop_a,
+ },
+ {
+ dataElement: 'Ix2HsbDMLea',
+ value: $.form.prop_b,
+ },
+ {
+ dataElement: 'eY5ehpbEsB7',
+ value: $.form.prop_c,
+ },
+ ],
+});
diff --git a/adaptors/library/jobs/DHIS2-Events-API.js b/adaptors/library/jobs/DHIS2-Events-API.js
index e1b02e6b8373..ff5175fdddcf 100644
--- a/adaptors/library/jobs/DHIS2-Events-API.js
+++ b/adaptors/library/jobs/DHIS2-Events-API.js
@@ -1,25 +1,30 @@
// ----
// Create new events in DHIS2 using a generic JSON message, submitted by
-// Taylor Downs @ OpenFn for demonstration porpoises.
+// Taylor Downs @ OpenFn, Co-authored by @mtuchi
// ---
-event(
- fields(
- field('program', 'eBAyeGv0exc'),
- field('orgUnit', 'DiszpKrYNg8'),
- field('eventDate', dataValue('meta.date')),
- field('status', 'COMPLETED'),
- field('storedBy', 'admin'),
- field('coordinate', {
- latitude: '59.8',
- longitude: '10.9',
- }),
- field('dataValues', function (state) {
- return [
- dataElement('qrur9Dvnyt5', dataValue('form.prop_a')(state)),
- dataElement('oZg33kd9taw', dataValue('form.prop_b')(state)),
- dataElement('msodh3rEMJa', dataValue('form.prop_c')(state)),
- ];
- })
- )
-);
+create('events', {
+ program: 'eBAyeGv0exc',
+ orgUnit: 'DiszpKrYNg8',
+ occurredAt: $.meta.date,
+ status: 'COMPLETED',
+ storedBy: 'admin',
+ geometry: {
+ type: 'POINT',
+ coordinates: [59.8, 10.9],
+ },
+ dataValues: [
+ {
+ dataElement: 'qrur9Dvnyt5',
+ value: $.form.prop_a,
+ },
+ {
+ dataElement: 'oZg33kd9taw',
+ value: $.form.prop_b,
+ },
+ {
+ dataElement: 'msodh3rEMJa',
+ value: $.form.prop_c,
+ },
+ ],
+});
diff --git a/adaptors/library/jobs/commcare-create-user.js b/adaptors/library/jobs/commcare-create-user.js
new file mode 100644
index 000000000000..1112e6e119ad
--- /dev/null
+++ b/adaptors/library/jobs/commcare-create-user.js
@@ -0,0 +1,18 @@
+// Create a user and submit their case to commcare
+
+post('user', {
+ username: 'janedoe42',
+ password: 'qwer12345',
+ first_name: 'Jane',
+ last_name: 'Doe',
+ default_phone_number: '+50253311399',
+ email: 'jdoe@example.org',
+});
+
+submitXls([{ name: $.data.username, feeling_sick: 'No' }], {
+ case_type: 'pregnancy',
+ search_field: 'case_id',
+ search_column: 'case_id',
+ name_column: 'name',
+ create_new_cases: 'on',
+});
diff --git a/adaptors/library/jobs/generate-pdf.js b/adaptors/library/jobs/generate-pdf.js
new file mode 100644
index 000000000000..6205e25c9389
--- /dev/null
+++ b/adaptors/library/jobs/generate-pdf.js
@@ -0,0 +1,26 @@
+// Generate a PDF document from a HTML string
+
+fn(state => {
+ const { data } = state;
+
+ state.pdfHTMLContent = `
+
+
Sales Report
+
Date: ${data.date}
+
Total Sales: $${data.totalSales}
+
+ `;
+
+ return state;
+});
+
+generatePDF($.pdfHTMLContent, {
+ sandbox: true,
+ filename: 'trials.pdf',
+});
+
+fn(state => {
+ const { data } = state;
+ console.log(`Download PDF in 48 hours from ${data.url}`);
+ return { ...state, pdfData: data };
+});
diff --git a/adaptors/library/jobs/kobotoolbox-get-submissions.js b/adaptors/library/jobs/kobotoolbox-get-submissions.js
new file mode 100644
index 000000000000..777d609699a6
--- /dev/null
+++ b/adaptors/library/jobs/kobotoolbox-get-submissions.js
@@ -0,0 +1,12 @@
+//Get all Kobo forms/projects, and then get form submissions
+//that were submitted after a specific time
+cursor($.cursor, { defaultValue: 'today' });
+
+getForms();
+
+each(
+ $.data,
+ getSubmissions($.data.uid, {
+ query: `{"_submission_time":{"$gte":"${$.cursor}"}}`,
+ })
+);
diff --git a/adaptors/library/jobs/kobotoolbox-get-suspected-patients.js b/adaptors/library/jobs/kobotoolbox-get-suspected-patients.js
new file mode 100644
index 000000000000..489168832751
--- /dev/null
+++ b/adaptors/library/jobs/kobotoolbox-get-suspected-patients.js
@@ -0,0 +1,13 @@
+// Get all submissions of a specific form/asset and get the percentage of patients suspected of having covid-19
+getSubmissions('aDReHdA7UuNBYsiCXQBr43');
+
+fn(state => {
+ const results = state.data?.results;
+
+ const suspectedPatients = results.filter(
+ item => item['Please_select_sample_type'] === 'suspected_covid_19'
+ );
+ const suspectedCovidPatientsPercentage =
+ (suspectedPatients.length / results.length) * 100;
+ return { ...state, suspectedCovidPatientsPercentage };
+});
diff --git a/adaptors/library/jobs/opencrvs-create-birth-notification.js b/adaptors/library/jobs/opencrvs-create-birth-notification.js
new file mode 100644
index 000000000000..45773e45d600
--- /dev/null
+++ b/adaptors/library/jobs/opencrvs-create-birth-notification.js
@@ -0,0 +1,566 @@
+// Create a birth notification in OpenCRVS
+
+fn(state => {
+ const motherId = util.uuid();
+ const comparisonId = util.uuid();
+ const childId = util.uuid();
+ const encounterId = util.uuid();
+ const fatherId = util.uuid();
+ const informantId = util.uuid();
+
+ state.birthNotificationEntryData = [
+ {
+ fullUrl: `urn:uuid:${comparisonId}`,
+ resource: {
+ identifier: {
+ system: 'urn:ietf:rfc:3986',
+ value: '8f793c5a-3d53-4c9b-898b-1c04759716c6',
+ },
+ resourceType: 'Composition',
+ status: 'final',
+ type: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/doc-types',
+ code: 'birth-notification',
+ },
+ ],
+ text: 'Birth Notification',
+ },
+ class: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/specs/classes',
+ code: 'crvs-document',
+ },
+ ],
+ text: 'CRVS Document',
+ },
+ subject: {
+ reference: `urn:uuid:${childId}`,
+ },
+ date: '2022-08-14T14:43:47.000Z',
+ author: [],
+ title: 'Birth Notification',
+ section: [
+ {
+ title: 'Child details',
+ code: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/specs/sections',
+ code: 'child-details',
+ },
+ ],
+ text: 'Child details',
+ },
+ entry: [
+ {
+ reference: `urn:uuid:${childId}`,
+ },
+ ],
+ },
+ {
+ title: 'Birth encounter',
+ code: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/specs/sections',
+ code: 'birth-encounter',
+ },
+ ],
+ text: 'Birth encounter',
+ },
+ entry: [
+ {
+ reference: `urn:uuid:${encounterId}`,
+ },
+ ],
+ },
+ {
+ title: "Mother's details",
+ code: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/specs/sections',
+ code: 'mother-details',
+ },
+ ],
+ text: "Mother's details",
+ },
+ entry: [
+ {
+ reference: `urn:uuid:${motherId}`,
+ },
+ ],
+ },
+ {
+ title: "Informant's details",
+ code: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/specs/sections',
+ code: 'informant-details',
+ },
+ ],
+ text: "Informant's details",
+ },
+ entry: [
+ {
+ reference: `urn:uuid:${informantId}`,
+ },
+ ],
+ },
+ {
+ title: "Father's details",
+ code: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/doc-sections',
+ code: 'father-details',
+ },
+ ],
+ text: "Father's details",
+ },
+ entry: [
+ {
+ reference: `urn:uuid:${fatherId}`,
+ },
+ ],
+ },
+ ],
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${util.uuid()}`,
+ resource: {
+ resourceType: 'Task',
+ status: 'draft',
+ intent: 'unknown',
+ identifier: [],
+ code: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/specs/types',
+ code: 'BIRTH',
+ },
+ ],
+ },
+ focus: {
+ reference: `urn:uuid:${comparisonId}`,
+ },
+ extension: [
+ {
+ url: 'http://opencrvs.org/specs/extension/contact-person',
+ valueString: 'MOTHER',
+ },
+ {
+ url: 'http://opencrvs.org/specs/extension/contact-person-phone-number',
+ valueString: '+260759205190',
+ },
+ {
+ url: 'http://opencrvs.org/specs/extension/contact-person-email',
+ valueString: 'axon@gmail.com',
+ },
+ {
+ url: 'http://opencrvs.org/specs/extension/timeLoggedMS',
+ valueInteger: 0,
+ },
+ {
+ url: 'http://opencrvs.org/specs/extension/in-complete-fields',
+ valueString: 'N/A',
+ },
+ {
+ url: 'http://opencrvs.org/specs/extension/regLastOffice',
+ valueReference: {
+ reference: 'Location/178e21a9-60ad-4283-bd49-f576f61a5648',
+ },
+ },
+ ],
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${childId}`,
+ resource: {
+ resourceType: 'Patient',
+ active: true,
+ name: [
+ {
+ use: 'en',
+ family: 'Tatke',
+ given: ['Harney'],
+ },
+ ],
+ gender: 'male',
+ birthDate: '2022-06-29',
+ deceasedBoolean: false,
+ multipleBirthBoolean: false,
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${motherId}`,
+ resource: {
+ resourceType: 'Patient',
+ active: true,
+ identifier: [
+ {
+ use: 'official',
+ type: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/specs/identifier-type',
+ code: 'NATIONAL_ID',
+ },
+ ],
+ },
+ value: '3624667568',
+ },
+ ],
+ name: [
+ {
+ use: 'en',
+ family: 'Ratke',
+ given: ['Mom'],
+ },
+ ],
+ gender: 'female',
+ telecom: [
+ {
+ use: 'mobile',
+ system: 'phone',
+ value: '+260759205190',
+ },
+ ],
+ birthDate: '2002-06-29',
+ deceasedBoolean: false,
+ multipleBirthInteger: 2,
+ maritalStatus: {
+ coding: [
+ {
+ system: 'http://hl7.org/fhir/StructureDefinition/marital-status',
+ code: 'M',
+ },
+ ],
+ text: 'MARRIED',
+ },
+ address: [
+ {
+ type: 'PRIMARY_ADDRESS',
+ line: [
+ '12',
+ 'Usual Street',
+ 'Usual Residental Area',
+ '',
+ '',
+ 'URBAN',
+ ],
+ city: 'Meghanland',
+ district: 'c51ea274-8ffa-4b0b-b3d3-12991e9f0630',
+ state: 'c5d7a275-3638-41a3-bd53-145bd9410fd6',
+ postalCode: '52275',
+ country: 'FAR',
+ },
+ ],
+ extension: [
+ {
+ url: 'http://opencrvs.org/specs/extension/patient-occupation',
+ valueString: 'Housewife',
+ },
+ {
+ url: 'http://hl7.org/fhir/StructureDefinition/patient-nationality',
+ extension: [
+ {
+ url: 'code',
+ valueCodeableConcept: {
+ coding: [
+ {
+ system: 'urn:iso:std:iso:3166',
+ code: 'FAR',
+ },
+ ],
+ },
+ },
+ {
+ url: 'period',
+ valuePeriod: {
+ start: '',
+ end: '',
+ },
+ },
+ ],
+ },
+ {
+ url: 'http://opencrvs.org/specs/extension/educational-attainment',
+ valueString: 'POST_SECONDARY_ISCED_4',
+ },
+ ],
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${informantId}`,
+ resource: {
+ resourceType: 'RelatedPerson',
+ relationship: {
+ coding: [
+ {
+ system:
+ 'http://hl7.org/fhir/ValueSet/relatedperson-relationshiptype',
+ code: 'MOTHER',
+ },
+ ],
+ },
+ patient: {
+ reference: `urn:uuid:${motherId}`,
+ },
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${fatherId}`,
+ resource: {
+ resourceType: 'Patient',
+ active: true,
+ identifier: [
+ {
+ use: 'official',
+ type: {
+ coding: [
+ {
+ system: 'http://opencrvs.org/specs/identifier-type',
+ code: 'NATIONAL_ID',
+ },
+ ],
+ },
+ value: '6848901132',
+ },
+ ],
+ name: [
+ {
+ use: 'en',
+ family: 'Ratke',
+ given: ['Dad'],
+ },
+ ],
+ gender: 'male',
+ telecom: [
+ {
+ use: 'mobile',
+ system: 'phone',
+ value: '+260759205190',
+ },
+ ],
+ birthDate: '2002-06-29',
+ deceasedBoolean: false,
+ multipleBirthInteger: 2,
+ maritalStatus: {
+ coding: [
+ {
+ system: 'http://hl7.org/fhir/StructureDefinition/marital-status',
+ code: 'M',
+ },
+ ],
+ text: 'MARRIED',
+ },
+ address: [
+ {
+ type: 'PRIMARY_ADDRESS',
+ line: [
+ '12',
+ 'Usual Street',
+ 'Usual Residental Area',
+ '',
+ '',
+ 'URBAN',
+ ],
+ city: 'Madgeland',
+ district: 'c51ea274-8ffa-4b0b-b3d3-12991e9f0630',
+ state: 'c5d7a275-3638-41a3-bd53-145bd9410fd6',
+ postalCode: '52275',
+ country: 'FAR',
+ },
+ ],
+ extension: [
+ {
+ url: 'http://opencrvs.org/specs/extension/patient-occupation',
+ valueString: 'Businessman',
+ },
+ {
+ url: 'http://hl7.org/fhir/StructureDefinition/patient-nationality',
+ extension: [
+ {
+ url: 'code',
+ valueCodeableConcept: {
+ coding: [
+ {
+ system: 'urn:iso:std:iso:3166',
+ code: 'FAR',
+ },
+ ],
+ },
+ },
+ {
+ url: 'period',
+ valuePeriod: {
+ start: '',
+ end: '',
+ },
+ },
+ ],
+ },
+ {
+ url: 'http://opencrvs.org/specs/extension/educational-attainment',
+ valueString: 'POST_SECONDARY_ISCED_4',
+ },
+ ],
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${encounterId}`,
+ resource: {
+ resourceType: 'Encounter',
+ status: 'finished',
+ location: [
+ {
+ location: {
+ reference: 'Location/704b9706-d729-4834-8656-05b562065deb',
+ },
+ },
+ ],
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${util.uuid()}`,
+ resource: {
+ resourceType: 'Observation',
+ status: 'final',
+ context: {
+ reference: `urn:uuid:${encounterId}`,
+ },
+ category: [
+ {
+ coding: [
+ {
+ system: 'http://hl7.org/fhir/observation-category',
+ code: 'procedure',
+ display: 'Procedure',
+ },
+ ],
+ },
+ ],
+ code: {
+ coding: [
+ {
+ system: 'http://loinc.org',
+ code: '57722-1',
+ display: 'Birth plurality of Pregnancy',
+ },
+ ],
+ },
+ valueQuantity: {
+ value: 'SINGLE',
+ },
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${util.uuid()}`,
+ resource: {
+ resourceType: 'Observation',
+ status: 'final',
+ context: {
+ reference: `urn:uuid:${encounterId}`,
+ },
+ category: [
+ {
+ coding: [
+ {
+ system: 'http://hl7.org/fhir/observation-category',
+ code: 'vital-signs',
+ display: 'Vital Signs',
+ },
+ ],
+ },
+ ],
+ code: {
+ coding: [
+ {
+ system: 'http://loinc.org',
+ code: '3141-9',
+ display: 'Body weight Measured',
+ },
+ ],
+ },
+ valueQuantity: {
+ value: 4,
+ unit: 'kg',
+ system: 'http://unitsofmeasure.org',
+ code: 'kg',
+ },
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${util.uuid()}`,
+ resource: {
+ resourceType: 'Observation',
+ status: 'final',
+ context: {
+ reference: `urn:uuid:${encounterId}`,
+ },
+ category: [
+ {
+ coding: [
+ {
+ system: 'http://hl7.org/fhir/observation-category',
+ code: 'procedure',
+ display: 'Procedure',
+ },
+ ],
+ },
+ ],
+ code: {
+ coding: [
+ {
+ system: 'http://loinc.org',
+ code: '73764-3',
+ display: 'Birth attendant title',
+ },
+ ],
+ },
+ valueString: 'PHYSICIAN',
+ },
+ },
+ {
+ fullUrl: `urn:uuid:${util.uuid()}`,
+ resource: {
+ resourceType: 'QuestionnaireResponse',
+ extension: [],
+ status: 'completed',
+ subject: {
+ reference: `urn:uuid:${encounterId}`,
+ },
+ item: [
+ {
+ text: 'birth.mother.mother-view-group.motherIdType',
+ linkId: '',
+ answer: [
+ {
+ valueString: 'NATIONAL_ID',
+ },
+ ],
+ },
+ {
+ text: 'birth.father.father-view-group.fatherIdType',
+ linkId: '',
+ answer: [
+ {
+ valueString: 'NATIONAL_ID',
+ },
+ ],
+ },
+ ],
+ },
+ },
+ ];
+
+ return state;
+});
+
+createBirthNotification($.birthNotificationEntryData);
diff --git a/adaptors/library/jobs/opencrvs-query-events.js b/adaptors/library/jobs/opencrvs-query-events.js
new file mode 100644
index 000000000000..31db943e6516
--- /dev/null
+++ b/adaptors/library/jobs/opencrvs-query-events.js
@@ -0,0 +1,20 @@
+// Get a birth record from OpenCRVS
+
+fn(state => {
+ const { data } = state;
+ state.birthQuery = {
+ event: 'birth',
+ registrationStatuses: ['REGISTERED'],
+ fatherFirstNames: data.fatherFirstNames,
+ motherFirstNames: data.motherFirstNames,
+ childGender: data.childGender,
+ dateOfRegistrationStart: data.dateOfRegistrationStart,
+ dateOfRegistrationEnd: data.dateOfRegistrationEnd,
+ declarationJurisdictionId: data.declarationJurisdictionId,
+ eventLocationId: data.eventLocationId,
+ };
+
+ return state;
+});
+
+queryEvents($.birthQuery, { count: 10, skip: 0 });
diff --git a/adaptors/library/jobs/satusehat-create-condition.js b/adaptors/library/jobs/satusehat-create-condition.js
new file mode 100644
index 000000000000..14481165e8a1
--- /dev/null
+++ b/adaptors/library/jobs/satusehat-create-condition.js
@@ -0,0 +1,91 @@
+// Map diagnosis objects from commcare into Condition resources
+
+fn(state => {
+ state.conditions = state.visit.diagnosis.map(diagnosis => {
+ const diagnosisName = diagnosis.fields.name;
+ const code = diagnosis.fields['icd_10_code'];
+
+ return {
+ resourceType: 'Condition',
+ clinicalStatus: {
+ coding: [
+ {
+ system: 'http://terminology.hl7.org/CodeSystem/condition-clinical',
+ code: 'active',
+ display: 'Active',
+ },
+ ],
+ },
+ verificationStatus: {
+ coding: [
+ {
+ system:
+ 'http://terminology.hl7.org/CodeSystem/condition-ver-status',
+ code: 'confirmed',
+ display: 'Confirmed',
+ },
+ ],
+ },
+ category: [
+ {
+ coding: [
+ {
+ system:
+ 'http://terminology.hl7.org/CodeSystem/condition-category',
+ code: 'encounter-diagnosis',
+ display: 'Encounter Diagnosis',
+ },
+ ],
+ },
+ ],
+ subject: {
+ reference: `Patient/${state.patient.ihs_number}`,
+ display: state.patient.full_name,
+ },
+ encounter: {
+ reference: `Encounter/${state.encounterId}`,
+ display:
+ state.patient.full_name +
+ "'s " +
+ state.visit.properties.visit_type +
+ ' on ' +
+ state.visit.properties.visit_date,
+ },
+ code: {
+ coding: [
+ {
+ system: 'http://hl7.org/fhir/sid/icd-10',
+ code: code,
+ display: diagnosisName,
+ },
+ ],
+ },
+ bodySite: [
+ {
+ coding: [
+ {
+ system: 'http://snomed.info/sct',
+ code: '74262004',
+ display: 'Oral cavity structure',
+ },
+ ],
+ },
+ ],
+ recorder: {
+ reference: state.encounter.resource.participant[0].individual.reference,
+ },
+ onsetDateTime: state.visit.visit_date,
+ recordedDate: state.visit.visit_date,
+ };
+ });
+ return state;
+});
+
+// Post our condition resources to Satusehat
+post('Condition', $.conditions);
+
+fn(state => {
+ // Store the newly created resources, created by the server, into state for the next step
+ state.conditions = state.data;
+ return state;
+});
diff --git a/adaptors/library/jobs/satusehat-create-encounter.js b/adaptors/library/jobs/satusehat-create-encounter.js
new file mode 100644
index 000000000000..c597a8176a37
--- /dev/null
+++ b/adaptors/library/jobs/satusehat-create-encounter.js
@@ -0,0 +1,102 @@
+// Create an encounter in Satusehat based on an incoming Commcare visit
+
+fn(state => {
+ state.encounterId = util.uuid();
+
+ state.encounter = {
+ id: state.encounterId,
+ resourceType: 'Encounter',
+ identifier: [
+ {
+ system: `http://sys-ids.kemkes.go.id/encounter/${state.visit.organizationId}`,
+ value: state.visit.case_id,
+ },
+ ],
+ status: 'arrived',
+ class: {
+ system: 'http://terminology.hl7.org/CodeSystem/v3-ActCode',
+ code: 'AMB',
+ display: 'ambulatory',
+ },
+ subject: {
+ reference: 'Patient/P00805884304',
+ display: 'Elizabeth Dior',
+ },
+ participant: [
+ {
+ type: [
+ {
+ coding: [
+ {
+ system:
+ 'http://terminology.hl7.org/CodeSystem/v3-ParticipationType',
+ code: 'ATND',
+ display: 'attender',
+ },
+ ],
+ },
+ ],
+ individual: {
+ reference: `Practitioner/N10000001`,
+ display: 'Voigt',
+ },
+ },
+ ],
+ period: {
+ start: state.visit.visit_date,
+ end: state.visit.visit_date,
+ },
+
+ location: [
+ {
+ location: {
+ reference: `Location/574d5ffd-2dc2-453b-a787-d1c63cc89ae4`,
+ display: `PLKB`,
+ },
+ extension: [
+ {
+ url: 'https://fhir.kemkes.go.id/r4/StructureDefinition/ServiceClass',
+ extension: [
+ {
+ url: 'value',
+ valueCodeableConcept: {
+ coding: [
+ {
+ system:
+ 'http://terminology.kemkes.go.id/CodeSystem/locationServiceClass-Outpatient',
+ code: 'reguler',
+ display: 'Kelas Reguler',
+ },
+ ],
+ },
+ },
+ ],
+ },
+ ],
+ },
+ ],
+ statusHistory: [
+ {
+ status: 'arrived',
+ period: {
+ start: state.visit.visit_date,
+ end: state.visit.visit_date,
+ },
+ },
+ ],
+ serviceProvider: {
+ reference: `Organization/${state.visit.organizationId}`,
+ },
+ };
+
+ return state;
+});
+
+// Post our encounter resources to Satusehat
+post('Encounter', $.encounter);
+
+fn(state => {
+ // Store the newly created resources, created by the server, into state for the next step
+ state.encounter = state.data;
+ return state;
+});
diff --git a/adaptors/library/jobs/satusehat-create-medication.js b/adaptors/library/jobs/satusehat-create-medication.js
new file mode 100644
index 000000000000..851ded15dac9
--- /dev/null
+++ b/adaptors/library/jobs/satusehat-create-medication.js
@@ -0,0 +1,71 @@
+// Create medication in Satusehat based on an incoming Commcare visit
+
+fn(state => {
+ // state.visit is a Commcare Visit record
+ const properties = state.visit.properties;
+ const medicineKeys = Object.keys(properties).filter(
+ key =>
+ key.startsWith('prescription_') &&
+ !key.includes('_amount') &&
+ !key.includes('_instruction') &&
+ !key.includes('_dose')
+ );
+
+ const medicationId = util.uuid();
+
+ // Map medication data for each relevant key
+ state.medication = medicineKeys.map(item => {
+ return {
+ resourceType: 'Medication',
+ id: state => medicationId,
+ meta: {
+ profile: [
+ 'https://fhir.kemkes.go.id/r4/StructureDefinition/Medication',
+ ],
+ },
+ identifier: [
+ {
+ system: `http://sys-ids.kemkes.go.id/medication/${state.visit.satusehatId}`,
+ use: 'usual',
+ value: item.fields['Nama'],
+ },
+ ],
+ code: {
+ coding: [
+ {
+ system: 'http://sys-ids.kemkes.go.id/kfa',
+ code: item.fields['kfa_codes'],
+ display: item.fields['Nama'],
+ },
+ ],
+ },
+ status: 'active',
+ extension: [
+ {
+ url: 'https://fhir.kemkes.go.id/r4/StructureDefinition/MedicationType',
+ valueCodeableConcept: {
+ coding: [
+ {
+ system:
+ 'http://terminology.kemkes.go.id/CodeSystem/medication-type',
+ code: 'NC',
+ display: 'Non-compound',
+ },
+ ],
+ },
+ },
+ ],
+ };
+ });
+
+ return state;
+});
+
+// Post our medication resources to Satusehat
+post('Medication', $.medication);
+
+fn(state => {
+ // Store the newly created resources, created by the server, into state for the next step
+ state.medication = state.data;
+ return state;
+});
diff --git a/adaptors/library/jobs/satusehat-create-observations.js b/adaptors/library/jobs/satusehat-create-observations.js
new file mode 100644
index 000000000000..384f83270a1c
--- /dev/null
+++ b/adaptors/library/jobs/satusehat-create-observations.js
@@ -0,0 +1,136 @@
+// Create observation in Satusehat based on an incoming Commcare visit
+
+function createObservationData(item, state, result) {
+ const patientReference = `Patient/${state.patient.ihs_number}`;
+ const encounterReference = `Encounter/${state.encounterId}`;
+ const encounterDisplay = `${state.patient.full_name}'s ${state.visit.properties.visit_type} on ${state.visit.properties.visit_date}`;
+ return {
+ resourceType: 'Observation',
+ status: 'final',
+ category: [
+ {
+ coding: item.category,
+ },
+ ],
+ code: {
+ coding: item.code,
+ },
+ subject: {
+ reference: patientReference,
+ },
+ performer: [
+ {
+ reference: `Practitioner/N10000001`,
+ display: 'Voigt',
+ },
+ ],
+ encounter: {
+ reference: encounterReference,
+ display: encounterDisplay,
+ },
+ effectiveDateTime: state.visit.visit_date,
+ ...result,
+ };
+}
+
+fn(state => {
+ // Filtering and mapping observations based on the visit.
+ state.observations = state.visit.observations.map(item => {
+ const { value_type: valueType, value } = item;
+ const dataObjects = [];
+
+ if (valueType === 'valueCodeableConcept') {
+ value.forEach(value => {
+ const result = {
+ [valueType]: {
+ coding: [JSON.parse(value.snomed_code)],
+ value: JSON.parse(value.value),
+ },
+ };
+ const data = createObservationData(
+ { ...item, id: value.id },
+ state,
+ result
+ );
+ dataObjects.push(data);
+ });
+ } else if (valueType === 'valueQuantity') {
+ const result = {
+ [valueType]: {
+ coding: [value.snomed_code],
+ value: value.value,
+ },
+ };
+ const data = createObservationData(
+ { ...item, id: value.id },
+ state,
+ result
+ );
+ dataObjects.push(data);
+ } else if (valueType === 'valueInteger') {
+ const result = {
+ [valueType]: {
+ coding: value.snomed_code,
+ value: value.value,
+ },
+ };
+ const data = createObservationData(
+ { ...item, id: value.id },
+ state,
+ result
+ );
+ dataObjects.push(data);
+ } else if (valueType === 'valueDateTime') {
+ const result = {
+ [valueType]: {
+ coding: [JSON.parse(value.snomed_code)],
+ value: new Date(JSON.parse(value.value)).toISOString(),
+ },
+ };
+ const data = createObservationData(
+ { ...item, id: value.id },
+ state,
+ result
+ );
+ dataObjects.push(data);
+ } else if (valueType === 'valueBoolean') {
+ const result = {
+ [valueType]: {
+ coding: [JSON.parse(value.snomed_code)],
+ value: JSON.parse(value.value) === 'never' ? false : true,
+ },
+ };
+ const data = createObservationData(
+ { ...item, id: value.id },
+ state,
+ result
+ );
+ dataObjects.push(data);
+ } else {
+ const result = {
+ [valueType]: {
+ coding: [value.snomed_code],
+ value: value.value,
+ },
+ };
+
+ if (result) {
+ const data = createObservationData(item, state, result);
+ dataObjects.push(data);
+ }
+ }
+
+ return dataObjects;
+ });
+
+ return state;
+});
+
+// Post our observation resources to Satusehat
+post('Observation', $.observations);
+
+// Store the newly created resources, created by the server, into state for the next step
+fn(state => {
+ state.observations = state.data;
+ return state;
+});
diff --git a/adaptors/library/staticExamples.json b/adaptors/library/staticExamples.json
index 02cd577b75e3..81447aed609e 100644
--- a/adaptors/library/staticExamples.json
+++ b/adaptors/library/staticExamples.json
@@ -4,6 +4,11 @@
"adaptor": "commcare",
"name": "Bulk upload obat lookup-table"
},
+ {
+ "expressionPath": "jobs/commcare-create-user",
+ "adaptor": "commcare",
+ "name": "Create user"
+ },
{
"expressionPath": "jobs/getKoboData",
"adaptor": "http",
@@ -30,6 +35,16 @@
"adaptor": "openhim",
"name": "Create encounter in OpenHIM"
},
+ {
+ "expressionPath": "jobs/opencrvs-create-birth-notification",
+ "adaptor": "opencrvs",
+ "name": "Create birth notification in OpenCRVS"
+ },
+ {
+ "expressionPath": "jobs/opencrvs-query-events",
+ "adaptor": "opencrvs",
+ "name": "Query an event in OpenCRVS"
+ },
{
"expressionPath": "jobs/complex-http-request-chains",
"adaptor": "http",
@@ -56,15 +71,35 @@
"adaptor": "salesforce",
"name": "Create SMS linked to contact"
},
+ {
+ "expressionPath": "jobs/satusehat-create-condition",
+ "adaptor": "satusehat",
+ "name": "Create a condition Record"
+ },
+ {
+ "expressionPath": "jobs/satusehat-create-encounter",
+ "adaptor": "satusehat",
+ "name": "Create an Encounter Record"
+ },
+ {
+ "expressionPath": "jobs/satusehat-create-observations",
+ "adaptor": "satusehat",
+ "name": "Create an Observation"
+ },
+ {
+ "expressionPath": "jobs/satusehat-create-medication",
+ "adaptor": "satusehat",
+ "name": "Create a Medication Record"
+ },
{
"expressionPath": "jobs/DHIS2-DataValues-API",
"adaptor": "dhis2",
- "name": "Add data values"
+ "name": "Create data values"
},
{
"expressionPath": "jobs/DHIS2-Events-API",
"adaptor": "dhis2",
- "name": "Add events"
+ "name": "Create new events"
},
{
"expressionPath": "jobs/ODK-Create-Many-Records-Moving-In-And-Out-Of-Repeat-Blocks",
@@ -105,5 +140,20 @@
"expressionPath": "jobs/async-findValue",
"adaptor": "postgresql",
"name": "Using findValue with an array of data"
+ },
+ {
+ "expressionPath": "jobs/generate-pdf",
+ "adaptor": "pdfshift",
+ "name": "Generate a PDF from a HTML string"
+ },
+ {
+ "expressionPath": "jobs/kobotoolbox-get-suspected-patients",
+ "adaptor": "kobotoolbox",
+ "name": "Create an asset in KoboToolbox"
+ },
+ {
+ "expressionPath": "jobs/kobotoolbox-get-submissions",
+ "adaptor": "kobotoolbox",
+ "name": "Extract submissions from KoboToolbox"
}
]
\ No newline at end of file
diff --git a/adaptors/mailchimp.md b/adaptors/mailchimp.md
new file mode 100644
index 000000000000..221d9cbde410
--- /dev/null
+++ b/adaptors/mailchimp.md
@@ -0,0 +1,41 @@
+---
+title: MailChimp Adaptor
+---
+
+## About Mailchimp
+
+[Mailchimp](https://mailchimp.com/) is a marketing automation platform that allows businesses to design, send, and manage email campaigns. It also provides tools for audience management, analytics, and integrations with other platforms to support marketing efforts.
+
+## Integration Options
+
+Mailchimp supports two primary integration options:
+
+**1. Rest API:** Mailchimp offers a REST API that enables external applications to interact with its services. This option is ideal for applications requiring scheduled or bulk synchronization with Mailchimp. Refer to the Mailchimp REST API [documentation](https://mailchimp.com/developer/marketing/api/) for detailed guidelines on endpoints and payload formats.
+
+**2. Webhook:** Webhook or Data Forwarding to push data from MailChimp to external systems ([see docs](https://mailchimp.com/developer/transactional/docs/webhooks/)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+When integrating with Mailchimp via OpenFn, authentication via **API Key** is supported ([see MC docs](https://mailchimp.com/developer/marketing/docs/fundamentals/#connecting-to-the-api). See this adaptor's [Configuration docs](/adaptors/packages/mailchimp-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "server": "us11",
+ "apiKey": "0eb22c7b4a1c5bcd789379bf8a92902d-us13"
+}
+```
+
+### Helpful Links
+1. [Developer Portal](https://mailchimp.com/developer/)
+2. [API Reference](https://mailchimp.com/developer/marketing/)
+3. [Webhook Setup Guide](https://mailchimp.com/developer/marketing/guides/set-up-webhooks/)
+
+### Implementation Examples
+
+1. GIFE Project - Mailchimp -> Salesforce sync: [https://github.com/OpenFn/gife](https://github.com/OpenFn/gife)
+
+
+
+
diff --git a/adaptors/mailgun.md b/adaptors/mailgun.md
new file mode 100644
index 000000000000..6195a400d204
--- /dev/null
+++ b/adaptors/mailgun.md
@@ -0,0 +1,38 @@
+---
+title: Mailgun Adaptor
+---
+
+## About Mailgun
+
+[Mailgun](https://www.mailgun.com/) is an email delivery service designed for developers and provides APIs for sending, receiving, and tracking emails.
+
+## Integration Options
+
+Mailgun supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** Mailgun has a REST API that enables external services like OpenFn to pull data from Mailgun, or push data from external apps to Mailgun. This option is suited for scheduled, bulk syncs or workflows that must update data in Mailgun with external information. See [functions](/adaptors/packages/mailgun-docs) for more on how to use this adaptor to work with the API.
+
+2. **Webhook:** Webhook or Data Forwarding to push data from Mailgun to external systems (see [docs](https://documentation.mailgun.com/docs/mailgun/user-manual/tracking-messages/#webhooks)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+1. See [Mailgun docs](https://developers.google.com/gmail/api/auth/scopes) for the latest on supported authentication methods.
+2. When integrating with Mailgun via OpenFn, there is one primary authentication method that is supported: **API Key**. See this adaptor's [Configuration docs](/adaptors/packages/mailgun-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "domain": "sandbox-123.mailgun.org",
+ "apiKey": "otherThiNGfSECret"
+}
+```
+
+### Helpful Links
+
+1. [Mailgun API Documentation](https://documentation.mailgun.com/)
+2. [Event Webhooks](https://documentation.mailgun.com/en/latest/api-events.html#event-webhooks)
+3. [Support and Community](https://help.mailgun.com/)
+
+
+
diff --git a/adaptors/mongodb.md b/adaptors/mongodb.md
new file mode 100644
index 000000000000..5c6f560b0428
--- /dev/null
+++ b/adaptors/mongodb.md
@@ -0,0 +1,36 @@
+---
+title: MongoDB Adaptor
+---
+
+## About MongoDB
+
+[MongoDB](https://www.mongodb.com/) is a NoSQL, document-oriented database that stores data in BSON (Binary JSON) format, enabling easy storage and retrieval of complex and hierarchical data structures
+
+## Integration Options
+
+The `mongodb` adaptor provides direct database connections for accessing data and executing SQL and standard database operations. See [functions](/adaptors/packages/mongodb-docs) for more on how to use this adaptor.
+
+
+## Authentication
+
+See the [MongoDB docs](https://www.mongodb.com/docs/) for the latest on supported authentication methods. When integrating with a MongoDB database via OpenFn, you authenticate via SSH using authorized database credentials. See this adaptor's [configuration docs](/adaptors/packages/mongodb-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "clusterHostname": "yourCluster-xxxyzzz.mongodb.net",
+ "username": "admin",
+ "password": "@secret(!)Pass"
+}
+```
+
+### Helpful Links
+
+1. [MongoDB documentation](https://www.mongodb.com/docs/)
+
+
+
+
+
+
diff --git a/adaptors/mssql.md b/adaptors/mssql.md
new file mode 100644
index 000000000000..50b809a17ac6
--- /dev/null
+++ b/adaptors/mssql.md
@@ -0,0 +1,37 @@
+---
+title: MSSQL Adaptor
+---
+
+## About MSSQL
+
+[Microsoft SQL Server](https://learn.microsoft.com/en-us/sql/?view=sql-server-ver16) (MSSQL) is a relational database management system (RDBMS) developed by Microsoft. It supports a wide variety of applications, including data warehousing, transaction processing, and business intelligence. It can be accessed and manipulated using SQL to extract or load data.
+
+## Integration Options
+
+The `mssql` adaptor provides direct database connections for accessing data and executing SQL and standard database operations. See [functions](/adaptors/packages/mssql-docs) for more on how to use this adaptor.
+
+
+## Authentication
+
+See [MSSQL docs](https://learn.microsoft.com/en-us/sql/?view=sql-server-ver16) for the latest on supported authentication methods. When integrating with an MSSQL database via OpenFn, you authenticate via SSH using authorized database credentials. See this adaptor's [Configuration docs](/adaptors/packages/mssql-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "server": "something.database.windows.net",
+ "database": "demo-db",
+ "userName": "admin",
+ "password": "@super(!)Password"
+}
+```
+
+### Helpful Links
+
+1. [MSSQL documentation](https://learn.microsoft.com/en-us/sql/?view=sql-server-ver16)
+
+
+
+
+
+
diff --git a/adaptors/mysql.md b/adaptors/mysql.md
new file mode 100644
index 000000000000..bc64964a31d9
--- /dev/null
+++ b/adaptors/mysql.md
@@ -0,0 +1,37 @@
+---
+title: MySQL Adaptor
+---
+
+## About MySQL
+
+MySQL is a free and open-source relational database management system. It can be accessed and manipulated using SQL to extract or load data.
+
+## Integration Options
+
+The `mysql` adaptor provides direct database connections for accessing data and executing SQL and standard database operations. See [functions](/adaptors/packages/mysql-docs) for more on how to use this adaptor.
+
+
+## Authentication
+
+See the [MySQL docs](https://dev.mysql.com/doc/) for the latest on supported authentication methods. When integrating with a MySQL database via OpenFn, you authenticate via SSH using authorized database credentials. See this adaptor's [configuration docs](/adaptors/packages/mysql-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "host": "some-host-url.compute-1.amazonaws.com",
+ "database": "demo-db",
+ "user": "admin-demo",
+ "password": "@super(!)Secretpass"
+}
+```
+
+### Helpful Links
+
+1. [MySQL documentation](https://dev.mysql.com/doc/)
+
+
+
+
+
+
diff --git a/adaptors/ocl.md b/adaptors/ocl.md
new file mode 100644
index 000000000000..416a233b7369
--- /dev/null
+++ b/adaptors/ocl.md
@@ -0,0 +1,38 @@
+---
+title: OCL Adaptor
+---
+
+## About OCL
+
+[OCL (Open Concept Lab)](https://openconceptlab.org/) is an open-source platform that provides a collaborative environment for creating, managing, and sharing standardized healthcare terminologies, dictionaries, and value sets.
+
+Relationships between concepts are defined in OCL as `mappings`. The API supports searching and editing concepts and mappings, building `sources`, and logically grouping concepts and mappings into `collections`. [See OCL docs](https://docs.openconceptlab.org/en/latest/oclapi/overview.html#overview) to learn more, access the [Swagger API](https://api.openconceptlab.org/swagger/), and to learn more about the cloud instance available at [https://openconceptlab.org](https://openconceptlab.org).
+
+## Integration Options
+
+**1. Rest API:** OCL offers a REST API that allows systems to interact with its concept dictionaries and value sets. With the API, you can:
+This option is suited for scheduled synchronization or workflows requiring regular updates to or from OCL. Refer to the [OCL REST API documentation](https://docs.openconceptlab.org/en/latest/oclapi/overview.html) for endpoint details and usage examples.
+
+**2. Bulk Export and Import**: OCL supports exporting and importing concept dictionaries and value sets in various formats (e.g., JSON, CSV). This allows for manual or automated bulk data integration. Use this option for one-time data transfer or systems without direct API connectivity.
+
+## Authentication
+When integrating with OCL via OpenFn, you must provide a username and password to generate an authorization token ([see OCL docs](https://docs.openconceptlab.org/en/latest/oclapi/overview.html#authentication-and-authorization)). See this adaptor's [Configuration docs](/adaptors/packages/ocl-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "hostUrl": "https://api.openconceptlab.org/",
+ "username": "usernmame",
+ "password": "supersecretpassword"
+}
+```
+## Helpful Links
+1. [OCL Developer Guide](https://docs.openconceptlab.org/)
+2. [OCL API Reference](https://docs.openconceptlab.org/en/latest/oclapi/apireference/index.html)
+3. [OCL Community Support](https://openconceptlab.org/category/community/)
+4. [OCL GitHub Repository](https://github.com/OpenConceptLab)
+
+
+## Implementation Examples
+1. OpenFn Prototype for Médecins Sans Frontières (MSF) LIME Project - OpenMRS -> OCL -> DHIS2 sync: https://github.com/OpenFn/openfn-lime
diff --git a/adaptors/odoo.md b/adaptors/odoo.md
new file mode 100644
index 000000000000..290cf475d994
--- /dev/null
+++ b/adaptors/odoo.md
@@ -0,0 +1,46 @@
+---
+title: Odoo Adaptor
+---
+
+## About Odoo
+
+[Odoo](https://odoo.com/) is a marketing automation platform that allows businesses to design, send, and manage email campaigns. It also provides tools for audience management, analytics, and integrations with other platforms to support marketing efforts.
+
+## Integration Options
+
+Odoo supports two primary integration options with OpenFn:
+
+**1. Direct DB Connection:** This is what _this_ adaptor supports to directly connect to your Odoo database.
+
+**2. Rest API:** Odoo offers a REST API that enables external applications to interact with its services, but this is a *paid option/may require an additional plugin*. This option is ideal for applications requiring scheduled or bulk synchronization with Odoo. Refer to the Odoo REST API [documentation](https://www.odoo.com/documentation/16.0/developer/api/external_api.html) for detailed guidelines on endpoints and payload formats.
+
+**3. Webhook:** Webhook or Data Forwarding to push data from Odoo to external systems ([see docs](https://www.odoo.com/documentation/16.0/developer/reference/webhooks.html)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+1. See [Odoo docs](https://www.odoo.com/documentation/16.0/developer/misc/api/odoo.html) for the latest on supported authentication methods.
+2. When integrating with Odoo via OpenFn, authentication via **User Credentials** is supported.
+3. See this adaptor's [Configuration docs](/adaptors/packages/odoo-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "password": "@some(!)Str0ngp4ss0w0rd",
+ "username": "test@openfn.org",
+ "database": "devel",
+ "baseUrl": "https://dev.newlogic-demo.com"
+}
+```
+
+### Helpful Links
+1. [Odoo Developer Portal](https://www.odoo.com/documentation)
+2. [API Reference](https://www.odoo.com/documentation/16.0/developer/api/external_api.html)
+3. [Webhook Setup Guide](https://www.odoo.com/documentation/16.0/developer/reference/webhooks.html)
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
+
diff --git a/adaptors/opencrvs.md b/adaptors/opencrvs.md
new file mode 100644
index 000000000000..6a49d23bf846
--- /dev/null
+++ b/adaptors/opencrvs.md
@@ -0,0 +1,62 @@
+---
+title: OpenCRVS Adaptor
+---
+
+## About OpenCRVS
+
+[OpenCRVS (Open Civil Registration and Vital Statistics)](https://www.opencrvs.org/)
+is an open-source solution designed to collect various population data. The
+system is used for recording the details of all major life events, such as
+births and deaths. It provides the foundation for human rights, government
+service delivery, and the measurement of development goals.
+
+## Integration Options
+
+OpenCRVS supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** OpenCRVS has a REST API that enables external services like
+ OpenFn to pull data from OpenCRVS, or push data from external apps to
+ OpenCRVS. This option is suited for scheduled, bulk syncs or workflows that
+ must update data in OpenCRVS with external information. See
+ [functions](/adaptors/packages/opencrvs-docs) for more on how to use this
+ adaptor to work with the API.
+
+2. **Webhook:** Webhook or Data Forwarding to push data from OpenCRVS to
+ external systems (see
+ [docs](https://documentation.opencrvs.org/technology/interoperability/webhook-clients)).
+ This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+See [OpenCRVS docs](https://documentation.opencrvs.org/) for the latest on
+supported authentication methods. When integrating with OpenCRVS via OpenFn,
+there are 2 primary authentication methods supported.
+
+Please note that OpenCRVS take is different `clientId` and `clientSecret` for
+each clients; i.e events notifications and search. Ensure to switch credentials
+for the different client requests. See
+[OpenCRVS authenticate client docs](https://documentation.opencrvs.org/technology/interoperability/authenticate-a-client)
+
+1. Basic Authentication. See this adaptor's
+ [Configuration docs](/adaptors/packages/opencrvs-configuration-schema) for
+ more on the required authentication parameters.
+2. Access token (requires access token created after authenticating in OpenCRVS)
+
+See platform docs on
+[managing credentials](/documentation/manage-projects/manage-credentials) for
+how to configure a credential in OpenFn. If working locally or if using a Raw
+JSON credential type, then your configuration will look something like this:
+
+```json
+{
+ "domain": "your-doman", //e.g openfn.opencrvs.dev
+ "clientId": "your-client-id", // e.g 12345678
+ "clientSecret": "your-client-secret-key", // e.g abcd123456
+ "access_token": "your-access-token" // Don't add accessToken if you're using client secret and client id
+}
+```
+
+### Helpful Links
+
+1. [OpenCRVS Documentation](https://documentation.opencrvs.org/)
+2. [OpenCRVS API Reference](https://documentation.opencrvs.org/technology/interoperability)
diff --git a/adaptors/openhim.md b/adaptors/openhim.md
new file mode 100644
index 000000000000..2021091a4dc5
--- /dev/null
+++ b/adaptors/openhim.md
@@ -0,0 +1,41 @@
+---
+title: OpenHIM Adaptor
+---
+
+## About OpenHIM
+
+[OpenHIM (Open Health Information Mediator)](https://openhim.org/) is an open-source, middleware solution designed to facilitate interoperability between health information systems. It acts as a mediator, handling data transformation, validation, and routing between various systems.
+
+## Integration Options
+
+OpenHIM supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** OpenHIM has a REST API that enables external services like OpenFn to pull data from OpenHIM, or push data from external apps to OpenHIM. This option is suited for scheduled, bulk syncs or workflows that must update data in OpenHIM with external information. See [functions](/adaptors/packages/openhim-docs) for more on how to use this adaptor to work with the API.
+
+2. **Webhook:** Webhook or Data Forwarding to push data from OpenHIM to external systems (see [docs](https://openhim.org/docs/user-guide/alerting-reports/)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+1. See [OpenHIM docs](https://openhim.org/documentation) for the latest on supported authentication methods.
+2. When integrating with OpenHIM via OpenFn, there is one primary authentication method that is supported: **Basic Authentication**. See this adaptor's [Configuration docs](/adaptors/packages/openhim-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "apiUrl": "http://openhim.com/api"
+}
+```
+
+### Helpful Links
+
+1. [OpenHIM Documentation](https://openhim.org/documentation)
+2. [OpenHIM API Reference](https://openhim.org/api)
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
+
diff --git a/adaptors/openimis.md b/adaptors/openimis.md
new file mode 100644
index 000000000000..39ab05e6131a
--- /dev/null
+++ b/adaptors/openimis.md
@@ -0,0 +1,43 @@
+---
+title: OpenIMIS Adaptor
+---
+
+## About OpenIMIS
+
+[OpenIMIS (Open Insurance Management Information System)](https://openimis.org/) is an open-source software solution designed to manage health financing schemes such as health insurance, results-based financing, and universal health coverage.
+
+## Integration Options
+
+OpenIMIS supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** OpenIMIS has a REST API that enables external services like OpenFn to pull data from OpenIMIS, or push data from external apps to OpenIMIS. This option is suited for scheduled, bulk syncs or workflows that must update data in OpenIMIS with external information. See [functions](/adaptors/packages/openimis-docs) for more on how to use this adaptor to work with the API.
+
+2. **Webhook:** OpenIMIS does not natively support webhooks as a standard feature. However, integrations and custom implementations can enable webhook-like behavior. More details can be found on the OpenMRS [documentation page](https://wiki.openmrs.org/).
+
+## Authentication
+
+1. See [OpenIMIS docs](https://docs.openimis.org/en/latest/user_manual/user_login/login.html) for the latest on supported authentication methods.
+2. When integrating with OpenIMIS via OpenFn, there is one primary authentication method that is supported: **Basic Authentication**. See this adaptor's [Configuration docs](/adaptors/packages/openimis-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "baseUrl": "https://demo.openimis.org",
+ "username": "test@openfn.org",
+ "password": "@some(!)Str0ngp4ss0w0rd"
+}
+```
+
+### Helpful Links
+
+1. [OpenIMIS Documentation](https://openimis.org/documentation)
+2. [API Reference (REST & GraphQL)](https://openimis.org/api)
+3. [Integration Guide](https://openimis.org/integration)
+
+
+### Implementation Examples
+
+_Coming soon!_
+
+
diff --git a/adaptors/openlmis.md b/adaptors/openlmis.md
new file mode 100644
index 000000000000..b9f576419b65
--- /dev/null
+++ b/adaptors/openlmis.md
@@ -0,0 +1,44 @@
+---
+title: OpenLMIS Adaptor
+---
+
+## About OpenLMIS
+
+[OpenLMIS (Open Logistics Management Information System)](https://openlmis.org/) is an open-source, electronic logistics management information system designed to improve the management and distribution of health commodities. It enables supply chain management, providing visibility and accuracy in inventory tracking, order management, and reporting.
+
+
+## Integration Options
+OpenLMIS v3 uses a micro-services architecture with different services each providing different APIs - [see docs](https://docs.openlmis.org/en/latest/components/).
+
+**Rest API:** OpenLMIS has a REST API that enables external services like OpenFn to pull data from OpenLMIS, or push data from external apps to OpenLMIS. This option is suited for scheduled, bulk syncs or workflows that must update data in OpenLMIS with external information. See [functions](/adaptors/packages/openlmis-docs) for more on how to use this adaptor to work with the API.
+
+## Authentication
+
+1. See [OpenLMIS docs](https://openlmis.github.io/openlmis-api/) for the latest on supported authentication methods.
+2. The [Auth Service](https://docs.openlmis.org/en/latest/components/authServiceDesign.html0) in OpenLMIS v3 is a stand-alone micro-service that implements **OAuth 2**. See this adaptor's [Configuration docs](/adaptors/packages/openlmis-configuration-schema) for more on the required authentication parameters.
+
+When configuring your credential, `clientId` and `clientSecret` are required inputs for the authentication. If working with a demo environment, OpenLMIS provides default values for these inputs, which should be changed for any production system. In the below example, we have included these default values available.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "password": "@some(!)Str0ngp4ss0w0rd",
+ "username": "administrator",
+ "baseUrl": "https://test.openlmis.org",
+ "clientId": "user-client", // Default value for demo systems
+ "clientSecret": "changeme" // Default value for demo systems
+}
+```
+
+### Helpful Links
+
+1. [OpenLMIS Documentation](https://openlmis.org/documentation/)
+2. [OpenLMIS API Guide](https://openlmis.github.io/openlmis-api/)
+3. [Integration Guide](https://openlmis.org/integration/)
+
+### Implementation Examples
+
+_Coming soon!_
+
+
diff --git a/adaptors/openmrs.md b/adaptors/openmrs.md
new file mode 100644
index 000000000000..521b2934ce6f
--- /dev/null
+++ b/adaptors/openmrs.md
@@ -0,0 +1,181 @@
+---
+title: OpenMRS Adaptor
+---
+
+## About OpenMRS
+
+[OpenMRS (Open Medical Record System)](https://openmrs.org/) is an open-source
+platform designed to manage electronic medical records (EMRs) in low-resource
+environments. It provides a framework that allows developers to extend its core
+functionality through custom modules and APIs.
+
+## Integration Options
+
+**1. Rest API:** OpenMRS offers a REST API that enables external applications to
+interact with its database and perform bulk operations. This option is ideal for
+applications requiring scheduled or bulk synchronization with OpenMRS. Refer to
+the OpenMRS REST API [documentation](https://wiki.openmrs.org/) for detailed
+guidelines on endpoints and payload formats.
+
+**2. Webhook:** OpenMRS does not natively support webhooks as a standard
+feature. However, the platform is highly extensible and allows for customization
+through its module system. More details can be found on the OpenMRS
+[documentation page](https://wiki.openmrs.org/).
+
+## Authentication
+
+When integrating with OpenMRS via OpenFn, **Basic Authentication** is supported.
+
+The `instanceUrl`, `username` an `password` properties are all required to
+access your OpenMRS instance.
+
+See [Managing Credentials](/documentation/manage-projects/manage-credentials) to
+configure a credential in OpenFn.
+
+If working locally or if using a Raw JSON credential type, then your
+configuration will look something like this:
+
+```
+{
+ "instanceUrl": "http://openmrs.com/instance/url",
+ "password":"test",
+ "username":"test"
+}
+```
+
+See this adaptor's
+[Configuration docs](/adaptors/packages/openmrs-configuration-schema) for more
+details.
+
+## API Basics
+
+There are three ways you can use the OpenMRS API, all available interchangeable
+in the same step in a workflow.
+
+Like other adaptors, the FHIR adaptor "returns" output by writing to
+`state.data`:
+
+```js
+get('patient/71075074-f02e-4270-89a3-f2dcda436f70');
+fn(state => {
+ console.log(state.data); // the downloaded patient is available here
+ return state;
+});
+```
+
+### REST API Operations
+
+The main namespace contains high-level helper functions, which are designed as
+the primary interface. These are designed to handle the most common use cases,
+like downloading OpenMRS resources. They will configure default values and
+handle some complications for you, like authorization and pagination.
+
+All requests generated by these operations are made against your instance URL
+plus `ws/rest/v1`.
+
+For a list of valid resource types, see the
+[OpenMRS REST API Docs](https://rest.openmrs.org/#openmrs-rest-api). These docs
+are a useful resource when using OpenMRS, as different resources have different
+usage rules.
+
+For example, to download all concepts:
+
+```js
+get('concept');
+// calls /ws/rest/v1/concept
+```
+
+Each resource in OpenMRS defines a number of query parameters. These can be
+passed as an options object as the second argument. For example, to set the
+[`source` parameter](https://rest.openmrs.org/#list-all-concepts) on the concept
+resource:
+
+```js
+get('concept', { source: 'SNOWMED' });
+// calls /ws/rest/v1/concept?source=SNOWMED
+```
+
+Most options are appended to the request URL as query parameters, so you can
+pass whatever parameters OpenMRS requires.
+
+The adaptor will download resources over multiple pages. Each individual request
+to OpenMRS will be logged, for debugging and optimization.
+
+Note that by default, the function imposes a download limit of 10,000 records.
+You can force all data to be downloaded by passing `max: Infinity`, like this:
+
+```js
+get('concept', { max: Infinity });
+```
+
+When paging, the adaptor will make multiple requests. By default each request
+will download a page of data according to the instance's configuration. To force
+a larger page size (and reduce the number of requests, pass `pageSize`)
+
+```js
+get('concept', { pageSize: 1000 });
+```
+
+To disable pagination and force a single request, set `limit`.
+
+You can page from a specific position by passing `startIndex`. This works with
+pagination and with `limit`.
+
+## HTTP Utility Functions
+
+The REST API operations are be opinionated, and in some cases can make too many
+assumptions about what you want to do.
+
+The operations in the `http` namespace are simpler, lower-level helpers and work
+more like curl or postman. They'll do exactly what you ask for them. They'll
+handle authorization for you but won't paginate or prepend anything to the
+requested path.
+
+To request a single page of concepts, you can do:
+
+```js
+http.get('ws/rest/v1/concept');
+```
+
+You can pass a query via the options object:
+
+```js
+http.get('ws/rest/v1/concept', {
+ query: {
+ limit: 1000,
+ source: 'SNOWMED',
+ },
+});
+```
+
+Using the http helpers, pagination must be implemented manually.
+
+The HTTP helpers write to things to your state object:
+
+- `state.data` is the response body returned by OpenMRS
+- `state.response` is the rest of the HTTP response (excluding the body),
+ including `statusCode`, `statusMessage` and `headers`.
+
+## FHIR Helpers
+
+The OpenMRS adaptor also provides a
+[FHIR helper API](https://docs.openfn.org/adaptors/packages/openmrs-docs#fhir),
+which allows you to make requests against FHIR endpoints and download FHIR data.
+
+```js
+fhir.get('Encounter', {
+ count: 100,
+ lastUpdated: 'ge2024-01-01T00:00:00Z',
+});
+```
+
+### Helpful Links
+
+1. [OpenMRS Developer Guide](https://openmrs.atlassian.net/wiki/spaces/docs/pages/25476048/Developer+Guide)
+2. Community Forums: [OpenMRS Talk](https://talk.openmrs.org/)
+
+### Implementation Examples
+
+1. OpenFn Prototype for Médecins Sans Frontières (MSF) LIME Project - OpenMRS ->
+ DHIS2 sync:
+ [https://github.com/OpenFn/openfn-lime](https://github.com/OpenFn/openfn-lime)
diff --git a/adaptors/openspp.md b/adaptors/openspp.md
new file mode 100644
index 000000000000..dd3b964ea4fc
--- /dev/null
+++ b/adaptors/openspp.md
@@ -0,0 +1,58 @@
+---
+title: OpenSSP Adaptor
+---
+
+## About OpenSPP
+
+[OpenSPP (Open Source Social Protection Platform)](https://openspp.org/) is an
+integrated and digital social protection information system that enables
+governments and humanitarian agencies to streamline the creation and management
+of assistance programs.
+
+## Integration Options
+
+OpenSPP supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** OpenSPP has a REST API that enables external services like
+ OpenFn to pull data from OpenSPP, or push data from external apps to OpenSPP.
+ This option is suited for scheduled, bulk syncs or workflows that must update
+ data in OpenSPP with external information. See
+ [functions](/adaptors/packages/openspp-docs) for more on how to use this
+ adaptor to work with the API.
+
+2. **Webhook:** Webhook or Data Forwarding to push data from OpenSPP to external
+ systems (see [docs](https://openspp.org/api)). This option is suited for
+ real-time, event-based data integration.
+
+## Authentication
+
+1. See [OpenSPP docs](https://openspp.org/security) for the latest on supported
+ authentication methods.
+2. When integrating with OpenSPP via OpenFn, there is one primary authentication
+ method that is supported: **Basic Authentication**. See this adaptor's
+ [Configuration docs](/adaptors/packages/openspp-configuration-schema) for
+ more on the required authentication parameters.
+
+See platform docs on
+[managing credentials](/documentation/manage-projects/manage-credentials) for
+how to configure a credential in OpenFn. If working locally or if using a Raw
+JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "password": "@some(!)Str0ngp4ss0w0rd",
+ "username": "test@openfn.org",
+ "database": "devel",
+ "baseUrl": "https://dev.newlogic-demo.com"
+}
+```
+
+### Helpful Links
+
+1. [OpenSPP Documentation](https://openspp.org/documentation)
+2. [API Reference](https://openspp.org/api)
+3. [Integration Guide](https://openspp.org/integration)
+
+### Implementation Examples
+
+_Coming soon!_
diff --git a/adaptors/pdfshift.md b/adaptors/pdfshift.md
new file mode 100644
index 000000000000..601c383de995
--- /dev/null
+++ b/adaptors/pdfshift.md
@@ -0,0 +1,41 @@
+---
+title: PDFShift Adaptor
+---
+
+## About PDFShift
+
+[PDFShift](https://pdfshift.io/) is a solution designed to automate document
+conversion tasks.
+
+## Integration Options
+
+PDFShift has a REST API that enables external services like OpenFn to pull data
+from PDFShift, or push data from external apps to PDFShift. This option is
+suited for scheduled, bulk syncs or workflows that must update data in PDFShift
+with external information. See [functions](/adaptors/packages/pdfshift-docs) for
+more on how to use this adaptor to work with the API.
+
+## Authentication
+
+See [PDFShift docs](https://docs.pdfshift.io/#authentication) for the latest on
+supported authentication methods. When integrating with PDFShift via OpenFn,
+only one primary authentication method is supported.
+
+Api Key (requires api key created after authenticating in PDFShift). See this
+adaptor's [Configuration docs](/adaptors/packages/pdfshift-configuration-schema)
+for the required authentication parameters.
+
+See platform docs on
+[managing credentials](/documentation/manage-projects/manage-credentials) for
+how to configure a credential in OpenFn. If working locally or if using a Raw
+JSON credential type, then your configuration will look something like this:
+
+```json
+{
+ "apiKey": "sk_563874gfvftdv2t28462763fy23d28"
+}
+```
+
+### Helpful Links
+
+1. [PDFShift Documentation](https://docs.pdfshift.io/#introduction)
diff --git a/adaptors/postgresql.md b/adaptors/postgresql.md
index 8a6f972d218e..d37d3998d720 100644
--- a/adaptors/postgresql.md
+++ b/adaptors/postgresql.md
@@ -91,7 +91,7 @@ See the [Job Library](/adaptors/library) for more sample jobs.
data recieved
- **Testing**: Create a robust test suite which outlines which tables should be
updated by each job run
- 
+ 
## Common Errors
diff --git a/adaptors/powerbi.md b/adaptors/powerbi.md
index 4fed4ab6eaed..01e3579343e0 100644
--- a/adaptors/powerbi.md
+++ b/adaptors/powerbi.md
@@ -30,6 +30,6 @@ established, OpenFn assists the iNGO with connecting this data to Power BI.
Refer to the diagram below for a visualization of the data flow cited in the
above use case.
-
+
diff --git a/adaptors/primero.md b/adaptors/primero.md
index a1545169db73..155ce35c995b 100644
--- a/adaptors/primero.md
+++ b/adaptors/primero.md
@@ -63,7 +63,7 @@ See the examples section more sample Primero jobs.
### Integration tips
- Data forwarding can be enabled in Primero. There is a webhook that can forward
- case information to a designated URL endpoint (e.g., OpenFn Inbox). This data
+ case information to a designated URL endpoint (e.g., OpenFn Inbox). This feature requires a backend configuration update that the Primero support team can help with. The data
forwarding can happen automatically on insert of a new case, as well as
on-demand when a user clicks the `Sync` button (which may be added to the page
layout if this feature is in use).
diff --git a/adaptors/rapidpro.md b/adaptors/rapidpro.md
new file mode 100644
index 000000000000..c062ebecabdc
--- /dev/null
+++ b/adaptors/rapidpro.md
@@ -0,0 +1,42 @@
+---
+title: RapidPro Adaptor
+---
+
+## About RapidPro
+
+[RapidPro](https://app.rapidpro.io/) is an open-source platform for building scalable, automated messaging workflows. It is widely used in development and humanitarian contexts for managing communication via SMS, social media, and other messaging channels.
+
+
+## Integration Options
+
+**RapidPro supports two primary integration options:**
+
+**1. Rest API:** RapidPro has an available REST API that enables external services like OpenFn to pull data RapidPro, or push data from external apps to RapidPro. This option suits scheduled, bulk syncs or workflows that must update data in RapidPro with external information. See [functions](/adaptors/packages/rapidpro-docs) for more on how to use this adaptor to work with the API.
+
+**2. Webhook:** RapidPro also has a Webhook or Data Forwarding to push data from Rapidpro to external systems. This option is suited for real-time, event-based data integration. Check out the RapidPro [developer documentation](https://docs.rapidpro.io/webhooks/) to learn how to set up a webhook to push data to OpenFn.
+
+## Authentication
+
+When integrating with RapidPro via OpenFn, one primary authentication method is supported: **Personal Access Token (PAT)**. See this adaptor's [Configuration docs](/adaptors/packages/rapidpro-configuration-schema) for more on required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "host": "https://app.rapidpro.io/",
+ "token": "#Super-sSCrecrete-token"
+}
+```
+
+### Helpful Links
+
+1. [RapidPro API documentation](https://rapidpro.io/api/v2/)
+2. [RapidPro Community](https://community.rapidpro.io/)
+
+### Implementation Examples
+
+1. Sample RapidPro -> DHIS2 sync: [https://github.com/OpenFn/rapidpro-dhis2](https://github.com/OpenFn/rapidpro-dhis2)
+
+
+
+
diff --git a/adaptors/redis.md b/adaptors/redis.md
new file mode 100644
index 000000000000..7f403cb205d2
--- /dev/null
+++ b/adaptors/redis.md
@@ -0,0 +1,38 @@
+---
+title: Redis Adaptor
+---
+
+## About Redis
+
+[Redis (Remote Dictionary Server)](https://redis.io/) is an open-source, in-memory data store used as a database, cache, and message broker. It is commonly used for caching, real-time analytics, session management, and pub/sub messaging.
+
+
+## Integration Options
+
+The `redis` adaptor provides direct database connections for accessing data and executing NoSQL and standard database operations. See [functions](/adaptors/packages/redis-docs) for more on how to use this adaptor.
+
+
+## Authentication
+
+See [Redis](https://redis.io/docs/latest/) for the latest on supported authentication methods. When integrating with a NoSQL database via OpenFn, you authenticate via SSH using authorized database credentials. See this adaptor's [Configuration docs](/adaptors/packages/redis-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "host": "redis.example.com",
+ "password": "@some(!)Str0ngp4ss0w0rd",
+ "username": "test@openfn.org"
+}
+```
+
+### Helpful Links
+
+1. [Redis Documentation](https://redis.io/documentation)
+
+
+### Implementation Examples
+
+_Coming soon!_
+
+
diff --git a/adaptors/salesforce.md b/adaptors/salesforce.md
index 378cf2eeca49..9c2051a19f6e 100644
--- a/adaptors/salesforce.md
+++ b/adaptors/salesforce.md
@@ -181,9 +181,9 @@ A "Salesforce" Credential record should include:
- Security Token (Salesforce will email you this when you set your password. If
you cannot find "security token" in your inbox, see below for how to reset
it.)
-- Login URL: If a production system, use `https://login.salesforce.com/` (unless
- you have a custom domain `https://domainName.salesforce.com/`). For sandbox
- environments, `https://test.salesforce.com/`.
+- Login URL: For a production system, use `https://login.salesforce.com`. For
+ sandbox environments, use `https://test.salesforce.com`. For custom domains,
+ use `.my.salesforce.com`

@@ -242,18 +242,29 @@ Please save this `security token` in your OpenFn `Credential`.
## Salesforce Limits
-1. If using the `bulk()` adaptor functions, [see Salesforce docs](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_bulkapi.htm) on Bulk API Limits.
-2. If using the standard `query()` function (_not_ bulk API), note the [Salesforce SOQL query limits](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_soslsoql.htm) for maximum rows returned per
-request (e.g., max `2,000` rows returned per query request) and the string character limit (e.g., `100,000` string character limit for regular SOQL queries, and `4,000` character limit if using `WHERE` clause in your query).
-3. [See Apex Governor limits](https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_gov_limits.htm) for standard DML limits (e.g., you can only process
-`10,000` rows per transaction if using standard insert/update/upsert operations and not bulk) and other limits for Salesforce automation and Apex that might be triggered
-by your OpenFn workflow.
-4. [See here](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_limits.htm) for how to query your org's specific limits, and [this article](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm) for
-how to monitor your API usage.
+1. If using the `bulk()` adaptor functions,
+ [see Salesforce docs](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_bulkapi.htm)
+ on Bulk API Limits.
+2. If using the standard `query()` function (_not_ bulk API), note the
+ [Salesforce SOQL query limits](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_soslsoql.htm)
+ for maximum rows returned per request (e.g., max `2,000` rows returned per
+ query request) and the string character limit (e.g., `100,000` string
+ character limit for regular SOQL queries, and `4,000` character limit if
+ using `WHERE` clause in your query).
+3. [See Apex Governor limits](https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_gov_limits.htm)
+ for standard DML limits (e.g., you can only process `10,000` rows per
+ transaction if using standard insert/update/upsert operations and not bulk)
+ and other limits for Salesforce automation and Apex that might be triggered
+ by your OpenFn workflow.
+4. [See here](https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_limits.htm)
+ for how to query your org's specific limits, and
+ [this article](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm)
+ for how to monitor your API usage.
## OpenFn Adaptors
-OpenFn supports a robust `salesforce` adaptor ([see adaptor source code](https://github.com/OpenFn/adaptors/blob/main/packages/salesforce/src/Adaptor.js))
+OpenFn supports a robust `salesforce` adaptor
+([see adaptor source code](https://github.com/OpenFn/adaptors/blob/main/packages/salesforce/src/Adaptor.js))
with a range of helper functions for common CRUD & upsert operations, and for
accessing the Salesforce bulk API.
diff --git a/adaptors/satusehat.md b/adaptors/satusehat.md
index 8be57505ede8..cc837ca3a59e 100644
--- a/adaptors/satusehat.md
+++ b/adaptors/satusehat.md
@@ -25,7 +25,7 @@ exchange
The following example shows a HTTP POST request to creating an `Encounter` FHIR
resource. Data was taken from the
-[Satusehate Postman Collection](https://www.postman.com/satusehat/satusehat-public/request/56uan96/encounter-create)
+[Satusehat Postman Collection](https://www.postman.com/satusehat/satusehat-public/request/56uan96/encounter-create)
```js
post('Encounter', {
@@ -125,7 +125,10 @@ The result will be written to `state.data`.
## Integration Options
1. **[REST APIs](https://satusehat.kemkes.go.id/platform/docs/id/postman-workshop/)**
- enable external services like OpenFn to pull data from Satusehat or push data to Satusehat from external apps. This option is suitable for scheduled workflows or those that need to update data in Satusehat with external information.
+ enable external services like OpenFn to pull data from Satusehat or push data
+ to Satusehat from external apps. This option is suitable for scheduled
+ workflows or those that need to update data in Satusehat with external
+ information.
## How to Extract or Modify Satusehat Data
@@ -166,7 +169,7 @@ See platform docs
how to configure a credential in OpenFn and see the below Satusehat credential
example.
-
+
If you're using the `Raw JSON` credential type, your configuration may look like
this:
diff --git a/adaptors/sftp.md b/adaptors/sftp.md
new file mode 100644
index 000000000000..ddeb65cb6b31
--- /dev/null
+++ b/adaptors/sftp.md
@@ -0,0 +1,39 @@
+---
+title: SFTP Adaptor
+---
+
+## About SFTP
+
+[SFTP (Secure File Transfer Protocol)](https://www.techtarget.com/searchcontentmanagement/definition/Secure-File-Transfer-Protocol-SSH-File-Transfer-Protocol) is a secure method for transferring files between systems over an encrypted SSH connection. It is widely used for securely uploading, downloading, and managing files on remote servers.
+
+Using this adaptor, you can read and write files (e.g., `csv`, `xls`, `json` files) saved on a SFTP server.
+
+## Integration Options
+
+**Direct File Transfers:** SFTP allows users to manually or programmatically transfer files between a local and remote system. See [functions](/adaptors/packages/sftp-docs) for more on how to use this adaptor to work with an SFTP server.
+
+## Authentication
+
+When integrating with a SFTP server via OpenFn, you can provide a `username` and `password` for an authorized user to authenticate. See this adaptor's [Configuration docs](/adaptors/packages/sftp-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "host": "191.173.128.88",
+ "username": "name",
+ "password": "pwd"
+}
+```
+
+### Helpful Links
+1. [Best Practices for SFTP](https://www.ssh.com/academy/ssh/sftp)
+
+### Implementation Examples
+
+1. Women for Women International - SFTP -> Salesforce sync: [https://github.com/OpenFn/women-for-women](https://github.com/OpenFn/women-for-women)
+
+
+
+
+
diff --git a/adaptors/surveycto.md b/adaptors/surveycto.md
index 9d623964cb50..5755d6a47c67 100644
--- a/adaptors/surveycto.md
+++ b/adaptors/surveycto.md
@@ -38,7 +38,7 @@ SurveyCTO provides a REST API which can be accessed via the OpenFn `surveycto`
adaptor. In SurveyCTO, admins can also configure webhooks to push data to
OpenFn/other external systems in real-time in CSV or JSON format.
-
+
### 1. Webhooks (for real-time integration):
@@ -63,7 +63,7 @@ To configure a webhook to push data to OpenFn:
[See this docs page](https://docs.surveycto.com/05-exporting-and-publishing-data/03-publishing-data-to-the-cloud/05.forms-to-webhooks.html)
for more on SurveyCTO webooks.
-
+
### 2. API integration (for scheduled and/or bulk data integration):
diff --git a/adaptors/tableau.md b/adaptors/tableau.md
index e7decefd2f57..24e4e29779c2 100644
--- a/adaptors/tableau.md
+++ b/adaptors/tableau.md
@@ -33,6 +33,6 @@ established, OpenFn assists the iNGO with connecting this data to Tableau.
Refer to the diagram below for a visualization of the data flow cited in the
above use case.
-
+
diff --git a/adaptors/telerivet.md b/adaptors/telerivet.md
new file mode 100644
index 000000000000..809db1da86ec
--- /dev/null
+++ b/adaptors/telerivet.md
@@ -0,0 +1,43 @@
+---
+title: Telerivet Adaptor
+---
+
+## About Telerivet
+
+[Telerivet](https://www.telerivet.com/) is a platform for sending, receiving, and automating SMS, voice calls, and mobile messaging through various channels. Telerivet is commonly used for customer notifications, marketing campaigns, and transactional messaging.
+
+## Integration Options
+
+Telerivet supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** Telerivet has a REST API that enables external services like OpenFn to pull data from Telerivet, or push data from external apps to Telerivet. This option is suited for scheduled, bulk syncs or workflows that must update data in Telerivet with external information. See [functions](/adaptors/packages/telerivet-docs) for more on how to use this adaptor to work with the API.
+
+2. **Webhook:** Webhook or Data Forwarding to push data from Telerivet to external systems (see [docs](https://telerivet.com/api/webhook)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+1. See [Telerivet docs](https://developers.google.com/gmail/api/auth/scopes) for the latest on supported authentication methods.
+2. When integrating with Telerivet via OpenFn, there is one primary authentication method that is supported: **API Key**. See this adaptor's [Configuration docs](/adaptors/packages/telerivet-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "projectId": "telerivet_project_id",
+ "apiKey": "telerivet_api_key"
+}
+```
+
+### Helpful Links
+
+1. [Telerivet API Documentation](https://telerivet.com/api)
+2. [Webhook Setup](https://telerivet.com/help/webhooks)
+
+
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
diff --git a/adaptors/twilio.md b/adaptors/twilio.md
new file mode 100644
index 000000000000..1246ca391529
--- /dev/null
+++ b/adaptors/twilio.md
@@ -0,0 +1,41 @@
+---
+title: Twilio Adaptor
+---
+
+## About Twilio
+
+[Twilio](https://www.twilio.com/en-us) is a cloud communications platform that enables developers to build and scale applications for voice, messaging, video, and other communication channels.
+
+## Integration Options
+
+Twilio supports 2 primary integration options with OpenFn:
+
+1. **Rest API:** Twilio has a REST API that enables external services like OpenFn to pull data from Twilio, or push data from external apps to Twilio. This option is suited for scheduled, bulk syncs or workflows that must update data in Twilio with external information. See [functions](/adaptors/packages/twilio-docs) for more on how to use this adaptor to work with the API.
+2. **Webhook:** Webhook or Data Forwarding to push data from Twilio to external systems (see [docs](https://www.twilio.com/docs/usage/webhooks)). This option is suited for real-time, event-based data integration.
+
+## Authentication
+
+1. See [Twilio docs](https://www.twilio.com/docs/) for the latest on supported authentication methods.
+2. When integrating with Twilio via OpenFn, there is one primary authentication method that is supported: **API Key**. See this adaptor's [Configuration docs](/adaptors/packages/twilio-configuration-schema) for more on the required authentication parameters.
+
+See platform docs on [managing credentials](/documentation/manage-projects/manage-credentials) for how to configure a credential in OpenFn. If working locally or if using a Raw JSON credential type, then your configuration will look something like this:
+
+```
+{
+ "accountSid": "account_id",
+ "authToken": "evenMoreSecret"
+}
+```
+
+### Helpful Links
+
+1. [Twilio API Documentation](https://www.twilio.com/docs/)
+2. [Webhook Setup](https://www.twilio.com/docs/usage/webhooks)
+
+
+### Implementation Examples
+
+_Coming soon!_
+
+
+
diff --git a/adaptors/varo.md b/adaptors/varo.md
new file mode 100644
index 000000000000..b495634676ac
--- /dev/null
+++ b/adaptors/varo.md
@@ -0,0 +1,30 @@
+---
+title: Varo Adaptor
+---
+
+## About Varo
+
+[Varo](https://www.varo-app.org/) is a simple and free smartphone app that
+collects and forwards useful cold chain information and temperature logs to the
+email inbox of your choice.
+
+## Integration Options
+
+This is a data transformation adaptor for converting Varo data to the WHO
+Equipment Monitoring System (EMS) standard.
+
+To integrate with Varo data, you can use the relevant OpenFn adaptor (e.g.,
+Gmail) for your chosen email inbox to query the Varo emails and attachments.
+
+
+
+## Authentication
+
+N/A - This adaptor is for data transformation only, and not for direct
+connection to the Varo app. See note in above section.
+
+## Helpful Links
+
+- Varo website: https://www.varo-app.org/
+- WHO documentation on EMS standard:
+ https://extranet.who.int/prequal/immunization-devices/e006-temperature-monitoring-devices
diff --git a/adaptors/whatsapp.md b/adaptors/whatsapp.md
new file mode 100644
index 000000000000..934ad5a0a769
--- /dev/null
+++ b/adaptors/whatsapp.md
@@ -0,0 +1,41 @@
+---
+title: WhatsApp Adaptor
+---
+
+## About WhatsApp
+
+The [WhatsApp Business Platform](https://developers.facebook.com/docs/whatsapp) enables organizations to communicate with their customers on WhatsApp at scale. It supports sending messages, notifications, and media, as well as receiving inbound customer messages through a secure and reliable API.
+
+This adaptor allows OpenFn users to integrate workflows with WhatsApp Business Cloud or On-Premise APIs to send and receive WhatsApp messages as part of automated processes.
+
+## Integration Options
+
+The WhatsApp Business API provides a [REST API](https://developers.facebook.com/docs/whatsapp) to manage message sending, conversations, and contacts. See [functions](/adaptors/packages/whatsapp-docs) for examples of how to use this adaptor to call the API.
+
+In addition, WhatsApp Business supports [Webhooks](https://developers.facebook.com/docs/whatsapp/cloud-api/webhooks/payload-examples) that deliver inbound messages and delivery status updates in real time. These can be consumed by OpenFn workflows via a [webhook event trigger](/documentation/build/triggers#webhook-event-triggers) when customers send messages.
+
+## Authentication
+Access to the WhatsApp Business API requires a `Permanent Access Token` (for Cloud API) or a generated `Bearer Token` (for On-Premise deployments). This token must be included as an Authorization Bearer Token in all requests.
+
+For Cloud API, you must also specify your `phone_number_id` and `whatsapp_business_account_id`.
+[See Meta WhatsApp Business docs](https://developers.facebook.com/docs/whatsapp/cloud-api/get-started) for detailed guidance.
+
+OpenFn users can use the `whatsapp` credential type when [creating a credential](/documentation/manage-projects/manage-credentials) on the app.
+
+If working locally or if using a `Raw JSON` credential type in OpenFn, your configuration will look something like this ([see configuration docs](/adaptors/packages/whatsapp-configuration-schema)):
+
+```json
+{
+ "baseUrl": "https://graph.facebook.com/v21.0",
+ "apiToken": "EAAJZC...your_long_lived_access_token",
+ "phoneNumberId": "123456789012345",
+ "wabaId": "987654321098765"
+}
+
+```
+
+## Helpful Links
+
+- WhatsApp Business API documentation: https://developers.facebook.com/docs/whatsapp
+- Cloud API Get Started guide: https://developers.facebook.com/docs/whatsapp/cloud-api/get-started
+- Webhook payload examples: https://developers.facebook.com/docs/whatsapp/cloud-api/webhooks/payload-examples
diff --git a/articles/2020-06-16-how-information-is-organized.md b/articles/2020-06-16-how-information-is-organized.md
index 42d3847b4945..e06576ee88c7 100644
--- a/articles/2020-06-16-how-information-is-organized.md
+++ b/articles/2020-06-16-how-information-is-organized.md
@@ -1,9 +1,7 @@
---
layout: post
title: 'How Information Is Organized... In Organizations'
-author: Taylor Downs
-author_url: https://github.com/taylordowns2000
-author_image_url: https://avatars.githubusercontent.com/taylordowns2000
+authors: taylor
tags: [how-to, tips]
image: /img/informationorganized.jpg
featured: false
@@ -61,7 +59,7 @@ technology.
### Boxes and crow's feet
-
+
The box is my favorite. It represents an entity in your data system. Entities
(like `teachers`) have attributes (like `name`, `phone number`, `date of birth`,
@@ -71,7 +69,7 @@ gender, etc. These are the fields on your teacher entity. By submitting one of
these forms, you’ll add a new teacher to your database. If you’re an Excel
person, the attributes are columns in your `teachers` table.
-
+
The crow’s foot is my second favorite. It’s used to show relationships between
entities. We know that teachers are related to the sessions that they conduct.
@@ -85,7 +83,7 @@ relationship. The crow’s foot (that little three-pronged fella) denotes the
many. One teacher can have many sessions. One session, however, can only have
one teacher. See the diagram below.
-
+
If we focus just on `teacher` and `session` and think back to MS Excel, we can
envision a `teachers` table and a `sessions` table. Let’s put them on different
@@ -131,4 +129,5 @@ interchangeable parts. Alas, the middle way is probably the best.
That’s all for now. More soon.
_Need help organizing or scaling your organization's information or process
-flows? Contact our team of ICT4D specialists at ._
+flows? Contact our team of ICT4D specialists at
+[support@openfn.org](mailto:support@openfn.org)._
diff --git a/articles/2020-06-24-three-questions-to-ask.md b/articles/2020-06-24-three-questions-to-ask.md
index 581a5633d472..7b78fb7e3634 100644
--- a/articles/2020-06-24-three-questions-to-ask.md
+++ b/articles/2020-06-24-three-questions-to-ask.md
@@ -1,9 +1,7 @@
---
layout: post
title: 'To Automate or Not to Automate? Ask Yourself These 3 Questions.'
-author: Aleksa Krolls
-author_url: https://github.com/aleksa-krolls
-author_image_url: https://avatars.githubusercontent.com/aleksa-krolls
+authors: aleksa
tags: [tips]
featured: true
---
diff --git a/articles/2020-07-02-allow-yourself-to-fail.md b/articles/2020-07-02-allow-yourself-to-fail.md
index dd869d274400..f9c0e4d1dde9 100644
--- a/articles/2020-07-02-allow-yourself-to-fail.md
+++ b/articles/2020-07-02-allow-yourself-to-fail.md
@@ -1,9 +1,7 @@
---
layout: post
title: 'Allow Yourself to Fail'
-author: Taylor Downs
-author_url: https://github.com/taylordowns2000
-author_image_url: https://avatars.githubusercontent.com/taylordowns2000
+authors: taylor
tags: [how-to, tips]
featured: false
---
@@ -64,11 +62,11 @@ creating S3 (**S**ecure, **S**table and **S**calable—more on that
So... when clients let me mess around with their jobs, I _always_ recommend we
design for idempotence. It's common sense when you're passing messages between
-two different systems that are bound to evolve, go offline, have a bad day, etc
+two different systems that are bound to evolve, go offline, have a bad day, etc.
— Taylor
-[Sign up](https://openfn.org/signup){: .btn} to set up a project today,
-absolutely free.
+[Sign up](https://openfn.org/register) to set up a project today, absolutely
+free.
-[Reach out](mailto:admin@openfn.org){: .btn} for more information.
+[Reach out](mailto:admin@openfn.org) for more information.
diff --git a/articles/2020-07-14-cron-is-better-than-a-timer.md b/articles/2020-07-14-cron-is-better-than-a-timer.md
index 687098933f0c..e96d793079a8 100644
--- a/articles/2020-07-14-cron-is-better-than-a-timer.md
+++ b/articles/2020-07-14-cron-is-better-than-a-timer.md
@@ -1,9 +1,7 @@
---
layout: post
title: 'Product News: Enhanced Scheduled/Periodic Job Control'
-author: Taylor Downs
-author_url: https://github.com/taylordowns2000
-author_image_url: https://avatars.githubusercontent.com/taylordowns2000
+authors: taylor
tags: [annoucement, tips]
featured: false
---
diff --git a/articles/2020-12-09-upsert-in-dhis2.md b/articles/2020-12-09-upsert-in-dhis2.md
index 1b2672feadd4..4b4e2857ed99 100644
--- a/articles/2020-12-09-upsert-in-dhis2.md
+++ b/articles/2020-12-09-upsert-in-dhis2.md
@@ -1,9 +1,7 @@
---
layout: post
title: 'Tracked entity instances in DHIS2'
-author: Taylor Downs
-author_url: https://github.com/taylordowns2000
-author_image_url: https://avatars.githubusercontent.com/taylordowns2000
+authors: taylor
tags: [how-to, tips]
featured: true
---
@@ -43,14 +41,14 @@ Because an upsert simultaneously UPDATES and INSERTS, it prevents duplicates.
Upserts are important and good because they cut down on the risk of duplicate
data entry and they also allow for transactions to be retried over and over to
ensure data integrity. That last bit is called “idempotency” and you can read
-about it [over here](https://blog.openfn.org/allow-yourself-to-fail/).
+about it [over here](2020-07-02-allow-yourself-to-fail.md).
Please don’t hesitate to reach out to one of OpenFn’s implementation specialists
if you’d like to learn more.
— Taylor
-[Sign up](https://openfn.org/signup){: .btn} to set up a project today,
-absolutely free.
+[Sign up](https://openfn.org/register) to set up a project today, absolutely
+free.
-[Reach out](mailto:admin@openfn.org){: .btn} for more information.
+[Reach out](mailto:admin@openfn.org) for more information.
diff --git a/articles/2021-02-03-hosted-or-local-deployment.md b/articles/2021-02-03-hosted-or-local-deployment.md
index bff10a740307..12ba03592c17 100644
--- a/articles/2021-02-03-hosted-or-local-deployment.md
+++ b/articles/2021-02-03-hosted-or-local-deployment.md
@@ -1,9 +1,7 @@
---
layout: post
title: 'Our Servers or Yours: Thinking through deployment options'
-author: Jed Goldstein
-author_url: https://github.com/jedbgold
-author_image_url: https://avatars.githubusercontent.com/jedbgold
+authors: jed
tags: [how-to, tips]
featured: true
---
diff --git a/articles/2021-02-17-syncing-options.md b/articles/2021-02-17-syncing-options.md
index e0ad727a5518..651d0a1a34fc 100644
--- a/articles/2021-02-17-syncing-options.md
+++ b/articles/2021-02-17-syncing-options.md
@@ -1,9 +1,7 @@
---
layout: post
title: 'Sync Like You Mean It: Thinking Through System “Syncing” Protocols'
-author: Jed Goldstein
-author_url: https://github.com/jedbgold
-author_image_url: https://avatars.githubusercontent.com/jedbgold
+authors: jed
tags: [how-to, tips]
featured: true
---
@@ -38,7 +36,7 @@ interface, i.e., when a real case-worker clicks the “refer” button) OpenFn
receives a small payload with case data and transmits it to OSCaR and vice
versa.
-
+
@@ -62,7 +60,7 @@ case might not get referred to Oscar!
### Scheduled Syncs
-
+
The second option considered, a bi-directional schedule dependent sync, solves
for the issue discussed above. On a scheduled basis (every 5 minutes, for
diff --git a/articles/2021-05-24-commcare-events.md b/articles/2021-05-24-commcare-events.md
index b94ed1e68879..5c7a2d5301c5 100644
--- a/articles/2021-05-24-commcare-events.md
+++ b/articles/2021-05-24-commcare-events.md
@@ -1,9 +1,7 @@
---
layout: post
title: 'Forms and Cases: CommCare and event-based integration'
-author: Taylor Downs
-author_url: https://github.com/taylordowns2000
-author_image_url: https://avatars.githubusercontent.com/taylordowns2000
+authors: taylor
tags: [how-to, tips]
featured: true
---
diff --git a/articles/2021-07-05-wrapping-my-head-around-jobs.md b/articles/2021-07-05-wrapping-my-head-around-jobs.md
index 65ced6180e76..a8dc78689e2b 100644
--- a/articles/2021-07-05-wrapping-my-head-around-jobs.md
+++ b/articles/2021-07-05-wrapping-my-head-around-jobs.md
@@ -1,9 +1,7 @@
---
layout: post
title: Wrapping my head around jobs
-author: Taylor Downs
-author_url: https://github.com/taylordowns2000
-author_image_url: https://avatars.githubusercontent.com/taylordowns2000
+authors: taylor
tags: [how-to, tips, jobs]
featured: true
---
diff --git a/articles/2021-10-08-improving-multistage-docker-builds-using-buildx.md b/articles/2021-10-08-improving-multistage-docker-builds-using-buildx.md
index 62f193b43c62..ce52cd219836 100644
--- a/articles/2021-10-08-improving-multistage-docker-builds-using-buildx.md
+++ b/articles/2021-10-08-improving-multistage-docker-builds-using-buildx.md
@@ -1,9 +1,7 @@
---
layout: post
title: Improving Multistage Docker Builds using Buildx
-author: Stuart Corbishley
-author_url: https://github.com/stuartc
-author_image_url: https://avatars.githubusercontent.com/stuartc
+authors: stu
tags: [how-to, docker, ci/cd]
featured: true
---
diff --git a/articles/2021-10-15-webpack-to-esbuild-part1.md b/articles/2021-10-15-webpack-to-esbuild-part1.md
index 2dcd7ce45829..caa10c9277c4 100644
--- a/articles/2021-10-15-webpack-to-esbuild-part1.md
+++ b/articles/2021-10-15-webpack-to-esbuild-part1.md
@@ -1,9 +1,7 @@
---
layout: post
title: Moving from Webpack to esbuild on Phoenix
-author: Stuart Corbishley
-author_url: https://github.com/stuartc
-author_image_url: https://avatars.githubusercontent.com/stuartc
+authors: stu
tags: [how-to, js, webpack, build, phoenix, ci/cd]
featured: true
---
diff --git a/articles/2021-10-22-testing-react-app-with-jest-hound.md b/articles/2021-10-22-testing-react-app-with-jest-hound.md
index 44d6e26c19b8..7f56ae2fc3ef 100644
--- a/articles/2021-10-22-testing-react-app-with-jest-hound.md
+++ b/articles/2021-10-22-testing-react-app-with-jest-hound.md
@@ -1,9 +1,7 @@
---
layout: post
title: Testing a React app, the blurred line between Unit, integration and E2E
-author: Chaiwa Berian
-author_url: https://github.com/chaiwa-berian
-author_image_url: https://avatars.githubusercontent.com/u/7937584?v=4
+authors: chaiwa
tags: [how-to, tips, testing, browser-testing, react, elixir]
featured: true
diff --git a/articles/2021-10-29-how-learning-javascript-helped-me-better-understand-jobs.md b/articles/2021-10-29-how-learning-javascript-helped-me-better-understand-jobs.md
index 23c6e77d575f..7918d56c73bc 100644
--- a/articles/2021-10-29-how-learning-javascript-helped-me-better-understand-jobs.md
+++ b/articles/2021-10-29-how-learning-javascript-helped-me-better-understand-jobs.md
@@ -1,19 +1,17 @@
---
layout: post
title: How learning JavaScript helps me better understand OpenFn jobs
-author: Aicha Diallo
-author_url: https://github.com/daissatou2
-author_image_url: https://avatars.githubusercontent.com/daissatou2
+authors: aicha
tags: [javascript, tips, jobs, learning]
featured: true
---
-OpenFn automation happens via [jobs](/documentation/build/workflows) which define
-specific steps ("operations") that OpenFn should perform. They're written in a
-[scripting language](https://www.openfn.org/core) that runs on top of (and has
-full access to) **Javascript**. A basic understanding of Javascript will take
-your job writing on OpenFn to the next level. To improve my limited knowledge of
-JavaScript, I have been taking Codecademy's
+OpenFn automation happens via [jobs](/documentation/build/workflows) which
+define specific steps ("operations") that OpenFn should perform. They're written
+in a basic scripting language that runs on top of (and has full access to)
+**Javascript**. A basic understanding of Javascript will take your job writing
+on OpenFn to the next level. To improve my limited knowledge of JavaScript, I
+have been taking Codecademy's
[Introduction to JavaScript Course](https://www.codecademy.com/learn/introduction-to-javascript).
diff --git a/articles/2022-06-07-workflow-automation.md b/articles/2022-06-07-workflow-automation.md
index f2d37220ed6d..5f8fcf2d64a5 100644
--- a/articles/2022-06-07-workflow-automation.md
+++ b/articles/2022-06-07-workflow-automation.md
@@ -2,9 +2,7 @@
layout: post
title:
'Workflow Automation; Why do it yourself when a program can do it for you?'
-author: Alexa de Vegvar
-author_url: https://github.com/alexadevegvar
-author_image_url: https://avatars.githubusercontent.com/alexadevegvar
+authors: alexa
tags: [automation, solution]
image: https://user-images.githubusercontent.com/105651463/172341536-ebde5ca0-12b8-4a8a-bb40-da9442701df3.png
featured: false
diff --git a/articles/2022-09-19-auth-security.md b/articles/2022-09-19-auth-security.md
index 9f2195e68fea..4c8bced5271c 100644
--- a/articles/2022-09-19-auth-security.md
+++ b/articles/2022-09-19-auth-security.md
@@ -1,17 +1,15 @@
---
layout: post
title: 'Secure by design: a roadmap to secure authentication and authorization'
-author: Amber Rignell
-author_url: https://github.com/amberrignell
-author_image_url: https://avatars.githubusercontent.com/amberrignell
+authors: amber
tags: [security, authentication, authorization]
image: https://user-images.githubusercontent.com/105651463/172341536-ebde5ca0-12b8-4a8a-bb40-da9442701df3.png
featured: false
---
-_In order to ensure that Global Goods and Digital Public Goods can enable impact
+In order to ensure that Global Goods and Digital Public Goods can enable impact
without doing any harm, it is crucial that their authentication and
-authorization systems are secure from the outset. _
+authorization systems are secure from the outset.
We’ve put together a resource to help other Digital Public Goods and Global
Goods achieve just this. You can view and comment on the resource
@@ -57,8 +55,8 @@ to implement such security features down the line.
As product manager at Open Function Group, I myself have recently been reviewing
and analyzing numerous resources to identify and prioritize features required to
get to fully-secure authentication and authorization for our newest product
-[Lightning](/documentation#openfn-v2-lightning-/)–and make sure that nothing falls
-through the cracks. I’ve also had the opportunity to gather insights from
+[Lightning](/documentation#openfn-v2-lightning-/)–and make sure that nothing
+falls through the cracks. I’ve also had the opportunity to gather insights from
Digital Public Good community forums (OpenMRS, DHIS2, OpenLMIS, etc) and speak
to other product managers and engineers from the Digital Public Goods community
to ask them about their learnings, and any tips they might have: Austin and
diff --git a/articles/authors.yml b/articles/authors.yml
new file mode 100644
index 000000000000..9b69f796c13b
--- /dev/null
+++ b/articles/authors.yml
@@ -0,0 +1,51 @@
+aleksa:
+ name: Aleksa Krolls
+ title: Co-Founder, Chief Customer Officer
+ page: true
+ email: aleksa@openfn.org
+ image_url: https://avatars.githubusercontent.com/aleksa-krolls
+ socials:
+ github: aleksa-krolls
+
+taylor:
+ name: Taylor Downs
+ title: Founder & CEO
+ page: true
+ image_url: https://avatars.githubusercontent.com/taylordowns2000
+ email: taylor@openfn.org
+ socials:
+ x: taylordowns2000
+ github: taylordowns2000
+
+jed:
+ name: Jed Goldstein
+ image_url: https://avatars.githubusercontent.com/jedbgold
+ socials:
+ github: jedbgold
+
+stu:
+ name: Stuart Corbishley
+ image_url: https://avatars.githubusercontent.com/stuartc
+ socials:
+ github: https://github.com/stuartc
+
+chaiwa:
+ name: Chaiwa Berian
+ image_url: https://avatars.githubusercontent.com/u/7937584?v=4
+ socials:
+ github: chaiwa-berian
+
+aicha:
+ name: Aicha Diallo
+ image_url: https://avatars.githubusercontent.com/daissatou2
+ socials:
+ github: daissatou2
+
+alexa:
+ name: Alexa de Vegvar
+
+amber:
+ name: Amber Rignell
+ image_url: https://avatars.githubusercontent.com/amberrignell
+ socials:
+ github: amberrignell
diff --git a/docs/build-for-developers/build-compliant-apps.md b/docs/build-for-developers/build-compliant-apps.md
index e6700e4dfb3c..02f208bd6abe 100644
--- a/docs/build-for-developers/build-compliant-apps.md
+++ b/docs/build-for-developers/build-compliant-apps.md
@@ -4,7 +4,7 @@ sidebar_label: Building compliant APIs
---
This section is for you if you are hoping to build or extend an existing
-application that can connect to OpenFn. We follow modern web-standard JSON api
+application that can connect to OpenFn. We follow modern web-standard JSON API
guidelines.
For your application to a be data provider (or "source") for OpenFn
@@ -17,7 +17,7 @@ of having requests be made and handled every X seconds, your servers and
OpenFn's servers will only work when new data is available.
For your application to be a consumer (or "destination") for OpenFn, you must
-either have a standard, JSON-based rest API or create a language-package that
+either have a standard, JSON-based REST API or create a language-package that
meets your API specifications.
## Sending data to OpenFn
@@ -31,7 +31,7 @@ curl -X POST \
-H "Content-Type: application/json" \
-H "Cache-Control: no-cache" \
-d '{"foo":"bar", "baz":"qux"}' \
- "https://staging.openfn.org/inbox/some-secret-api-key"
+ "https://app.openfn.org/i/some-secret-inbox-uuid"
```
OpenFn will respond with a 200 and an empty JSON object in the event of a
diff --git a/docs/build-for-developers/cli-collections.md b/docs/build-for-developers/cli-collections.md
new file mode 100644
index 000000000000..fee0e3ae08c7
--- /dev/null
+++ b/docs/build-for-developers/cli-collections.md
@@ -0,0 +1,264 @@
+---
+title: Collections CLI Usage
+sidebar_label: Collections
+slug: /collections-cli
+---
+
+The OpenFn CLI includes support for reading from and writing to
+[Collections](/documentation/build/collections): a key/value store built into
+OpenFn.
+
+:::caution Versions
+
+Collections support was added to the CLI in version 1.9.0.
+
+Run `npm install -g @openfn/cli` to update or install.
+
+:::
+
+You can use the CLI to:
+
+- Explore the contents of Collections without running a Workflow
+- Experiment with query syntax to get the keys you need
+- Update mapping objects and lookup tables from local (or source-controlled)
+ files
+- Manually remove unneeded data
+
+:::tip
+
+Got feedback? Want more Collections support in the CLI? Post a Feature Request
+to [community.openfn.org](https://community.openfn.org/c/feature-requests)!
+
+:::
+
+Get started with the Collections API with `openfn collections --help`
+
+You'll need a Personal Access Token (PAT) to access a collection. You'll also
+need to ensure a collection has been created before you can read or write to
+it - see
+[Managing Collections](/documentation/build/collections#managing-collections)
+
+:::info Trying to use Collections in a CLI workflow?
+
+These docs explain how to use the `openfn collections` CLI command.
+
+If you're running an expression or workflow through the CLI, you need to use the
+collections adaptor - check out the
+[Collections Adaptor Docs](/adaptors/collections#cli-usage) for detauls
+
+:::
+
+## Getting a PAT
+
+Data inside Collections is securely stored under a Project, and access is
+strictly only allowed to users with access to that Project. So if you want to
+access a Collection, you have to tell the server who you are.
+
+We do this using Personal Access Tokens. See
+[Create and Manage API Tokens](/documentation/api-tokens#about-api-tokens) for
+more details.
+
+One you have a PAT, you need to pass it in to the CLI. The easiest way to do
+this is to set your `OPENFN_PAT` env var, which the CLI will use automatically.
+
+If you're using multiple access tokens, you can pass `--token` to the CLI to
+override the default.
+
+```bash
+openfn collections get my-collection \* --token $MY_OPENFN_PAT
+```
+
+:::tip
+
+The rest of this guide assumes that the `OPENFN_PAT` env var has been set. So
+long as it has, as you're using a server which has a `my-collection` collection,
+all examples will work.
+
+:::
+
+## Setting a server
+
+By default, the CLI will point to our primary platform at
+https://app.openfn.org.
+
+If you're running from open source or using a different deployment, you'll also
+need to tell the CLI which Collections server to use.
+
+You can do this by passing `--lightning` directly:
+
+```bash
+openfn collections get my-collection \* --lightning http://localhost:4000
+```
+
+Or by setting the `OPENFN_ENDPOINT` environment variable.
+
+:::tip
+
+To see which server the CLI is using, ask for debug-level logging in your
+output:
+
+```bash
+openfn collections get my-collection \* --log debug
+```
+
+:::
+
+## Fetching items
+
+You can fetch items from a Collection by passing a collection name and a key, or
+key pattern (like `*` for "everything", or `2024*` for keys starting with
+`2024`)
+
+```bash
+openfn collections get
+```
+
+For example, to get everything from `my-collection`, run:
+
+```bash
+openfn collections get my-collection \*
+```
+
+:::tip
+
+In unix shells (MacOS or Linux), the `*` character has special meaning. So if
+you want to get all items, you have to escape it or quote it:
+
+```
+openfn collections get my-collection \*
+```
+
+Including `*` in a pattern string should still work:
+
+```
+openfn collections get my-collection 2024*
+```
+
+:::
+
+Collections are saved as strings, but will be serialized to JSON in the output.
+
+By default the CLI will log downloaded values to your shell. To write to disk,
+pass `--output` or `-o` with a file path relative to your working directory:
+
+```bash
+openfn collections get my-collection \* -o /tmp/my_collection.json
+```
+
+To format the output to make it easier to read, add the `--pretty` flag for
+pretty-printing:
+
+```bash
+openfn collections get my-collection \* -o /tmp/my_collection.json --pretty
+```
+
+It's important to understand that the output works a bit differently if you're
+getting one specific item with a single key or getting many items with a
+key-pattern.
+
+A single key always returns its value "raw" or "verbatim", without the key
+attached. So for a key `item-1` which holds a JSON object as a value, then this:
+
+```bash
+openfn collections get my-collection item-1
+```
+
+Will download and save something like this:
+
+```js
+{
+ "id": "item-1"
+ /* ... other properties of the value */
+}
+```
+
+If you use a key-pattern to retrieve data, the value is output in multi-item
+mode: which is a JSON object where the key is the item's key, and the value is
+the item's value:
+
+So if we get all items whose key starts with `item-`:
+
+```bash
+$ openfn collections get my-collection item-1*
+```
+
+The resulting data will look like this:
+
+```json
+{
+ "item-1": {
+ "id": "item-1"
+ /* ... other properties of the value */
+ },
+ "item-10": {
+ "id": "item-10"
+ /* ... other properties of the value */
+ }
+}
+```
+
+## Uploading items
+
+You can use the `collections` command to upload data to a collection. When
+uploading, values always come from a file on disk. In this example we'll use
+JSON files, but if you're uploading a single value it doesn't have to be valid
+JSON.
+
+The `set` command has two modes. To upload a single item, use:
+
+```bash
+openfn collections set
+```
+
+This will read the data in `path/to/value.json` as a string, and upsert it under
+the provided key. Key patterns are not supported.
+
+To bulk upsert multiple values, use:
+
+```bash
+openfn collections set --items
+```
+
+The `items.json` file must contain a JSON object where the keys are item keys
+and the values are item values (just like the multi-item get command returns):
+
+```json
+{
+ "item-1": {
+ "id": "item-1"
+ /* ... other properties of the value */
+ },
+ "item-10": {
+ "id": "item-10"
+ /* ... other properties of the value */
+ }
+}
+```
+
+:::tip
+
+Remember that Collections always uses an _upsert_ strategy when uploading new
+items.
+
+This means that if a key does not exist, it will be created and assigned a
+value. If it already exists, its value will be updated.
+
+:::
+
+## Removing items
+
+You can also remove items from a collection with the `collections remove`
+command:
+
+```bash
+openfn collections remove
+```
+
+Key-patterns are supported and allow you to remove multiple keys.
+
+Use `--dry-run` to get a list of the keys that would be deleted without actually
+running the delete:
+
+```bash
+openfn collections remove my-collection 2024* --dry-run
+```
diff --git a/docs/build-for-developers/cli-intro.md b/docs/build-for-developers/cli-intro.md
index abd0dc4d64c1..944c7712318a 100644
--- a/docs/build-for-developers/cli-intro.md
+++ b/docs/build-for-developers/cli-intro.md
@@ -56,58 +56,59 @@ openfn test
The word `openfn` will invoke the CLI. The word `test` will invoke the test
command.
-Expand to see the expected output.
-
- [CLI] ♦ Versions:
- ▸ node.js 18.12.1
- ▸ cli 1.0.0
- [CLI] ℹ Running test workflow...
-
- [CLI] ℹ Execution plan:
- [CLI] ℹ {
- "options": {
- "start": "start"
- },
- "workflow": {
- "steps": [
- {
- "id": "start",
- "state": {
- "data": {
- "defaultAnswer": 42
- }
- },
- "expression": "const fn = () => (state) => { console.log('Starting computer...'); return state; }; fn()",
- "next": {
+
+Expand to see the expected output.
+
+```
+[CLI] ♦ Versions:
+ ▸ node.js 18.12.1
+ ▸ cli 1.0.0
+[CLI] ℹ Running test workflow...
+[CLI] ℹ Execution plan:
+[CLI] ℹ {
+ "options": {
+ "start": "start"
+ },
+ "workflow": {
+ "steps": [
+ {
+ "id": "start",
+ "state": {
+ "data": {
+ "defaultAnswer": 42
+ }
+ "expression": "const fn = () => (state) => { console.log('Starting computer...'); return state; }; fn()",
+ "next": {
"calculate": "!state.error"
- }
- },
- {
- "id": "calculate",
- "expression": "const fn = () => (state) => { console.log('Calculating to life, the universe, and everything..'); return state }; fn()",
- "next": {
- "result": true
- }
- },
- {
- "id": "result",
- "expression": "const fn = () => (state) => ({ data: { answer: state.data.answer || state.data.defaultAnswer } }); fn()"
- }
- ]
- }
- }
-
- [CLI] ✔ Compiled all expressions in workflow
- [R/T] ℹ Executing undefined
- [R/T] ℹ Starting step start
- [JOB] ℹ Starting computer...
- [R/T] ✔ Completed step start in 1ms
- [R/T] ℹ Starting step calculate
- [JOB] ℹ Calculating to life, the universe, and everything..
- [R/T] ✔ Completed step calculate in 1ms
- [R/T] ℹ Starting step result
- [R/T] ✔ Completed step result in 0ms
- [CLI] ✔ Result: 42
+ }
+ },
+ {
+ "id": "calculate",
+ "expression": "const fn = () => (state) => { console.log('Calculating to life, the universe, and everything..'); return state }; fn()",
+ "next": {
+ "result": true
+ }
+ },
+ {
+ "id": "result",
+ "expression": "const fn = () => (state) => ({ data: { answer: state.data.answer || state.data.defaultAnswer } }); fn()"
+ }
+ ]
+ }
+}
+
+[CLI] ✔ Compiled all expressions in workflow
+[R/T] ℹ Executing undefined
+[R/T] ℹ Starting step start
+[JOB] ℹ Starting computer...
+[R/T] ✔ Completed step start in 1ms
+[R/T] ℹ Starting step calculate
+[JOB] ℹ Calculating to life, the universe, and everything..
+[R/T] ✔ Completed step calculate in 1ms
+[R/T] ℹ Starting step result
+[R/T] ✔ Completed step result in 0ms
+[CLI] ✔ Result: 42
+```
diff --git a/docs/build/ai-assistant.md b/docs/build/ai-assistant.md
index 2214039d1610..6ab1c62a19a3 100644
--- a/docs/build/ai-assistant.md
+++ b/docs/build/ai-assistant.md
@@ -4,23 +4,34 @@ sidebar_label: AI Assistant
---
The OpenFn AI Assistant provides a chat interface with an AI Model to help you
-build workflows.
+build workflows. **Check out
+[this video](https://www.youtube.com/watch?v=3L_cGl9tWRc&ab_channel=OpenFn.org)
+for an introduction on how it works.**
You can use it to draft, proofread and debug job code, diagnose errors, and
understand the capabilities of the platform.
-
+:::info Build AI-Driven Workflows on OpenFn
-:::caution Assistant Unavailable? Can't find it?
+Check out OpenFn [adaptors](/adaptors) to build OpenFn workflows that
+orchestrate interactions with LLMs (like ChatGPT and Claude) and turn AI-powered
+decision-making into action and automated execution.
-For local deployments of OpenFn, the instance administrator needs to configure the AI
-Assistant before it is available. See
+:::
+
+
+
+:::caution Assistant Unavailable? Can't find it?
+
+For local deployments of OpenFn, the instance administrator needs to configure
+the AI Assistant before it is available. See
[deployment docs](https://github.com/OpenFn/lightning/blob/main/DEPLOYMENT.md#ai-chat)
here for help or contact the super user of your instance.
The Assistant is available on app.openfn.org, with usage credits subject your
-project's plan. See [openfn.org/pricing](https://www.openfn.org/pricing) or contact [support@openfn.org](mailto:support@openfn.org)
-for more information on paid plans for usage of OpenFn's cloud-hosted platform.
+project's plan. See [openfn.org/pricing](https://www.openfn.org/pricing) or
+contact [support@openfn.org](mailto:support@openfn.org) for more information on
+paid plans for usage of OpenFn's cloud-hosted platform.
:::
@@ -36,20 +47,20 @@ versions of the assistant may be rolled out to other pages.
All chat sessions are shared between all users of the project. You can start a
new chat session at any time, or open an old one.
-Input data and run-time job logs not sent to the model. But we do include
-your step code so that the Assistant can provide a contextually relevant answer.
+Input data and run-time job logs not sent to the model. But we do include your
+step code so that the Assistant can provide a contextually relevant answer.
## A Note on Responsible AI Usage
The AI assistant is built on emerging Large Language Model (terminology). Like
-other LLMS and chatbots, its capabilities are impressive, but imperfect.
+other LLMs and chatbots, its capabilities are impressive, but imperfect.
Remember that ultimately, all responses are generated by an algorithm and YOU,
the human in charge, are responsible for how its output is used. You should
consider all responses critically and verify the output where possible.
-You can read more about our approach to AI in our
-[Responsible AI Policy](https://www.openfn.org/ai).
+**You can read more about our approach to AI in our
+[Responsible AI Policy](https://www.openfn.org/ai).**
## How to access the AI Assistant
@@ -68,9 +79,10 @@ You can close a chat session by clicking the `(X)` button on the top right of
the chat interface, which will take you back to the list of sessions for this
step.
-:::caution Feedback or Questions about the Assistant?
+:::caution Feedback or Questions about the Assistant?
-We welcome your questions or feedback on [community.openfn.org](https://community.openfn.org/),
-or contact [support@openfn.org](mailto:support@openfn.org) with private queries.
+We welcome your questions or feedback on
+[community.openfn.org](https://community.openfn.org/), or contact
+[support@openfn.org](mailto:support@openfn.org) with private queries.
:::
diff --git a/docs/build/collections.md b/docs/build/collections.md
index 9de9bce3335b..b4ab95677690 100644
--- a/docs/build/collections.md
+++ b/docs/build/collections.md
@@ -4,7 +4,9 @@ sidebar_label: Collections
---
Collections provides a high-volume, high-performance storage solution built into
-OpenFn.
+OpenFn. Check out
+**[this video](https://www.youtube.com/watch?v=iXkkkzratzY&t=3s&ab_channel=OpenFn.org)**
+for an introduction.
Collections is suitable for buffering, caching and aggregating data from
Webhooks, storing large mapping files, and sharing state between workflows.
@@ -55,10 +57,20 @@ uploaded to a collection using the CLI.
## Collections Basics
-Data is stored in Collections as key-value pairs, where the key is a unique
-identifier for some data (like a UUID, or timestamp). The value is always a
-string - although JSON objects will be automatically serialized to a string
-using the Collections API.
+:::tip
+
+The Collections API is automatically available to all Workflows and does not
+require any credentials. Authentication with the OpenFn platform is managed for
+you.
+
+You can use the Collections API with any adaptor.
+
+:::
+
+Data is stored as key-value pairs, where the key is a unique identifier for some
+data (like a UUID, or timestamp). The value is always saved as a string
+(although you can pass JSON-compatible objects directly, which will be
+automatically serialized by the Collections API).
Keys can be fetched in bulk and filtered by _pattern_. For example, the pattern
`2024*` will match all keys which start with `2024`. Designing keys to have an
@@ -74,6 +86,36 @@ collections.get('openfn-patient-registrations', '2024*').then(state => {
});
```
+Returned items are written to state.data as an array of `[{ key, value }]`
+pairs:
+
+```js
+{
+ "data": {
+ "20240102-5901257": {
+ "name": "Tom Waits",
+ "id": "5901257",
+ },
+ "20240213-0183216": {
+ "name": "Billie Holiday",
+ "id": "0183216",
+ }
+ }
+}
+```
+
+If fetching a single item (i.e. no `*` in the key), it will be written directly
+to `state.data` with no key:
+
+```js
+{
+ "data": {
+ "name": "Billie Holiday",
+ "id": "0183216",
+ }
+}
+```
+
Every key permanently saves its creation date, so as well as fetching by
key-pattern, you can also filter keys by date. This example fetches all keys
created before 30th September 2024:
@@ -102,7 +144,11 @@ collections.each(
);
```
-New values are uploaded to a collection through `collections.set`:
+Values are uploaded to a collection through `collections.set`. All sets are
+"upserts" - meaning that new keys will be created for values that don't exist,
+and values will be updated for keys that _do_ exist.
+
+The example below sets a single item:
```js
collections.set('openfn-demo', 'commcare-fhir-value-mappings', {
@@ -115,11 +161,21 @@ collections.set('openfn-demo', 'commcare-fhir-value-mappings', {
});
```
+If setting multiple values at once, pass a key generator function instead of an
+id to generate a key for each item. For example, if several value are saved in
+an array on `state.data`:
+
+```js
+collections.set('openfn-demo', (patient, state) => patient.id, $.patients);
+```
+
+The key generator will be called with each value and must return a string key.
+
## Managing Collections
Collections can be created, destroyed or renamed from the Admin menu.
-
+
Before it can be used, a collection must be created. Collection names must be
unique to the deployment, so we recommend using your organisation (and maybe
diff --git a/docs/build/credentials.md b/docs/build/credentials.md
index 29eb0e5bba00..328b37160b20 100644
--- a/docs/build/credentials.md
+++ b/docs/build/credentials.md
@@ -10,7 +10,7 @@ A Credentials' values can only be viewed or edited by a single user — their "o
(the user that created that credential). All the collaborators on a Project can
choose from all credentials for the Project when defining a job.
-
+
### Create a new Credential
@@ -77,13 +77,13 @@ Users of the hosted OpenFn platform SaaS can post on
Note the credential selects only required scopes for Google Sheets.
-
+
#### e.g., Salesforce OAuth Credential
Note that you can choose which scopes to access on Salesforce.
-
+
:::tip
diff --git a/docs/build/editing-locally.md b/docs/build/editing-locally.md
index 416176471668..d4c9dbf95587 100644
--- a/docs/build/editing-locally.md
+++ b/docs/build/editing-locally.md
@@ -19,7 +19,7 @@ machine:
2. Clone the repo from GitHub. Depending on how you're connecting, grab the
HTTPS or SSH URL of the repository.
-
+
:::tip
@@ -45,14 +45,14 @@ for more info.)
6. To edit your steps, use a code editor. We recommend
[Visual Studio Code](https://code.visualstudio.com/download).
-
+
7. When using VS Code, make sure you install the
[Prettier VSCode Extension](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode)
and set it as the default formatter in Settings as seen below. This will apply
the correct code formatting to the files you change.
-
+
8. Once you're done, you can check which files you changed with `git status`.
diff --git a/docs/build/limits.md b/docs/build/limits.md
index c35bf2ab1977..7deac3fd863b 100644
--- a/docs/build/limits.md
+++ b/docs/build/limits.md
@@ -2,14 +2,38 @@
title: Limits
---
-OpenFn has a number of configurable limits that help ensure smooth operation.
-This page details what those limits are and how they can be adjusted.
-
-:::tip Increasing limits
-
-Contact enterprise@openfn.org to increase limits on standard plans, configure
-custom limits on dedicated deployments, or get help adjusting limits on your
-locally deployment instance.
+OpenFn cloud hosted instance has a number of limits that help ensure smooth
+operation. The table below shows the limits for different plans. For a more
+detailed list of limits, see the
+[OpenFn pricing page](https://openfn.org/pricing). For self hosted instances,
+these limits are configurable. See the
+[deployment guide](https://openfn.github.io/lightning/deployment.html#limits)
+for more details.
+
+| Feature | Description | DPG | Free | Core | Growth | Scale | Unlimited |
+| --------------------------- | ------------------------------------------------------------ | ------------ | ------- | ------ | ------- | --------- | --------- |
+| Runs | Maximum number of runs allowed per month | Unlimited | 100 | 2,000 | 5,000 | 10,000 | Unlimited |
+| Workflow Execution Duration | Maximum time a workflow can run before being killed | Configurable | 60 secs | 5 mins | 20 mins | 30 mins | 30 mins |
+| Memory Usage | Maximum memory allowed per workflow attempt | Configurable | 128MB | 256MB | 512GB | 1GB | 1GB |
+| Dataclip Size | Maximum size for data clips persisted run states | Configurable | 512KB | 2MB | 10MB | 10MB | 10MB |
+| AI Assistant | Maximum AI tokens available | Configurable | 500K | 1.5M | 5M | 10M | 10M |
+| Data Collections (Storage) | Maximum storage for data collections | Configurable | 1MB | 5MB | 10MB | 50MB | 50MB |
+| Data Collections (Number) | Maximum number of data collections per project | Configurable | 2 | 5 | 10 | Unlimited | Unlimited |
+| Concurrency Control | Allowing users to control concurrency limits for the project | Configurable | Yes | Yes | Yes | Yes | Yes |
+
+
+
+:::tip Increasing limits for cloud hosted and managed instances
+
+For standard plans, you can increase your limits by upgrading to a higher plan
+by following the
+[upgrade plan instructions](/documentation/hosted/overview#upgrading-your-subscription).
+
+For custom limits or upgrades in dedicated deployments, contact
+enterprise@openfn.org.
:::
@@ -23,9 +47,9 @@ badge as your attempt state.
> _Instance superusers can control this limit the `MAX_RUN_DURATION` environment
> variable._
-## Memory Usage (500 MB)
+## Memory Usage (1GB)
-Each workflow attempt may not use more than `500 MB` of memory. You can view the
+Each workflow attempt may not use more than `1GB` of memory. You can view the
maximum memory usage of each attempt by clicking on the attempt ID. If an
attempt exceeds this limit, it will be killed by the worker and you'll see a
`Killed:OOM` badge as your attempt state.
diff --git a/docs/build/paths.md b/docs/build/paths.md
index 58b70813c961..4724101ca33a 100644
--- a/docs/build/paths.md
+++ b/docs/build/paths.md
@@ -21,7 +21,7 @@ proceed to the next Step when executed:
4. **Matches a JavaScript Expression**: the next Step will only run if an
expression evaluates to be true
-
+
## Writing JavaScript Expressions for Custom Path Conditions
@@ -36,7 +36,7 @@ This is a regular Javsacript expression with `state` in scope. If the expression
evaluates to true (or anything _truthy_), the Path will be followed and the next
Step will be executed.
-
+
Examples of valid conditions include:
diff --git a/docs/build/steps/step-design-intro.md b/docs/build/steps/step-design-intro.md
index 67cd9277bed0..4eca012be4e4 100644
--- a/docs/build/steps/step-design-intro.md
+++ b/docs/build/steps/step-design-intro.md
@@ -17,7 +17,7 @@ In short, to design a Workflow Step, you will need to follow the below list of
actions, and consider summarizing your design specifications in a
[workflow diagram](/documentation/design/design-workflow).
-
+
## 1. Determine your Inputs/Outputs
@@ -37,7 +37,7 @@ To get started:
source app (input) & destination app (output).
2. Paste the metadata into an Excel spreadsheet to create a mapping sheet:
-
+
3. Map the source and destinationdata elements & define rules for data cleaning
and transformation. Consider:
diff --git a/docs/build/steps/step-editor.md b/docs/build/steps/step-editor.md
index 02ffa8c0220c..99863dff01bd 100644
--- a/docs/build/steps/step-editor.md
+++ b/docs/build/steps/step-editor.md
@@ -8,7 +8,9 @@ Inspector interface.
:::tip
-If you're writing jobs on the platform app (Lightning), you can use the [AI Assistant](/documentation/build/ai-assistant) to help you. You'll find it in the Inspector.
+If you're writing jobs on the platform app (Lightning), you can use the
+[AI Assistant](/documentation/build/ai-assistant) to help you. You'll find it in
+the Inspector.
:::
@@ -49,3 +51,9 @@ include `Logs` and an `Output`.
See [Writing Jobs docs](/documentation/jobs/job-writing-guide) for more on
writing custom logic, and see [this article](/documentation/jobs/state) for more
on the concept of "state" when writing Jobs and building OpenFn Workflows.
+
+## Keyboard Shortcuts
+
+From the inspector you can perform certain common actions (e.g., save, run, sync
+to github) using keystrokes. Check out the full list of keyboard shortcuts
+[here](/documentation/keyboard-shortcuts).
diff --git a/docs/build/steps/steps.md b/docs/build/steps/steps.md
index 18b48ae00152..07d30fabaaba 100644
--- a/docs/build/steps/steps.md
+++ b/docs/build/steps/steps.md
@@ -23,7 +23,7 @@ click on an existing Step to view or configure its key components.
To properly configure a Step, you must understand its basic anatomy.
-
+
A Step includes these key components:
diff --git a/docs/build/triggers.md b/docs/build/triggers.md
index 25185f3f721f..ac20d4fbd134 100644
--- a/docs/build/triggers.md
+++ b/docs/build/triggers.md
@@ -16,7 +16,7 @@ These triggers are fired by "pushing" data to OpenFn (i.e., by sending an HTTP
The triggering HTTP request might be sent via a webhook in an external app,
another OpenFn workflow, or manually (i.e., via cURL request).
-
+
To learn about how to add an additional layer of security to your Webhook
Trigger by adding authentication, head over to our
@@ -35,7 +35,7 @@ These Triggers enable users to “pull” data from connected systems. You can p
a standard schedule (e.g., every day, or every month), or define a custom
schedule using cron expressions.
-
+
:::tip Help with cron expressions
@@ -91,7 +91,7 @@ fn(state => {
});
```
-## Kafka Triggers
+## Kafka Triggers (beta) 🚧
The Kafka trigger allows OpenFn users to build Workflows triggered by messages
published by a Kafka cluster. The triggers make use of Kafka consumer groups
@@ -101,11 +101,11 @@ converts them to `Input` dataclips that are used to initialize a Work Order.
:::info For self-hosted OpenFn deployments
Instance administrators have to enable Kafka for their instance by setting
- `KAFKA_TRIGGERS_ENABLED=yes` in the environment variable.
+`KAFKA_TRIGGERS_ENABLED=yes` in the environment variable.
:::
-
+
:::info What is Kafka?
@@ -157,3 +157,32 @@ configuration modal.**
Learn how the initial `state` (and `Input`) for Kafka-triggered Workflows gets
built [here](../jobs/state#kafka-triggered-runs).
+
+### Known "sharp edges" on the Kafka trigger feature
+
+We'll monitor bug/exception reports, perform user interviews and collect feature
+requests during the beta to determine which of these rough-edges are worth
+ironing out, and how to do so assuming that the community wants to support Kafka
+triggers going forward. Please don't hesitate to reach out on
+[community.openfn.org](https://community.openfn.org) to make your voice heard!
+
+1. Performance settings are out of the _end-user's_ control and can only be set
+ at instance-level. As there is quite a close relationship between cluster and
+ consumer settings, this may prove to be an obstacle as users will not be able
+ to tune their consumers to align with their individual clusters in large
+ multi-tenant deployments.
+2. If a run could not be turned into a Workflow (due to persistence errors or
+ hitting the hard limit), these will not be visible to the end-user and may be
+ lost forever (i.e. the cluster thinks Lightning has them but it doesn't and
+ they eventually rotate off the cluster)
+3. Errors are written to the log and to Sentry; nothing is visible to the
+ end-user.
+4. Reprocessing dropped messages isn't practical if you are not writing failed
+ messages to some sort of persistent file storage.
+5. If a consumer group is disconnected from a cluster long enough for the
+ cluster to forget the last message, we may not be able to prevent duplicates
+ making it through (as a result of instance-wide de-duplication settings).
+6. We cannot provide concurrency _and_ honour message sequence that is based on
+ the Kafka message key mechanism. If a user wants to guarantee the Kafka
+ message sequence, they must enable 1-at-a-time processing on OpenFn by
+ turning their workflow concurrency down to 1.
diff --git a/docs/build/workflow-snapshots.md b/docs/build/workflow-snapshots.md
index a771df983809..81e8923869a2 100644
--- a/docs/build/workflow-snapshots.md
+++ b/docs/build/workflow-snapshots.md
@@ -14,7 +14,7 @@ Snapshots are created in 2 ways:
### How can I view a snapshot?
To view a snapshot, go to the `History` page. Expand a work order to see the runs included.
-
+
From the expanded history view, there are two ways to visualize snapshots:
1. By inspecting a step in the run
@@ -24,22 +24,22 @@ From the expanded history view, there are two ways to visualize snapshots:
Click on the inspect icon in front of the step you would like to view.
-
+
This will open the [inspector screen](../build/steps/step-editor.md) for that step in the run with all associated artifacts: logs and input/output data. On the inspector, you'll notice that you're in a read-only mode, and hovering on the workflow snapshot ID chip, you’ll see a message that reads “You are viewing a snapshot of this workflow that was taken on ….”
-
+
To view the corresponding canvas for this snapshot, close this inspector view by clicking the `X` on the top right corner of the page. This will open the associated canvas with the step selected as shown below.
-
+
From the canvas, you can inspect any step by clicking on the step and opening the inspector for the run associated with the step and snapshot.
#### Viewing a snapshot from the Run view
From the expanded history view, click on the run ID to open the run view.
-
+
From this view, click on the Workflow name (Simple Flow) to open the workflow canvas for this snapshot.
Similar to viewing a snapshot by inspecting a step, you can click the inspect icon in front of the steps to open the inspector for the step.
@@ -49,9 +49,9 @@ Snapshots are read-only and serve as a reference for the state of a workflow whe
When you switch to the latest version, the snapshot ID tag will change to blue colour and the text will be `latest`.
-
+
-
+
### Retrying a Snapshot
When retrying a run with a snapshot, the retry will be executed with the latest version of the workflow and job code. You cannot retry a workflow with an earlier snapshot, only with the latest version.
diff --git a/docs/build/workflows-api.md b/docs/build/workflows-api.md
new file mode 100644
index 000000000000..618db1dd9c02
--- /dev/null
+++ b/docs/build/workflows-api.md
@@ -0,0 +1,172 @@
+---
+title: Workflows API
+sidebar_label: Workflows API
+---
+
+The Workflows API allows you to programmatically create and modify Workflows.
+
+You can use the Workflows API with the `http` adaptor or curl.
+
+:::info Version Compatibility
+
+The Workflows API was introduced in version 2.10.10, January 2025
+
+:::
+
+## Authentication
+
+All requests must be Authenticated.
+
+Authentication uses the Authorization header with a Personal Access Token (PAT)
+from the app.
+
+If you're using the http adaptor, set the `access_token` on the credential to
+your PAT.
+
+If you are using curl, add the bearer token (in the example below the token
+will be expanded from an env var):
+
+```
+curl -H "Authorization: Bearer $OPENFN_PAT" https://app.openfn.org/api/projects//workflows
+```
+
+## REST API
+
+The Workflow API has the following RESTful structure:
+
+- `GET /api/projects/:projectId/workflows` - get a list of workflows for a
+ project. Returns an array of Workflows.
+- `GET /api/projects/:projectId/workflows/:workflowId` - get a single workflow
+ by id. Returns a single Workflow.
+- `PUT /api/projects/:projectId/workflows` - create a new Workflow. Include
+ workflow JSON in the body. Returns the updated Workflow JSON.
+- `PUT /api/projects/:projectId/workflows/:workflowId` - update a Workflow. This
+ will replace the existing workflow with the JSON in the body. Returns the
+ updated Workflow JSON.
+- `PATCH /api/projects/:projectId/workflows/:workflowId` - partially update a
+ Workflow. The existing workflow will be updated with the JSON in the body.
+
+## Workflow Structure
+
+A Workflow has the following structure:
+
+```
+{
+ "name": "My Workflow",
+ "id": "a414cb3b-e387-4c4f-b8de-70d51f1160da",
+ "project_id": "79efba60-072a-4d4f-8d6c-22dfd3852176",
+ "edges": [
+ {
+ "id": "759fe475-ed23-4914-8a6d-155968bc0aa1"
+ "condition_type": "always",
+ "enabled": true,
+ "source_job_id": null,
+ "source_trigger_id": "c79ce46c-ab0f-4f5b-bf2d-fed52aef2a41",
+ "target_job_id": "26304a1e-267b-4bc9-940f-171db1905885",
+ }
+ ],
+ "jobs": [
+ {
+ "id": "26304a1e-267b-4bc9-940f-171db1905885",
+ "body": "/* job code goes here */",
+ "name": "my-job",
+ "adaptor": "@openfn/language-common@latest",
+ }
+ ],
+ "triggers": [
+ {
+ "id": "c79ce46c-ab0f-4f5b-bf2d-fed52aef2a41",
+ "comment": null,
+ "custom_path": null,
+ "cron_expression": null,
+ "type": "webhook",
+ "enabled": true
+ }
+ ],
+}
+```
+
+When creating a new Workflow, the server will generate UUIDs for the workflow
+and all steps and edges. You can use any id string you like in the creation of
+new nodes and edges - so long as id usage is consistent.
+
+When matching a PUT or PATCH request, new steps and edges MUST be assigned
+UUIDs. If using the `http` adaptor, you can use `util.uuid()` for this (see the
+example below).
+
+You MUST ensure that any steps and triggers referenced by an edge are defined
+within the same workflow.
+
+## HTTP Adaptor Examples
+
+You will need to create a credential with `access_token` set to your Personal
+Access Token (PAT) and `baseUrl` set to your OpenFn instance (ie,
+`"https://app.openfn.org"`)
+
+Create new Workflow:
+
+```js
+post(`/api/projects/${$.projectId}/workflows`, {
+ body: {
+ name: 'My Workflow',
+ edges: [
+ {
+ source_trigger_id: 'trigger-1',
+ target_job_id: 'job-1',
+ condition_type: 'always',
+ },
+ ],
+ jobs: [
+ {
+ id: 'job-1',
+ name: 'My Job',
+ body: '/* job code goes here */',
+ adaptor: '@openfn/language-common@latest',
+ },
+ ],
+ triggers: [
+ {
+ id: 'trigger-1',
+ type: 'webhook',
+ enabled: true,
+ },
+ ],
+ },
+ headers: { 'content-type': 'application/json' },
+});
+```
+
+The resulting workflow with updated UUIDs and metadata will be written to
+`state.data.workflow`.
+
+Add a new step and edge to an existing Workflow:
+
+```js
+fn(state => {
+ const jobId = util.uuid();
+ state.diff = {
+ edges: [
+ {
+ id: util.uuid(),
+ source_job_id: 'c79ce46c-ab0f-4f5b-bf2d-fed52aef2a41',
+ target_job_id: jobId,
+ condition_type: 'always',
+ },
+ ],
+ jobs: [
+ {
+ id: jobId,
+ body: '/* job code goes here */',
+ adaptor: '@openfn/language-common@latest',
+ },
+ ],
+ };
+ return state;
+});
+patch(`/api/projects/${$.projectId}/workflows/${$.workflowId}`, {
+ body: $.diff,
+ headers: { 'content-type': 'application/json' },
+});
+```
+
+The resulting workflow will be written to `state.data.workflow`.
diff --git a/docs/build/workflows.md b/docs/build/workflows.md
index a65cd42d1fa1..f22bd14d61dd 100644
--- a/docs/build/workflows.md
+++ b/docs/build/workflows.md
@@ -27,44 +27,123 @@ Check out the video overview below to learn how to create a Workflow.
### Merging branches and Skipping Steps
-The workflow builder allows branch merging and skipping steps. To merge two or more steps into one step or to skip some steps:
+The workflow builder allows branch merging and skipping steps. To merge two or
+more steps into one step or to skip some steps:
1. Hover on the step you want to merge or initiate a skip
-2. You will see a link 🔗 icon beside the new step icon.
-3. Click on the link icon and drag to create a path
+2. You will see a plus icon
+3. Click on the plus icon and drag to create a path
4. Drop the new path on the desired step in your workflow
-
+
+
+:::note Looping is not supported
+
+Looping workflows are not supported so you have to connect paths to downstream
+steps. When using branching and skipping paths, you can use edge conditions like
+with any other step.
-:::note
-Looping workflows are not supported so you have to connect paths to downstream steps.
-When using branching and skipping paths, you can use edge conditions like with any other step.
:::
## Run Workflows
-To run a Workflow, you can either activate the Trigger (e.g., send a request to
-the Webhook Event Trigger's URL, or wait for the cron timer to be activated), or
-run your workflow manually. Check out the video below for how to run your
-workflow manually.
+Workflows will run automatically when they are "enabled"—i.e., when their
+trigger is turned on. A webhook trigger will run your workflow whenever a
+request is received at that trigger's URL, and a cron trigger will run a
+workflow whenever its cron schedule matches the current time.
+
+:::info
+
+Please note that workflows are disabled by default. When you are ready to have
+your workflow running, you have to manually enable the workflow.
+
+:::
+
+### Enabling or Disabling a Workflow
+
+There are two ways to disable or enable a workflow in your OpenFn project:
+
+1. via the workflow state toggle
+2. via the workflow trigger
+
+#### Via the workflow state toggle
+
+You can enable or disable your workflow by using the toggle button located on
+the corresponding row in the project workflows list or the toggle on the
+navigation bar in the workflow canvas.
+
+The screenshot below shows an enabled workflow in the workflow list.
+
+
+
+The screenshot below shows a disabled workflow in the workflow canvas.
+
+
+
+#### Via the workflow trigger
+
+To enable or disable a workflow via the workflow trigger, select the trigger
+icon on the canvas and use the toggle in the configuration panel to toggle the
+workflow state.
+
+
+
+### Manual Runs
+
+Check out the video for a quick overview.
+
+
+
+You can run a workflow manually in three ways:
-
+#### With an empty input
-## Turn off or disable Workflows
+This is the default behavior and the input dataclip for your run will be `{}`.
-To "turn off" or disable a Workflow:
+
-1. Open the Workflow
-2. Click on the Trigger
-3. Select the **Disable this trigger** checkbox
-4. Select **Save** to save your changes and turn off / disable your workflow
+#### With a custom input
-
+You can type, copy/paste, or import (browse your file system or drag & drop) any
+file with valid JSON.
+
+
+
+#### With an existing input
+
+You can pick from a list of previous inputs that were used to run this step.
+
+
+
+### Named Dataclips
+
+Dataclips (custom inputs, step results, webhook requests) can be named to make
+them easier to find and use for testing.
+
+:::info Named dataclips aren't erased
+
+Named dataclips will not be removed alongside other project history when your
+retention period is reached. They will be stored indefinitely.
+
+:::
+
+Assign your dataclip a name by clicking the label field.
+
+
+
+After assigning names to your inputs you can search for them by their name on
+the search bar.
+
+
+
+Filter only named inputs by clicking the tag button.
+
+
## Limit Concurrency
-Workflow **concurrency** is the number pf runs will be allowed for a given
-workflow **_at the same time_**. In OpenFn, project administrators and editors
+Workflow **concurrency** is the number of runs that will be allowed for a given
+workflow **_at the same time_**. In OpenFn, project owners and administrators
are able to limit the maximum number of the runs that can be executed at the
same time for a workflow. You might do this to ensure "one at a time" serial
processing or to keep a fast OpenFn workflow from overwhelming the API rate
@@ -72,12 +151,8 @@ limit of some other connected system.
:::info
-Note that this setting allows administrators to _limit_ the maximum concurrency
-for a workflow, but the **global maximum** (i.e., the highest concurrency that
-can be reached if a workflow is _not_ limited by a project administrator) will
-be controlled by your OpenFn instance superuser. This global maximum will be
-determined by the computing power and throughput made available to your
-installation.
+Please check to make sure that your parallel execution is not disabled for your
+project as it will override the workflow level concurrency limit.
:::
@@ -107,4 +182,22 @@ canvas.
2. In the modal, enter the maximum concurrency limit
3. Click save.
-
+
+
+### Log Outputs
+
+For data security and compliance purposes, the log output of a workflow run can
+be configured to disable logging `console.log()` statements. This can be done
+via the workflow configuration modal by a project owner or administrator.
+
+1. Click on the settings icon.
+2. In the modal, toggle the **Allow `console.log()` usage** switch to disable
+ logging `console.log()` statements. By default, this is enabled.
+
+
+
+## Keyboard Shortcuts
+
+From the canvas you can perform certain common actions (e.g., save) using
+keystrokes. Check out the full list of keyboard shortcuts
+[here](/documentation/keyboard-shortcuts).
diff --git a/docs/build/working-with-branches.md b/docs/build/working-with-branches.md
index 80491624b056..b11814e219f3 100644
--- a/docs/build/working-with-branches.md
+++ b/docs/build/working-with-branches.md
@@ -44,14 +44,14 @@ repo to your local folder.
6. On GitHub, you can create a Pull Request to get your changes reviewed and
approved.
- 
+ 
- 
+ 
7. As you keep working with branches, make sure you check which branch you're on
with `git status`.
-
+
8. To keep your local copy up to date with the remote repo, switch to `main`
with `git checkout main` and hit `git pull` to pull any changes.
diff --git a/docs/contribute/impact-tracker.md b/docs/contribute/impact-tracker.md
new file mode 100644
index 000000000000..a99a0cad223f
--- /dev/null
+++ b/docs/contribute/impact-tracker.md
@@ -0,0 +1,76 @@
+---
+title: Impact Tracker
+id: impact
+---
+
+## Introduction
+
+OpenFn is a free and open-source Digital Public Good. Many users are unable to
+contribute financially or by participating in our product development community,
+but by sending these nightly anonymous aggregate usage reports they ensure the
+long-term sustainability of the project by:
+
+1. allowing us to understand the needs of our users,
+2. by better demonstrating our impact,
+3. and by helping us secure continued donor support.
+
+:::success Anonymous Impact Tracker
+
+Head to [openfn.org/impact](https://www.openfn.org/impact) to see it action.
+
+:::
+
+How does this work? These metrics ([see below](#the-data-yes-all-of-it)) are
+anonymous and submitted by instance operators around the world. When someone
+launches OpenFn, the first thing they see is a message like the one below,
+explaining exactly what aggregate anonymous data they're sending and where
+they're sending it.
+
+Per the installation instructions, instance admins can opt-out of metrics
+reporting at any time via the `USAGE_TRACKING_ENABLED` environment variable, but
+most prefer to contribute!
+
+## The data. (Yes, all of it.)
+
+If a metrics client admin sends anonymous usage data to any instance of a
+metrics server, this is what will be sent:
+
+```json
+{
+ "version": "2",
+ "instance": {
+ "version": "v2.4.2:match:f1bd9ae",
+ "hashed_uuid": "4CE189B993247E94FD2A9EDD28CEC9C9D5A7125AB85F4586A6C994D89DCC0979",
+ "no_of_users": 137,
+ "operating_system": "linux",
+ "no_of_active_users": 70
+ },
+ "projects": [
+ {
+ "workflows": [
+ {
+ "no_of_jobs": 1,
+ "no_of_runs": 6,
+ "hashed_uuid": "C08DD42A9DF75A017001429240D0E6C425BA89AF03C134A05E631CDB0A53FA87",
+ "no_of_steps": 6,
+ "no_of_active_jobs": 1
+ },
+ {
+ "no_of_jobs": 4,
+ "no_of_runs": 6,
+ "hashed_uuid": "BD70CCCA4D953D1B59F7803CC24A4EE84CFD21DE4F46CBA85FB3FA41AACA4EAD",
+ "no_of_steps": 24,
+ "no_of_active_jobs": 4
+ },
+ ... more workflows
+ ],
+ "hashed_uuid": "71A5B39B570E1E9156B73997C327E5A2FABD06507CE3FEBF85128016446FCD49",
+ "no_of_users": 6,
+ "no_of_active_users": 6
+ },
+ ... more projects
+ ],
+ "report_date": "2024-04-25",
+ "generated_at": "2024-04-26T01:30:00.876776Z"
+}
+```
diff --git a/docs/contribute/openfn-roadmap.md b/docs/contribute/openfn-roadmap.md
deleted file mode 100644
index e2b49175d205..000000000000
--- a/docs/contribute/openfn-roadmap.md
+++ /dev/null
@@ -1,174 +0,0 @@
----
-title: Roadmap and Product Ethos
-sidebar_label: Roadmap
----
-
-## Introduction
-
-This page details the planned roadmaps for the key products in the OpenFn
-product suite, including Lightning, Adaptors, and this Docs site. While this
-page will be updated periodically, you can track our realtime roadmap and
-progress [here](#what-are-we-currently-working-on).
-
-## Our approach to product development
-
-At OpenFn, we follow the [Shape Up approach](https://basecamp.com/shapeup) to
-help our small engineering team build _meaningul_ products and features faster
-without compomising on quality. With Shape Up in place, we typcially commit to
-_projects_ that can be delivered in a 4-6 week period with multiple releases
-based on QA approval within the building cycle. We also proiritize feedback and
-feature requests from our users over features in the backlog.
-
-:::success Transparency and building what matters
-
-Note that it's fairly rare for us to commit to delivering a specific feature
-more than 2 or 3 months in the future. We are committed to constantly evluating
-what our user base needs most and spending the few resources we have to deliver
-value to them—we simply can't guarantee that what sounds like the "7th most
-important feature to build" now will still be on our list in 6 months.
-
-At the same time, we strive to be as transparent and inclusive as we can in our
-planning processes. We have a big backlog of feature requests and GitHub issues
-(bug reports, stubs, even partially shaped epics or projects) that are getting
-voted up, commented on, and used as inspiration when we're deciding what to
-prioritize next.
-
-Read on to learn more about how we work, how you can see what's coming, and how
-you can get involved!
-
-:::
-
-## What are we currently working on?
-
-All of our team's work is tracked publicly using a GitHub Project. Three key
-views give you up-to-the-minute insights on what we're doing and what's on our
-immediate roadmap.
-
-### See [**_Now_**](https://github.com/orgs/OpenFn/projects/3/views/1) 🚧 for what's been funded and is currently being built
-
-### See [**_Next_**](https://github.com/orgs/OpenFn/projects/3/views/2) 📝 for what's being shaped and _may_ be funded in the next cycle
-
-### See [**_Epics_**](https://github.com/orgs/OpenFn/projects/3/views/7) 🤔 for a list of projects that we're considering, roughly-prioritized
-
-### See [**_Bugs_**](https://github.com/orgs/OpenFn/projects/3/views/22) 🐞 for reported bugs
-
-We will update this site periodically (ideally after each cycle, typically
-lasting between 2-6 weeks) to reflect our progress on major items. You can also
-keep track of all new features, changes, and bug fixes in our
-[Changelog](https://openfn.github.io/lightning/changelog.html).
-
-## How to **get involved**
-
-We use [Canny](https://openfn.canny.io/feature-requests) to receive, track,
-engage and manage new feature requests from the community of users of OpenFn
-globally whilst giving users the ability to the upvote their favoritie and
-mission critical feature request.
-
-### Upvote features 👍
-
-1. Go to [openfn.canny.io](https://openfn.canny.io/feature-requests)
-2. Scroll down or use the filter and search features to see existing feature
- requests
-3. Click on the (^) beside the request to upvote
-4. If you want more upvotes for this feature request, share a link to the
- feature with your network
-
-### Request a new feature 💡
-
-1. Go to [openfn.canny.io](https://openfn.canny.io/feature-requests)
-2. Provide a very clear, concise and descriptive title for the feature (e.g.,
- "Make the new workflow button green")
-3. Describe the feature request in detail and why it's important to you
-4. Share the feature request on the OpenFn community and across your
- professional network for upvotes
-
-:::info Tip
-
-When describing the feature, it is very helpful to help us understand the
-problem, proposed solution (if any) and similar solutions we might glean
-insights from _if they exist_.
-
-:::
-
-### Open an issue or bug directly
-
-If you prefer the direct approach, you can search across all tracked issues in
-OpenFn's GitHub org [here](https://github.com/OpenFn), comment on them, or even
-pick them up to work on yourself. If you don't find what you're looking for,
-please go ahead an create an issue in the relevant repo. We'll do our best to
-respond promptly!
-
-## Summary Roadmaps
-
-#### How to interpret `Status` values in the roadmap
-
-- `Tracked` means we're thinking about this, but it hasn't been
- designed/scoped/shaped
-- `Shaped` means it's been scoped and ready to be picked up by an engineer
-- `In dev` means it's currently being worked on by an engineer (see
- [_NOW_](#what-are-we-currently-working-on))
-- `Delivered` means it's been released (see
- [Changelog](https://openfn.github.io/lightning/changelog.html))
-
-### Lightning Roadmap
-
-OpenFn/Lightning is the fully open-source workflow automation platform at the
-core of the OpenFn Digital Public Good (learn more about the product
-[here](/documentation#openfn-v2-lightning-)).
-
-| **Feature** | **`Status`** | **Target Timeline** | **Related Links** | **Description** |
-| --------------------------------------------------------------------- | ------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| 1. Enable workflow snapshotting | Delivered | Q3 '24 | [Issue 1680](https://github.com/OpenFn/lightning/issues/1680) | Keep a snapshot of a snapshot based on a run or saved changes of the workflow. Allow users to be able to view snapshot and switch to the latest version of the workflow from a snapshot mode. |
-| 2. Enable configurable concurrency by workflow | Delivered | Q3 '24 | [Issue 2002](https://github.com/OpenFn/lightning/issues/2022) | Allow users to control the limit of concurrent runs per workflow in a project. |
-| 3. New workflow triggers (Kafka) | Delivered | Q3 '24 | [Issue 1801](https://github.com/OpenFn/lightning/issues/1801) | Enable a new trigger type that allows users to run workflows based on messages from a Kafka cluster. |
-| 4. AI-enabled assitants | Delivered | Q3 '24 | [Issue 2193](https://github.com/OpenFn/lightning/issues/2193) | LLM based AI assitant that supports users with job writing, debugging and co-piloting their workflow design process. |
-| 5. Invite new users as collaborators | Delivered | Q3 '24 | [Issue 1835](https://github.com/OpenFn/lightning/issues/1835) | Invite users who do not have OpenFn accounts to projects as collaborators. |
-| 6. Allow users to create projects | Delivered | Q3 '24 | [Issue 1700](https://github.com/OpenFn/Lightning/issues/1700) | All users to create new projects from their dashboard. |
-| 7. Allow users to export workorder history | Delivered | Q3 '24 | [Issue 1698](https://github.com/OpenFn/lightning/issues/1698) | Allow project users to be able to export workorder history. The workorder history contains ALL logs and data clips (Input and Output) associated with runs in a workorder. |
-| 8. Project datastores and buffers | Shaped | Q4 '24 | [Issue 2190](https://github.com/OpenFn/lightning/issues/2190) | Allow users to configure a data store or buffer that allows temporary of storage of data that can be used in a workflow. |
-| 9. Make User Onboarding better | Shaped | Q4 '24 | [Issue 2523](https://github.com/OpenFn/Lightning/issues/2523) | Control what is printed in run logs by specifying log levels and allow users to disable printing console.logs, for data privacy once workflows are handling production data. |
-| 10. Snapshots and Audit Trail | Shaped | Q4 '24 | [Issue 2526](https://github.com/OpenFn/Lightning/issues/2526) | Control what is printed in run logs by specifying log levels and allow users to disable printing console.logs, for data privacy once workflows are handling production data. |
-| 11. Implement Monaco Editor | Tracked | Q4 '24 | [Issue 2126](https://github.com/OpenFn/Lightning/issues/2126) | Implement monaco editor for code editor or log viewing across the platform |
-| 12. Allow users to switch between sandbox and production modes | Tracked | Q4 '24 | [Issue 2524](https://github.com/OpenFn/Lightning/issues/2524) and [Issue 2525](https://github.com/OpenFn/Lightning/issues/2525) | Allow users to be able to switch between sandbox and production modes in their projects. |
-| 13. Control log outputs | Tracked | Q4 '24 | [Issue 1755](https://github.com/OpenFn/Lightning/issues/1755) | Control what is printed in run logs by specifying log levels and allow users to disable printing console.logs, for data privacy once workflows are handling production data. |
-| 14. Enable manual workflow triggers | Tracked | Q1 '25 | [Issue 2155](https://github.com/OpenFn/lightning/issues/2155) | Add funtionality that allows users to manually trigger workflow with JSON/CSV data as input data clip |
-| 15. Redesign workflow canvas and inspector interface | Tracked | Q1 '25 | [Issue 2021](https://github.com/OpenFn/lightning/issues/2021) | Redesign the workflow canvas and inspector interface to make workflow design better to help user build workflow faster and easier. |
-| 16. Improved History page filter | Tracked | Q1 '25 | [Issue 1791](https://github.com/OpenFn/lightning/issues/1791) | Extend workorder history page and enable cascading filtering. This is useful for debuging, failure recovery and auditability of the workflow. |
-| 17. Enhanced websocket worker Monitoring | Tracked | Q1 '25 | [Issue 608](https://github.com/OpenFn/kit/issues/608) | Give users better visibility of what's happening on inside the worker, especially when an error occurs during run execution. |
-| 18. Expanded Audit Trail and Node Authentication (ATNA) functionality | Tracked | Q1 '25 | [Issue 271](https://github.com/OpenFn/Lightning/issues/271) | Extend audit trail functionality to cover more aspects of ATNA, reference [OpenHIE IOL requirement IOLWF-1](https://guides.ohie.org/arch-spec/openhie-component-specifications-1/openhie-interoperability-layer-iol#openhie-iol-workflow-requirements). |
-
-_You can follow Lightning's progress and track delivered features in the
-[Changelog](https://openfn.github.io/lightning/changelog.html)._
-
-### Adaptors Roadmap
-
-OpenFn's open-source adaptors can connect any application, including web APIs,
-databases, and even raw data files, enabling interoperability with any
-information system ([read more](/adaptors/)). Adaptors, alongside OpenFn's
-workflow engine, enable automated workflows that cut across digital systems.
-
-| **Feature** | **`Status`** | **Target Timeline** | **Related Links** | **Description** |
-| ------------------------------------------------------------------------------------------------- | ------------ | ------------------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| 1. Add "magic" functions to existing, in-demand adaptors | Delivered | Q1 2024 | [Issue 243](https://github.com/OpenFn/adaptors/issues/243) | Add functions, dynamic lists, and shortcuts to fast-track workflow configuration for key adaptors including HTTP, [DHIS2](https://dhis2.org/), [CommCare](https://www.dimagi.com/commcare/), & [OpenMRS](https://openmrs.org/) |
-| 2. New [`OpenMRS`](https://openmrs.org/) adaptor version | Delivered | Q2 2024 | [See existing adaptor docs](/adaptors/packages/openmrs-readme) | To ensure compliance with OpenMRS v3 |
-| 3. Enhancements to [`FHIR`](http://www.hl7.org/fhir/) & [`OpenHIM`](http://openhim.org/) adaptors | Delivered | Q3 2024 | See existing adaptors for [FHIR](/adaptors/packages/fhir-docs) and [OpenHIM](/adaptors/packages/openhim-docs) | To rebuild the existing 2021 [OpenFn Instant-OpenHIE reference demo](/documentation/get-started/standards#openhie-standard-architecture) to highlight the exchange of data between existing non-FHIR digital health tools and a HAPI FHIR server. (OpenFn Lightning is OpenHIE-compliant and can be used as a workflow engine for the OpenHIE Interoperability layer - [learn more here](/documentation#openfn-v2-lightning-#standards-and-compliance-matter).) We also want to demonstrate data exchange between existing non-FHIR digital health tools and key components of Google’s [Open Health Stack](https://developers.google.com/open-health-stack) and [Cloud Healthcare API](https://cloud.google.com/healthcare-api/docs/concepts/fhir) |
-| 4. Enhancements to the [`OCL`](https://openconceptlab.org/) adaptor | Tracked | Q4 2024 | [See existing adaptor docs](/adaptors/packages/ocl-readme) | To ensure that mappings stored in OCLs can be more easily access and processed as inputs in OpenFn/Lightning workflows |
-| 5. Implement [MOSIP](https://mosip.io/#1) Adaptor | Tracked | Q1 2025 | [Issue 737](https://github.com/OpenFn/adaptors/issues/737) | Enable OpenFn workflows to integrate with the MOSIP platform for identity management use cases across countries. |
-| 6. Implement [OpenCRVS](https://www.opencrvs.org/) Adaptor | Tracked | Q1 2025 | [Issue 736](https://github.com/OpenFn/adaptors/issues/736) | Enable OpenFn workflows to integrate with OpenCRVS for CRVS workflows.|
-| 7. Implement [ArcGIS](https://www.arcgis.com/) Adaptor | Tracked | Q1 2025 | [Issue 738](https://github.com/OpenFn/adaptors/issues/738) | Enable Geospatial data management in OpenFn Workflows.|
-| 8. Support Personal Access Tokens in DHIS2 | Tracked | Q1 2025 | [Issue 697](https://github.com/OpenFn/adaptors/issues/697) | Extend the DHIS2 Adaptor to support Personal Access Tokens (PAT).|
-
-### Docs Roadmap
-
-| **Feature** | **`Status`** | **Target Timeline** | **Related Links** | **Description** |
-| ----------------------------------------------------------- | ------------ | ------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| 1. OpenFn and the [OpenHIE](https://ohie.org/) architecture | Delivered | 2024 | [See current docs](/documentation#openfn-v2-lightning-#standards-and-compliance-matter) | New page dedicated to how OpenHIE aligns with OpenHIE architecture; expansion of the existing small section on standards |
-| 2. New Lightning User Guidance | In Progress | 2024 | To be hosted on docs.openfn.org | New documentation, videos, and other user guidance on how to use OpenFn/Lightning and how to migrate existing OpenFn/platform projects to Lightning (the new OpenFn "v2") |
-
-## Have questions, feedback or found a bug?
-
-We encourage users to post their questions on the OpenFn Community at
-[community.openfn.org](https://community.openfn.org), or consider creating
-issues for bugs via product repository. You can also independetly start
-contributing to the OpenFn software, adaptors, or documentation by getting
-started [here](./writing-code.md).
diff --git a/docs/contribute/roadmap.md b/docs/contribute/roadmap.md
new file mode 100644
index 000000000000..386a8d8f5603
--- /dev/null
+++ b/docs/contribute/roadmap.md
@@ -0,0 +1,186 @@
+---
+title: Roadmap and Product Ethos
+sidebar_label: Roadmap
+---
+
+## Introduction
+
+This page details the planned roadmaps for the key products in the OpenFn
+product suite, including Lightning, Adaptors, and this Docs site. While this
+page will be updated periodically, you can track our realtime roadmap and
+progress [here](#what-are-we-currently-working-on).
+
+## Our approach to product development
+
+At OpenFn, we follow the [Shape Up approach](https://basecamp.com/shapeup) to
+help our small engineering team build _meaningful_ products and features faster
+without compromising on quality. With Shape Up in place, we typically commit to
+_projects_ that can be delivered in a 4-6 week period with multiple releases
+based on QA approval within the building cycle. We also prioritize feedback and
+feature requests from our users over features in the backlog.
+
+:::success Transparency and building what matters
+
+Note that it's fairly rare for us to commit to delivering a specific feature
+more than 2 or 3 months in the future. We are committed to constantly evaluating
+what our user base needs most and spending the few resources we have to deliver
+value to them—we simply can't guarantee that what sounds like the "7th most
+important feature to build" now will still be on our list in 6 months.
+
+At the same time, we strive to be as transparent and inclusive as we can in our
+planning processes. We have a big backlog of feature requests and GitHub issues
+(bug reports, stubs, even partially shaped epics or projects) that are getting
+voted up, commented on, and used as inspiration when we're deciding what to
+prioritize next.
+
+Read on to learn more about how we work, how you can see what's coming, and how
+you can get involved!
+
+:::
+
+## What are we currently working on?
+
+All of our team's work is tracked publicly using a GitHub Project. Three key
+views give you up-to-the-minute insights on what we're doing and what's on our
+immediate roadmap.
+
+### See [**_Now_**](https://github.com/orgs/OpenFn/projects/3/views/24?layout=table&sortedBy%5Bdirection%5D=desc&sortedBy%5BcolumnId%5D=Status) 🚧 for what's currently being built
+
+### See [**_Next_**](https://github.com/orgs/OpenFn/projects/3/views/2?layout=table&sortedBy%5Bdirection%5D=desc&sortedBy%5BcolumnId%5D=Status) ⏭️ for what's being considered for the next sprint
+
+### See [**_Epics_**](https://github.com/orgs/OpenFn/projects/3/views/7) 🤔 for a list of projects that we're considering, roughly-prioritized
+
+### See [**_Bugs_**](https://github.com/orgs/OpenFn/projects/3/views/22) 🐞 for known bugs we're tracking
+
+We will update this site monthly to reflect our progress on major items. You can
+also keep track of all new features, changes, and bug fixes in real-time via our
+[Changelog](https://openfn.github.io/lightning/changelog.html).
+
+## How to **get involved**
+
+We collect feedback and new feature requests via our
+[Community](https://community.openfn.org/c/feature-requests/) site. This allows
+OpenFn core team and users to track, engage by upvoting their favorite and
+mission critical feature requests.
+
+:::info Join our weekly product update
+
+We encourage you to join our weekly product updates where we present what's new
+and coming next. It's also a good opportunity for you to ask us questions about
+OpenFn. The call holds every Friday at 11 AM GMT (London)
+[here](https://meet.google.com/vaw-qvfq-mru) -
+(https://meet.google.com/vaw-qvfq-mru). You can also
+[add our events to your calendar](https://calendar.google.com/calendar/u/0?cid=Y182Y2Y4NWY0NjlhNWVlMzA4NzEwMWE5MWNhYmRjZTRkMDZlZDU1OGY1OTM3ZGUzNTQ0NWNkYmQ2NDFhMDY3MGFjQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20)
+so you never miss out on our demos, webinars, and product updates.
+
+:::
+
+### Upvote features 👍
+
+1. Go to the community
+ [feature request board](https://community.openfn.org/c/feature-requests/)
+2. Scroll down or use the filter and search features to see existing feature
+ requests
+3. Click on the (Vote) button beside the title of the request to upvote
+4. If you want more upvotes for this feature request, share a link to the
+ feature with your network
+
+### Request a new feature 💡
+
+1. Go to the community
+ [feature request board](https://community.openfn.org/c/feature-requests/)
+2. Click on `+ New Topic` to create a new request.
+3. Provide a very clear, concise and descriptive title for the feature (e.g.,
+ "Make the new workflow button green")
+4. Describe the feature request in detail and why it's important to you. Helpful
+ if you can add reference images and links.
+5. Share the feature request on across your professional network for upvotes
+
+:::info Tip
+
+When describing the feature, it is very helpful to help us understand the
+problem, proposed solution (if any) and similar solutions we might glean
+insights from _if they exist_.
+
+:::
+
+### Open an issue or bug directly
+
+If you prefer the direct approach, you can search across all tracked issues in
+OpenFn's GitHub org [here](https://github.com/OpenFn), comment on them, or even
+pick them up to work on yourself. If you don't find what you're looking for,
+please go ahead an create an issue in the relevant repo. We'll do our best to
+respond promptly!
+
+
+
+## Have questions, feedback or found a bug?
+
+We encourage users to post their questions on the OpenFn Community at
+[community.openfn.org](https://community.openfn.org), or consider creating
+issues for bugs via product repository. You can also independently start
+contributing to the OpenFn software, adaptors, or documentation by getting
+started [here](./writing-code.md).
diff --git a/docs/contribute/style-guide.md b/docs/contribute/style-guide.md
index b3bde792e12b..3860a6f35e37 100644
--- a/docs/contribute/style-guide.md
+++ b/docs/contribute/style-guide.md
@@ -80,7 +80,7 @@ Strikethrough uses two tildes. ~~Scratch this.~~
Or leave it empty and use the [link text itself].
URLs and URLs in angle brackets will automatically get turned into links.
-http://www.example.com/ or and sometimes example.com
+http://www.example.com/ or <http://www.example.com/> and sometimes example.com
(but not on GitHub, for example).
Some text to show that the reference links can follow later.
diff --git a/docs/contribute/writing-code.md b/docs/contribute/writing-code.md
index 74c19bd587c2..51fc6555f523 100644
--- a/docs/contribute/writing-code.md
+++ b/docs/contribute/writing-code.md
@@ -12,11 +12,11 @@ There are three ways you can contribute to the OpenFn DPG:
### 1. Build or extend OpenFn adaptors
- Requires knowledge of Javascript and Typescript
-- See the [README.md](https://github.com/OpenFn/lightning#contribute-to-this-project) to learn how to contribute
+- See the [README.md](https://github.com/OpenFn/adaptors#contributing) to learn how to contribute
### 2. Add or improve a feature on the OpenFn Lightning platform
- Requires knowledge of Elixir and Pheonix Liveview
-- See the [README.md](https://github.com/OpenFn/adaptors#contributing) to learn how to contribute
+- See the [README.md](https://github.com/OpenFn/lightning#contribute-to-this-project) to learn how to contribute
### 3. Add to or improve our documentation
diff --git a/docs/deploy/options.md b/docs/deploy/options.md
index 37bdd4455ea2..bef0b3c53e2a 100644
--- a/docs/deploy/options.md
+++ b/docs/deploy/options.md
@@ -17,7 +17,7 @@ We're committed to a **no vendor lock-in** experience.
:::
-| Pathway | Free | Standard | Dedicated | Do-it-yourself (DIY) |
+| Pathway | Free Cloud | OpenFn Cloud | Dedicated | Do-it-yourself (DIY) |
| :-------------------: | :------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------: |
| Description | Go live today on OpenFn.org for small scale projects | Scale up and down, pay only for what you need | A dedicated, unrestricted OpenFn installation anywhere in the world on our servers or yours | Deploy and manage your own solutions with OpenFn |
| License | Free forever with usage limitations | **SaaS** [plans](https://www.openfn.org/pricing); contact enterprise@openfn.org for custom/invoice agreements | **SDaaS** includes deployment, maintenance, security patches, upgrades, and troubleshooting as a service; contact enterprise@openfn.org | LGPLv3 means use freely as part of any closed or open-source solution, but make all _derivative_ works open source |
diff --git a/docs/design/api-discovery.md b/docs/design/api-discovery.md
index 6418833f3d0a..5eb03234ff98 100644
--- a/docs/design/api-discovery.md
+++ b/docs/design/api-discovery.md
@@ -12,7 +12,7 @@ APIs tell applications how to communicate. An API is the “messenger” that:
2. Delivers your request to the provider that you’re requesting it from, and then
3. Delivers the response back to you
-|  |
+|  |
|:--:|
| *[Source](https://snipcart.com/blog/integrating-apis-introduction)*|
@@ -72,7 +72,7 @@ Be on the lookout for API limits. Documentation will often have a dedicated sect
The output of API discovery should be a “technical” workflow diagram. This diagram is different from the functional workflow diagram produced during [“Discovery”](/documentation/design/discovery) in that it captures the technical specifications for how to integrate with target applications. These specifications include the specific methods/operations (e.g., GET, POST) and the database/API names of the target resources (i.e., specific API endpoints or database tables).
-
+
__When drafting your technical specifications, consider the following:__
1. __Plan for failure. Your workflows will fail. Consider what happens when they do…__
@@ -95,4 +95,4 @@ __When drafting your technical specifications, consider the following:__
Check out the technical workflow diagram below for syncing forms submissions from KoboToolBox to DHIS2. The original functional diagram can be found [here](/documentation/design/discovery#workflow-requirements-gathering).
-
\ No newline at end of file
+
\ No newline at end of file
diff --git a/docs/design/design-overview.md b/docs/design/design-overview.md
index 9e4b4b30d798..c05ae99ea193 100644
--- a/docs/design/design-overview.md
+++ b/docs/design/design-overview.md
@@ -14,19 +14,19 @@ Before you dive in, make sure you have a clear understanding of these key terms
### Workflow
The set of instructions that determine how to solve a problem or accomplish a task. They are often broken down into smaller, independent tasks.
-
+
### Workflow Automation
The use of software to perform these tasks independently, in accordance with predefined business rules, and without the need for human input.
-
+
### Data Integration
The process of combining data from different sources into a centralized view. Data integration is a means of achieving workflow automation. Its tasks may be streamlined, automated, and managed by a workflow automation tool.
-
+
## Introduction
diff --git a/docs/design/design-workflow.md b/docs/design/design-workflow.md
index 8d59bdd916e7..39cbf415bdba 100644
--- a/docs/design/design-workflow.md
+++ b/docs/design/design-workflow.md
@@ -54,4 +54,4 @@ See the below example BPMN diagram for the user story:
> contacts in my SMS campaign configured on RapidPro to send them automated
> alerts and program updates.
-
+
diff --git a/docs/design/discovery.md b/docs/design/discovery.md
index b2b3e6329625..fe42390af2d2 100644
--- a/docs/design/discovery.md
+++ b/docs/design/discovery.md
@@ -72,7 +72,7 @@ notation).
each time a form is submitted (i.e. real-time sync). There are a maximum of 5000
patients registered in Kobo per month.
-
+
### Technical feasibility assessment
@@ -107,7 +107,7 @@ Example: Only one instance of PatientCare and DHIS2 exist for this integration
and they have already been built with REST APIs. They are both hosted on
PatientCare managed servers that require IP whitelisting for access.
-
+
### Capacity assessment
@@ -155,10 +155,10 @@ requirements. Check out the example solution architecture diagrams below.
**Example 1:**
-
+
**Example 2:**
-|  |
+|  |
| :------------------------------------------------------------------------------------------------------------------------------------------------: |
| _[Source](https://lucid.app/lucidchart/1e997197-2d67-4393-8394-a532d83561b2/edit?invitationId=inv_85b809a1-6fbd-4275-abdc-618fbd56e90d&page=0_0#)_ |
diff --git a/docs/design/mapping-specs.md b/docs/design/mapping-specs.md
index ca2804ea639d..8e0e6bebce67 100644
--- a/docs/design/mapping-specs.md
+++ b/docs/design/mapping-specs.md
@@ -19,7 +19,7 @@ For each automation step in your workflow, you will document which data elements
(or metadata) will be referenced, as well as the “rules” for how these data
elements should be mapped, reassigned, cleaned, transformed, and/or calculated.
-
+
**To draft a data element mapping specification, you’ll need to…**
@@ -31,7 +31,7 @@ elements should be mapped, reassigned, cleaned, transformed, and/or calculated.
file with “dummy” data.
3. Start “mapping” the data elements and recording transformation rules!
-|  |
+|  |
| :--------------------------------------------------------: |
| _The data mapping process for data integration solutions._ |
@@ -143,7 +143,7 @@ second step in the workflow is dedicated to querying this list of global
mappings from the app in which they're stored to dynamically fetch the latest
globals every time the workflow is run.
-
+
:::tip
diff --git a/docs/design/overview.md b/docs/design/overview.md
index 0a044ac83118..51e216f2ff3b 100644
--- a/docs/design/overview.md
+++ b/docs/design/overview.md
@@ -9,7 +9,7 @@ This docs page is under construction. Check back later for the complete docs, or
:::
# Getting started with workflow automation design for OpenFn projects
-Overview of design process and key outputs/artefacts...
+Overview of design process and key outputs/artifacts...
**Integration design begins with the functional or business requirements (not
the technical bits).** Therefore, you do not need to be an IT consultant or
@@ -116,7 +116,7 @@ See the below example BPMN diagram for the user story:
> contacts in my SMS campaign configured on RapidPro to send them automated
> alerts and program updates.
-
+
## 3. Map data elements to be exchanged
diff --git a/docs/design/workflow-specs.md b/docs/design/workflow-specs.md
index 07aac631c680..7b102e809844 100644
--- a/docs/design/workflow-specs.md
+++ b/docs/design/workflow-specs.md
@@ -17,7 +17,7 @@ Given these, you’ll be ready to finalize your workflow specifications and hand
Each “task” or “step” in the OpenFn swimlane of your technical diagram can be implemented as a distinct operation in your workflow configuration. In the example diagram below, you might implement 1 job with 3 chained operations, or 3 jobs with 1 operation each.
-
+
__The workflow specifications should link to all of the design artifacts and highlight the following:__
1. The required number of OpenFn jobs and the function of each
diff --git a/docs/get-started/home.md b/docs/get-started/home.md
index 4674abf153ba..c4af9efb3157 100644
--- a/docs/get-started/home.md
+++ b/docs/get-started/home.md
@@ -78,7 +78,7 @@ entirely self-hosted platform.
Version 2 leverages the same tried-and-trusted core technology as the OpenFn v1
and comes with an improved, visual interface for building integrations.
-
+
**Check out the
[OpenFn v2 Basics playlist](https://www.youtube.com/watch?v=U0MXYRXkDnI&list=PL1pD3-abjHJ0L01RjouO2xOWKtEUYi8e4&ab_channel=OpenFn.org)**
diff --git a/docs/get-started/security-compliance.md b/docs/get-started/security-compliance.md
index 77a1a7ea33fa..4a3e4205c636 100644
--- a/docs/get-started/security-compliance.md
+++ b/docs/get-started/security-compliance.md
@@ -14,31 +14,112 @@ NGOs worldwide.
✓ Robust, configurable security settings to ensure compliance with your policies
-✓ Build “zero-persistence” data pipelines to fully control where data is store
+✓ Build “zero-persistence” data pipelines to fully control where data is stored
✓ Security implementation training & guidance for your project teams
([read more](../get-started/security.md))
See our main website to learn more about OpenFn
-[Security & Trust](https://openfn.org/trust).
+[Security & Trust](https://www.openfn.org/trust) and
+[Compliance](https://www.openfn.org/compliance).
+
+## Compliance
+
+OpenFn implementations are highly configurable and can be deployed anywhere to
+help ensure compliance with your country- or organization-specific data privacy
+and security policies.
+
+**For more on how we think about compliance–especially with policies like GDPR
+or HIPAA–check out our [Compliance](https://www.openfn.org/compliance) web
+page.** Contact [our core team](mailto:support@openfn.org) if interested in
+consultation and advisory on how to deploy and configure your OpenFn
+implementation to ensure 100% compliance.
## OpenFn and data storage
-In your digital ecosystem, **OpenFn serves as a data processing and transfer
-solution—not as a data storage service.**
+In your digital ecosystem, typically **OpenFn serves as a data processing and
+transfer solution—not as a data storage service.**
-As an open source Digital Public Good, OpenFn can be deployed anywhere and
-Workflows can be configured to adhere to your organization's specific data
-sharing agreements and security policies.
+As an open source Digital Public Good, OpenFn can be deployed anywhere
+([see docs](../deploy/options.md)) and workflows can be configured to adhere to
+your organization's specific data sharing agreements and security policies.
Consult the `Manage Projects` docs pages for more on project and
[data storage settings](../manage-projects/io-data-storage.md).
-See the below diagram for an example architecture where even the cloud-hosted
-OpenFn platform can be configured as a **“zero-persistence” data pipeline** to
-ensure compliance with data security and residency requirements. This enables
-partners to rapidly configure and pilot projects using the turnkey OpenFn
-cloud-hosted platform, before migrating to a local deployment when they’re ready
-to scale.
+See the below diagram for an example architecture where even the OpenFn Cloud
+can be configured as a **“zero-persistence” data pipeline** to ensure compliance
+with data security and residency requirements. This enables partners to rapidly
+configure and pilot projects using the turnkey OpenFn cloud-hosted platform,
+before migrating to a local deployment when they’re ready to scale.
+
+
+
+To delete your project data at any time, you can
+[delete your project](../manage-projects/platform-mgmt.md) or
+[delete your account](../manage-users/user-profile.md).
+
+## Encryption
+
+OpenFn Cloud uses a security-oriented Cloud SQL product for data storage that
+guarantees 256-bit encryption at rest and we only allow connections with
+TLS/SSL.
+
+Platform encryption:
+
+- 256-bit Advanced Encryption Standard
+- SSL/TLS encryption in transit
+- Credentials/secrets encrypted on disk
+
+Learn more at [openfn.org/trust](https://www.openfn.org/trust#encryption).
+
+## Credentials
+
+[Credentials](../manage-projects/manage-credentials.md), used to grant OpenFn
+API access to your various technologies, are encrypted at rest so that, in the
+unlikely event of a database breach, without access to multiple, independently
+secured boxes an attacker would be unable to read your authentication
+information.
+
+Connections to your destination applications are only made over HTTPS, using SSL
+and basic authentication in most cases—with the technical connection
+specifications being determined by the REST endpoint of the application to which
+you are connecting. Technical documentation for individual adaptors can be found
+in the [Adaptor docs](/adaptors) or in their respective repositories on Github
+at [github.com/OpenFn/adaptors](https://github.com/OpenFn/adaptors).
+
+Credentials can only be viewed by you (the creator), and are loaded into your
+private runtime for job execution. You can delete these credentials at any time
+and they will be purged from the system.
+[See docs](../manage-users/user-credentials.md) for more on OpenFn credentials
+management and sharing.
+
+## User Access Management and RBAC
+
+OpenFn supports user access management through **role-based access control
+(RBAC)**, allowing admins to assign granular permissions at both the environment
+and project levels. Roles (e.g., Admin, Editor, Viewer) control who can view,
+edit, run, or manage workflows and credentials. Access can be restricted to
+specific projects or environment configurations, with support for audit logs and
+scoped API tokens to ensure security and compliance.
+
+When new users are invited to work on your Project as Collaborators, they are
+assigned a role that determines their permissions. See docs on
+[Collaboration](../manage-projects/collaboration.md) and
+[User Roles](../manage-projects/user-roles-permissions.md) for more information.
+
+When users register for the platform, they will be prompted to create a secure
+password. OpenFn super administrators can also enable
+[Multi-Factor Authentication](../manage-users/user-profile.md), password expiry,
+and stale account lockout.
+
+:::info More OpenFn Security Questions?
+
+First, be sure to consult the [Trust](https://www.openfn.org/trust) and
+[Compliance](https://www.openfn.org/compliance) pages on our website, as well as
+[Security Implementation Guidebook](../get-started/security.md).
+
+Ask questions on [Community](https://community.openfn.org/) or
+[contact our core team](mailto:security@openfn.org) for private queries.
-
+:::
diff --git a/docs/get-started/standards.md b/docs/get-started/standards.md
index 9972963b4212..620530cc5176 100644
--- a/docs/get-started/standards.md
+++ b/docs/get-started/standards.md
@@ -3,10 +3,11 @@ sidebar_label: Standards
title: Standards & OpenFn
---
-OpenFn follows global standards for open source software and for workflow engine solutions. Read on to learn how OpenFn complies with specific standards.
-
+OpenFn follows global standards for open source software and for workflow engine
+solutions. Read on to learn how OpenFn complies with specific standards.
## Digital Public Good
+
OpenFn is recognised by the
[Ditial Public Goods Alliance](https://digitalpublicgoods.net/) as a Digital
Public Good, or "DPG".
@@ -26,7 +27,8 @@ You can read more about the DPG standard
## Global Good for Health
OpenFn is one of 36 software applications that have been recognised as a Digital
-Square [Global Good for Health](https://wiki.digitalsquare.io/index.php/What_are_Global_Goods#:~:text=Digital%20Square%20Global%20Goods%20are,scale%2C%20are%20used%20across%20multiple).
+Square
+[Global Good for Health](https://wiki.digitalsquare.io/index.php/What_are_Global_Goods#:~:text=Digital%20Square%20Global%20Goods%20are,scale%2C%20are%20used%20across%20multiple).
:::info Global Goods for Health Definition
@@ -44,7 +46,8 @@ You can read more about Global Goods for Health
## OpenHIE Standard Architecture
-OpenFn is considered a OpenHIE reference technology and is compliant with the OpenHIE standard architecture for digital health implementations.
+OpenFn is considered a OpenHIE reference technology and is compliant with the
+OpenHIE standard architecture for digital health implementations.
_This section assumes you are familiar with the OpenHIE specification–a
reference framework that makes sharing health data across information systems
@@ -54,10 +57,10 @@ possible through a Health Information Exchange (“HIE”). To learn more, check
### OpenFn and OpenHIE
-The OpenFn platform v2 ([OpenFn/lightning](https://github.com/OpenFn/)) is an OpenHIE-compliant **_workflow engine_** used to (1)
-automate complex business processes that cut across digital systems (including
-OpenHIE components _and_ point of care systems), and to (2) handle data mapping
-and transformation.
+The OpenFn platform v2 ([OpenFn/lightning](https://github.com/OpenFn/)) is an
+OpenHIE-compliant **_workflow engine_** used to (1) automate complex business
+processes that cut across digital systems (including OpenHIE components _and_
+point of care systems), and to (2) handle data mapping and transformation.
If your organization is implementing the OpenHIE standard architecture, then
OpenFn provides a workflow engine that interfaces with your interoperability
@@ -77,7 +80,7 @@ used as a fully OpenHIE-compliant **_interoperability layer _**because it does
not leverage the IHE ATNA profile (see
[requirement IOL-WF1](https://guides.ohie.org/arch-spec/openhie-component-specifications-1/openhie-interoperability-layer-iol#openhie-iol-workflow-requirements)).
-
+
_For an overview of OpenFn Lightning and how it fits into OpenHIE, see our
[introduction for the OpenHIE showcase](https://www.youtube.com/watch?v=PTRRZBYtqyc)_
@@ -122,48 +125,94 @@ leveraging OpenFn’s out-of-box DHIS2 adaptor and reusable workflow templates t
quickly develop automation that reformats data received from CommCare and maps
it to the DHIS2 data model.
-
+
## GovStack
-OpenFn is compliant with [GovStack's standard specification](https://govstack.gitbook.io/bb-workflow/2-description) for workflow engines.
-
+OpenFn is compliant with
+[GovStack's standard specification](https://govstack.gitbook.io/bb-workflow/2-description)
+for workflow engines.
## Pricinciples for Digital Development
-OpenFn was designed for the social sector and has been actively prioritizing the [Principles of Digital Development](https://digitalprinciples.org/) since its inception.
+OpenFn was designed for the social sector and has been actively prioritizing the
+[Principles of Digital Development](https://digitalprinciples.org/) since its
+inception.
OpenFn solutions are:
-- **interoperable** (connect any application);
-- **reusable** (utilize existing OpenFn configurations as templates, or easily share, copy, and modify your own configurations; see docs.openfn.org/library);
-- **sustainable** (flexible implementation options with no lock-in);
-- **scalable** (OpenFn leverages enterprise-grade tech to handle high data volumes and provides a range of deployment options to ensure total solution ownership on any server);
-- **promote open standards and open access** (through our open-source software, documentation, and features to help users implement open standards in their information exchange solutions), and
-- **address privacy & security**.
+
+- **interoperable** (connect any application);
+- **reusable** (utilize existing OpenFn configurations as templates, or easily
+ share, copy, and modify your own configurations; see docs.openfn.org/library);
+- **sustainable** (flexible implementation options with no lock-in);
+- **scalable** (OpenFn leverages enterprise-grade tech to handle high data
+ volumes and provides a range of deployment options to ensure total solution
+ ownership on any server);
+- **promote open standards and open access** (through our open-source software,
+ documentation, and features to help users implement open standards in their
+ information exchange solutions), and
+- **address privacy & security**.
## FHIR for health data exchange
-[FHIR](https://www.hl7.org/fhir/) (pronounced "fire" 🔥) is a standard for health care data exchange, published by HL7®.
+[FHIR](https://www.hl7.org/fhir/) (pronounced "fire" 🔥) is a standard for
+health care data exchange, published by HL7®.
+
+OpenFn is used by health organizations to connect multiple FHIR- and non-FHIR
+compliant systems in a secure, stable, and scalable manner. OpenFn can
+facilitate 2 categories of FHIR workflows:
+
+### 1. Non-FHIR to FHIR Data Exchange
+
+OpenFn users can configure workflows to convert non-FHIR data to FHIR-compliant
+formats, and then route to FHIR systems.
-OpenFn is used by health organizations to connect multiple FHIR- and non-FHIR compliant systems in a secure, stable, and scalable manner. OpenFn can facilitate 2 categories of FHIR workflows:
+For example, get data from CommCare mobile app, convert to FHIR, and send to
+national health system's FHIR store.
+
-### 1. Non-FHIR to FHIR
+### 2. FHIR to FHIR Data Exchange
-OpenFn users can configure Workflows to convert non-FHIR data to FHIR-compliant formats, and then route to FHIR systems.
+OpenFn users can also configure Workflows to automate the exchange and routing
+of _already_ FHIR-compliant data to other FHIR-compliant systems.
-For example, get data from CommCare mobile app, convert to FHIR, and send to national health system's FHIR store.
-
+For example, get data from OpenMRS's FHIR API, and forward to the national
+health system's FHIR store (no data transformation needed).
-### 2.FHIR to FHIR
+
-OpenFn users can also configure Workflows to automate the exchange and routing of _already_ FHIR-compliant data to other FHIR-compliant systems.
+## FHIR Adaptors
-For example, get data from OpenMRS's FHIR API, and forward to the national health system's FHIR store (no data transformation needed).
+OpenFn [adaptors](/adaptors) fast-track integration setup with target
+applications (including FHIR endpoints!). The core team is currently working on
+a suite of FHIR-specific adaptors to enable interoperability with FHIR systems.
+
+The [fhir-4 adaptor](/adaptors/fhir-4) makes it easy to access and modify data
+held on any [FHIR r4](https://www.hl7.org/fhir/R4/) compliant server. It also
+provides full code-assist to developers while creating specific resource
+definitions, simplifying data-entry and mapping logic.
+
+We also provide a generic [fhir](/adaptors/fhir) adaptor that's compatible with
+all FHIR versions.
+
+:::info Fhir 4 support
+
+The `fhir-4` adaptor is new to OpenFn as of March 2025. It introduces richer
+levels of support from the generic [fhir](/adaptors/fhir)adaptor. Support for
+other FHIR versions is coming soon
+
+:::
-
+See the
+[Adaptors Wiki](https://github.com/OpenFn/adaptors/wiki/Generating-Fhir-Adaptors)
+to learn how to build your own FHIR adpator specific to your target FHIR
+Implementation Guide
## Other Data Standards
-OpenFn Workflows can automate data transformation, cleaning, and formatting rules to ensure compliance with _your_ organization's specific standards.
+OpenFn Workflows can automate data transformation, cleaning, and formatting
+rules to ensure compliance with _your_ organization's specific standards.
-Ask on the [community](https://community.openfn.org) to explore how OpenFn can be leverage to help automate application and enforcement of other data standards.
+Ask on the [community](https://community.openfn.org) to explore how OpenFn can
+be leverage to help automate application and enforcement of other data
+standards.
diff --git a/docs/get-started/terminology.md b/docs/get-started/terminology.md
index de96712881c5..88f1bae70e80 100644
--- a/docs/get-started/terminology.md
+++ b/docs/get-started/terminology.md
@@ -45,6 +45,13 @@ Workflows are the **"what to do"** part of automation!
:::
+A workflow is a structured sequence of tasks, processes, or actions that are
+executed automatically based on predefined rules, triggers, and logic.
+
+When working with AI, workflows provide the structured execution needed to turn
+LLM insights into real-world actions, while AI agents enable more dynamic
+decision-making within workflows.
+
A Workflow is a collection of a Trigger, Steps, Paths, and custom logic
connected together to automate a specific business process or task. A Workflow
is configured via the Canvas in the web app, or locally (via code).
@@ -162,7 +169,7 @@ OpenMRS, and your system administrator will have been notified that one of those
patients couldn’t be created (or whatever more robust error-handling you’ve set
up will take place.)
-
+
:::note
@@ -192,7 +199,7 @@ fail, so it needs to be retried to successfully process).
Runs have start times, end times, logs, and status codes that indicate when they
took place, what they did, and whether or not they succeeded.
-
+
Imagine that a Workflow is configured to create a new patient in OpenMRS
whenever a new case is opened in CommCare. Today if 1 patient is created, then:
@@ -215,14 +222,14 @@ activities performed when running a Workflow or specific Step.
OpenFn developers can control what appears in Logs by editing `console.log(...)`
statements in the Workflow job expressions of individual Steps.
-
+
## History
On the platform, the History page provides a list of all of the Work Orders and
Runs that have been processed in a Project.
-
+
## Inspector
@@ -231,7 +238,7 @@ workflows.
The Inspector has 3 key interfaces: `Input`, `Editor`, & `Output`.
-
+
### Input
diff --git a/docs/jobs/job-writing-guide.md b/docs/jobs/job-writing-guide.md
index e76de74db9a8..e45a186770c1 100644
--- a/docs/jobs/job-writing-guide.md
+++ b/docs/jobs/job-writing-guide.md
@@ -925,7 +925,7 @@ In a cron workflow, OpenFn will pass the previous state into the next state - so
state persists across runs. We can take advantage of that to pick up where we
left off.
-You can use the [`cursor()`](adaptors/packages/common-docs#cursor) operation,
+You can use the [`cursor()`](/adaptors/packages/common-docs#cursor) operation,
which is built-in to most adaptors, to make cursor management easier.
diff --git a/docs/jobs/state.md b/docs/jobs/state.md
index 1811a741a8ba..086d398f09d9 100644
--- a/docs/jobs/state.md
+++ b/docs/jobs/state.md
@@ -12,7 +12,7 @@ to read from and write to.
The final state form a Job must always be a serializable Javascript object (ie,
a JSON object). Any non-serializable keys will be removed.
-
+
:::tip A note on terminology
@@ -60,40 +60,42 @@ state controls what is output by the run at the end of all of these operations.
Best practice is to include a final state cleanup step that removes any data
that should not persist between runs or be output (like PII), for example:
- // get data from a data source
- get('https://jsonplaceholder.typicode.com/users')
-
- // store retrieved data in state for use later in job
- fn(state => {
- state.users = state.data;
- return state;
- });
-
- // get more data from another data source
- get('https://jsonplaceholder.typicode.com/posts')
-
- // store additional retrieved data in state for use later in job
- fn(state => {
- state.posts = state.data;
- return state;
- });
-
- // compare data
- fn(state => {
- if (state.users.length > state.posts.length) {
- // do something based on the comparison
- }
- return state;
- });
-
- // cleanup state at the end before finshing job
- fn(state => {
- state.data = null
- state.users = null
- state.posts = null
-
- return state;
- });
+```js
+// get data from a data source
+get('https://jsonplaceholder.typicode.com/users')
+
+// store retrieved data in state for use later in job
+fn(state => {
+ state.users = state.data;
+ return state;
+});
+
+// get more data from another data source
+get('https://jsonplaceholder.typicode.com/posts')
+
+// store additional retrieved data in state for use later in job
+fn(state => {
+ state.posts = state.data;
+ return state;
+});
+
+// compare data
+fn(state => {
+ if (state.users.length > state.posts.length) {
+ // do something based on the comparison
+ }
+ return state;
+});
+
+// cleanup state at the end before finshing job
+fn(state => {
+ state.data = null
+ state.users = null
+ state.posts = null
+
+ return state;
+});
+```
### Webhook triggered runs
@@ -195,4 +197,4 @@ on state, keyed by the ID of the job that failed.
See the below diagram for a visual description of how state might be passed
between Steps in a Workflow.
-
+
diff --git a/docs/keyboard-shortcuts.md b/docs/keyboard-shortcuts.md
new file mode 100644
index 000000000000..32542b8a82ad
--- /dev/null
+++ b/docs/keyboard-shortcuts.md
@@ -0,0 +1,34 @@
+---
+title: Keyboard Shortcuts
+keywords: [keystrokes, keyboard shortcuts, shortcuts]
+---
+
+Keyboard shortcuts (keystrokes) allow you to perform common actions without
+taking your hands off the keyboard. 🤓
+
+## Platform Shortcuts
+
+| Command | Availability | Mac | Linux/Windows | Notes |
+| ------------------------ | ----------------- | ---------------- | ------------------ | ---------------------------------------------------------------------------------------------------------------------------- |
+| Save Workflow | Canvas, Inspector | `⌘+s` | `Ctrl+s` |
+| Save & Sync Workflow | Canvas, Inspector | `⌘+Shift+s` | `Ctrl+Shift+s` | When opting to sync, you'll be prompted to either enter a commit message or use the default message provided. |
+| Run | Inspector | `⌘+Return` | `Ctrl+Enter` | This will save (but not sync) your workflow and run it from the current step with the default workorder grouping behavior.\* |
+| Run _(alternate action)_ | Inspector | `⌘+Shift+Return` | `Ctrl+Shift+Enter` | This will save (but not sync) your workflow and create a new workorder from the current step. |
+
+\*If you are viewing an existing work order and create a run from the inspector,
+that run will be associated with the existing work order—this is the default
+behavior. (Think of this as "retrying".) Sometimes, for auditing purposes, it's
+helpful to create an entirely new work order. This can be done by using the
+alternate run button.
+
+## Selected Editor Shortcuts
+
+See the command pallette (right click or press `F1`) for the full list. Note
+that to access these shortcuts you must click into a specific editor—there are
+multiple editors in the Inspector interface.
+
+| Command | Availability | Mac | Linux/Windows |
+| -------------------- | --------------------- | ---------------- | ------------- |
+| View Editor Commands | Inspector, Run Viewer | `F1` | `F1` |
+| Format Code | Inspector | `Shift+Option+F` | `Shift+Alt+F` |
+| Comment Code Out/In | Inspector | `⌘+/` | `Ctrl+/` |
diff --git a/docs/manage-projects/collaboration.md b/docs/manage-projects/collaboration.md
index 90e0d04b8b16..524a3ef31aa1 100644
--- a/docs/manage-projects/collaboration.md
+++ b/docs/manage-projects/collaboration.md
@@ -49,11 +49,11 @@ associated, you will be requested to authorize OpenFn to create an account for
them and send them an invite to your project. Click `Invite new user` to proceed
with the invitation.
-
+
-
+
-
+
:::note
diff --git a/docs/manage-projects/io-data-storage.md b/docs/manage-projects/io-data-storage.md
index a249347567c4..c415a1c7a561 100644
--- a/docs/manage-projects/io-data-storage.md
+++ b/docs/manage-projects/io-data-storage.md
@@ -56,12 +56,12 @@ To export work order history for your project, open the project and click on
`History` on the side menu. On the History page, scroll to the bottom of the
work order history table and click on the cloud icon (see image below).
-
+
Clicking on the download icon will prompt a confirmation modal for the export.
If confirmed, a background process will be initiated for the export.
-
+
When the export is complete, an email will be sent to your OpenFn user's
associated email address.
@@ -84,7 +84,7 @@ On the `History Export`s page, you will see the list of exports showing your
latest request and previous requests with the other information such as
filename, date of export, user who requested the export and the status.
-
+
:::caution Configuring storage for exports
diff --git a/docs/manage-projects/link-to-gh.md b/docs/manage-projects/link-to-gh.md
index 34231205b45f..548755fc7d27 100644
--- a/docs/manage-projects/link-to-gh.md
+++ b/docs/manage-projects/link-to-gh.md
@@ -34,7 +34,7 @@ To configure your project to use Github sync, follow the following steps:
2. If you have not already connected your OpenFn user account to GitHub, do so
by clicking the **"Connect your OpenFn account to GitHub"** button.
-
+
3. Choose which GitHub installation to use for your project or follow tip below
to update your installations.
@@ -56,7 +56,7 @@ To configure your project to use Github sync, follow the following steps:
4. Choose your preferred repository and branch you'd like to connect your
project
-
+
5. **_Optionally_**, if you _first want to sync from GitHub to OpenFn and
already have config file_, add a filepath to an existing project
@@ -101,16 +101,16 @@ interface. After clicking that link, you can follow the steps below:
1. Click **"Configure"** or **"Install"**.
-
+
2. Then select the GitHub account that owns the repository you want to connect
to.
-
+
3. Select the repository to sync with and hit **"Save"**.
-
+
4. When you're done making changes on GitHub, head back to OpenFn and refresh
the connection lists with the the 🔄 button next to the drop down list of
@@ -133,10 +133,36 @@ code in the `project.yaml` file on your repository.
:::info
Your OpenFn project can be represented as code and packaged as project.yaml
-which is called the prokect spec. See the
+which is called the project spec. See the
[portability documentation](/documentation/deploy/portability) to learn more.
-::: To configure your project to sync to GitHub, follow these steps:
+:::
+
+After successfully setting up your project connection to GitHub as detailed
+[above](#managing-github-permissions), you can initiate subsequent syncs either
+via the Canvas, the Inspector or the version control page in the project
+settings.
+
+To initiate a sync via the Canvas or Inspector, press `Ctrl+Shift+s`. (Or
+`⌘+Shift+s` on Mac; see
+[keyboard shortcuts](/documentation/keyboard-shortcuts).) You can also click the
+dropdown icon beside the save button to select `Save & Sync`. When you click
+Save & Sync, you'll see a confirmation modal with an option for you to customize
+the commit message.
+
+
+
+:::info Sync is a "project level" action
+
+When you trigger `Save & Sync` in a workflow, your new changes and _previous_
+uncommitted changes (if any) to your project's resources (including other
+workflows) will be committed to GitHub. I.e., if there are other, uncommitted
+changes by either you or someone else to other workflows in the project, they’ll
+also show up in that sync also.
+
+:::
+
+To configure your project to sync to GitHub via the project settings:
1. Go to the Project where you made edits to your Workflow(s), and then navigate
to the `Project Settings` page
@@ -145,7 +171,7 @@ which is called the prokect spec. See the
3. Click the button `Initiate Sync to Branch` to trigger a sync to the connected
Github repository
-
+
### Sync from GitHub to OpenFn
@@ -257,7 +283,6 @@ your-git-monorepo
└── projectSpec.yaml
```
-
:::tip A sync in time, saves nine
#### Syncing Changes from OpenFn to GitHub
@@ -275,12 +300,12 @@ repository. A GitHub action is automatically triggered to push changes to OpenFn
ensuring that future syncs are not affected. Changes can include adding,
renaming, deleting a file or updating a file path. :::
-
## Troubleshooting
### Github Sync Error: Unexpected inputs provided: ["snapshots"]
-If you installed GitHub sync before July 17th, 2024 you may need to update your `.github/workflows/openfn-pull.yml` file to match:
+If you installed GitHub sync before July 17th, 2024 you may need to update your
+`.github/workflows/openfn-pull.yml` file to match:
```
on:
diff --git a/docs/manage-projects/manage-credentials.md b/docs/manage-projects/manage-credentials.md
index 6ad145c9daf1..ac5ce9d93701 100644
--- a/docs/manage-projects/manage-credentials.md
+++ b/docs/manage-projects/manage-credentials.md
@@ -6,12 +6,12 @@ You can view the Credentials related to a Project under the Project
`Settings > Credentials` page. In this article, you'll learn how to manage your
credentials related to a project.
-### View all Project Credentials
+## View all Project Credentials
Via this `Credentials` page, you can see a list of all Credentials, including
the name, type, and owner, and whether they are for a production environment.
-
+
:::info Viewing credential secrets
@@ -21,7 +21,7 @@ etc.).
:::
-### Create a new Credential
+## Create a new Credential
1. Click on the `New Credential` button, and choose the type of app you want to
connect.
@@ -32,12 +32,12 @@ etc.).
{ "loginUrl": "https://random-app.com", "username": "test", "password": "pwd" }
```
-
+
3. Click `Configure Credentials` and add the authentication details for your
app. The credential form will indicate which fields are required.
-
+
:::tip Not sure how to fill in all credential details?
@@ -51,30 +51,121 @@ ask on [Community](https://community.openfn.org).
3. Click `Save` and you'll see it listed in your `Credentials` page. You can now
use it across the Project when building and running Workflows.
-
+
-### Share Credentials
+## Keychain Credentials (Variable Auth)
+
+Keychain credentials allow for a single job to make use of multiple credentials.
+They work by inspecting the data in the job's runtime state (i.e., state.data)
+and checking for the value of a predetermined identifier. Based on that value,
+present in the data for a given source message, for example, another credential
+will be selected and applied for that particular job run.
+
+Keychain credentials allow for a single job to make use of multiple credentials.
+They work by inspecting the data in the initial input for a given run (e.g.,
+state.data) and checking for the value of a predetermined identifier.
+
+Imagine you have 2 credentials in your project:
+
+1. Taylor’s Login, External ID: abc123, Body:
+ `{ username: “tay”, password: “shhhhh” }`
+
+2. Roina’s Login, External ID: def456, Body:
+ `{ username: “ro”, password: “veryshh” }`
+
+And a job that uses a “Keychain Credential” with a path of `$.data.myId`.
+
+If a job in your workflow uses the “Keychain Credential” and the initial
+dataclip for a given run looks like this:
+
+```json
+{
+ "data": {
+ "content": "Hello world",
+ "myId": "abc123"
+ }
+}
+```
+
+Then the keychain credential will search for abc123 in your project credentials
+and provide those secrets to the same job. I.e., the will run with the “Taylor’s
+Login” credential.
+
+If another run is executed, and the initial dataclip for that run is:
+
+```json
+{
+ "data": {
+ "content": "Goodbye!",
+ "myId": "def456"
+ }
+}
+```
+
+The same job will run with the “Roina’s Login” credential.
+
+:::info Notes & Limitations
+
+Since credential secrets are fetched at the start of a run (not the start of a
+step) it’s not currently possible to resolve keychain credentials based on data
+that is added to state later in a run. I.e., the data must be present in the
+input dataclip for the whole run, not the input dataclip of the step that uses
+the keychain credential.
+
+:::
+
+### Create a Keychain Credential
+
+1. On the `Credentials` page under project settings, click on the dropdown icon
+ on the `Add New` button to select the Keychain option:
+
+ 
+
+2. Name your keychain credential and assign it a JSONPath expression. You can
+ also select a default credential to use when the JSONPath expression doesn't
+ match:
+
+ 
+
+3. Assign an external ID for your keychain to access by creating a new
+ credential or editing an existing credential:
+
+ 
+
+4. Now on a job in your workflow, you are able to select and use a keychain
+ credential:
+
+ 
+
+5. You can now reference your keychain in your input to use it:
+
+ 
+
+## Share Credentials
If you own a Credential, you can choose which Project has access to it. To
update which Projects your Credential is shared with, follow the steps on the
[User Credentials docs page](/documentation/user-credentials).
-### `Raw JSON` Credentials
+## `Raw JSON` Credentials
Raw credentials are valid JSON documents which are passed into a job's runtime
state. Note that owners of these credentials will be able to view them, in their
entirety, in the clear.
-Raw credentials will work with any adaptor, so long as that adaptor's required `configuration` keys (e.g., `baseUrl`) are specified in your credential. See the "configuration schema" docs for each adaptor to see what is required for that app.
+Raw credentials will work with any adaptor, so long as that adaptor's required
+`configuration` keys (e.g., `baseUrl`) are specified in your credential. See the
+"configuration schema" docs for each adaptor to see what is required for that
+app.
:::info Use `Raw JSON` to specify custom credential inputs
-Use the `Raw JSON` credential type if you would like to store secrets that
-are not standard inputs in an adaptor's credential form. For example, if my REST
-API requires a `client_id` instead of a `username`, then my `configuration`
-schema might look like the below code snippet. Because `client_id` isn't an
-option in the default `Http` credential form, I can create my own custom credential
-using the `Raw JSON` type.
+Use the `Raw JSON` credential type if you would like to store secrets that are
+not standard inputs in an adaptor's credential form. For example, if my REST API
+requires a `client_id` instead of a `username`, then my `configuration` schema
+might look like the below code snippet. Because `client_id` isn't an option in
+the default `Http` credential form, I can create my own custom credential using
+the `Raw JSON` type.
:::
@@ -88,3 +179,9 @@ Example Raw JSON credential body or `configuration`:
"customInput": "whateverYouWant"
}
```
+
+## Credentials Security
+
+All credentials are stored encrypted at rest, and credential secrets can only be
+viewed by credential owners. See OpenFn
+[Security docs](../get-started/security-compliance.md) for more information.
diff --git a/docs/manage-projects/notifications.md b/docs/manage-projects/notifications.md
index 5927e507bbf5..7df6c7aead4f 100644
--- a/docs/manage-projects/notifications.md
+++ b/docs/manage-projects/notifications.md
@@ -16,14 +16,14 @@ workflows.
You can enable failure alerts to receive email notifications when a job fails
under `Project Settings > Collaboration`.
-
+
The email notification includes the logs and a link to the failed run that you
can inspect and begin troubleshooting.
-
+
-
+
### Email Digests
@@ -31,9 +31,9 @@ Also in `Project Settings > Collaboration`, you can choose to receive daily,
weekly or monthly email digests from a project, summarizing successful and
failed runs for each of your workflows.
-
+
-
+
:::note
diff --git a/docs/manage-projects/oauth.md b/docs/manage-projects/oauth.md
index da2caeb9cc7e..db6fdb725e45 100644
--- a/docs/manage-projects/oauth.md
+++ b/docs/manage-projects/oauth.md
@@ -4,12 +4,6 @@ sidebar_label: OAuth Authentication
slug: /oauth
---
-Some applications require OAuth as an authentication method for connecting with
-third party applications making requests via their APIs. This guide walks you
-through how to set up and manage OAuth clients.
-
-### OAuth Authentication
-
Some applications require [OAuth](https://oauth.net/2/) as an authentication
method for connecting with third-party applications and making requests via
their APIs. OpenFn allows you to connect with applications using their OAuth
@@ -17,9 +11,9 @@ authentication. To use this feature in your OpenFn workflows, you need to set up
OAuth clients and credentials for your instances or projects. This guide walks
you through the management of OAuth clients and credentials.
-### Setting up an OAuth client
+## OAuth Clients
-#### What is an OAuth client and when do I need it?
+### What is an OAuth client and when do I need it?
By setting up OAuth for an application, you authorize OpenFn to connect and
interact with this application within a set of scopes defined by you. For
@@ -47,13 +41,13 @@ Oauth clients can be set up either on the
If you do not have an OAuth client already set up for your project, you will see
an empty section with a button prompting you to create a client as shown below.
-
+
Alternatively, you will see the list of existing OAuth clients you have access
to. To create a new client, click the `Add new` button and select
`OAuth client [Advanced]` in the dropdown.
-
+
:::tip
@@ -63,8 +57,8 @@ application. (Note: You should substitue `https://app.openfn.org/` with _your_
OpenFn's deployment base URL if you're not using app.openfn.org.)
For app-specific guidance (e.g., how to set up an Oauth Client
-[for Google Sheets](./adaptors/googlesheets)), refer to the relevant
-[Adaptor documentation](./adaptors) for app-specific guidance
+[for Google Sheets](../adaptors/googlesheets)), refer to the relevant
+[Adaptor documentation](../adaptors) for app-specific guidance
:::
@@ -78,9 +72,9 @@ A super user has the privilege to share OAuth clients with projects in two ways:
They can do this in the OAuth client configuration modal either when creating
the client, or via editing it.
-
+
-#### Making OAuth clients global
+### Making OAuth clients global
When an OAuth client is global, users in the instance can have access to it and
can create credentials from it.
@@ -92,16 +86,18 @@ save changes. All projects on the instance can now access the client and users
with owner, admin and editor rights on these projects can now create credentials
from the client.
-
+
-#### Sharing OAuth clients with projects
+### Sharing OAuth clients with projects
To share an OAuth client with specific projects, scroll down to
`Manage Project Access` section in the OAuth client configuration modal. Select
the project dropdown and select a project and click the add button to grant the
project access to the client.
-
+
+
+## Oauth Credentials
### Creating a credential from an OAuth client
@@ -113,9 +109,9 @@ credentials and are associated with clients.
select `Credential` from the dropdown or click on the
`create a new credential` button.
-
+
-
+
2. Then, in the credential type modal, find and select the Oauth client to use
for creating the OAuth credential. This will open a new modal for you to
@@ -140,7 +136,7 @@ OAuth clients should have the required permissions in the application.
To delete a credential/client, simply click `Delete`.
-
+
A confirmation message pops up to confirm your action.
@@ -157,3 +153,5 @@ Go to the docs on
[managing user credentials](../manage-users/user-credentials.md) to learn more
about credential management for the applications you are integrating with on
OpenFn.
+
+### Example Oauth Client Configuration
diff --git a/docs/manage-projects/platform-mgmt.md b/docs/manage-projects/platform-mgmt.md
index b4eccc77829e..062fa4c2e770 100644
--- a/docs/manage-projects/platform-mgmt.md
+++ b/docs/manage-projects/platform-mgmt.md
@@ -11,10 +11,11 @@ contains their Workflows, Credentials and Collaborators scoped to that project.
We introduced a `Projects` table in `v2.7.14` to help users manage their OpenFn
projects in a table view. This is the new page that you'll see everytime your
-login to your OpenFn account. When users click `Projects` in the menu sidebar,
-you will see the list of projects where you have been granted collaborator access.
+login to your OpenFn account. When users click `Projects` in the menu sidebar,
+you will see the list of projects where you have been granted collaborator
+access.
-
+
## Creating a new Project
@@ -31,19 +32,43 @@ To create a new Project, follow the following steps:
:::info For cloud hosted users on app.openfn.org
1. Projects in a billing account should have unique names.
-2. New projects CANNOT be on a free plan.
+2. Users each get one free starter project. To create a new project you'll need
+ to have a valid payment method and select a plan.
:::
-
+
## Updating Project Information
You can access your Project's information under `Settings` (see app menu
sidebar). Setup allows you to view or edit the project name and description.
-
+
You can also export your entire project "as code" - either to save it, or to
edit your project locally. You can learn more about this feature on our
[Portability page](/documentation/deploy/portability).
+
+## Managing Project Concurrency
+
+OpenFn supports concurrent runs of workflows and projects. This means that
+multiple runs of the same workflow or project can be executed at the same time,
+provided they are configured to run in parallel.
+
+To manage project concurrency, you can use the `Concurrency` section in the
+project settings.
+
+
+
+You can enable or disable parallel execution for a project. When parallel
+execution is disabled, only one run of a workflow in the project can be executed
+at a time.
+
+:::info Project vs workflow level concurrency
+
+Project level concurrency overrides workflow level concurrency. This means that
+if parallel execution is disabled for a project, workflow concurrency settings
+will be ignored.
+
+:::
diff --git a/docs/manage-projects/staging-prod.md b/docs/manage-projects/staging-prod.md
index 806150d4c255..af1a4caf2f18 100644
--- a/docs/manage-projects/staging-prod.md
+++ b/docs/manage-projects/staging-prod.md
@@ -10,30 +10,30 @@ It's a safe and efficient practice to use separate production and staging/testin
1. First, create a `Production` and a `Staging` project on OpenFn (2 projects)
-
+
2. Choose/create a GitHub repo for your project, and create a `staging` branch
-
+
3. Connect your projects to the `main` and `staging` respectively - use [this guide](../manage-projects/link-to-gh.md) to set up the connection
4. In each repo, create an empty `.js` file for your job. Make sure they have the same name and path on each repo (e.g. `upsert-contacts.js`). These will store the code for the job they'll be linked to in the next step.
5. When you connected the branches to your projects in step 3 above, there was a `spec.yaml` file automatically created on the branch after the first sync (along with two other configuration files). Open these files on GitHub, and locate your job in the file. Replace the contents of `body` with: `path: {path to the related js file}`. Do this on both your `main` and `staging` branches.
-
-
+
+
6. You're now all set up!
7. To sync a change from your Staging project to Production **using the OpenFn app**, go to your `Staging` project on OpenFn and make edits to your job. Then go to your project's `Settings` > Sync to `GitHub`, and click `Initiate Sync to Branch`.
8. Alternatively, you can make edits directly to the job code on Github, and commit them to the `staging` branch on Github.
9. Once you've committed edits to your `staging` branch, on Github you'll see an update that there have been recent changes. Click `Compare & pull request`.
-
+
10. Create a Pull Request. This will automatically include all changes that happened to the files on the staging branch.
-
+
11. Depending on your team's Github workflow, either have someone to approve and merge the Pull Request, or click `Merge pull request`.
@@ -61,11 +61,11 @@ You can find more information on this setup in our [Github docs](../manage-proje
2. When this is set up, create a new `staging` branch on Github based on your existing production `main` branch that stores your current project. To do this, on your Github repo click into `Branches` (where it show `1 Branch` in the screenshot below).
-
+
3. Click `New branch`, give it a name like `staging`, make sure the source is `main` if you have multiple branches already. Then click `Create new branch`.
-
+
4. Navigate over to your new `staging` branch. **Here comes an important step. Note how the new branch now contains the 3 configurations files (`config.json`, `spec.yaml` and `state.json`) that were present on the main branch. Delete these from the `staging` branch.** New ones specific to the staging branch will be created in the subsequent steps.
diff --git a/docs/manage-projects/user-roles-permissions.md b/docs/manage-projects/user-roles-permissions.md
index 05b292df099c..f4c6836bdacc 100644
--- a/docs/manage-projects/user-roles-permissions.md
+++ b/docs/manage-projects/user-roles-permissions.md
@@ -8,49 +8,50 @@ are assigned a `Role` that determines their permissions. The four available
Roles are: Owner (**only 1 per project**), Admin, Editor & Viewer. Check out the
table below for the permissions available to each role.
-| Context | Action | Owner | Admin | Editor | Viewer |
-| :-------- | :------------------------------------------------------------------ | :----------------: | :----------------: | :----------------: | :----------------: | --- |
-| Workflows | Create a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: | " |
-| Workflows | Edit a Job in a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: | " |
-| Workflows | Add/remove webhook authentication method for Workflow | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Workflows | Delete a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: | " |
-| Workflows | Run from the Inspector | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: | " |
-| Workflows | Select the 5 latest Inputs for a Job in a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: | " |
-| History | View/search/filter on the History page | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| History | View a Run from the Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| History | View a Input from a Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| History | Run from the Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: | " |
-| Settings | View Project name | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| Settings | Edit Project name | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Settings | View Project description | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| Settings | Edit Project description | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Settings | Export Project | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| Settings | Delete a Project | :heavy_check_mark: | :x: | :x: | :x: | " |
-| Settings | View Project Credentials, type, and owner | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| Settings | Add/remove webhook authentication method for Project | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Settings | Change MFA requirement for Project | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Settings | Add/remove Project Collaborator | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Settings | View Project Collaborators (project_users, role, digest and alerts) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| Settings | Edit digest and alerts for themselves | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | " |
-| Settings | Edit digest and alerts for others | :x: | :x: | :x: | :x: | " |
-| Settings | Change Input/Output Dataclip storage policy | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Settings | Change History retention period | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Settings | Update GitHub project/repo connection | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: | " |
-| Settings | Initiate GitHub sync | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: | " |
+| Context | Action | Owner | Admin | Editor | Viewer |
+| :-------- | :------------------------------------------------------------------ | :----------------- | :----------------- | :----------------- | :----------------- |
+| Workflows | Create a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| Workflows | Edit a Job in a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| Workflows | Add/remove webhook authentication method for Workflow | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Workflows | Delete a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| Workflows | Run from the Inspector | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| Workflows | Select the 5 latest Inputs for a Job in a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| History | View/search/filter on the History page | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| History | View a Run from the Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| History | View a Input from a Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| History | Run from the Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| Settings | View Project name | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Settings | Edit Project name | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Settings | View Project description | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Settings | Edit Project description | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Settings | Export Project | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Settings | Delete a Project | :heavy_check_mark: | :x: | :x: | :x: |
+| Settings | View Project Credentials, type, and owner | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Settings | Add/remove webhook authentication method for Project | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Settings | Change MFA requirement for Project | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Settings | Add/remove Project Collaborator | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Settings | View Project Collaborators (project_users, role, digest and alerts) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Settings | Edit digest and alerts for themselves | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| Settings | Edit digest and alerts for others | :x: | :x: | :x: | :x: |
+| Settings | Change Input/Output Dataclip storage policy | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Settings | Change History retention period | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Settings | Update GitHub project/repo connection | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
+| Settings | Initiate GitHub sync | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |
### Super User privileges
Every OpenFn instance has a user with a Super User role that enables them to
-have full administrative control of the platform. This includes management of users,
-projects, audit trail, and third-party authentication, with the below Super User
-privileges:
+have full administrative control of the platform. This includes management of
+users, projects, audit trail, and third-party authentication, with the below
+Super User privileges:
| Aspect | Description | Features/Permissions |
| ------------------ | ------------------------------------------------------- | ---------------------------------------------------------------- |
-| User Management | The management of users on an OpenFn instance | Creating, editing, removing users |
+| User Management | The management of users on an OpenFn instance | Creating, editing, removing users |
| Project Management | How projects are created and managed on the instance | Create, delete, edit a project, assign users |
| Authentication | Third-party access management for users on the instance | Set up OpenID Auth for the instance |
| Audit Trailing | Auditability and change management | View history of relevant user actions on the instance for audits |
-If you're using the hosted OpenFn platform (e.g., app.openfn.org), contact [support@openfn.org](mailto:support@openfn.org)
-if you need to get in touch with the super user to request new projects or configuration changes.
+If you're using the hosted OpenFn platform (e.g., app.openfn.org), contact
+[support@openfn.org](mailto:support@openfn.org) if you need to get in touch with
+the super user to request new projects or configuration changes.
diff --git a/docs/manage-projects/webhook-auth.md b/docs/manage-projects/webhook-auth.md
index 572632b162c8..f2fa3365673c 100644
--- a/docs/manage-projects/webhook-auth.md
+++ b/docs/manage-projects/webhook-auth.md
@@ -24,26 +24,26 @@ You can add a new Authentication Method under `Webhook Security` of your
`Project Settings`. The authentication you set up here can then be used in any
of your Workflows within this Project.
-
+
After clicking `New auth method`, choose the type - Basic HTTP or API Key
Authentication.
-
+
#### Basic Auth
For Basic Auth, give it a name, choose a username and password, and hit
`Create Auth Method`.
-
+
#### API Key
For API Key, just choose a name, and click `Create Auth Method`. An API key is
generated for you.
-
+
You can edit or delete your auth methods on this page as well.
@@ -52,9 +52,9 @@ You can edit or delete your auth methods on this page as well.
Once you added an auth method to a webhook, it will show up under
`Linked Triggers`.
-
+
-
+
### Adding Authentication Via a Workflow
@@ -69,7 +69,7 @@ these.
Once you've added an auth method, it will show up in your Webhook Trigger
configuration.
-
+
Only requests using these required auth details will be able to send data to
your Workflow.
diff --git a/docs/manage-projects/workflow-dashboard.md b/docs/manage-projects/workflow-dashboard.md
index de5d0fed2753..e64e90a8f6cf 100644
--- a/docs/manage-projects/workflow-dashboard.md
+++ b/docs/manage-projects/workflow-dashboard.md
@@ -12,13 +12,13 @@ On your main Workflows page you can get an overview of the number of the Work
Orders and Runs in your project, and the number and ratio of successes and
failures.
-
+
For further investigation, clicking on the number of Work Orders for the
Workflows as shown bring you to the History of that Workflow. For example, here
are the 7 Work Orders in a failed state:
-
+
Check out our
[History docs](https://docs.openfn.org/documentation/monitor-history/activity-history)
diff --git a/docs/manage-users/api-tokens.md b/docs/manage-users/api-tokens.md
index 470464a96a51..0561af575696 100644
--- a/docs/manage-users/api-tokens.md
+++ b/docs/manage-users/api-tokens.md
@@ -23,13 +23,13 @@ will also have Admin permissions).
You can manage your tokens in your User Profile.
-
+
-
+
1. Click `Generate New Token` to create a new one.
-
+
2. Make sure you copy your new token right away. You won't be able to view or
copy it later.
diff --git a/docs/manage-users/user-credentials.md b/docs/manage-users/user-credentials.md
index d30ac3fd41a6..3c1c39a8f6f8 100644
--- a/docs/manage-users/user-credentials.md
+++ b/docs/manage-users/user-credentials.md
@@ -10,15 +10,15 @@ You can manage all the Credentials you own under the `Credentials` page of your
The `Credentials` page of your `User Settings` allows you to add, view, edit or delete the Credentials you own. It is the central place to manage your Credentials across all projects you collaborate on.
-
+
-
+
For guidance on how to set up a new Credential, head over to our [Manage Credentials](../manage-projects/manage-credentials.md) page.
You can update the name and login details of a Credential after clicking `Edit`.
-
+
### Share Credentials
@@ -27,7 +27,7 @@ You can also allow multiple projects to have access to a Credential you own.
To add or remove project access, click `Edit` on the Credential you want to share, and choose the project from the dropdown under `Project Access`.
-
+
:::info Shared Credentials remain secret
diff --git a/docs/manage-users/user-profile.md b/docs/manage-users/user-profile.md
index 8968535e9aa6..599daf8fdede 100644
--- a/docs/manage-users/user-profile.md
+++ b/docs/manage-users/user-profile.md
@@ -7,21 +7,21 @@ slug: /user-profile
This article walks you through how to view and update your user information in
your User Profile.
-
+
### Change Email and Password
You can change the email address associated with your Profile, and update your
password.
-
+
### Enable Multi-Factor Authentication
By enabling Multi-Factor Authentication, you can add an additional layer of
security to your account by requiring more than just a password for logging in.
-
+
You can link your account to an authenticator app or a browser extension like
1Password or Authy. Once configured, the app will generate a one-time password
@@ -37,4 +37,30 @@ app to set it up.
Your User Profile is also the place where you can delete your OpenFn account.
-
+
+
+To delete your account, click on the **"Delete my account"** button. You will be
+asked to confirm the deletion by entering your email address and clicking
+**"Delete Account"**.
+
+When you confirm your intent to delete your account, your account will be
+scheduled for deletion based on the grace period that your instance
+administrator has set.
+
+:::info
+
+The grace period is the amount of time you have to change your mind and cancel
+the deletion before your account is actually deleted. Default is 7 days.
+
+:::
+
+#### Account Deletion and Auditing
+
+Please note that your account will not be permanently removed from an instance
+if it has been used to manually create work orders or runs until that related
+activity has been deleted. In such cases, you form part of a given project's
+audit trail and the instance administrator may not be allowed to permanently
+delete your account.
+
+If you are using https://app.openfn.org you will need to cancel any active
+subscriptions before you can delete your account.
diff --git a/docs/migration/automated-migration.md b/docs/migration/automated-migration.md
index 8e0dbd258e0e..b088e4e6a439 100644
--- a/docs/migration/automated-migration.md
+++ b/docs/migration/automated-migration.md
@@ -62,7 +62,7 @@ See below example `config.json` file to use as a template.
project. Start by copying the project ID from the URL of your v2 project like
so:
-
+
6. Then run the `openfn pull` CLI command below with the project ID and the path
to your `config.json` file created above.
@@ -77,19 +77,38 @@ project configuration, including any Workflows already configured on v2).
7. Open up your exported v1 `project.yaml` file, copy everything from under
`workflows:`
-
+
8. Paste the copied config in the bottom of `workflows` section of your the
newly created v2 `project.yaml`. (You are manually copying over the v1 config
over to your v2 project's Workflows.)
-
+
-
+
-9. Finally, once you're happy with your new v2 `project.yaml` file, it's time to
- deploy the new config to your v2. Run the following command in the CLI to
- _deploy_.
+9. You may want to disable all of your workflows at this point, so that when
+ they're deployed to v2 they are in a disabled state by default. This also
+ allows you to import all of your workflows regardless of how many active
+ workflows your v2 subscription allows for. To do this, find all the
+ `triggers` in your `project.yaml` (by default, they will be set to `true`),
+ and set them to `false`:
+
+```yaml
+workflows:
+ Sample-Workflow:
+ name: Sample Workflow
+ jobs:
+
+ triggers:
+ webhook:
+ type: webhook
+ enabled: false
+```
+
+10. Finally, once you're happy with your new v2 `project.yaml` file, it's time
+ to deploy the new config to your v2 project. Run the following command in
+ the CLI to _deploy_.
```
openfn deploy -c config.json
@@ -97,7 +116,7 @@ openfn deploy -c config.json
When prompted, confirm you want to deploy by typing `y` ("yes").
-10. If successfully, verify the new Project config on your v2 app.
+10. If successful, verify the new Project config on your v2 app.
:::tip Questions?
diff --git a/docs/migration/converting-triggers.md b/docs/migration/converting-triggers.md
index c3ba7b223b9b..ca1f0a14f7e4 100644
--- a/docs/migration/converting-triggers.md
+++ b/docs/migration/converting-triggers.md
@@ -18,9 +18,8 @@ v1: Message Filters, Cron Triggers, Flow Triggers, and Fail Triggers.
### Converting Cron Triggers
Setting up a
-[Cron Trigger on v2](../build/triggers.md#cron-triggers-formerly-timers)
-works just the same as on
-[v1](../../versioned_docs/version-legacy/build/triggers.md):
+[Cron Trigger on v2](../build/triggers.md#cron-triggers-formerly-timers) works
+just the same as on [v1](../../versioned_docs/version-legacy/build/triggers.md):
when you're building a Workflow, select Cron Schedule as Trigger type, and set
the frequency.
@@ -30,9 +29,9 @@ With a Flow trigger, we can execute a job upon success of another specified job.
With a Fail trigger, the job will run if an another specified job failed.
On v2, we achieve the same conditional behavior with
-[Path Conditions](../build/paths.md): a job can run (1) always, (2), on
-success of another job, (3) on failure of another job, or (4) on a custom
-condition - we'll get to this last one in the next section.
+[Path Conditions](../build/paths.md): a job can run (1) always, (2), on success
+of another job, (3) on failure of another job, or (4) on a custom condition -
+we'll get to this last one in the next section.
So, if on v1 you had a Flow Trigger, on v2 you'll need to choose the
`On Success` condition between your jobs. If you had a Fail Trigger, select the
@@ -44,15 +43,34 @@ V1 Message Filters work with webhooks: if an external application sends data to
your project, you can check whether the incoming message meets (or doesn't meet)
certain criteria, and execute jobs accordingly.
-V2 works on the same principle, with a somewhat different setup. Once you've
-configured your [Webhook](../build/triggers.md#webhook-event-triggers),
-you can use a custom Path Condition that matches a JavaScript expression to
-decide whether a subsequent job should be executed or not.
+V2 works on the same principle, with a somewhat different setup.
+
+#### One Webhook per Workflow
+
+On V1, you had one unified Inbox for your entire project, where all messages
+would arrive to the same shared webhook URL from every source application. These
+were then picked up by the different jobs based on Message Filter Triggers.
+
+
+
+The main difference on V2 is that each workflow that operates with a webhook
+trigger has its own unique webhook URL to receive data to.
+
+
+
+This will mean that in your source applications you will probably need to configure multiple webhooks/forwarders, one for each of your recieving workflows, instead of the previous common Inbox one.
+
+#### Path Conditions
+
+Once you've configured your
+[Webhook](../build/triggers.md#webhook-event-triggers), you can use a custom
+Path Condition that matches a JavaScript expression to decide whether a
+subsequent job should be executed or not.
For example, if on v1 you had a Message Filter trigger that looked like this:
-
+
The matching v2 JavaScript Path Condition would look like this:
-
+
diff --git a/docs/migration/migration-steps.md b/docs/migration/migration-steps.md
index c5022b9ccb07..0d3ff12b93aa 100644
--- a/docs/migration/migration-steps.md
+++ b/docs/migration/migration-steps.md
@@ -37,6 +37,14 @@ decisions. For customized migration support, ask your questions on our
version control. Follow [this guide](../manage-projects/link-to-gh.md) to
learn how it works and set it up.
+ :::warning Turn off GitHub sync on v1 before setting it up on v2 If you're
+ using the same GitHub repo and branch for both your v1 and v2 project,
+ disable GitHub sync on v1 _before_ you enable it on v2. Otherwise every
+ change you still make on v1 will trigger a GitHub > OpenFn sync on v2,
+ overwriting any changes you may not have synced yet from your v2 project to GitHub.
+
+ :::
+
:::warning GitHub sync works differently in v2
Once the GitHub sync is live, consider that all changes will be synced with
@@ -72,10 +80,12 @@ decisions. For customized migration support, ask your questions on our
adjusted your Project Settings.
14. When all Workflows run successfully, update each Step in your Workflows to
use a "production" Credential to connect to live systems.
-15. While you're testing, you may be using [Path Conditions](../build/paths.md) to allow
- only test data, such as `test_case == yes`. If you then want to exclude test
- data from your production systems, don't forget to update edge conditions,
- eg. `test_case == no`. Check out [this guide(https://docs.openfn.org/documentation/converting-triggers#converting-message-filters)] for a specific example.
+15. While you're testing, you may be using [Path Conditions](../build/paths.md)
+ to allow only test data, such as `test_case == yes`. If you then want to
+ exclude test data from your production systems, don't forget to update edge
+ conditions, eg. `test_case == no`. Check out [this
+ guide(https://docs.openfn.org/documentation/converting-triggers#converting-message-filters)]
+ for a specific example.
16. If webhooks are used in your source applications, update the webhook
configurations in these apps to point to your v2 OpenFn Workflows (you can
locate your Workflow's new webhook endpoint URL by clicking n the Trigger).
@@ -83,10 +93,14 @@ decisions. For customized migration support, ask your questions on our
Workflows and monitor usage on your
[Workflows Dashboard](../manage-projects/workflow-dashboard.md). Now time to
shut down your v1 project.
-18. Turn "off" your Jobs on v1 and delete the GitHub connection in your v1
- Project Settings to disable version control.
-19. You have the option to export some of your v1 data: `Messages` and `Run History` for
- reference or archival. To do this, visit the `Downloads` page in your v1 project. Your most recent downloadable `receipts archives` and `runs archives` are at the bottom of the export list. You can generate a new export by navigating to the `Inbox` or `Activity History` page, filtering your view to query the desired data to export, and then select the cloud ☁ icon to `Export to csv`.
+18. Turn "off" your Jobs on v1.
+19. You have the option to export some of your v1 data: `Messages` and
+ `Run History` for reference or archival. To do this, visit the `Downloads`
+ page in your v1 project. Your most recent downloadable `receipts archives`
+ and `runs archives` are at the bottom of the export list. You can generate a
+ new export by navigating to the `Inbox` or `Activity History` page,
+ filtering your view to query the desired data to export, and then select the
+ cloud ☁ icon to `Export to csv`.
20. Finally, when ready, request to delete your project on v1. To do this, go to
your v1 `Project Settings` and select the `Delete Project` button.
diff --git a/docs/monitor-history/activity-history.md b/docs/monitor-history/activity-history.md
index 2d96a7803f41..2f620ec2d1f4 100644
--- a/docs/monitor-history/activity-history.md
+++ b/docs/monitor-history/activity-history.md
@@ -14,7 +14,7 @@ The `History` page provides a list of all of the
[Runs](/documentation/get-started/terminology#run) that have been processed in a
Project.
-
+
## Workflow execution: Work Orders and Runs
@@ -34,7 +34,7 @@ OpenFn Workflows are executed as follows:
`Run` will be created. If successful, then both the Run and related Work
Order will be updated with a `success` status.
-
+
Check out the other pages in this docs section to learn more about inspecting
Runs, troubleshooting, and rerunning failed Runs.
@@ -46,7 +46,7 @@ input/output dataclips or run logs contain specific text strings. By default,
the system will search run logs only but you can select to search any or all of
three options:
-
+
1. OpenFn UUIDs for workorders, runs, or steps
2. Input/Output dataclip bodies
@@ -57,6 +57,19 @@ search is applied. This method of searching allows you to find work orders
quickly and allows for partial string matches across all text in the run logs
and across the "keys" and "values" of your dataclips.
+:::caution Very large/complex input dataclips may not be indexed
+
+It's not currently possible to create `ts_vector` indexes larger than 1MB, and
+as a result very large or complex input dataclips may not appear in search
+results. This typically won't happen until you're nearing 10MB of JSON, but the
+number of distinct lexemes & positions in your JSON will impact the final index
+size.
+
+More at the Postgres
+["text search limitations" docs page](https://www.postgresql.org/docs/current/textsearch-limitations.html).
+
+:::
+
Partial string matching works best at the start of words, so if you're looking
for items matching `"newPatient"` it's better to search for `"newPat"` than for
`"tient"`. (When in doubt, whole words or IDs produce the best results.)
diff --git a/docs/monitor-history/troubleshooting.md b/docs/monitor-history/troubleshooting.md
index 06db41bb4e76..32f7d0a542e3 100644
--- a/docs/monitor-history/troubleshooting.md
+++ b/docs/monitor-history/troubleshooting.md
@@ -1,42 +1,61 @@
---
-title: Troubleshooting
-sidebar_label: Troubleshooting
+title: Logs & Troubleshooting
+sidebar_label: Logs & Troubleshooting
+keywords:
+ - runs
+ - logs
+ - log levels
+ - status codes
+ - exit codes
+ - troubleshooting
---
-This page provides troubleshooting tips for *OpenFn v2 platform* users.
+This page provides troubleshooting tips for _OpenFn v2 platform_ users.
## Runs
One of the most helpful pages for troubleshooting on OpenFn is the
-[History](./activity-history.md) page.
-This page provides a list of all of the runs executed for a Work Order and their status. Project administrators can troubleshoot errors by clicking into the run to review the run details. Learn more about runs [here](./inspect-runs.md) here.
+[History](./activity-history.md) page. This page provides a list of all of the
+runs executed for a Work Order and their status. Project administrators can
+troubleshoot errors by clicking into the run to review the run details. Learn
+more about runs [here](./inspect-runs.md) here.
### Status codes
-Every run will have a status code. The status code is a way for OpenFn to classify
-the run status and can help you troubleshoot errors. Learn more about OpenFn
-status codes and what each one means [here](./status-codes.md).
+Every run will have a status code. The status code is a way for OpenFn to
+classify the run status and can help you troubleshoot errors. Learn more about
+OpenFn status codes and what each one means [here](./status-codes.md).
### The time it took for the workflow to fail
The run will also record how long it took before the workflow failed. This
-information helps users understand if the workflow is taking longer than it should
-and is especially helpful with errors that involve timeouts. You can use the run
-to determine at which operation the workflow is timing out and determine if the workflow
-performance can be optimized.
+information helps users understand if the workflow is taking longer than it
+should and is especially helpful with errors that involve timeouts. You can use
+the run to determine at which operation the workflow is timing out and determine
+if the workflow performance can be optimized.
### Run logs
-As workflows are developed it is important to log details which will make testing and
-troubleshooting much easier in the future. Keep reading for the two most
-important parts of a run log!
+As workflows are developed it is important to log details which will make
+testing and troubleshooting much easier in the future.
+
+#### Log Levels
+
+
+
+| Level | Description |
+| ------- | ------------------------------------------------------------------------------------------------------------------------------------ |
+| `debug` | Shows all logs, including system-level stuff produced by the "runtime" and the output of a user-entered `console.debug()` statement. |
+| `info` | The default log level, shows key information produced by the adaptors or `console.log()`/`console.info()` statements. |
+| `warn` | Hides most of the noise and only shows major run events (step start/end), adaptor warnings, or `console.warn()` statements. |
+| `error` | Hides all but major run events, adaptor errors, and `console.error()` statements. |
#### Mappings
-If possible, the logs should be written so that you can see exactly what was mapped between
-the source system and the destination system. In summary,
-the log can have a **"Data received from source system"** section and a **"Data
-to be uploaded to destination system"** section.
+If possible, the logs should be written so that you can see exactly what was
+mapped between the source system and the destination system. In summary, the log
+can have a **"Data received from source system"** section and a **"Data to be
+uploaded to destination system"** section.
These logs can help admins verify that the source data and the data being
uploaded to the destination system is correct. For example, seeing in the logs
@@ -61,13 +80,13 @@ Other error messages are not as clear and can take some time to debug:
`TypeError [Error]: Cannot read property 'split' of undefined`
-**`TypeErrors`** usually indicate that the job received a part of the input
-that it wasn't expecting, or there is a syntax error in your job code. It means
-that the job needs to be updated to know how to handle the input. In this
-case, the job received an old version of the Commcare form which was missing a
-field on which the job called the `split` function. You can determine this by
-reviewing the job for which fields the split function is being called on and
-checking that they are all present in the message.
+**`TypeErrors`** usually indicate that the job received a part of the input that
+it wasn't expecting, or there is a syntax error in your job code. It means that
+the job needs to be updated to know how to handle the input. In this case, the
+job received an old version of the Commcare form which was missing a field on
+which the job called the `split` function. You can determine this by reviewing
+the job for which fields the split function is being called on and checking that
+they are all present in the message.
The more you test and troubleshoot with a particular system, the more familiar
with its error messages you become.
@@ -82,15 +101,18 @@ their error messages [here](/adaptors#connect-anything).
## Leveraging search and filtering in OpenFn
-Leverage the various search functionalities in OpenFn to find the right runs to support your troubleshooting. You can search on the History page across OpenFn IDs, Inputs, and/or Logs.
-
-Check out this [video](https://youtu.be/XIUykmLCxwQ?si=hquc8rPTJrAZkbbD) for how to use Search.
-
+Leverage the various search functionalities in OpenFn to find the right runs to
+support your troubleshooting. You can search on the History page across OpenFn
+IDs, Inputs, and/or Logs.
+Check out this [video](https://youtu.be/XIUykmLCxwQ?si=hquc8rPTJrAZkbbD) for how
+to use Search.
## Sign up for email alerts
-You can turn on notifications to receive [email alerts](../manage-projects/notifications.md) when a workflow fails and subscribe to digests that summarize project activity.
+You can turn on notifications to receive
+[email alerts](../manage-projects/notifications.md) when a workflow fails and
+subscribe to digests that summarize project activity.
## More
diff --git a/docs/tutorials/commcare-to-db.md b/docs/tutorials/commcare-to-db.md
index ca1067b4ef1c..2a0161f9aaa1 100644
--- a/docs/tutorials/commcare-to-db.md
+++ b/docs/tutorials/commcare-to-db.md
@@ -24,7 +24,7 @@ also follow along with the prebuilt solution. Follow along at the links below:**
- Username: testuser
- Password: 123
-
+
3. [Public report that shows records in the PostgreSQL database](https://analytics.openfn.org/public/question/095449a9-5696-463c-a4fb-24614c9f08a5)
@@ -45,7 +45,7 @@ data on patients registered!
:::
-
+
**This integration can be broken up into two parts:**
@@ -68,7 +68,7 @@ workflow. After data forwarding is set up, it happens automatically, **_in
real-time for all forms and cases_**. Learn more about configuring a webhook
[here](/adaptors/commcare#webhook-forward-cases-andor-forms-from-commcare-to-openfn-using-rest-service).
-
+
### Option 2: Extracting Commcare data via the REST API
@@ -108,7 +108,7 @@ Make sure you have copied the webhook URL from your OpenFn workflow into CommCar
for how to create your own `mapping specification document` to map data
elements to be exchanged.
-
+
@@ -160,7 +160,7 @@ configuration according to your mapping specifications.
automatically run.
5. Check out the `History` and ensure that the work order was successful.
-
+
:::info
@@ -177,7 +177,7 @@ configuration according to your mapping specifications.
4. **Finally, refresh your database and check out the new submission data!**
-
+
While this guide is specifically for PostgreSQL databases, you can generally
follow these same steps for other database types (e.g., MS SQL or MySQL)—simply
diff --git a/docs/tutorials/http-to-googlesheets.md b/docs/tutorials/http-to-googlesheets.md
index 52fc900a8d45..ff1abea1c410 100644
--- a/docs/tutorials/http-to-googlesheets.md
+++ b/docs/tutorials/http-to-googlesheets.md
@@ -96,7 +96,16 @@ Create a new Googlesheet `Credential` using your Google account's email. (Make
sure this Google user has edit access to the GoogleSheet you want to integrate
with.)
-For this demo, we have configured the Googlesheet
+:::info Don't see a GoogleSheets credential option?
+
+If your instance superuser hasn't configured a global Oauth client, you may need
+to set one up for yourself. Learn about Oauth Clients
+[here](/documentation/oauth#oauth-clients) and specifics for a GoogleSheet
+Client [here](/adaptors/googlesheets#permissions-scopes).
+
+:::
+
+For this demo, we configured the Googlesheet
[like this](https://docs.google.com/spreadsheets/d/1gT4cpHSDQp8A_JIX_5lqTLTwV0xBo_u8u3ZNWALmCLc/edit?usp=sharing)
to store the `users` data.
diff --git a/docusaurus.config.js b/docusaurus.config.js
index 9acc96fa06e5..c7fd6482c441 100644
--- a/docusaurus.config.js
+++ b/docusaurus.config.js
@@ -18,6 +18,15 @@ module.exports = {
},
themes: ['@docusaurus/theme-mermaid'],
themeConfig: {
+ metadata: [
+ {name: 'keywords', content: 'OpenFn, workflow automation, ICT4D, integration, automation, documentation'},
+ {name: 'description', content: 'OpenFn documentation - The leading digital public good for workflow automation, making ICT4D more efficient.'},
+ {name: 'twitter:card', content: 'summary_large_image'},
+ {name: 'twitter:site', content: '@openfn'},
+ {name: 'twitter:title', content: 'OpenFn Documentation'},
+ {name: 'twitter:description', content: 'The leading digital public good for workflow automation, OpenFn makes ICT4D more efficient.'},
+ {name: 'twitter:image', content: 'https://docs.openfn.org/img/og-image.png'},
+ ],
algolia: {
appId: 'O729P2PJGT',
apiKey: 'a929e9c0d8fb90f86a1f39f8b6f1816e',
@@ -144,6 +153,12 @@ module.exports = {
googleTagManager: {
containerId: 'GTM-5RNRM5NC',
},
+ sitemap: {
+ changefreq: 'weekly',
+ priority: 0.5,
+ ignorePatterns: ['/tags/**'],
+ filename: 'sitemap.xml',
+ },
},
],
],
@@ -180,6 +195,22 @@ module.exports = {
},
},
],
+ [
+ '@docusaurus/plugin-client-redirects',
+ {
+ redirects: [
+ {
+ to: '/documentation/contribute/roadmap',
+ from: '/documentation/contribute/openfn-roadmap',
+ },
+ // Redirect from multiple old paths to the new path
+ // {
+ // to: '/docs/newDoc2',
+ // from: ['/docs/oldDocFrom2019', '/docs/legacyDocFrom2016'],
+ // },
+ ],
+ },
+ ],
],
scripts: [
{
diff --git a/generate-adaptors/index.js b/generate-adaptors/index.js
index 7a0da52ce5e3..88b4d2b66c85 100644
--- a/generate-adaptors/index.js
+++ b/generate-adaptors/index.js
@@ -88,6 +88,22 @@ function pushToPaths(name) {
}
function generateJsDoc(a) {
+ // Add line break before tags and escape curly braces outside of code blocks
+ let docsContent = JSON.parse(a.docs).replace(/<\/dt>/g, '\n');
+
+ // Split content by code blocks (both inline ` and multi-line ```)
+ const codeBlockRegex = /(```[\s\S]*?```|`[^`]*`)/g;
+ const parts = docsContent.split(codeBlockRegex);
+
+ // Escape curly braces only in non-code parts (odd indices are code blocks)
+ for (let i = 0; i < parts.length; i++) {
+ if (i % 2 === 0) { // Non-code parts
+ parts[i] = parts[i].replace(/{/g, '\\{').replace(/}/g, '\\}');
+ }
+ }
+
+ docsContent = parts.join('');
+
return `---
title: ${a.name}@${a.version}
id: ${a.name}-docs
@@ -97,7 +113,7 @@ keywords:
${a.functions.length > 0 ? '- ' : ''}${a.functions.join('\r\n - ')}
---
-${JSON.parse(a.docs)}`;
+${docsContent}`;
}
function generateChangelog(a) {
@@ -209,7 +225,6 @@ async function buildAdaptors(monorepoPath) {
console.log('Generating adaptors docs via JSDoc...');
adaptors.map(a => {
if (!a.name) {
- console.warn('WARNING: No name for ', a);
return;
}
diff --git a/package.json b/package.json
index 46fd827deaf1..f7252ab4bd3b 100644
--- a/package.json
+++ b/package.json
@@ -7,7 +7,7 @@
"generate-library": "docusaurus generate-library",
"generate-adaptors": "docusaurus generate-adaptors",
"start": "docusaurus generate-adaptors & docusaurus start",
- "start:dev": "docusaurus generate-adaptors -m && docusaurus start",
+ "start:dev": "docusaurus generate-adaptors -m && docusaurus generate-library && docusaurus start",
"start-offline": "docusaurus start",
"build": "docusaurus build",
"swizzle": "docusaurus swizzle",
@@ -17,21 +17,26 @@
},
"dependencies": {
"@babel/helper-module-transforms": "^7.17.7",
- "@docusaurus/core": "2.4.3",
- "@docusaurus/plugin-google-gtag": "2.4.3",
- "@docusaurus/plugin-google-tag-manager": "^3.5.2",
- "@docusaurus/preset-classic": "2.4.3",
- "@docusaurus/theme-mermaid": "2.4.3",
- "@mdx-js/react": "^1.6.21",
+ "@docusaurus/core": "3.8.1",
+ "@docusaurus/plugin-client-redirects": "^3.8.1",
+ "@docusaurus/plugin-google-gtag": "3.8.1",
+ "@docusaurus/plugin-google-tag-manager": "3.8.1",
+ "@docusaurus/plugin-sitemap": "^3.8.1",
+ "@docusaurus/preset-classic": "3.8.1",
+ "@docusaurus/theme-mermaid": "3.8.1",
+ "@mdx-js/react": "^3.0.0",
+ "axios": "^1.6.0",
"clsx": "^1.1.1",
"compare-versions": "^5.0.1",
"date-fns": "2.21.3",
+ "prism-react-renderer": "^2.1.0",
"raw-loader": "^4.0.2",
- "react": "^16.8.4",
- "react-dom": "^16.8.4",
+ "react": "^18.2.0",
+ "react-dom": "^18.2.0",
"react-flow-renderer": "^9.6.8",
"react-particles": "^2.5.3",
"react-player": "^2.9.0",
+ "sharp": "^0.34.2",
"tsparticles": "^2.5.3"
},
"browserslist": {
@@ -47,19 +52,24 @@
]
},
"devDependencies": {
- "@docusaurus/module-type-aliases": "2.4.3",
- "@tsconfig/docusaurus": "^1.0.2",
+ "@docusaurus/module-type-aliases": "3.8.1",
+ "@docusaurus/tsconfig": "3.8.1",
+ "@docusaurus/types": "3.8.1",
"@types/node": "^14.14.22",
- "@types/react": "^17.0.0",
+ "@types/react": "^18.2.29",
"@types/react-helmet": "^6.1.0",
"@types/react-router-dom": "^5.1.7",
"chokidar": "^3.6.0",
- "typescript": "^4.1.3"
+ "glob": "^11.0.3",
+ "typescript": "~5.2.2"
},
"resolutions": {
"trim": "^0.0.3",
"got": "^11.8.5",
"cytoscape": "3.28.1"
},
+ "engines": {
+ "node": ">=18.0"
+ },
"packageManager": "yarn@3.2.3"
}
diff --git a/scripts/optimize-images.js b/scripts/optimize-images.js
new file mode 100644
index 000000000000..36468117b806
--- /dev/null
+++ b/scripts/optimize-images.js
@@ -0,0 +1,53 @@
+const fs = require('fs').promises;
+const path = require('path');
+const sharp = require('sharp');
+const glob = require('glob');
+
+const ROOT_DIR = path.join(__dirname, '..');
+const IMAGES_DIR = path.join(ROOT_DIR, 'static/img');
+
+async function optimizeImages() {
+ // Find all PNG, JPG, and JPEG images
+ const images = glob.sync('**/*.{png,jpg,jpeg}', {
+ cwd: IMAGES_DIR,
+ ignore: ['**/node_modules/**']
+ });
+
+ console.log(`Found ${images.length} images to optimize`);
+
+ // Convert each image to WebP
+ for (const image of images) {
+ const inputPath = path.join(IMAGES_DIR, image);
+ const outputPath = inputPath.replace(/\.(png|jpg|jpeg)$/, '.webp');
+
+ try {
+ await sharp(inputPath).webp().toFile(outputPath);
+ console.log(`Converted ${image} to WebP`);
+
+ // Delete the original image
+ await fs.unlink(inputPath);
+ console.log(`Deleted original ${image}`);
+ } catch (error) {
+ console.error(`Error processing ${image}:`, error);
+ }
+ }
+
+ // Update references in markdown files
+ const mdFiles = glob.sync('**/*.md', {
+ cwd: ROOT_DIR,
+ ignore: ['**/node_modules/**']
+ });
+
+ for (const mdFile of mdFiles) {
+ const filePath = path.join(ROOT_DIR, mdFile);
+ let content = await fs.readFile(filePath, 'utf8');
+
+ // Replace image extensions in markdown
+ content = content.replace(/\.(png|jpg|jpeg)(?=\))/g, '.webp');
+
+ await fs.writeFile(filePath, content, 'utf8');
+ console.log(`Updated references in ${mdFile}`);
+ }
+}
+
+optimizeImages().catch(console.error);
\ No newline at end of file
diff --git a/sidebars-main.js b/sidebars-main.js
index 73552380fb76..57698b0ee577 100644
--- a/sidebars-main.js
+++ b/sidebars-main.js
@@ -70,6 +70,7 @@ module.exports = {
'build/working-with-branches',
'build/troubleshooting',
'build/workflow-snapshots',
+ 'build/workflows-api',
],
},
{
@@ -111,6 +112,7 @@ module.exports = {
],
},
'hosted/overview',
+ 'keyboard-shortcuts',
],
},
@@ -122,6 +124,7 @@ module.exports = {
'build-for-developers/cli-usage',
'build-for-developers/cli-walkthrough',
'build-for-developers/cli-challenges',
+ 'build-for-developers/cli-collections',
],
},
{
@@ -143,7 +146,8 @@ module.exports = {
type: 'category',
label: 'Contribute',
items: [
- 'contribute/openfn-roadmap',
+ 'contribute/roadmap',
+ 'contribute/impact',
'contribute/writing-code',
'contribute/writing-docs',
'contribute/style-guide',
diff --git a/src/css/custom.css b/src/css/custom.css
index 4930c472bb32..731ebccb56da 100644
--- a/src/css/custom.css
+++ b/src/css/custom.css
@@ -5,7 +5,7 @@
* work well for content-centric websites.
*/
-@import url('https://fonts.googleapis.com/css?family=Roboto&display=swap');
+@import url('https://fonts.googleapis.com/css?family=Inter&display=swap');
/* @import url('https://fonts.googleapis.com/css2?family=IBM+Plex+Sans&display=swap'); */
/* You can override the default Infima variables here. */
@@ -18,7 +18,7 @@
--ifm-color-primary-lighter: #48a9f5;
--ifm-color-primary-lightest: #70bbf7;
--ifm-code-font-size: 95%;
- font-family: 'Roboto';
+ font-family: 'Inter', sans-serif;
/* font-family: 'IBM Plex Sans', sans-serif; */
}
diff --git a/static/img/1_branch.png b/static/img/1_branch.png
deleted file mode 100644
index 4f4a528499d0..000000000000
Binary files a/static/img/1_branch.png and /dev/null differ
diff --git a/static/img/1_branch.webp b/static/img/1_branch.webp
new file mode 100644
index 000000000000..a51056836d12
Binary files /dev/null and b/static/img/1_branch.webp differ
diff --git a/static/img/2.1_new_account_dashboard.png b/static/img/2.1_new_account_dashboard.png
deleted file mode 100644
index 8b6d21b5f875..000000000000
Binary files a/static/img/2.1_new_account_dashboard.png and /dev/null differ
diff --git a/static/img/2.1_new_account_dashboard.webp b/static/img/2.1_new_account_dashboard.webp
new file mode 100644
index 000000000000..4b2af8886faf
Binary files /dev/null and b/static/img/2.1_new_account_dashboard.webp differ
diff --git a/static/img/2.2_inbox_url.png b/static/img/2.2_inbox_url.png
deleted file mode 100644
index 727c28366535..000000000000
Binary files a/static/img/2.2_inbox_url.png and /dev/null differ
diff --git a/static/img/2.2_inbox_url.webp b/static/img/2.2_inbox_url.webp
new file mode 100644
index 000000000000..f9eb107072d7
Binary files /dev/null and b/static/img/2.2_inbox_url.webp differ
diff --git a/static/img/2.3_kobo_rest.png b/static/img/2.3_kobo_rest.png
deleted file mode 100644
index 89f16052afdc..000000000000
Binary files a/static/img/2.3_kobo_rest.png and /dev/null differ
diff --git a/static/img/2.3_kobo_rest.webp b/static/img/2.3_kobo_rest.webp
new file mode 100644
index 000000000000..c4fec5acb6b3
Binary files /dev/null and b/static/img/2.3_kobo_rest.webp differ
diff --git a/static/img/2.4_kobo_rest.png b/static/img/2.4_kobo_rest.png
deleted file mode 100644
index e125c95597a4..000000000000
Binary files a/static/img/2.4_kobo_rest.png and /dev/null differ
diff --git a/static/img/2.4_kobo_rest.webp b/static/img/2.4_kobo_rest.webp
new file mode 100644
index 000000000000..e2834f6d85ba
Binary files /dev/null and b/static/img/2.4_kobo_rest.webp differ
diff --git a/static/img/2.5_open_kobo_form.png b/static/img/2.5_open_kobo_form.png
deleted file mode 100644
index 6a82b69a14d9..000000000000
Binary files a/static/img/2.5_open_kobo_form.png and /dev/null differ
diff --git a/static/img/2.5_open_kobo_form.webp b/static/img/2.5_open_kobo_form.webp
new file mode 100644
index 000000000000..efd7df93df64
Binary files /dev/null and b/static/img/2.5_open_kobo_form.webp differ
diff --git a/static/img/2.6_inbox.png b/static/img/2.6_inbox.png
deleted file mode 100644
index 649d9bc22265..000000000000
Binary files a/static/img/2.6_inbox.png and /dev/null differ
diff --git a/static/img/2.6_inbox.webp b/static/img/2.6_inbox.webp
new file mode 100644
index 000000000000..0131ef6d5634
Binary files /dev/null and b/static/img/2.6_inbox.webp differ
diff --git a/static/img/2.7_message.png b/static/img/2.7_message.png
deleted file mode 100644
index 2dc0bd2ad1ef..000000000000
Binary files a/static/img/2.7_message.png and /dev/null differ
diff --git a/static/img/2.7_message.webp b/static/img/2.7_message.webp
new file mode 100644
index 000000000000..f74113e2ed7e
Binary files /dev/null and b/static/img/2.7_message.webp differ
diff --git a/static/img/2.8_common_data_point.png b/static/img/2.8_common_data_point.png
deleted file mode 100644
index fe935b878839..000000000000
Binary files a/static/img/2.8_common_data_point.png and /dev/null differ
diff --git a/static/img/2.8_common_data_point.webp b/static/img/2.8_common_data_point.webp
new file mode 100644
index 000000000000..a83dcdb7879b
Binary files /dev/null and b/static/img/2.8_common_data_point.webp differ
diff --git a/static/img/3.1_create_credentials.png b/static/img/3.1_create_credentials.png
deleted file mode 100644
index def24ed14d6e..000000000000
Binary files a/static/img/3.1_create_credentials.png and /dev/null differ
diff --git a/static/img/3.1_create_credentials.webp b/static/img/3.1_create_credentials.webp
new file mode 100644
index 000000000000..98bdd89726f0
Binary files /dev/null and b/static/img/3.1_create_credentials.webp differ
diff --git a/static/img/3.2_select_credential_type.png b/static/img/3.2_select_credential_type.png
deleted file mode 100644
index 23a0c6bd6411..000000000000
Binary files a/static/img/3.2_select_credential_type.png and /dev/null differ
diff --git a/static/img/3.2_select_credential_type.webp b/static/img/3.2_select_credential_type.webp
new file mode 100644
index 000000000000..05bc07b794c5
Binary files /dev/null and b/static/img/3.2_select_credential_type.webp differ
diff --git a/static/img/4.1_new_job.png b/static/img/4.1_new_job.png
deleted file mode 100644
index 302357cc75bb..000000000000
Binary files a/static/img/4.1_new_job.png and /dev/null differ
diff --git a/static/img/4.1_new_job.webp b/static/img/4.1_new_job.webp
new file mode 100644
index 000000000000..dd5cd55ec296
Binary files /dev/null and b/static/img/4.1_new_job.webp differ
diff --git a/static/img/4.2_new_trigger.png b/static/img/4.2_new_trigger.png
deleted file mode 100644
index bcd8fd16062a..000000000000
Binary files a/static/img/4.2_new_trigger.png and /dev/null differ
diff --git a/static/img/4.2_new_trigger.webp b/static/img/4.2_new_trigger.webp
new file mode 100644
index 000000000000..5a7e4a17a356
Binary files /dev/null and b/static/img/4.2_new_trigger.webp differ
diff --git a/static/img/4.3_trigger_message.png b/static/img/4.3_trigger_message.png
deleted file mode 100644
index 4baddc4644cf..000000000000
Binary files a/static/img/4.3_trigger_message.png and /dev/null differ
diff --git a/static/img/4.3_trigger_message.webp b/static/img/4.3_trigger_message.webp
new file mode 100644
index 000000000000..beb056cd3e96
Binary files /dev/null and b/static/img/4.3_trigger_message.webp differ
diff --git a/static/img/4.4_adaptor_operation.png b/static/img/4.4_adaptor_operation.png
deleted file mode 100644
index 1b28e2d55700..000000000000
Binary files a/static/img/4.4_adaptor_operation.png and /dev/null differ
diff --git a/static/img/4.4_adaptor_operation.webp b/static/img/4.4_adaptor_operation.webp
new file mode 100644
index 000000000000..3f473632c092
Binary files /dev/null and b/static/img/4.4_adaptor_operation.webp differ
diff --git a/static/img/4.5_sheets_id.png b/static/img/4.5_sheets_id.png
deleted file mode 100644
index a0a7a7cff2e8..000000000000
Binary files a/static/img/4.5_sheets_id.png and /dev/null differ
diff --git a/static/img/4.5_sheets_id.webp b/static/img/4.5_sheets_id.webp
new file mode 100644
index 000000000000..bb8e5847f5d1
Binary files /dev/null and b/static/img/4.5_sheets_id.webp differ
diff --git a/static/img/4.6_select_values.png b/static/img/4.6_select_values.png
deleted file mode 100644
index ec9c88a5b369..000000000000
Binary files a/static/img/4.6_select_values.png and /dev/null differ
diff --git a/static/img/4.6_select_values.webp b/static/img/4.6_select_values.webp
new file mode 100644
index 000000000000..e139e8a3cd31
Binary files /dev/null and b/static/img/4.6_select_values.webp differ
diff --git a/static/img/4.7_save_and_run.png b/static/img/4.7_save_and_run.png
deleted file mode 100644
index 7a51462ba651..000000000000
Binary files a/static/img/4.7_save_and_run.png and /dev/null differ
diff --git a/static/img/4.7_save_and_run.webp b/static/img/4.7_save_and_run.webp
new file mode 100644
index 000000000000..2fc0fc24f36b
Binary files /dev/null and b/static/img/4.7_save_and_run.webp differ
diff --git a/static/img/5.1_autoprocess.png b/static/img/5.1_autoprocess.png
deleted file mode 100644
index 4ce70b58058e..000000000000
Binary files a/static/img/5.1_autoprocess.png and /dev/null differ
diff --git a/static/img/5.1_autoprocess.webp b/static/img/5.1_autoprocess.webp
new file mode 100644
index 000000000000..5670b1b7afdc
Binary files /dev/null and b/static/img/5.1_autoprocess.webp differ
diff --git a/static/img/BPMN_example.png b/static/img/BPMN_example.png
deleted file mode 100644
index 0664c20e2175..000000000000
Binary files a/static/img/BPMN_example.png and /dev/null differ
diff --git a/static/img/BPMN_example.webp b/static/img/BPMN_example.webp
new file mode 100644
index 000000000000..d05f816ac154
Binary files /dev/null and b/static/img/BPMN_example.webp differ
diff --git a/static/img/access_security1.png b/static/img/access_security1.png
deleted file mode 100644
index d087390e5101..000000000000
Binary files a/static/img/access_security1.png and /dev/null differ
diff --git a/static/img/access_security1.webp b/static/img/access_security1.webp
new file mode 100644
index 000000000000..0aa07bbae814
Binary files /dev/null and b/static/img/access_security1.webp differ
diff --git a/static/img/access_security_collaboration1.png b/static/img/access_security_collaboration1.png
deleted file mode 100644
index 251f301b580e..000000000000
Binary files a/static/img/access_security_collaboration1.png and /dev/null differ
diff --git a/static/img/access_security_collaboration1.webp b/static/img/access_security_collaboration1.webp
new file mode 100644
index 000000000000..e97a4c700b15
Binary files /dev/null and b/static/img/access_security_collaboration1.webp differ
diff --git a/static/img/activity_history_cc.png b/static/img/activity_history_cc.png
deleted file mode 100644
index 3702da3af54d..000000000000
Binary files a/static/img/activity_history_cc.png and /dev/null differ
diff --git a/static/img/activity_history_cc.webp b/static/img/activity_history_cc.webp
new file mode 100644
index 000000000000..30b0c69d5767
Binary files /dev/null and b/static/img/activity_history_cc.webp differ
diff --git a/static/img/activity_history_final.png b/static/img/activity_history_final.png
deleted file mode 100644
index 0851bd68028f..000000000000
Binary files a/static/img/activity_history_final.png and /dev/null differ
diff --git a/static/img/activity_history_final.webp b/static/img/activity_history_final.webp
new file mode 100644
index 000000000000..555a5d0f53c2
Binary files /dev/null and b/static/img/activity_history_final.webp differ
diff --git a/static/img/activity_history_success.png b/static/img/activity_history_success.png
deleted file mode 100644
index e9a0b1c71f5b..000000000000
Binary files a/static/img/activity_history_success.png and /dev/null differ
diff --git a/static/img/activity_history_success.webp b/static/img/activity_history_success.webp
new file mode 100644
index 000000000000..a666e969524e
Binary files /dev/null and b/static/img/activity_history_success.webp differ
diff --git a/static/img/adaptor_choice_openfn.png b/static/img/adaptor_choice_openfn.png
deleted file mode 100644
index 07632f42a3c8..000000000000
Binary files a/static/img/adaptor_choice_openfn.png and /dev/null differ
diff --git a/static/img/adaptor_choice_openfn.webp b/static/img/adaptor_choice_openfn.webp
new file mode 100644
index 000000000000..8e28747389cb
Binary files /dev/null and b/static/img/adaptor_choice_openfn.webp differ
diff --git a/static/img/adaptor_npm.png b/static/img/adaptor_npm.png
deleted file mode 100644
index 768c4f1758bf..000000000000
Binary files a/static/img/adaptor_npm.png and /dev/null differ
diff --git a/static/img/adaptor_npm.webp b/static/img/adaptor_npm.webp
new file mode 100644
index 000000000000..5932aa5ff33c
Binary files /dev/null and b/static/img/adaptor_npm.webp differ
diff --git a/static/img/add_collab.png b/static/img/add_collab.png
deleted file mode 100644
index 0220465b970b..000000000000
Binary files a/static/img/add_collab.png and /dev/null differ
diff --git a/static/img/add_collab.webp b/static/img/add_collab.webp
new file mode 100644
index 000000000000..447ff1a2b6e7
Binary files /dev/null and b/static/img/add_collab.webp differ
diff --git a/static/img/ai-assistant.png b/static/img/ai-assistant.png
deleted file mode 100644
index ffb4fdce27ed..000000000000
Binary files a/static/img/ai-assistant.png and /dev/null differ
diff --git a/static/img/ai-assistant.webp b/static/img/ai-assistant.webp
new file mode 100644
index 000000000000..6c77f171fe4b
Binary files /dev/null and b/static/img/ai-assistant.webp differ
diff --git a/static/img/ai-workflow-example.webp b/static/img/ai-workflow-example.webp
new file mode 100644
index 000000000000..0bf45d03c374
Binary files /dev/null and b/static/img/ai-workflow-example.webp differ
diff --git a/static/img/airport.jpeg b/static/img/airport.jpeg
deleted file mode 100644
index 4367ba8027d0..000000000000
Binary files a/static/img/airport.jpeg and /dev/null differ
diff --git a/static/img/airport.webp b/static/img/airport.webp
new file mode 100644
index 000000000000..856e15b41ab6
Binary files /dev/null and b/static/img/airport.webp differ
diff --git a/static/img/akira-chix.jpg b/static/img/akira-chix.jpg
deleted file mode 100644
index 05fc6db91217..000000000000
Binary files a/static/img/akira-chix.jpg and /dev/null differ
diff --git a/static/img/akira-chix.webp b/static/img/akira-chix.webp
new file mode 100644
index 000000000000..27ac2e4bf905
Binary files /dev/null and b/static/img/akira-chix.webp differ
diff --git a/static/img/anatomy_of_step.png b/static/img/anatomy_of_step.png
deleted file mode 100644
index 5d5876d8b59c..000000000000
Binary files a/static/img/anatomy_of_step.png and /dev/null differ
diff --git a/static/img/anatomy_of_step.webp b/static/img/anatomy_of_step.webp
new file mode 100644
index 000000000000..a8c5ef919693
Binary files /dev/null and b/static/img/anatomy_of_step.webp differ
diff --git a/static/img/api_diagram.png b/static/img/api_diagram.png
deleted file mode 100644
index 279224135421..000000000000
Binary files a/static/img/api_diagram.png and /dev/null differ
diff --git a/static/img/api_diagram.webp b/static/img/api_diagram.webp
new file mode 100644
index 000000000000..0168064409bf
Binary files /dev/null and b/static/img/api_diagram.webp differ
diff --git a/static/img/api_example.png b/static/img/api_example.png
deleted file mode 100644
index a477e3e08964..000000000000
Binary files a/static/img/api_example.png and /dev/null differ
diff --git a/static/img/api_example.webp b/static/img/api_example.webp
new file mode 100644
index 000000000000..7f228879c479
Binary files /dev/null and b/static/img/api_example.webp differ
diff --git a/static/img/apps.png b/static/img/apps.png
deleted file mode 100644
index 69599aca698c..000000000000
Binary files a/static/img/apps.png and /dev/null differ
diff --git a/static/img/apps.webp b/static/img/apps.webp
new file mode 100644
index 000000000000..e3caa12a7690
Binary files /dev/null and b/static/img/apps.webp differ
diff --git a/static/img/ase.jpg b/static/img/ase.jpg
deleted file mode 100644
index efd787310224..000000000000
Binary files a/static/img/ase.jpg and /dev/null differ
diff --git a/static/img/ase.webp b/static/img/ase.webp
new file mode 100644
index 000000000000..33afd822b1d7
Binary files /dev/null and b/static/img/ase.webp differ
diff --git a/static/img/assign_externalID.webp b/static/img/assign_externalID.webp
new file mode 100644
index 000000000000..69dc45b5e05c
Binary files /dev/null and b/static/img/assign_externalID.webp differ
diff --git a/static/img/authentication-vs-athorization.png b/static/img/authentication-vs-athorization.png
deleted file mode 100644
index 8930c5f334ee..000000000000
Binary files a/static/img/authentication-vs-athorization.png and /dev/null differ
diff --git a/static/img/authentication-vs-athorization.webp b/static/img/authentication-vs-athorization.webp
new file mode 100644
index 000000000000..629b63d423b6
Binary files /dev/null and b/static/img/authentication-vs-athorization.webp differ
diff --git a/static/img/box5.png b/static/img/box5.png
deleted file mode 100644
index 3ef16f432980..000000000000
Binary files a/static/img/box5.png and /dev/null differ
diff --git a/static/img/box5.webp b/static/img/box5.webp
new file mode 100644
index 000000000000..5db0419e3be7
Binary files /dev/null and b/static/img/box5.webp differ
diff --git a/static/img/brazil_students.png b/static/img/brazil_students.png
deleted file mode 100644
index 901319488991..000000000000
Binary files a/static/img/brazil_students.png and /dev/null differ
diff --git a/static/img/brazil_students.webp b/static/img/brazil_students.webp
new file mode 100644
index 000000000000..382c029b2135
Binary files /dev/null and b/static/img/brazil_students.webp differ
diff --git a/static/img/bulk_reprocess.png b/static/img/bulk_reprocess.png
deleted file mode 100644
index 41ebfc3afbc1..000000000000
Binary files a/static/img/bulk_reprocess.png and /dev/null differ
diff --git a/static/img/bulk_reprocess.webp b/static/img/bulk_reprocess.webp
new file mode 100644
index 000000000000..ced4cbf667f9
Binary files /dev/null and b/static/img/bulk_reprocess.webp differ
diff --git a/static/img/calendar.jpg b/static/img/calendar.jpg
deleted file mode 100644
index 658323656303..000000000000
Binary files a/static/img/calendar.jpg and /dev/null differ
diff --git a/static/img/calendar.webp b/static/img/calendar.webp
new file mode 100644
index 000000000000..e5e1f53c66af
Binary files /dev/null and b/static/img/calendar.webp differ
diff --git a/static/img/cambodia-kids.png b/static/img/cambodia-kids.png
deleted file mode 100644
index f6528ff3645f..000000000000
Binary files a/static/img/cambodia-kids.png and /dev/null differ
diff --git a/static/img/cambodia-kids.webp b/static/img/cambodia-kids.webp
new file mode 100644
index 000000000000..84fb686eb142
Binary files /dev/null and b/static/img/cambodia-kids.webp differ
diff --git a/static/img/cambodia_child.png b/static/img/cambodia_child.png
deleted file mode 100644
index 018e6dc7016d..000000000000
Binary files a/static/img/cambodia_child.png and /dev/null differ
diff --git a/static/img/cambodia_child.webp b/static/img/cambodia_child.webp
new file mode 100644
index 000000000000..735d442e90c5
Binary files /dev/null and b/static/img/cambodia_child.webp differ
diff --git a/static/img/cambodia_dashboard.png b/static/img/cambodia_dashboard.png
deleted file mode 100644
index 427146ce5f98..000000000000
Binary files a/static/img/cambodia_dashboard.png and /dev/null differ
diff --git a/static/img/cambodia_dashboard.webp b/static/img/cambodia_dashboard.webp
new file mode 100644
index 000000000000..23ba7d7292f6
Binary files /dev/null and b/static/img/cambodia_dashboard.webp differ
diff --git a/static/img/cambodia_school.png b/static/img/cambodia_school.png
deleted file mode 100644
index a5a2a4918f3a..000000000000
Binary files a/static/img/cambodia_school.png and /dev/null differ
diff --git a/static/img/cambodia_school.webp b/static/img/cambodia_school.webp
new file mode 100644
index 000000000000..51b83a708522
Binary files /dev/null and b/static/img/cambodia_school.webp differ
diff --git a/static/img/carisfoundation.jpeg b/static/img/carisfoundation.jpeg
deleted file mode 100644
index d4b334ca6bd4..000000000000
Binary files a/static/img/carisfoundation.jpeg and /dev/null differ
diff --git a/static/img/carisfoundation.webp b/static/img/carisfoundation.webp
new file mode 100644
index 000000000000..02590eb3572a
Binary files /dev/null and b/static/img/carisfoundation.webp differ
diff --git a/static/img/case-referral-history.png b/static/img/case-referral-history.png
deleted file mode 100644
index 3ef980da6082..000000000000
Binary files a/static/img/case-referral-history.png and /dev/null differ
diff --git a/static/img/case-referral-history.webp b/static/img/case-referral-history.webp
new file mode 100644
index 000000000000..0f691e473a9a
Binary files /dev/null and b/static/img/case-referral-history.webp differ
diff --git a/static/img/case_referral_workflow.png b/static/img/case_referral_workflow.png
deleted file mode 100644
index 6c58ad6d51c5..000000000000
Binary files a/static/img/case_referral_workflow.png and /dev/null differ
diff --git a/static/img/case_referral_workflow.webp b/static/img/case_referral_workflow.webp
new file mode 100644
index 000000000000..c7bb9fcac144
Binary files /dev/null and b/static/img/case_referral_workflow.webp differ
diff --git a/static/img/cc-postgres.png b/static/img/cc-postgres.png
deleted file mode 100644
index 13ba374e0b54..000000000000
Binary files a/static/img/cc-postgres.png and /dev/null differ
diff --git a/static/img/cc-postgres.webp b/static/img/cc-postgres.webp
new file mode 100644
index 000000000000..6b2ad9c40aa6
Binary files /dev/null and b/static/img/cc-postgres.webp differ
diff --git a/static/img/chaining-operations.png b/static/img/chaining-operations.png
deleted file mode 100644
index 08844adafda2..000000000000
Binary files a/static/img/chaining-operations.png and /dev/null differ
diff --git a/static/img/chaining-operations.webp b/static/img/chaining-operations.webp
new file mode 100644
index 000000000000..13607161aa8d
Binary files /dev/null and b/static/img/chaining-operations.webp differ
diff --git a/static/img/collaboration.png b/static/img/collaboration.png
deleted file mode 100644
index 3ed7a1b67d33..000000000000
Binary files a/static/img/collaboration.png and /dev/null differ
diff --git a/static/img/collaboration.webp b/static/img/collaboration.webp
new file mode 100644
index 000000000000..b2b644c16b8c
Binary files /dev/null and b/static/img/collaboration.webp differ
diff --git a/static/img/collections_admin.png b/static/img/collections_admin.png
deleted file mode 100644
index d86a0dd63527..000000000000
Binary files a/static/img/collections_admin.png and /dev/null differ
diff --git a/static/img/collections_admin.webp b/static/img/collections_admin.webp
new file mode 100644
index 000000000000..994e9f426c7e
Binary files /dev/null and b/static/img/collections_admin.webp differ
diff --git a/static/img/commcare-data-model.png b/static/img/commcare-data-model.png
deleted file mode 100644
index c5a3abd1a40f..000000000000
Binary files a/static/img/commcare-data-model.png and /dev/null differ
diff --git a/static/img/commcare-data-model.webp b/static/img/commcare-data-model.webp
new file mode 100644
index 000000000000..294c97c93a2d
Binary files /dev/null and b/static/img/commcare-data-model.webp differ
diff --git a/static/img/commcare_credential.png b/static/img/commcare_credential.png
deleted file mode 100644
index 329a846ef923..000000000000
Binary files a/static/img/commcare_credential.png and /dev/null differ
diff --git a/static/img/commcare_credential.webp b/static/img/commcare_credential.webp
new file mode 100644
index 000000000000..57079fe981a8
Binary files /dev/null and b/static/img/commcare_credential.webp differ
diff --git a/static/img/commcare_credential_edit.png b/static/img/commcare_credential_edit.png
deleted file mode 100644
index da99b8427885..000000000000
Binary files a/static/img/commcare_credential_edit.png and /dev/null differ
diff --git a/static/img/commcare_credential_edit.webp b/static/img/commcare_credential_edit.webp
new file mode 100644
index 000000000000..5024ff96c29b
Binary files /dev/null and b/static/img/commcare_credential_edit.webp differ
diff --git a/static/img/commcare_data_forwarding_overview.png b/static/img/commcare_data_forwarding_overview.png
deleted file mode 100644
index 819493b0925e..000000000000
Binary files a/static/img/commcare_data_forwarding_overview.png and /dev/null differ
diff --git a/static/img/commcare_data_forwarding_overview.webp b/static/img/commcare_data_forwarding_overview.webp
new file mode 100644
index 000000000000..0f79d9a6c13a
Binary files /dev/null and b/static/img/commcare_data_forwarding_overview.webp differ
diff --git a/static/img/commcare_forward_cases.png b/static/img/commcare_forward_cases.png
deleted file mode 100644
index 2b8e18511220..000000000000
Binary files a/static/img/commcare_forward_cases.png and /dev/null differ
diff --git a/static/img/commcare_forward_cases.webp b/static/img/commcare_forward_cases.webp
new file mode 100644
index 000000000000..1030b15ec88d
Binary files /dev/null and b/static/img/commcare_forward_cases.webp differ
diff --git a/static/img/commecare_connection_settings.png b/static/img/commecare_connection_settings.png
deleted file mode 100644
index 88266902724d..000000000000
Binary files a/static/img/commecare_connection_settings.png and /dev/null differ
diff --git a/static/img/commecare_connection_settings.webp b/static/img/commecare_connection_settings.webp
new file mode 100644
index 000000000000..5bcc32f248bb
Binary files /dev/null and b/static/img/commecare_connection_settings.webp differ
diff --git a/static/img/commecare_forward_forms.png b/static/img/commecare_forward_forms.png
deleted file mode 100644
index 9f8bc1551b8d..000000000000
Binary files a/static/img/commecare_forward_forms.png and /dev/null differ
diff --git a/static/img/commecare_forward_forms.webp b/static/img/commecare_forward_forms.webp
new file mode 100644
index 000000000000..20de58dc9ed3
Binary files /dev/null and b/static/img/commecare_forward_forms.webp differ
diff --git a/static/img/configuring-concurrency.png b/static/img/configuring-concurrency.png
deleted file mode 100644
index 415486e0c784..000000000000
Binary files a/static/img/configuring-concurrency.png and /dev/null differ
diff --git a/static/img/configuring-concurrency.webp b/static/img/configuring-concurrency.webp
new file mode 100644
index 000000000000..2051ceac2485
Binary files /dev/null and b/static/img/configuring-concurrency.webp differ
diff --git a/static/img/configuring-kafka.png b/static/img/configuring-kafka.png
deleted file mode 100644
index e4f97397d423..000000000000
Binary files a/static/img/configuring-kafka.png and /dev/null differ
diff --git a/static/img/configuring-kafka.webp b/static/img/configuring-kafka.webp
new file mode 100644
index 000000000000..06b725b0fc1e
Binary files /dev/null and b/static/img/configuring-kafka.webp differ
diff --git a/static/img/configuring-log-outputs.webp b/static/img/configuring-log-outputs.webp
new file mode 100644
index 000000000000..c223ae44c44f
Binary files /dev/null and b/static/img/configuring-log-outputs.webp differ
diff --git a/static/img/configuring-project-concurrency.webp b/static/img/configuring-project-concurrency.webp
new file mode 100644
index 000000000000..f3f87984edc7
Binary files /dev/null and b/static/img/configuring-project-concurrency.webp differ
diff --git a/static/img/confirm_export.png b/static/img/confirm_export.png
deleted file mode 100644
index 8b7d9e249f45..000000000000
Binary files a/static/img/confirm_export.png and /dev/null differ
diff --git a/static/img/confirm_export.webp b/static/img/confirm_export.webp
new file mode 100644
index 000000000000..99d1f3a619f1
Binary files /dev/null and b/static/img/confirm_export.webp differ
diff --git a/static/img/connect-account-to-github.png b/static/img/connect-account-to-github.png
deleted file mode 100644
index 793ca8f0278d..000000000000
Binary files a/static/img/connect-account-to-github.png and /dev/null differ
diff --git a/static/img/connect-account-to-github.webp b/static/img/connect-account-to-github.webp
new file mode 100644
index 000000000000..83c18279310a
Binary files /dev/null and b/static/img/connect-account-to-github.webp differ
diff --git a/static/img/create-project-modal.png b/static/img/create-project-modal.png
deleted file mode 100644
index 13c03ace6d77..000000000000
Binary files a/static/img/create-project-modal.png and /dev/null differ
diff --git a/static/img/create-project-modal.webp b/static/img/create-project-modal.webp
new file mode 100644
index 000000000000..ff48e5d0173b
Binary files /dev/null and b/static/img/create-project-modal.webp differ
diff --git a/static/img/create_new_cred.png b/static/img/create_new_cred.png
deleted file mode 100644
index 292a6739fd90..000000000000
Binary files a/static/img/create_new_cred.png and /dev/null differ
diff --git a/static/img/create_new_cred.webp b/static/img/create_new_cred.webp
new file mode 100644
index 000000000000..63c4237091b8
Binary files /dev/null and b/static/img/create_new_cred.webp differ
diff --git a/static/img/create_new_oauth_client.png b/static/img/create_new_oauth_client.png
deleted file mode 100644
index 88a068da144d..000000000000
Binary files a/static/img/create_new_oauth_client.png and /dev/null differ
diff --git a/static/img/create_new_oauth_client.webp b/static/img/create_new_oauth_client.webp
new file mode 100644
index 000000000000..1c3a7dcaeb89
Binary files /dev/null and b/static/img/create_new_oauth_client.webp differ
diff --git a/static/img/create_pr.png b/static/img/create_pr.png
deleted file mode 100644
index 5701cb9b0291..000000000000
Binary files a/static/img/create_pr.png and /dev/null differ
diff --git a/static/img/create_pr.webp b/static/img/create_pr.webp
new file mode 100644
index 000000000000..70e7cc2f97f0
Binary files /dev/null and b/static/img/create_pr.webp differ
diff --git a/static/img/credentials.png b/static/img/credentials.png
deleted file mode 100644
index 325eb579a52e..000000000000
Binary files a/static/img/credentials.png and /dev/null differ
diff --git a/static/img/credentials.webp b/static/img/credentials.webp
new file mode 100644
index 000000000000..db27482ea3a0
Binary files /dev/null and b/static/img/credentials.webp differ
diff --git a/static/img/cron_trigger.png b/static/img/cron_trigger.png
deleted file mode 100644
index 4f4ef93e0e05..000000000000
Binary files a/static/img/cron_trigger.png and /dev/null differ
diff --git a/static/img/cron_trigger.webp b/static/img/cron_trigger.webp
new file mode 100644
index 000000000000..759c797eba8a
Binary files /dev/null and b/static/img/cron_trigger.webp differ
diff --git a/static/img/cronhourglass.jpg b/static/img/cronhourglass.jpg
deleted file mode 100644
index 4531ec1797b8..000000000000
Binary files a/static/img/cronhourglass.jpg and /dev/null differ
diff --git a/static/img/cronhourglass.webp b/static/img/cronhourglass.webp
new file mode 100644
index 000000000000..12f361c4dd63
Binary files /dev/null and b/static/img/cronhourglass.webp differ
diff --git a/static/img/crowsfeet2.png b/static/img/crowsfeet2.png
deleted file mode 100644
index e11a1f066613..000000000000
Binary files a/static/img/crowsfeet2.png and /dev/null differ
diff --git a/static/img/crowsfeet2.webp b/static/img/crowsfeet2.webp
new file mode 100644
index 000000000000..194ae86e8cda
Binary files /dev/null and b/static/img/crowsfeet2.webp differ
diff --git a/static/img/custom.webp b/static/img/custom.webp
new file mode 100644
index 000000000000..22972587db3b
Binary files /dev/null and b/static/img/custom.webp differ
diff --git a/static/img/data-element-mapping.png b/static/img/data-element-mapping.png
deleted file mode 100644
index 3d33fdd5d3dc..000000000000
Binary files a/static/img/data-element-mapping.png and /dev/null differ
diff --git a/static/img/data-element-mapping.webp b/static/img/data-element-mapping.webp
new file mode 100644
index 000000000000..dcb758778ddf
Binary files /dev/null and b/static/img/data-element-mapping.webp differ
diff --git a/static/img/data_integration.png b/static/img/data_integration.png
deleted file mode 100644
index 84b589b1ef7b..000000000000
Binary files a/static/img/data_integration.png and /dev/null differ
diff --git a/static/img/data_integration.webp b/static/img/data_integration.webp
new file mode 100644
index 000000000000..3720942a9dce
Binary files /dev/null and b/static/img/data_integration.webp differ
diff --git a/static/img/db_config.png b/static/img/db_config.png
deleted file mode 100644
index 8187e6309dd7..000000000000
Binary files a/static/img/db_config.png and /dev/null differ
diff --git a/static/img/db_config.webp b/static/img/db_config.webp
new file mode 100644
index 000000000000..763b57403dff
Binary files /dev/null and b/static/img/db_config.webp differ
diff --git a/static/img/deployment_paths.png b/static/img/deployment_paths.png
deleted file mode 100644
index 7f122f47c8ec..000000000000
Binary files a/static/img/deployment_paths.png and /dev/null differ
diff --git a/static/img/deployment_paths.webp b/static/img/deployment_paths.webp
new file mode 100644
index 000000000000..94b87f243689
Binary files /dev/null and b/static/img/deployment_paths.webp differ
diff --git a/static/img/designforfailure.jpg b/static/img/designforfailure.jpg
deleted file mode 100644
index e115020feed1..000000000000
Binary files a/static/img/designforfailure.jpg and /dev/null differ
diff --git a/static/img/designforfailure.webp b/static/img/designforfailure.webp
new file mode 100644
index 000000000000..ae267044e3c2
Binary files /dev/null and b/static/img/designforfailure.webp differ
diff --git a/static/img/digest_email.png b/static/img/digest_email.png
deleted file mode 100644
index b40cf9e234b6..000000000000
Binary files a/static/img/digest_email.png and /dev/null differ
diff --git a/static/img/digest_email.webp b/static/img/digest_email.webp
new file mode 100644
index 000000000000..68b9fb372cdf
Binary files /dev/null and b/static/img/digest_email.webp differ
diff --git a/static/img/disable-wf.png b/static/img/disable-wf.png
deleted file mode 100644
index 5a5aa8b6871e..000000000000
Binary files a/static/img/disable-wf.png and /dev/null differ
diff --git a/static/img/disable-wf.webp b/static/img/disable-wf.webp
new file mode 100644
index 000000000000..cfad2238f466
Binary files /dev/null and b/static/img/disable-wf.webp differ
diff --git a/static/img/edit_job_vscode.png b/static/img/edit_job_vscode.png
deleted file mode 100644
index c47aa504a552..000000000000
Binary files a/static/img/edit_job_vscode.png and /dev/null differ
diff --git a/static/img/edit_job_vscode.webp b/static/img/edit_job_vscode.webp
new file mode 100644
index 000000000000..51fd30c3b4fc
Binary files /dev/null and b/static/img/edit_job_vscode.webp differ
diff --git a/static/img/edit_steps.png b/static/img/edit_steps.png
deleted file mode 100644
index 6aaac825bafb..000000000000
Binary files a/static/img/edit_steps.png and /dev/null differ
diff --git a/static/img/edit_steps.webp b/static/img/edit_steps.webp
new file mode 100644
index 000000000000..fee3f44bdbcb
Binary files /dev/null and b/static/img/edit_steps.webp differ
diff --git a/static/img/empty.webp b/static/img/empty.webp
new file mode 100644
index 000000000000..22a6463197ef
Binary files /dev/null and b/static/img/empty.webp differ
diff --git a/static/img/example-workflow-state.png b/static/img/example-workflow-state.png
deleted file mode 100644
index a2de4a5bbd15..000000000000
Binary files a/static/img/example-workflow-state.png and /dev/null differ
diff --git a/static/img/example-workflow-state.webp b/static/img/example-workflow-state.webp
new file mode 100644
index 000000000000..62f8f9beda96
Binary files /dev/null and b/static/img/example-workflow-state.webp differ
diff --git a/static/img/existing.webp b/static/img/existing.webp
new file mode 100644
index 000000000000..44d68b5c6a6a
Binary files /dev/null and b/static/img/existing.webp differ
diff --git a/static/img/export-messages.png b/static/img/export-messages.png
deleted file mode 100644
index c75a5331a2e6..000000000000
Binary files a/static/img/export-messages.png and /dev/null differ
diff --git a/static/img/export-messages.webp b/static/img/export-messages.webp
new file mode 100644
index 000000000000..f673ca8e33d5
Binary files /dev/null and b/static/img/export-messages.webp differ
diff --git a/static/img/export-runs.png b/static/img/export-runs.png
deleted file mode 100644
index 0be739a41c2a..000000000000
Binary files a/static/img/export-runs.png and /dev/null differ
diff --git a/static/img/export-runs.webp b/static/img/export-runs.webp
new file mode 100644
index 000000000000..c5f04000fe48
Binary files /dev/null and b/static/img/export-runs.webp differ
diff --git a/static/img/exportcsv.png b/static/img/exportcsv.png
deleted file mode 100644
index 9053d78f4468..000000000000
Binary files a/static/img/exportcsv.png and /dev/null differ
diff --git a/static/img/exportcsv.webp b/static/img/exportcsv.webp
new file mode 100644
index 000000000000..9155e176f28b
Binary files /dev/null and b/static/img/exportcsv.webp differ
diff --git a/static/img/exportruns.png b/static/img/exportruns.png
deleted file mode 100644
index cca0d9052cb0..000000000000
Binary files a/static/img/exportruns.png and /dev/null differ
diff --git a/static/img/exportruns.webp b/static/img/exportruns.webp
new file mode 100644
index 000000000000..1279239e08c7
Binary files /dev/null and b/static/img/exportruns.webp differ
diff --git a/static/img/farmers.jpg b/static/img/farmers.jpg
deleted file mode 100644
index ca89ce39aa70..000000000000
Binary files a/static/img/farmers.jpg and /dev/null differ
diff --git a/static/img/farmers.webp b/static/img/farmers.webp
new file mode 100644
index 000000000000..8f32df1a05ca
Binary files /dev/null and b/static/img/farmers.webp differ
diff --git a/static/img/form_workflow.png b/static/img/form_workflow.png
deleted file mode 100644
index 7c51ace8b051..000000000000
Binary files a/static/img/form_workflow.png and /dev/null differ
diff --git a/static/img/form_workflow.webp b/static/img/form_workflow.webp
new file mode 100644
index 000000000000..c717236574c9
Binary files /dev/null and b/static/img/form_workflow.webp differ
diff --git a/static/img/foundational-functional-identity.png b/static/img/foundational-functional-identity.png
deleted file mode 100644
index 554645fab527..000000000000
Binary files a/static/img/foundational-functional-identity.png and /dev/null differ
diff --git a/static/img/foundational-functional-identity.webp b/static/img/foundational-functional-identity.webp
new file mode 100644
index 000000000000..14c7a99ce9aa
Binary files /dev/null and b/static/img/foundational-functional-identity.webp differ
diff --git a/static/img/functional_example.png b/static/img/functional_example.png
deleted file mode 100644
index 9480c3fdb4d4..000000000000
Binary files a/static/img/functional_example.png and /dev/null differ
diff --git a/static/img/functional_example.webp b/static/img/functional_example.webp
new file mode 100644
index 000000000000..d75f8b649f98
Binary files /dev/null and b/static/img/functional_example.webp differ
diff --git a/static/img/george-washington-bridge.jpg b/static/img/george-washington-bridge.jpg
deleted file mode 100644
index 9e6de42ba3e8..000000000000
Binary files a/static/img/george-washington-bridge.jpg and /dev/null differ
diff --git a/static/img/george-washington-bridge.webp b/static/img/george-washington-bridge.webp
new file mode 100644
index 000000000000..d97b6bfc9905
Binary files /dev/null and b/static/img/george-washington-bridge.webp differ
diff --git a/static/img/git-status.png b/static/img/git-status.png
deleted file mode 100644
index 122171c37291..000000000000
Binary files a/static/img/git-status.png and /dev/null differ
diff --git a/static/img/git-status.webp b/static/img/git-status.webp
new file mode 100644
index 000000000000..5ca0a67bf6fa
Binary files /dev/null and b/static/img/git-status.webp differ
diff --git a/static/img/git_clone_url.png b/static/img/git_clone_url.png
deleted file mode 100644
index 78518cded4ac..000000000000
Binary files a/static/img/git_clone_url.png and /dev/null differ
diff --git a/static/img/git_clone_url.webp b/static/img/git_clone_url.webp
new file mode 100644
index 000000000000..bd6c2e1067f8
Binary files /dev/null and b/static/img/git_clone_url.webp differ
diff --git a/static/img/github-options.png b/static/img/github-options.png
deleted file mode 100644
index 5edc06c012e9..000000000000
Binary files a/static/img/github-options.png and /dev/null differ
diff --git a/static/img/github-options.webp b/static/img/github-options.webp
new file mode 100644
index 000000000000..b3d4e5511a59
Binary files /dev/null and b/static/img/github-options.webp differ
diff --git a/static/img/google-oauth2.png b/static/img/google-oauth2.png
deleted file mode 100644
index 5dda596fd50a..000000000000
Binary files a/static/img/google-oauth2.png and /dev/null differ
diff --git a/static/img/google-oauth2.webp b/static/img/google-oauth2.webp
new file mode 100644
index 000000000000..654a1b283279
Binary files /dev/null and b/static/img/google-oauth2.webp differ
diff --git a/static/img/history-page-annotated.png b/static/img/history-page-annotated.png
deleted file mode 100644
index d4afd6e14220..000000000000
Binary files a/static/img/history-page-annotated.png and /dev/null differ
diff --git a/static/img/history-page-annotated.webp b/static/img/history-page-annotated.webp
new file mode 100644
index 000000000000..8fc345c5bf62
Binary files /dev/null and b/static/img/history-page-annotated.webp differ
diff --git a/static/img/history_exports_page.png b/static/img/history_exports_page.png
deleted file mode 100644
index 766899e1289c..000000000000
Binary files a/static/img/history_exports_page.png and /dev/null differ
diff --git a/static/img/history_exports_page.webp b/static/img/history_exports_page.webp
new file mode 100644
index 000000000000..d84f63bd754c
Binary files /dev/null and b/static/img/history_exports_page.webp differ
diff --git a/static/img/history_page_cloud.png b/static/img/history_page_cloud.png
deleted file mode 100644
index df1d0788e6b6..000000000000
Binary files a/static/img/history_page_cloud.png and /dev/null differ
diff --git a/static/img/history_page_cloud.webp b/static/img/history_page_cloud.webp
new file mode 100644
index 000000000000..e2abcff0d2e4
Binary files /dev/null and b/static/img/history_page_cloud.webp differ
diff --git a/static/img/hourglass.jpg b/static/img/hourglass.jpg
deleted file mode 100644
index 761eabd62876..000000000000
Binary files a/static/img/hourglass.jpg and /dev/null differ
diff --git a/static/img/hourglass.webp b/static/img/hourglass.webp
new file mode 100644
index 000000000000..60ee628ea5c4
Binary files /dev/null and b/static/img/hourglass.webp differ
diff --git a/static/img/identity-lifecycle.png b/static/img/identity-lifecycle.png
deleted file mode 100644
index a7525a667a86..000000000000
Binary files a/static/img/identity-lifecycle.png and /dev/null differ
diff --git a/static/img/identity-lifecycle.webp b/static/img/identity-lifecycle.webp
new file mode 100644
index 000000000000..96039420487e
Binary files /dev/null and b/static/img/identity-lifecycle.webp differ
diff --git a/static/img/ikapa2.jpg b/static/img/ikapa2.jpg
deleted file mode 100644
index 2516fcb916bf..000000000000
Binary files a/static/img/ikapa2.jpg and /dev/null differ
diff --git a/static/img/ikapa2.webp b/static/img/ikapa2.webp
new file mode 100644
index 000000000000..dd5c8dc73421
Binary files /dev/null and b/static/img/ikapa2.webp differ
diff --git a/static/img/inbox-security.png b/static/img/inbox-security.png
deleted file mode 100644
index d2e7ead3ffef..000000000000
Binary files a/static/img/inbox-security.png and /dev/null differ
diff --git a/static/img/inbox-security.webp b/static/img/inbox-security.webp
new file mode 100644
index 000000000000..2354fca3449b
Binary files /dev/null and b/static/img/inbox-security.webp differ
diff --git a/static/img/inbox_filter.png b/static/img/inbox_filter.png
deleted file mode 100644
index 5ff82a5e3e2d..000000000000
Binary files a/static/img/inbox_filter.png and /dev/null differ
diff --git a/static/img/inbox_filter.webp b/static/img/inbox_filter.webp
new file mode 100644
index 000000000000..9dfb55964a0a
Binary files /dev/null and b/static/img/inbox_filter.webp differ
diff --git a/static/img/informationorganized.jpg b/static/img/informationorganized.jpg
deleted file mode 100644
index 61949fdf5ffd..000000000000
Binary files a/static/img/informationorganized.jpg and /dev/null differ
diff --git a/static/img/informationorganized.webp b/static/img/informationorganized.webp
new file mode 100644
index 000000000000..0b0339b47649
Binary files /dev/null and b/static/img/informationorganized.webp differ
diff --git a/static/img/inspect.png b/static/img/inspect.png
deleted file mode 100644
index d32bc2a25a5c..000000000000
Binary files a/static/img/inspect.png and /dev/null differ
diff --git a/static/img/inspect.webp b/static/img/inspect.webp
new file mode 100644
index 000000000000..6c4502798164
Binary files /dev/null and b/static/img/inspect.webp differ
diff --git a/static/img/inspector_interfaces.png b/static/img/inspector_interfaces.png
deleted file mode 100644
index 0e38ff5ba350..000000000000
Binary files a/static/img/inspector_interfaces.png and /dev/null differ
diff --git a/static/img/inspector_interfaces.webp b/static/img/inspector_interfaces.webp
new file mode 100644
index 000000000000..b9e88b0f3ac2
Binary files /dev/null and b/static/img/inspector_interfaces.webp differ
diff --git a/static/img/install_cc_app.png b/static/img/install_cc_app.png
deleted file mode 100644
index 5ea4b93019a1..000000000000
Binary files a/static/img/install_cc_app.png and /dev/null differ
diff --git a/static/img/install_cc_app.webp b/static/img/install_cc_app.webp
new file mode 100644
index 000000000000..3e224efddf1d
Binary files /dev/null and b/static/img/install_cc_app.webp differ
diff --git a/static/img/integration-toolkit.png b/static/img/integration-toolkit.png
deleted file mode 100644
index 22ee2e99087c..000000000000
Binary files a/static/img/integration-toolkit.png and /dev/null differ
diff --git a/static/img/integration-toolkit.webp b/static/img/integration-toolkit.webp
new file mode 100644
index 000000000000..f25a7a992f66
Binary files /dev/null and b/static/img/integration-toolkit.webp differ
diff --git a/static/img/invite-new-users.png b/static/img/invite-new-users.png
deleted file mode 100644
index 4e30c554c5c1..000000000000
Binary files a/static/img/invite-new-users.png and /dev/null differ
diff --git a/static/img/invite-new-users.webp b/static/img/invite-new-users.webp
new file mode 100644
index 000000000000..b2c9dd8ef4e6
Binary files /dev/null and b/static/img/invite-new-users.webp differ
diff --git a/static/img/itau_integration_diagram.png b/static/img/itau_integration_diagram.png
deleted file mode 100644
index f01f0724745b..000000000000
Binary files a/static/img/itau_integration_diagram.png and /dev/null differ
diff --git a/static/img/itau_integration_diagram.webp b/static/img/itau_integration_diagram.webp
new file mode 100644
index 000000000000..e9cf999769ba
Binary files /dev/null and b/static/img/itau_integration_diagram.webp differ
diff --git a/static/img/itau_social.png b/static/img/itau_social.png
deleted file mode 100644
index 889871742805..000000000000
Binary files a/static/img/itau_social.png and /dev/null differ
diff --git a/static/img/itau_social.webp b/static/img/itau_social.webp
new file mode 100644
index 000000000000..5d5762209c1b
Binary files /dev/null and b/static/img/itau_social.webp differ
diff --git a/static/img/kenya-joe.jpg b/static/img/kenya-joe.jpg
deleted file mode 100644
index a32b83650176..000000000000
Binary files a/static/img/kenya-joe.jpg and /dev/null differ
diff --git a/static/img/kenya-joe.webp b/static/img/kenya-joe.webp
new file mode 100644
index 000000000000..b4e0d9717c62
Binary files /dev/null and b/static/img/kenya-joe.webp differ
diff --git a/static/img/keychain_credential_dropdown.webp b/static/img/keychain_credential_dropdown.webp
new file mode 100644
index 000000000000..4734668647db
Binary files /dev/null and b/static/img/keychain_credential_dropdown.webp differ
diff --git a/static/img/keychain_input.webp b/static/img/keychain_input.webp
new file mode 100644
index 000000000000..f23901cf613f
Binary files /dev/null and b/static/img/keychain_input.webp differ
diff --git a/static/img/keychain_modal.webp b/static/img/keychain_modal.webp
new file mode 100644
index 000000000000..afb784a7ce25
Binary files /dev/null and b/static/img/keychain_modal.webp differ
diff --git a/static/img/keychain_selection.webp b/static/img/keychain_selection.webp
new file mode 100644
index 000000000000..c499f235a15c
Binary files /dev/null and b/static/img/keychain_selection.webp differ
diff --git a/static/img/kgvk.jpg b/static/img/kgvk.jpg
deleted file mode 100644
index 0a4d24877119..000000000000
Binary files a/static/img/kgvk.jpg and /dev/null differ
diff --git a/static/img/kgvk.v2.png b/static/img/kgvk.v2.png
deleted file mode 100644
index 8a6f2a0ecf03..000000000000
Binary files a/static/img/kgvk.v2.png and /dev/null differ
diff --git a/static/img/kgvk.v2.webp b/static/img/kgvk.v2.webp
new file mode 100644
index 000000000000..775abc50ba58
Binary files /dev/null and b/static/img/kgvk.v2.webp differ
diff --git a/static/img/kgvk.webp b/static/img/kgvk.webp
new file mode 100644
index 000000000000..726b210b8461
Binary files /dev/null and b/static/img/kgvk.webp differ
diff --git a/static/img/kobo-deploy.png b/static/img/kobo-deploy.png
deleted file mode 100644
index 64004876c474..000000000000
Binary files a/static/img/kobo-deploy.png and /dev/null differ
diff --git a/static/img/kobo-deploy.webp b/static/img/kobo-deploy.webp
new file mode 100644
index 000000000000..d8ef47e468fd
Binary files /dev/null and b/static/img/kobo-deploy.webp differ
diff --git a/static/img/kobo-new.png b/static/img/kobo-new.png
deleted file mode 100644
index 5afbc82420a6..000000000000
Binary files a/static/img/kobo-new.png and /dev/null differ
diff --git a/static/img/kobo-new.webp b/static/img/kobo-new.webp
new file mode 100644
index 000000000000..0e19c9cb8410
Binary files /dev/null and b/static/img/kobo-new.webp differ
diff --git a/static/img/kobo-project-space.png b/static/img/kobo-project-space.png
deleted file mode 100644
index 42982f79de2f..000000000000
Binary files a/static/img/kobo-project-space.png and /dev/null differ
diff --git a/static/img/kobo-project-space.webp b/static/img/kobo-project-space.webp
new file mode 100644
index 000000000000..9ca1d5bbac5e
Binary files /dev/null and b/static/img/kobo-project-space.webp differ
diff --git a/static/img/lightning-custom-input.png b/static/img/lightning-custom-input.png
deleted file mode 100644
index 2f3e9b0e4138..000000000000
Binary files a/static/img/lightning-custom-input.png and /dev/null differ
diff --git a/static/img/lightning-custom-input.webp b/static/img/lightning-custom-input.webp
new file mode 100644
index 000000000000..fea287ee8106
Binary files /dev/null and b/static/img/lightning-custom-input.webp differ
diff --git a/static/img/lightning-dhis2-editor.png b/static/img/lightning-dhis2-editor.png
deleted file mode 100644
index 2bc0dce49fc9..000000000000
Binary files a/static/img/lightning-dhis2-editor.png and /dev/null differ
diff --git a/static/img/lightning-dhis2-editor.webp b/static/img/lightning-dhis2-editor.webp
new file mode 100644
index 000000000000..cf57daaecc56
Binary files /dev/null and b/static/img/lightning-dhis2-editor.webp differ
diff --git a/static/img/lightning-dhis2-job.png b/static/img/lightning-dhis2-job.png
deleted file mode 100644
index d9822ab4a5ae..000000000000
Binary files a/static/img/lightning-dhis2-job.png and /dev/null differ
diff --git a/static/img/lightning-dhis2-job.webp b/static/img/lightning-dhis2-job.webp
new file mode 100644
index 000000000000..d23ee497bb49
Binary files /dev/null and b/static/img/lightning-dhis2-job.webp differ
diff --git a/static/img/lightning-sample-workflow.png b/static/img/lightning-sample-workflow.png
deleted file mode 100644
index d122afe53ea9..000000000000
Binary files a/static/img/lightning-sample-workflow.png and /dev/null differ
diff --git a/static/img/lightning-sample-workflow.webp b/static/img/lightning-sample-workflow.webp
new file mode 100644
index 000000000000..1b2dd11d2b3c
Binary files /dev/null and b/static/img/lightning-sample-workflow.webp differ
diff --git a/static/img/lightning-work-order-expanded.png b/static/img/lightning-work-order-expanded.png
deleted file mode 100644
index 79f6ce76c7db..000000000000
Binary files a/static/img/lightning-work-order-expanded.png and /dev/null differ
diff --git a/static/img/lightning-work-order-expanded.webp b/static/img/lightning-work-order-expanded.webp
new file mode 100644
index 000000000000..fd392d50ab86
Binary files /dev/null and b/static/img/lightning-work-order-expanded.webp differ
diff --git a/static/img/lightning-workflows-page.png b/static/img/lightning-workflows-page.png
deleted file mode 100644
index 353db9c71340..000000000000
Binary files a/static/img/lightning-workflows-page.png and /dev/null differ
diff --git a/static/img/lightning-workflows-page.webp b/static/img/lightning-workflows-page.webp
new file mode 100644
index 000000000000..5e39e9ba1085
Binary files /dev/null and b/static/img/lightning-workflows-page.webp differ
diff --git a/static/img/lightning_add_cred.png b/static/img/lightning_add_cred.png
deleted file mode 100644
index 6592230e6f0a..000000000000
Binary files a/static/img/lightning_add_cred.png and /dev/null differ
diff --git a/static/img/lightning_add_cred.webp b/static/img/lightning_add_cred.webp
new file mode 100644
index 000000000000..3e910f9d7496
Binary files /dev/null and b/static/img/lightning_add_cred.webp differ
diff --git a/static/img/lightning_add_repo_secrets.png b/static/img/lightning_add_repo_secrets.png
deleted file mode 100644
index b2ed7e8403e1..000000000000
Binary files a/static/img/lightning_add_repo_secrets.png and /dev/null differ
diff --git a/static/img/lightning_add_repo_secrets.webp b/static/img/lightning_add_repo_secrets.webp
new file mode 100644
index 000000000000..510f01d0581a
Binary files /dev/null and b/static/img/lightning_add_repo_secrets.webp differ
diff --git a/static/img/lightning_api_auth.png b/static/img/lightning_api_auth.png
deleted file mode 100644
index d74688071e41..000000000000
Binary files a/static/img/lightning_api_auth.png and /dev/null differ
diff --git a/static/img/lightning_api_auth.webp b/static/img/lightning_api_auth.webp
new file mode 100644
index 000000000000..343216539635
Binary files /dev/null and b/static/img/lightning_api_auth.webp differ
diff --git a/static/img/lightning_architecture.png b/static/img/lightning_architecture.png
deleted file mode 100644
index 49e9f76f1ba3..000000000000
Binary files a/static/img/lightning_architecture.png and /dev/null differ
diff --git a/static/img/lightning_architecture.webp b/static/img/lightning_architecture.webp
new file mode 100644
index 000000000000..a9552e01b091
Binary files /dev/null and b/static/img/lightning_architecture.webp differ
diff --git a/static/img/lightning_audit.png b/static/img/lightning_audit.png
deleted file mode 100644
index a71bacffa5b6..000000000000
Binary files a/static/img/lightning_audit.png and /dev/null differ
diff --git a/static/img/lightning_audit.webp b/static/img/lightning_audit.webp
new file mode 100644
index 000000000000..703102d93ac9
Binary files /dev/null and b/static/img/lightning_audit.webp differ
diff --git a/static/img/lightning_auth_from_workflow.png b/static/img/lightning_auth_from_workflow.png
deleted file mode 100644
index 39c79c4046c1..000000000000
Binary files a/static/img/lightning_auth_from_workflow.png and /dev/null differ
diff --git a/static/img/lightning_auth_from_workflow.webp b/static/img/lightning_auth_from_workflow.webp
new file mode 100644
index 000000000000..c6c6fdb3cc1e
Binary files /dev/null and b/static/img/lightning_auth_from_workflow.webp differ
diff --git a/static/img/lightning_auth_project_settings.png b/static/img/lightning_auth_project_settings.png
deleted file mode 100644
index e8332356ef82..000000000000
Binary files a/static/img/lightning_auth_project_settings.png and /dev/null differ
diff --git a/static/img/lightning_auth_project_settings.webp b/static/img/lightning_auth_project_settings.webp
new file mode 100644
index 000000000000..3ea77024796e
Binary files /dev/null and b/static/img/lightning_auth_project_settings.webp differ
diff --git a/static/img/lightning_basic_auth.png b/static/img/lightning_basic_auth.png
deleted file mode 100644
index d668bc1dc033..000000000000
Binary files a/static/img/lightning_basic_auth.png and /dev/null differ
diff --git a/static/img/lightning_basic_auth.webp b/static/img/lightning_basic_auth.webp
new file mode 100644
index 000000000000..60d5c1da4d21
Binary files /dev/null and b/static/img/lightning_basic_auth.webp differ
diff --git a/static/img/lightning_build.png b/static/img/lightning_build.png
deleted file mode 100644
index e4420a4ccfeb..000000000000
Binary files a/static/img/lightning_build.png and /dev/null differ
diff --git a/static/img/lightning_build.webp b/static/img/lightning_build.webp
new file mode 100644
index 000000000000..f28d637947a4
Binary files /dev/null and b/static/img/lightning_build.webp differ
diff --git a/static/img/lightning_build_run_audit.png b/static/img/lightning_build_run_audit.png
deleted file mode 100644
index 20d2152213ed..000000000000
Binary files a/static/img/lightning_build_run_audit.png and /dev/null differ
diff --git a/static/img/lightning_build_run_audit.webp b/static/img/lightning_build_run_audit.webp
new file mode 100644
index 000000000000..a56f191790c9
Binary files /dev/null and b/static/img/lightning_build_run_audit.webp differ
diff --git a/static/img/lightning_change_email_pw.png b/static/img/lightning_change_email_pw.png
deleted file mode 100644
index 5ac732630cdb..000000000000
Binary files a/static/img/lightning_change_email_pw.png and /dev/null differ
diff --git a/static/img/lightning_change_email_pw.webp b/static/img/lightning_change_email_pw.webp
new file mode 100644
index 000000000000..a194bf2cd22c
Binary files /dev/null and b/static/img/lightning_change_email_pw.webp differ
diff --git a/static/img/lightning_choose_auth_method.png b/static/img/lightning_choose_auth_method.png
deleted file mode 100644
index 7a5d9c6b94ed..000000000000
Binary files a/static/img/lightning_choose_auth_method.png and /dev/null differ
diff --git a/static/img/lightning_choose_auth_method.webp b/static/img/lightning_choose_auth_method.webp
new file mode 100644
index 000000000000..3d86d7964fbd
Binary files /dev/null and b/static/img/lightning_choose_auth_method.webp differ
diff --git a/static/img/lightning_choose_cred_type.png b/static/img/lightning_choose_cred_type.png
deleted file mode 100644
index bb58aba6f54a..000000000000
Binary files a/static/img/lightning_choose_cred_type.png and /dev/null differ
diff --git a/static/img/lightning_choose_cred_type.webp b/static/img/lightning_choose_cred_type.webp
new file mode 100644
index 000000000000..2faafb80b705
Binary files /dev/null and b/static/img/lightning_choose_cred_type.webp differ
diff --git a/static/img/lightning_connect_to_gh (1).png b/static/img/lightning_connect_to_gh (1).png
deleted file mode 100644
index db0129f4b20d..000000000000
Binary files a/static/img/lightning_connect_to_gh (1).png and /dev/null differ
diff --git a/static/img/lightning_connect_to_gh (1).webp b/static/img/lightning_connect_to_gh (1).webp
new file mode 100644
index 000000000000..2cb6241c5201
Binary files /dev/null and b/static/img/lightning_connect_to_gh (1).webp differ
diff --git a/static/img/lightning_connect_to_gh.png b/static/img/lightning_connect_to_gh.png
deleted file mode 100644
index db0129f4b20d..000000000000
Binary files a/static/img/lightning_connect_to_gh.png and /dev/null differ
diff --git a/static/img/lightning_connect_to_gh.webp b/static/img/lightning_connect_to_gh.webp
new file mode 100644
index 000000000000..2cb6241c5201
Binary files /dev/null and b/static/img/lightning_connect_to_gh.webp differ
diff --git a/static/img/lightning_cred_edit_view.png b/static/img/lightning_cred_edit_view.png
deleted file mode 100644
index fc7a01fa71f8..000000000000
Binary files a/static/img/lightning_cred_edit_view.png and /dev/null differ
diff --git a/static/img/lightning_cred_edit_view.webp b/static/img/lightning_cred_edit_view.webp
new file mode 100644
index 000000000000..aa04b8b16b79
Binary files /dev/null and b/static/img/lightning_cred_edit_view.webp differ
diff --git a/static/img/lightning_credentials_overview.png b/static/img/lightning_credentials_overview.png
deleted file mode 100644
index 72a16fae21cc..000000000000
Binary files a/static/img/lightning_credentials_overview.png and /dev/null differ
diff --git a/static/img/lightning_credentials_overview.webp b/static/img/lightning_credentials_overview.webp
new file mode 100644
index 000000000000..ab4838b09ef7
Binary files /dev/null and b/static/img/lightning_credentials_overview.webp differ
diff --git a/static/img/lightning_delete_account_cropped.png b/static/img/lightning_delete_account_cropped.png
deleted file mode 100644
index 3bebde18ab18..000000000000
Binary files a/static/img/lightning_delete_account_cropped.png and /dev/null differ
diff --git a/static/img/lightning_delete_account_cropped.webp b/static/img/lightning_delete_account_cropped.webp
new file mode 100644
index 000000000000..b3634603a38e
Binary files /dev/null and b/static/img/lightning_delete_account_cropped.webp differ
diff --git a/static/img/lightning_digest.png b/static/img/lightning_digest.png
deleted file mode 100644
index 419d7b59585a..000000000000
Binary files a/static/img/lightning_digest.png and /dev/null differ
diff --git a/static/img/lightning_digest.webp b/static/img/lightning_digest.webp
new file mode 100644
index 000000000000..69357641e928
Binary files /dev/null and b/static/img/lightning_digest.webp differ
diff --git a/static/img/lightning_edit_auth_method.png b/static/img/lightning_edit_auth_method.png
deleted file mode 100644
index e345fa245335..000000000000
Binary files a/static/img/lightning_edit_auth_method.png and /dev/null differ
diff --git a/static/img/lightning_edit_auth_method.webp b/static/img/lightning_edit_auth_method.webp
new file mode 100644
index 000000000000..08715b8f8fc1
Binary files /dev/null and b/static/img/lightning_edit_auth_method.webp differ
diff --git a/static/img/lightning_edit_user_credential.png b/static/img/lightning_edit_user_credential.png
deleted file mode 100644
index eea934ce8771..000000000000
Binary files a/static/img/lightning_edit_user_credential.png and /dev/null differ
diff --git a/static/img/lightning_edit_user_credential.webp b/static/img/lightning_edit_user_credential.webp
new file mode 100644
index 000000000000..df223329edc4
Binary files /dev/null and b/static/img/lightning_edit_user_credential.webp differ
diff --git a/static/img/lightning_editor.png b/static/img/lightning_editor.png
deleted file mode 100644
index 695a0b5c6768..000000000000
Binary files a/static/img/lightning_editor.png and /dev/null differ
diff --git a/static/img/lightning_editor.webp b/static/img/lightning_editor.webp
new file mode 100644
index 000000000000..ffd1632d1284
Binary files /dev/null and b/static/img/lightning_editor.webp differ
diff --git a/static/img/lightning_editor_1.png b/static/img/lightning_editor_1.png
deleted file mode 100644
index bb04be380cdb..000000000000
Binary files a/static/img/lightning_editor_1.png and /dev/null differ
diff --git a/static/img/lightning_editor_1.webp b/static/img/lightning_editor_1.webp
new file mode 100644
index 000000000000..9564537fc73d
Binary files /dev/null and b/static/img/lightning_editor_1.webp differ
diff --git a/static/img/lightning_enable_MFA.png b/static/img/lightning_enable_MFA.png
deleted file mode 100644
index 50aa87074dcc..000000000000
Binary files a/static/img/lightning_enable_MFA.png and /dev/null differ
diff --git a/static/img/lightning_enable_MFA.webp b/static/img/lightning_enable_MFA.webp
new file mode 100644
index 000000000000..c20b13c96ba6
Binary files /dev/null and b/static/img/lightning_enable_MFA.webp differ
diff --git a/static/img/lightning_failed_run.png b/static/img/lightning_failed_run.png
deleted file mode 100644
index 350116f7bc14..000000000000
Binary files a/static/img/lightning_failed_run.png and /dev/null differ
diff --git a/static/img/lightning_failed_run.webp b/static/img/lightning_failed_run.webp
new file mode 100644
index 000000000000..b251f921626e
Binary files /dev/null and b/static/img/lightning_failed_run.webp differ
diff --git a/static/img/lightning_failed_work_orders.png b/static/img/lightning_failed_work_orders.png
deleted file mode 100644
index 917398868056..000000000000
Binary files a/static/img/lightning_failed_work_orders.png and /dev/null differ
diff --git a/static/img/lightning_failed_work_orders.webp b/static/img/lightning_failed_work_orders.webp
new file mode 100644
index 000000000000..4006bd8909d7
Binary files /dev/null and b/static/img/lightning_failed_work_orders.webp differ
diff --git a/static/img/lightning_failure_alert.png b/static/img/lightning_failure_alert.png
deleted file mode 100644
index cc82afb2a4c6..000000000000
Binary files a/static/img/lightning_failure_alert.png and /dev/null differ
diff --git a/static/img/lightning_failure_alert.webp b/static/img/lightning_failure_alert.webp
new file mode 100644
index 000000000000..fac1d30b785d
Binary files /dev/null and b/static/img/lightning_failure_alert.webp differ
diff --git a/static/img/lightning_failure_email.png b/static/img/lightning_failure_email.png
deleted file mode 100644
index cb15ac8122d0..000000000000
Binary files a/static/img/lightning_failure_email.png and /dev/null differ
diff --git a/static/img/lightning_failure_email.webp b/static/img/lightning_failure_email.webp
new file mode 100644
index 000000000000..e482b3e7ee08
Binary files /dev/null and b/static/img/lightning_failure_email.webp differ
diff --git a/static/img/lightning_gh_configure.png b/static/img/lightning_gh_configure.png
deleted file mode 100644
index dc8555f55f04..000000000000
Binary files a/static/img/lightning_gh_configure.png and /dev/null differ
diff --git a/static/img/lightning_gh_configure.webp b/static/img/lightning_gh_configure.webp
new file mode 100644
index 000000000000..12ee8df402e8
Binary files /dev/null and b/static/img/lightning_gh_configure.webp differ
diff --git a/static/img/lightning_gh_initate_sync.png b/static/img/lightning_gh_initate_sync.png
deleted file mode 100644
index 3b697f8ae81f..000000000000
Binary files a/static/img/lightning_gh_initate_sync.png and /dev/null differ
diff --git a/static/img/lightning_gh_initate_sync.webp b/static/img/lightning_gh_initate_sync.webp
new file mode 100644
index 000000000000..c1cd1aac13f1
Binary files /dev/null and b/static/img/lightning_gh_initate_sync.webp differ
diff --git a/static/img/lightning_gh_install_openfn.png b/static/img/lightning_gh_install_openfn.png
deleted file mode 100644
index 918ea2c9c096..000000000000
Binary files a/static/img/lightning_gh_install_openfn.png and /dev/null differ
diff --git a/static/img/lightning_gh_install_openfn.webp b/static/img/lightning_gh_install_openfn.webp
new file mode 100644
index 000000000000..0f45835440ac
Binary files /dev/null and b/static/img/lightning_gh_install_openfn.webp differ
diff --git a/static/img/lightning_gh_permissions.png b/static/img/lightning_gh_permissions.png
deleted file mode 100644
index b2b4e909efee..000000000000
Binary files a/static/img/lightning_gh_permissions.png and /dev/null differ
diff --git a/static/img/lightning_gh_permissions.webp b/static/img/lightning_gh_permissions.webp
new file mode 100644
index 000000000000..0cba6a5728ec
Binary files /dev/null and b/static/img/lightning_gh_permissions.webp differ
diff --git a/static/img/lightning_history.png b/static/img/lightning_history.png
deleted file mode 100644
index c84e3e6c69c6..000000000000
Binary files a/static/img/lightning_history.png and /dev/null differ
diff --git a/static/img/lightning_history.webp b/static/img/lightning_history.webp
new file mode 100644
index 000000000000..b36c4c9c0fc2
Binary files /dev/null and b/static/img/lightning_history.webp differ
diff --git a/static/img/lightning_history_expanded.png b/static/img/lightning_history_expanded.png
deleted file mode 100644
index 59c120210fde..000000000000
Binary files a/static/img/lightning_history_expanded.png and /dev/null differ
diff --git a/static/img/lightning_history_expanded.webp b/static/img/lightning_history_expanded.webp
new file mode 100644
index 000000000000..13c40d4a4d82
Binary files /dev/null and b/static/img/lightning_history_expanded.webp differ
diff --git a/static/img/lightning_history_failure.png b/static/img/lightning_history_failure.png
deleted file mode 100644
index 7d59d55274be..000000000000
Binary files a/static/img/lightning_history_failure.png and /dev/null differ
diff --git a/static/img/lightning_history_failure.webp b/static/img/lightning_history_failure.webp
new file mode 100644
index 000000000000..5bb188f8f80b
Binary files /dev/null and b/static/img/lightning_history_failure.webp differ
diff --git a/static/img/lightning_input_data.png b/static/img/lightning_input_data.png
deleted file mode 100644
index 6706b893227f..000000000000
Binary files a/static/img/lightning_input_data.png and /dev/null differ
diff --git a/static/img/lightning_input_data.webp b/static/img/lightning_input_data.webp
new file mode 100644
index 000000000000..580863c035c7
Binary files /dev/null and b/static/img/lightning_input_data.webp differ
diff --git a/static/img/lightning_linked_triggers.png b/static/img/lightning_linked_triggers.png
deleted file mode 100644
index b55255d616ad..000000000000
Binary files a/static/img/lightning_linked_triggers.png and /dev/null differ
diff --git a/static/img/lightning_linked_triggers.webp b/static/img/lightning_linked_triggers.webp
new file mode 100644
index 000000000000..ae89a0778138
Binary files /dev/null and b/static/img/lightning_linked_triggers.webp differ
diff --git a/static/img/lightning_linked_triggers2.png b/static/img/lightning_linked_triggers2.png
deleted file mode 100644
index 2b048d66f0d2..000000000000
Binary files a/static/img/lightning_linked_triggers2.png and /dev/null differ
diff --git a/static/img/lightning_linked_triggers2.webp b/static/img/lightning_linked_triggers2.webp
new file mode 100644
index 000000000000..82a0dd9d7d29
Binary files /dev/null and b/static/img/lightning_linked_triggers2.webp differ
diff --git a/static/img/lightning_manual_run.png b/static/img/lightning_manual_run.png
deleted file mode 100644
index cbb0ea12c722..000000000000
Binary files a/static/img/lightning_manual_run.png and /dev/null differ
diff --git a/static/img/lightning_manual_run.webp b/static/img/lightning_manual_run.webp
new file mode 100644
index 000000000000..030eead338a8
Binary files /dev/null and b/static/img/lightning_manual_run.webp differ
diff --git a/static/img/lightning_new_api_token.png b/static/img/lightning_new_api_token.png
deleted file mode 100644
index 82400e9bd947..000000000000
Binary files a/static/img/lightning_new_api_token.png and /dev/null differ
diff --git a/static/img/lightning_new_api_token.webp b/static/img/lightning_new_api_token.webp
new file mode 100644
index 000000000000..3fa793982f79
Binary files /dev/null and b/static/img/lightning_new_api_token.webp differ
diff --git a/static/img/lightning_new_attempt.png b/static/img/lightning_new_attempt.png
deleted file mode 100644
index 5f345f3ff1d8..000000000000
Binary files a/static/img/lightning_new_attempt.png and /dev/null differ
diff --git a/static/img/lightning_new_attempt.webp b/static/img/lightning_new_attempt.webp
new file mode 100644
index 000000000000..d764468fcc05
Binary files /dev/null and b/static/img/lightning_new_attempt.webp differ
diff --git a/static/img/lightning_new_cred_ready.png b/static/img/lightning_new_cred_ready.png
deleted file mode 100644
index 70b50ce88ea6..000000000000
Binary files a/static/img/lightning_new_cred_ready.png and /dev/null differ
diff --git a/static/img/lightning_new_cred_ready.webp b/static/img/lightning_new_cred_ready.webp
new file mode 100644
index 000000000000..8c9ea53853b0
Binary files /dev/null and b/static/img/lightning_new_cred_ready.webp differ
diff --git a/static/img/lightning_no_api_token (1).png b/static/img/lightning_no_api_token (1).png
deleted file mode 100644
index 8c2bbae39d04..000000000000
Binary files a/static/img/lightning_no_api_token (1).png and /dev/null differ
diff --git a/static/img/lightning_no_api_token (1).webp b/static/img/lightning_no_api_token (1).webp
new file mode 100644
index 000000000000..f91dbd66dd74
Binary files /dev/null and b/static/img/lightning_no_api_token (1).webp differ
diff --git a/static/img/lightning_no_api_token.png b/static/img/lightning_no_api_token.png
deleted file mode 100644
index 8c2bbae39d04..000000000000
Binary files a/static/img/lightning_no_api_token.png and /dev/null differ
diff --git a/static/img/lightning_no_api_token.webp b/static/img/lightning_no_api_token.webp
new file mode 100644
index 000000000000..f91dbd66dd74
Binary files /dev/null and b/static/img/lightning_no_api_token.webp differ
diff --git a/static/img/lightning_openfn_select_repo_branch.png b/static/img/lightning_openfn_select_repo_branch.png
deleted file mode 100644
index 07709ef4ed9a..000000000000
Binary files a/static/img/lightning_openfn_select_repo_branch.png and /dev/null differ
diff --git a/static/img/lightning_openfn_select_repo_branch.webp b/static/img/lightning_openfn_select_repo_branch.webp
new file mode 100644
index 000000000000..372e0dec072a
Binary files /dev/null and b/static/img/lightning_openfn_select_repo_branch.webp differ
diff --git a/static/img/lightning_path_trigger_example.png b/static/img/lightning_path_trigger_example.png
deleted file mode 100644
index ca6a0e91b585..000000000000
Binary files a/static/img/lightning_path_trigger_example.png and /dev/null differ
diff --git a/static/img/lightning_path_trigger_example.webp b/static/img/lightning_path_trigger_example.webp
new file mode 100644
index 000000000000..19227e79719c
Binary files /dev/null and b/static/img/lightning_path_trigger_example.webp differ
diff --git a/static/img/lightning_preview.png b/static/img/lightning_preview.png
deleted file mode 100644
index 23c1462a0a9f..000000000000
Binary files a/static/img/lightning_preview.png and /dev/null differ
diff --git a/static/img/lightning_preview.webp b/static/img/lightning_preview.webp
new file mode 100644
index 000000000000..e1d7cf38b5dc
Binary files /dev/null and b/static/img/lightning_preview.webp differ
diff --git a/static/img/lightning_project_overview.png b/static/img/lightning_project_overview.png
deleted file mode 100644
index cef02485d887..000000000000
Binary files a/static/img/lightning_project_overview.png and /dev/null differ
diff --git a/static/img/lightning_project_overview.webp b/static/img/lightning_project_overview.webp
new file mode 100644
index 000000000000..83696cc343f6
Binary files /dev/null and b/static/img/lightning_project_overview.webp differ
diff --git a/static/img/lightning_rerun_downstream_job.png b/static/img/lightning_rerun_downstream_job.png
deleted file mode 100644
index 36ce94757d92..000000000000
Binary files a/static/img/lightning_rerun_downstream_job.png and /dev/null differ
diff --git a/static/img/lightning_rerun_downstream_job.webp b/static/img/lightning_rerun_downstream_job.webp
new file mode 100644
index 000000000000..b88a2eee5079
Binary files /dev/null and b/static/img/lightning_rerun_downstream_job.webp differ
diff --git a/static/img/lightning_retry.png b/static/img/lightning_retry.png
deleted file mode 100644
index 40ccce431b79..000000000000
Binary files a/static/img/lightning_retry.png and /dev/null differ
diff --git a/static/img/lightning_retry.webp b/static/img/lightning_retry.webp
new file mode 100644
index 000000000000..8b35ddda8820
Binary files /dev/null and b/static/img/lightning_retry.webp differ
diff --git a/static/img/lightning_select_user_profile.png b/static/img/lightning_select_user_profile.png
deleted file mode 100644
index 908d168f7666..000000000000
Binary files a/static/img/lightning_select_user_profile.png and /dev/null differ
diff --git a/static/img/lightning_select_user_profile.webp b/static/img/lightning_select_user_profile.webp
new file mode 100644
index 000000000000..2b7279789363
Binary files /dev/null and b/static/img/lightning_select_user_profile.webp differ
diff --git a/static/img/lightning_setup.png b/static/img/lightning_setup.png
deleted file mode 100644
index de6500bc1d80..000000000000
Binary files a/static/img/lightning_setup.png and /dev/null differ
diff --git a/static/img/lightning_setup.webp b/static/img/lightning_setup.webp
new file mode 100644
index 000000000000..5b0eef515c87
Binary files /dev/null and b/static/img/lightning_setup.webp differ
diff --git a/static/img/lightning_share_cred_with_project.png b/static/img/lightning_share_cred_with_project.png
deleted file mode 100644
index 97ce35fe18be..000000000000
Binary files a/static/img/lightning_share_cred_with_project.png and /dev/null differ
diff --git a/static/img/lightning_share_cred_with_project.webp b/static/img/lightning_share_cred_with_project.webp
new file mode 100644
index 000000000000..0def0b0ebe8d
Binary files /dev/null and b/static/img/lightning_share_cred_with_project.webp differ
diff --git a/static/img/lightning_trigger_example.png b/static/img/lightning_trigger_example.png
deleted file mode 100644
index cc3a85c881f9..000000000000
Binary files a/static/img/lightning_trigger_example.png and /dev/null differ
diff --git a/static/img/lightning_trigger_example.webp b/static/img/lightning_trigger_example.webp
new file mode 100644
index 000000000000..1fae7885ab02
Binary files /dev/null and b/static/img/lightning_trigger_example.webp differ
diff --git a/static/img/lightning_user_profile_api_tokens.png b/static/img/lightning_user_profile_api_tokens.png
deleted file mode 100644
index ff158492b369..000000000000
Binary files a/static/img/lightning_user_profile_api_tokens.png and /dev/null differ
diff --git a/static/img/lightning_user_profile_api_tokens.webp b/static/img/lightning_user_profile_api_tokens.webp
new file mode 100644
index 000000000000..c8dd7d12ccd1
Binary files /dev/null and b/static/img/lightning_user_profile_api_tokens.webp differ
diff --git a/static/img/lightning_user_profile_credentials.png b/static/img/lightning_user_profile_credentials.png
deleted file mode 100644
index f891e1772923..000000000000
Binary files a/static/img/lightning_user_profile_credentials.png and /dev/null differ
diff --git a/static/img/lightning_user_profile_credentials.webp b/static/img/lightning_user_profile_credentials.webp
new file mode 100644
index 000000000000..c7565f1db524
Binary files /dev/null and b/static/img/lightning_user_profile_credentials.webp differ
diff --git a/static/img/lightning_view_creds_not_edit.png b/static/img/lightning_view_creds_not_edit.png
deleted file mode 100644
index 56a28d0f5339..000000000000
Binary files a/static/img/lightning_view_creds_not_edit.png and /dev/null differ
diff --git a/static/img/lightning_view_creds_not_edit.webp b/static/img/lightning_view_creds_not_edit.webp
new file mode 100644
index 000000000000..98673c5b4d71
Binary files /dev/null and b/static/img/lightning_view_creds_not_edit.webp differ
diff --git a/static/img/lightning_weekly_digest.png b/static/img/lightning_weekly_digest.png
deleted file mode 100644
index 37454d6d2363..000000000000
Binary files a/static/img/lightning_weekly_digest.png and /dev/null differ
diff --git a/static/img/lightning_weekly_digest.webp b/static/img/lightning_weekly_digest.webp
new file mode 100644
index 000000000000..8a52915f0da2
Binary files /dev/null and b/static/img/lightning_weekly_digest.webp differ
diff --git a/static/img/lightning_workflow_dashboard.png b/static/img/lightning_workflow_dashboard.png
deleted file mode 100644
index 225c5774849b..000000000000
Binary files a/static/img/lightning_workflow_dashboard.png and /dev/null differ
diff --git a/static/img/lightning_workflow_dashboard.webp b/static/img/lightning_workflow_dashboard.webp
new file mode 100644
index 000000000000..1322f8a2ec3f
Binary files /dev/null and b/static/img/lightning_workflow_dashboard.webp differ
diff --git a/static/img/lightning_workflow_trigger_added.png b/static/img/lightning_workflow_trigger_added.png
deleted file mode 100644
index 216b10c32cc0..000000000000
Binary files a/static/img/lightning_workflow_trigger_added.png and /dev/null differ
diff --git a/static/img/lightning_workflow_trigger_added.webp b/static/img/lightning_workflow_trigger_added.webp
new file mode 100644
index 000000000000..bec6339ae0d5
Binary files /dev/null and b/static/img/lightning_workflow_trigger_added.webp differ
diff --git a/static/img/log-levels.webp b/static/img/log-levels.webp
new file mode 100644
index 000000000000..1cfb5f8879ce
Binary files /dev/null and b/static/img/log-levels.webp differ
diff --git a/static/img/logs_run.png b/static/img/logs_run.png
deleted file mode 100644
index 97554bf5bfdc..000000000000
Binary files a/static/img/logs_run.png and /dev/null differ
diff --git a/static/img/logs_run.webp b/static/img/logs_run.webp
new file mode 100644
index 000000000000..c053ac33f423
Binary files /dev/null and b/static/img/logs_run.webp differ
diff --git a/static/img/make.png b/static/img/make.png
deleted file mode 100644
index 568832d76944..000000000000
Binary files a/static/img/make.png and /dev/null differ
diff --git a/static/img/make.webp b/static/img/make.webp
new file mode 100644
index 000000000000..dfe87e2921cc
Binary files /dev/null and b/static/img/make.webp differ
diff --git a/static/img/manage_project_access.png b/static/img/manage_project_access.png
deleted file mode 100644
index 107c56c17957..000000000000
Binary files a/static/img/manage_project_access.png and /dev/null differ
diff --git a/static/img/manage_project_access.webp b/static/img/manage_project_access.webp
new file mode 100644
index 000000000000..235c903bb1dc
Binary files /dev/null and b/static/img/manage_project_access.webp differ
diff --git a/static/img/mapping_example.png b/static/img/mapping_example.png
deleted file mode 100644
index a45dab214cf2..000000000000
Binary files a/static/img/mapping_example.png and /dev/null differ
diff --git a/static/img/mapping_example.webp b/static/img/mapping_example.webp
new file mode 100644
index 000000000000..8e8cd84bab18
Binary files /dev/null and b/static/img/mapping_example.webp differ
diff --git a/static/img/mapping_process.png b/static/img/mapping_process.png
deleted file mode 100644
index a8b790824f89..000000000000
Binary files a/static/img/mapping_process.png and /dev/null differ
diff --git a/static/img/mapping_process.webp b/static/img/mapping_process.webp
new file mode 100644
index 000000000000..06639ba84590
Binary files /dev/null and b/static/img/mapping_process.webp differ
diff --git a/static/img/metabase.png b/static/img/metabase.png
deleted file mode 100644
index bb05006f8961..000000000000
Binary files a/static/img/metabase.png and /dev/null differ
diff --git a/static/img/metabase.webp b/static/img/metabase.webp
new file mode 100644
index 000000000000..21d74af8e0ad
Binary files /dev/null and b/static/img/metabase.webp differ
diff --git a/static/img/migration_existing-workflows.png b/static/img/migration_existing-workflows.png
deleted file mode 100644
index 08006586546a..000000000000
Binary files a/static/img/migration_existing-workflows.png and /dev/null differ
diff --git a/static/img/migration_existing-workflows.webp b/static/img/migration_existing-workflows.webp
new file mode 100644
index 000000000000..0015da12a179
Binary files /dev/null and b/static/img/migration_existing-workflows.webp differ
diff --git a/static/img/migration_workflow_pasted.png b/static/img/migration_workflow_pasted.png
deleted file mode 100644
index db3dc28e6f22..000000000000
Binary files a/static/img/migration_workflow_pasted.png and /dev/null differ
diff --git a/static/img/migration_workflow_pasted.webp b/static/img/migration_workflow_pasted.webp
new file mode 100644
index 000000000000..874cc99c0018
Binary files /dev/null and b/static/img/migration_workflow_pasted.webp differ
diff --git a/static/img/myagrograph.jpg b/static/img/myagrograph.jpg
deleted file mode 100644
index 098d79edc668..000000000000
Binary files a/static/img/myagrograph.jpg and /dev/null differ
diff --git a/static/img/myagrograph.webp b/static/img/myagrograph.webp
new file mode 100644
index 000000000000..ff7c0dcf9df7
Binary files /dev/null and b/static/img/myagrograph.webp differ
diff --git a/static/img/nalibali.jpg b/static/img/nalibali.jpg
deleted file mode 100644
index b74e3b259aa0..000000000000
Binary files a/static/img/nalibali.jpg and /dev/null differ
diff --git a/static/img/nalibali.webp b/static/img/nalibali.webp
new file mode 100644
index 000000000000..0df8858e501d
Binary files /dev/null and b/static/img/nalibali.webp differ
diff --git a/static/img/name_dataclip.webp b/static/img/name_dataclip.webp
new file mode 100644
index 000000000000..90ed8aa64b75
Binary files /dev/null and b/static/img/name_dataclip.webp differ
diff --git a/static/img/new_branch.png b/static/img/new_branch.png
deleted file mode 100644
index f2db132703de..000000000000
Binary files a/static/img/new_branch.png and /dev/null differ
diff --git a/static/img/new_branch.webp b/static/img/new_branch.webp
new file mode 100644
index 000000000000..a5c09521ecfa
Binary files /dev/null and b/static/img/new_branch.webp differ
diff --git a/static/img/new_cred.png b/static/img/new_cred.png
deleted file mode 100644
index 09308f6a0533..000000000000
Binary files a/static/img/new_cred.png and /dev/null differ
diff --git a/static/img/new_cred.webp b/static/img/new_cred.webp
new file mode 100644
index 000000000000..e40a7700447e
Binary files /dev/null and b/static/img/new_cred.webp differ
diff --git a/static/img/notifications.png b/static/img/notifications.png
deleted file mode 100644
index 68b9318e8fca..000000000000
Binary files a/static/img/notifications.png and /dev/null differ
diff --git a/static/img/notifications.webp b/static/img/notifications.webp
new file mode 100644
index 000000000000..6b5f4ec78b12
Binary files /dev/null and b/static/img/notifications.webp differ
diff --git a/static/img/oauth_client_edit.png b/static/img/oauth_client_edit.png
deleted file mode 100644
index 6ffa0e21a421..000000000000
Binary files a/static/img/oauth_client_edit.png and /dev/null differ
diff --git a/static/img/oauth_client_edit.webp b/static/img/oauth_client_edit.webp
new file mode 100644
index 000000000000..b3c49198da17
Binary files /dev/null and b/static/img/oauth_client_edit.webp differ
diff --git a/static/img/oauth_dropdown.png b/static/img/oauth_dropdown.png
deleted file mode 100644
index 9fca57652ce6..000000000000
Binary files a/static/img/oauth_dropdown.png and /dev/null differ
diff --git a/static/img/oauth_dropdown.webp b/static/img/oauth_dropdown.webp
new file mode 100644
index 000000000000..d93895635460
Binary files /dev/null and b/static/img/oauth_dropdown.webp differ
diff --git a/static/img/objectmodel3.png b/static/img/objectmodel3.png
deleted file mode 100644
index 11125cabd535..000000000000
Binary files a/static/img/objectmodel3.png and /dev/null differ
diff --git a/static/img/objectmodel3.webp b/static/img/objectmodel3.webp
new file mode 100644
index 000000000000..8756802f9cd3
Binary files /dev/null and b/static/img/objectmodel3.webp differ
diff --git a/static/img/ofg.png b/static/img/ofg.png
deleted file mode 100644
index d0c4fceaeab9..000000000000
Binary files a/static/img/ofg.png and /dev/null differ
diff --git a/static/img/ofg.webp b/static/img/ofg.webp
new file mode 100644
index 000000000000..9e6c18f16668
Binary files /dev/null and b/static/img/ofg.webp differ
diff --git a/static/img/ona_dashboard.png b/static/img/ona_dashboard.png
deleted file mode 100644
index 933c9a7d39fa..000000000000
Binary files a/static/img/ona_dashboard.png and /dev/null differ
diff --git a/static/img/ona_dashboard.webp b/static/img/ona_dashboard.webp
new file mode 100644
index 000000000000..1cf2acf049e4
Binary files /dev/null and b/static/img/ona_dashboard.webp differ
diff --git a/static/img/ona_integration.png b/static/img/ona_integration.png
deleted file mode 100644
index c001933b62c5..000000000000
Binary files a/static/img/ona_integration.png and /dev/null differ
diff --git a/static/img/ona_integration.webp b/static/img/ona_integration.webp
new file mode 100644
index 000000000000..7337da66a61c
Binary files /dev/null and b/static/img/ona_integration.webp differ
diff --git a/static/img/openfn-color.png b/static/img/openfn-color.png
deleted file mode 100644
index c8c1b59ea7c1..000000000000
Binary files a/static/img/openfn-color.png and /dev/null differ
diff --git a/static/img/openfn-color.webp b/static/img/openfn-color.webp
new file mode 100644
index 000000000000..3c416e632523
Binary files /dev/null and b/static/img/openfn-color.webp differ
diff --git a/static/img/openfn_dpg.png b/static/img/openfn_dpg.png
deleted file mode 100644
index 24aae9b95c01..000000000000
Binary files a/static/img/openfn_dpg.png and /dev/null differ
diff --git a/static/img/openfn_dpg.webp b/static/img/openfn_dpg.webp
new file mode 100644
index 000000000000..acc0dcee4bad
Binary files /dev/null and b/static/img/openfn_dpg.webp differ
diff --git a/static/img/openfn_prod_staging.png b/static/img/openfn_prod_staging.png
deleted file mode 100644
index 43f4f26204a8..000000000000
Binary files a/static/img/openfn_prod_staging.png and /dev/null differ
diff --git a/static/img/openfn_prod_staging.webp b/static/img/openfn_prod_staging.webp
new file mode 100644
index 000000000000..1ee4fb9b43d4
Binary files /dev/null and b/static/img/openfn_prod_staging.webp differ
diff --git a/static/img/openfn_surveycto_trigger.png b/static/img/openfn_surveycto_trigger.png
deleted file mode 100644
index 4bdd1404c19f..000000000000
Binary files a/static/img/openfn_surveycto_trigger.png and /dev/null differ
diff --git a/static/img/openfn_surveycto_trigger.webp b/static/img/openfn_surveycto_trigger.webp
new file mode 100644
index 000000000000..d93601310de9
Binary files /dev/null and b/static/img/openfn_surveycto_trigger.webp differ
diff --git a/static/img/openhie_architecture.png b/static/img/openhie_architecture.png
deleted file mode 100644
index 75266286ba8c..000000000000
Binary files a/static/img/openhie_architecture.png and /dev/null differ
diff --git a/static/img/openhie_architecture.webp b/static/img/openhie_architecture.webp
new file mode 100644
index 000000000000..2294547272b3
Binary files /dev/null and b/static/img/openhie_architecture.webp differ
diff --git a/static/img/option1.png b/static/img/option1.png
deleted file mode 100644
index 2f69175f76fd..000000000000
Binary files a/static/img/option1.png and /dev/null differ
diff --git a/static/img/option1.webp b/static/img/option1.webp
new file mode 100644
index 000000000000..d6bf252526f5
Binary files /dev/null and b/static/img/option1.webp differ
diff --git a/static/img/os4h-dashboard.png b/static/img/os4h-dashboard.png
deleted file mode 100644
index bdae5ca0808b..000000000000
Binary files a/static/img/os4h-dashboard.png and /dev/null differ
diff --git a/static/img/os4h-dashboard.webp b/static/img/os4h-dashboard.webp
new file mode 100644
index 000000000000..f07ea81081df
Binary files /dev/null and b/static/img/os4h-dashboard.webp differ
diff --git a/static/img/os4h-flow.png b/static/img/os4h-flow.png
deleted file mode 100644
index 78bc648a4b6a..000000000000
Binary files a/static/img/os4h-flow.png and /dev/null differ
diff --git a/static/img/os4h-flow.webp b/static/img/os4h-flow.webp
new file mode 100644
index 000000000000..feca0a8afa8c
Binary files /dev/null and b/static/img/os4h-flow.webp differ
diff --git a/static/img/os4h-meet.png b/static/img/os4h-meet.png
deleted file mode 100644
index 767358e05fdd..000000000000
Binary files a/static/img/os4h-meet.png and /dev/null differ
diff --git a/static/img/os4h-meet.webp b/static/img/os4h-meet.webp
new file mode 100644
index 000000000000..f19e8f7f374e
Binary files /dev/null and b/static/img/os4h-meet.webp differ
diff --git a/static/img/os4h-team.png b/static/img/os4h-team.png
deleted file mode 100644
index 93a084451117..000000000000
Binary files a/static/img/os4h-team.png and /dev/null differ
diff --git a/static/img/os4h-team.webp b/static/img/os4h-team.webp
new file mode 100644
index 000000000000..231e2ae3f2ab
Binary files /dev/null and b/static/img/os4h-team.webp differ
diff --git a/static/img/passing-state-steps.png b/static/img/passing-state-steps.png
deleted file mode 100644
index 27bee14930ee..000000000000
Binary files a/static/img/passing-state-steps.png and /dev/null differ
diff --git a/static/img/passing-state-steps.webp b/static/img/passing-state-steps.webp
new file mode 100644
index 000000000000..4d9da6983aab
Binary files /dev/null and b/static/img/passing-state-steps.webp differ
diff --git a/static/img/path_conditions.png b/static/img/path_conditions.png
deleted file mode 100644
index 3648c209d7a1..000000000000
Binary files a/static/img/path_conditions.png and /dev/null differ
diff --git a/static/img/path_conditions.webp b/static/img/path_conditions.webp
new file mode 100644
index 000000000000..a54bd42b8854
Binary files /dev/null and b/static/img/path_conditions.webp differ
diff --git a/static/img/path_js_expression.png b/static/img/path_js_expression.png
deleted file mode 100644
index 53c16ba7aa10..000000000000
Binary files a/static/img/path_js_expression.png and /dev/null differ
diff --git a/static/img/path_js_expression.webp b/static/img/path_js_expression.webp
new file mode 100644
index 000000000000..bc0a3bf83480
Binary files /dev/null and b/static/img/path_js_expression.webp differ
diff --git a/static/img/path_main.png b/static/img/path_main.png
deleted file mode 100644
index 268149285b04..000000000000
Binary files a/static/img/path_main.png and /dev/null differ
diff --git a/static/img/path_main.webp b/static/img/path_main.webp
new file mode 100644
index 000000000000..632780d756fa
Binary files /dev/null and b/static/img/path_main.webp differ
diff --git a/static/img/path_staging.png b/static/img/path_staging.png
deleted file mode 100644
index 21f719468ec5..000000000000
Binary files a/static/img/path_staging.png and /dev/null differ
diff --git a/static/img/path_staging.webp b/static/img/path_staging.webp
new file mode 100644
index 000000000000..9c279ca710a5
Binary files /dev/null and b/static/img/path_staging.webp differ
diff --git a/static/img/prettier.png b/static/img/prettier.png
deleted file mode 100644
index 01065321592d..000000000000
Binary files a/static/img/prettier.png and /dev/null differ
diff --git a/static/img/prettier.webp b/static/img/prettier.webp
new file mode 100644
index 000000000000..687ae97643d9
Binary files /dev/null and b/static/img/prettier.webp differ
diff --git a/static/img/primero-ona-dash-example.png b/static/img/primero-ona-dash-example.png
deleted file mode 100644
index ddf3214a8384..000000000000
Binary files a/static/img/primero-ona-dash-example.png and /dev/null differ
diff --git a/static/img/primero-ona-dash-example.webp b/static/img/primero-ona-dash-example.webp
new file mode 100644
index 000000000000..cbea1bbe8599
Binary files /dev/null and b/static/img/primero-ona-dash-example.webp differ
diff --git a/static/img/primero_oscar.png b/static/img/primero_oscar.png
deleted file mode 100644
index 5c171f960e8e..000000000000
Binary files a/static/img/primero_oscar.png and /dev/null differ
diff --git a/static/img/primero_oscar.webp b/static/img/primero_oscar.webp
new file mode 100644
index 000000000000..14a7772b5bf3
Binary files /dev/null and b/static/img/primero_oscar.webp differ
diff --git a/static/img/projectid.png b/static/img/projectid.png
deleted file mode 100644
index 27f893e93f41..000000000000
Binary files a/static/img/projectid.png and /dev/null differ
diff --git a/static/img/projectid.webp b/static/img/projectid.webp
new file mode 100644
index 000000000000..49540d9d6c22
Binary files /dev/null and b/static/img/projectid.webp differ
diff --git a/static/img/projects-table.png b/static/img/projects-table.png
deleted file mode 100644
index 15ad8f8b1b25..000000000000
Binary files a/static/img/projects-table.png and /dev/null differ
diff --git a/static/img/projects-table.webp b/static/img/projects-table.webp
new file mode 100644
index 000000000000..1baa16dc8fd1
Binary files /dev/null and b/static/img/projects-table.webp differ
diff --git a/static/img/pull-request-2.png b/static/img/pull-request-2.png
deleted file mode 100644
index ae0aa3ed4577..000000000000
Binary files a/static/img/pull-request-2.png and /dev/null differ
diff --git a/static/img/pull-request-2.webp b/static/img/pull-request-2.webp
new file mode 100644
index 000000000000..0b3b1261b03a
Binary files /dev/null and b/static/img/pull-request-2.webp differ
diff --git a/static/img/pull-request.png b/static/img/pull-request.png
deleted file mode 100644
index 360f1975d29b..000000000000
Binary files a/static/img/pull-request.png and /dev/null differ
diff --git a/static/img/pull-request.webp b/static/img/pull-request.webp
new file mode 100644
index 000000000000..5f2296eba32d
Binary files /dev/null and b/static/img/pull-request.webp differ
diff --git a/static/img/questions1.jpeg b/static/img/questions1.jpeg
deleted file mode 100644
index e8833f096aa0..000000000000
Binary files a/static/img/questions1.jpeg and /dev/null differ
diff --git a/static/img/questions1.webp b/static/img/questions1.webp
new file mode 100644
index 000000000000..1b970be328ea
Binary files /dev/null and b/static/img/questions1.webp differ
diff --git a/static/img/recover_password.gif b/static/img/recover_password.gif
deleted file mode 100644
index 5ba367c33d3d..000000000000
Binary files a/static/img/recover_password.gif and /dev/null differ
diff --git a/static/img/reprocess-messages.png b/static/img/reprocess-messages.png
deleted file mode 100644
index ca79a4cbf9d5..000000000000
Binary files a/static/img/reprocess-messages.png and /dev/null differ
diff --git a/static/img/reprocess-messages.webp b/static/img/reprocess-messages.webp
new file mode 100644
index 000000000000..3e2dfefbeac5
Binary files /dev/null and b/static/img/reprocess-messages.webp differ
diff --git a/static/img/reprocess-runs.png b/static/img/reprocess-runs.png
deleted file mode 100644
index 6063d9d9a896..000000000000
Binary files a/static/img/reprocess-runs.png and /dev/null differ
diff --git a/static/img/reprocess-runs.webp b/static/img/reprocess-runs.webp
new file mode 100644
index 000000000000..389a3120a644
Binary files /dev/null and b/static/img/reprocess-runs.webp differ
diff --git a/static/img/reprocess_msgs.png b/static/img/reprocess_msgs.png
deleted file mode 100644
index b8deffece119..000000000000
Binary files a/static/img/reprocess_msgs.png and /dev/null differ
diff --git a/static/img/reprocess_msgs.webp b/static/img/reprocess_msgs.webp
new file mode 100644
index 000000000000..84ec1920297c
Binary files /dev/null and b/static/img/reprocess_msgs.webp differ
diff --git a/static/img/retrybutton.png b/static/img/retrybutton.png
deleted file mode 100644
index b044b1fdfc4a..000000000000
Binary files a/static/img/retrybutton.png and /dev/null differ
diff --git a/static/img/retrybutton.webp b/static/img/retrybutton.webp
new file mode 100644
index 000000000000..6f0eb2747cbf
Binary files /dev/null and b/static/img/retrybutton.webp differ
diff --git a/static/img/roads.jpg b/static/img/roads.jpg
deleted file mode 100644
index 63e3fd4ca2e5..000000000000
Binary files a/static/img/roads.jpg and /dev/null differ
diff --git a/static/img/roads.webp b/static/img/roads.webp
new file mode 100644
index 000000000000..1a1ceef3e135
Binary files /dev/null and b/static/img/roads.webp differ
diff --git a/static/img/round-logo.png b/static/img/round-logo.png
deleted file mode 100644
index a37bc61259ea..000000000000
Binary files a/static/img/round-logo.png and /dev/null differ
diff --git a/static/img/round-logo.webp b/static/img/round-logo.webp
new file mode 100644
index 000000000000..88495ec9ed95
Binary files /dev/null and b/static/img/round-logo.webp differ
diff --git a/static/img/run_view_logs.png b/static/img/run_view_logs.png
deleted file mode 100644
index 23ec90259730..000000000000
Binary files a/static/img/run_view_logs.png and /dev/null differ
diff --git a/static/img/run_view_logs.webp b/static/img/run_view_logs.webp
new file mode 100644
index 000000000000..227a22a43c13
Binary files /dev/null and b/static/img/run_view_logs.webp differ
diff --git a/static/img/runs_retry.png b/static/img/runs_retry.png
deleted file mode 100644
index 5d8dd4063004..000000000000
Binary files a/static/img/runs_retry.png and /dev/null differ
diff --git a/static/img/runs_retry.webp b/static/img/runs_retry.webp
new file mode 100644
index 000000000000..7b8f4a4db2c0
Binary files /dev/null and b/static/img/runs_retry.webp differ
diff --git a/static/img/runtimetrigger1.png b/static/img/runtimetrigger1.png
deleted file mode 100644
index b795fccb8eb0..000000000000
Binary files a/static/img/runtimetrigger1.png and /dev/null differ
diff --git a/static/img/runtimetrigger1.webp b/static/img/runtimetrigger1.webp
new file mode 100644
index 000000000000..71823e26e2f3
Binary files /dev/null and b/static/img/runtimetrigger1.webp differ
diff --git a/static/img/salesforce-cred.png b/static/img/salesforce-cred.png
deleted file mode 100644
index 1eb52e492256..000000000000
Binary files a/static/img/salesforce-cred.png and /dev/null differ
diff --git a/static/img/salesforce-cred.webp b/static/img/salesforce-cred.webp
new file mode 100644
index 000000000000..f898cb5ffd35
Binary files /dev/null and b/static/img/salesforce-cred.webp differ
diff --git a/static/img/salesforce-oauth2.png b/static/img/salesforce-oauth2.png
deleted file mode 100644
index ac6c573cbf98..000000000000
Binary files a/static/img/salesforce-oauth2.png and /dev/null differ
diff --git a/static/img/salesforce-oauth2.webp b/static/img/salesforce-oauth2.webp
new file mode 100644
index 000000000000..af44dd1ec903
Binary files /dev/null and b/static/img/salesforce-oauth2.webp differ
diff --git a/static/img/salesforce-old-cred.png b/static/img/salesforce-old-cred.png
deleted file mode 100644
index fe5267d1b2ff..000000000000
Binary files a/static/img/salesforce-old-cred.png and /dev/null differ
diff --git a/static/img/salesforce-old-cred.webp b/static/img/salesforce-old-cred.webp
new file mode 100644
index 000000000000..b88b1941eb8d
Binary files /dev/null and b/static/img/salesforce-old-cred.webp differ
diff --git a/static/img/sample-bpmn.png b/static/img/sample-bpmn.png
deleted file mode 100644
index adec747b5f7f..000000000000
Binary files a/static/img/sample-bpmn.png and /dev/null differ
diff --git a/static/img/sample-bpmn.webp b/static/img/sample-bpmn.webp
new file mode 100644
index 000000000000..2bdd5ffa5150
Binary files /dev/null and b/static/img/sample-bpmn.webp differ
diff --git a/static/img/sample-cce-workflow.webp b/static/img/sample-cce-workflow.webp
new file mode 100644
index 000000000000..a1f7844aef22
Binary files /dev/null and b/static/img/sample-cce-workflow.webp differ
diff --git a/static/img/satusehat_credential_edit.png b/static/img/satusehat_credential_edit.png
deleted file mode 100644
index b6d45e9e6a1b..000000000000
Binary files a/static/img/satusehat_credential_edit.png and /dev/null differ
diff --git a/static/img/satusehat_credential_edit.webp b/static/img/satusehat_credential_edit.webp
new file mode 100644
index 000000000000..13198df8b993
Binary files /dev/null and b/static/img/satusehat_credential_edit.webp differ
diff --git a/static/img/save-and-sync.webp b/static/img/save-and-sync.webp
new file mode 100644
index 000000000000..499cf11862af
Binary files /dev/null and b/static/img/save-and-sync.webp differ
diff --git a/static/img/save_db_job.gif b/static/img/save_db_job.gif
deleted file mode 100644
index cf179adbabc1..000000000000
Binary files a/static/img/save_db_job.gif and /dev/null differ
diff --git a/static/img/save_run_job_cc.gif b/static/img/save_run_job_cc.gif
deleted file mode 100644
index 88955044bab8..000000000000
Binary files a/static/img/save_run_job_cc.gif and /dev/null differ
diff --git a/static/img/search-options.png b/static/img/search-options.png
deleted file mode 100644
index 34a523f157f9..000000000000
Binary files a/static/img/search-options.png and /dev/null differ
diff --git a/static/img/search-options.webp b/static/img/search-options.webp
new file mode 100644
index 000000000000..844446214497
Binary files /dev/null and b/static/img/search-options.webp differ
diff --git a/static/img/search_dataclip_by_name.webp b/static/img/search_dataclip_by_name.webp
new file mode 100644
index 000000000000..35e10115f9d7
Binary files /dev/null and b/static/img/search_dataclip_by_name.webp differ
diff --git a/static/img/security-token.png b/static/img/security-token.png
deleted file mode 100644
index 2d7e6c4a22c7..000000000000
Binary files a/static/img/security-token.png and /dev/null differ
diff --git a/static/img/security-token.webp b/static/img/security-token.webp
new file mode 100644
index 000000000000..a6943d59f6ca
Binary files /dev/null and b/static/img/security-token.webp differ
diff --git a/static/img/select_workflow_to_add.png b/static/img/select_workflow_to_add.png
deleted file mode 100644
index 22e696853819..000000000000
Binary files a/static/img/select_workflow_to_add.png and /dev/null differ
diff --git a/static/img/select_workflow_to_add.webp b/static/img/select_workflow_to_add.webp
new file mode 100644
index 000000000000..8d76ca0e4466
Binary files /dev/null and b/static/img/select_workflow_to_add.webp differ
diff --git a/static/img/settings_credentials.png b/static/img/settings_credentials.png
deleted file mode 100644
index 089a0f295351..000000000000
Binary files a/static/img/settings_credentials.png and /dev/null differ
diff --git a/static/img/settings_credentials.webp b/static/img/settings_credentials.webp
new file mode 100644
index 000000000000..acf2c7de01b7
Binary files /dev/null and b/static/img/settings_credentials.webp differ
diff --git a/static/img/sffh.png b/static/img/sffh.png
deleted file mode 100644
index 49d4ffa9a84f..000000000000
Binary files a/static/img/sffh.png and /dev/null differ
diff --git a/static/img/sffh.webp b/static/img/sffh.webp
new file mode 100644
index 000000000000..e15ccff50245
Binary files /dev/null and b/static/img/sffh.webp differ
diff --git a/static/img/share_oauth_client.png b/static/img/share_oauth_client.png
deleted file mode 100644
index 59d2cc10b606..000000000000
Binary files a/static/img/share_oauth_client.png and /dev/null differ
diff --git a/static/img/share_oauth_client.webp b/static/img/share_oauth_client.webp
new file mode 100644
index 000000000000..c6aac78b4617
Binary files /dev/null and b/static/img/share_oauth_client.webp differ
diff --git a/static/img/show_only_named_dataclips.webp b/static/img/show_only_named_dataclips.webp
new file mode 100644
index 000000000000..1183c12c4c3b
Binary files /dev/null and b/static/img/show_only_named_dataclips.webp differ
diff --git a/static/img/sinapis.jpg b/static/img/sinapis.jpg
deleted file mode 100644
index f9adadde629b..000000000000
Binary files a/static/img/sinapis.jpg and /dev/null differ
diff --git a/static/img/sinapis.webp b/static/img/sinapis.webp
new file mode 100644
index 000000000000..5ae7a89a8509
Binary files /dev/null and b/static/img/sinapis.webp differ
diff --git a/static/img/snapshots1.png b/static/img/snapshots1.png
deleted file mode 100644
index 639cbfee3312..000000000000
Binary files a/static/img/snapshots1.png and /dev/null differ
diff --git a/static/img/snapshots1.webp b/static/img/snapshots1.webp
new file mode 100644
index 000000000000..500bcee20f8f
Binary files /dev/null and b/static/img/snapshots1.webp differ
diff --git a/static/img/snapshots2.png b/static/img/snapshots2.png
deleted file mode 100644
index dc44843bf3f6..000000000000
Binary files a/static/img/snapshots2.png and /dev/null differ
diff --git a/static/img/snapshots2.webp b/static/img/snapshots2.webp
new file mode 100644
index 000000000000..88d9a7ce6929
Binary files /dev/null and b/static/img/snapshots2.webp differ
diff --git a/static/img/snapshots3.png b/static/img/snapshots3.png
deleted file mode 100644
index 8ec782bdba3f..000000000000
Binary files a/static/img/snapshots3.png and /dev/null differ
diff --git a/static/img/snapshots3.webp b/static/img/snapshots3.webp
new file mode 100644
index 000000000000..4e839aa06c27
Binary files /dev/null and b/static/img/snapshots3.webp differ
diff --git a/static/img/snapshots4.png b/static/img/snapshots4.png
deleted file mode 100644
index 46bbde2de691..000000000000
Binary files a/static/img/snapshots4.png and /dev/null differ
diff --git a/static/img/snapshots4.webp b/static/img/snapshots4.webp
new file mode 100644
index 000000000000..4a3327558a42
Binary files /dev/null and b/static/img/snapshots4.webp differ
diff --git a/static/img/snapshots5.png b/static/img/snapshots5.png
deleted file mode 100644
index 27de4d926b9e..000000000000
Binary files a/static/img/snapshots5.png and /dev/null differ
diff --git a/static/img/snapshots5.webp b/static/img/snapshots5.webp
new file mode 100644
index 000000000000..bf1159f4431b
Binary files /dev/null and b/static/img/snapshots5.webp differ
diff --git a/static/img/snapshots6.png b/static/img/snapshots6.png
deleted file mode 100644
index a4e70646f60e..000000000000
Binary files a/static/img/snapshots6.png and /dev/null differ
diff --git a/static/img/snapshots6.webp b/static/img/snapshots6.webp
new file mode 100644
index 000000000000..d0b0053addd3
Binary files /dev/null and b/static/img/snapshots6.webp differ
diff --git a/static/img/solution_diagram1.png b/static/img/solution_diagram1.png
deleted file mode 100644
index 7650448f0baf..000000000000
Binary files a/static/img/solution_diagram1.png and /dev/null differ
diff --git a/static/img/solution_diagram1.webp b/static/img/solution_diagram1.webp
new file mode 100644
index 000000000000..0250e59529fb
Binary files /dev/null and b/static/img/solution_diagram1.webp differ
diff --git a/static/img/solution_diagram2.png b/static/img/solution_diagram2.png
deleted file mode 100644
index 6b5e9ec3a837..000000000000
Binary files a/static/img/solution_diagram2.png and /dev/null differ
diff --git a/static/img/solution_diagram2.webp b/static/img/solution_diagram2.webp
new file mode 100644
index 000000000000..f1b3ca61d896
Binary files /dev/null and b/static/img/solution_diagram2.webp differ
diff --git a/static/img/srcfolder.png b/static/img/srcfolder.png
deleted file mode 100644
index aa7c2bc3fd84..000000000000
Binary files a/static/img/srcfolder.png and /dev/null differ
diff --git a/static/img/srcfolder.webp b/static/img/srcfolder.webp
new file mode 100644
index 000000000000..3870b73b4f47
Binary files /dev/null and b/static/img/srcfolder.webp differ
diff --git a/static/img/staging_prod_branches_gh.png b/static/img/staging_prod_branches_gh.png
deleted file mode 100644
index e0ce3f8c2fd0..000000000000
Binary files a/static/img/staging_prod_branches_gh.png and /dev/null differ
diff --git a/static/img/staging_prod_branches_gh.webp b/static/img/staging_prod_branches_gh.webp
new file mode 100644
index 000000000000..ab440d65296f
Binary files /dev/null and b/static/img/staging_prod_branches_gh.webp differ
diff --git a/static/img/staging_pushes.png b/static/img/staging_pushes.png
deleted file mode 100644
index 5b6e4e703b61..000000000000
Binary files a/static/img/staging_pushes.png and /dev/null differ
diff --git a/static/img/staging_pushes.webp b/static/img/staging_pushes.webp
new file mode 100644
index 000000000000..dcf48e2ab662
Binary files /dev/null and b/static/img/staging_pushes.webp differ
diff --git a/static/img/state-javascript.png b/static/img/state-javascript.png
deleted file mode 100644
index 60a1ed0c1f4e..000000000000
Binary files a/static/img/state-javascript.png and /dev/null differ
diff --git a/static/img/state-javascript.webp b/static/img/state-javascript.webp
new file mode 100644
index 000000000000..fdfeca2d704d
Binary files /dev/null and b/static/img/state-javascript.webp differ
diff --git a/static/img/step-anatomy.png b/static/img/step-anatomy.png
deleted file mode 100644
index b47a82be234b..000000000000
Binary files a/static/img/step-anatomy.png and /dev/null differ
diff --git a/static/img/step-anatomy.webp b/static/img/step-anatomy.webp
new file mode 100644
index 000000000000..7537da01c1ae
Binary files /dev/null and b/static/img/step-anatomy.webp differ
diff --git a/static/img/survey_db_powerbi.png b/static/img/survey_db_powerbi.png
deleted file mode 100644
index 0f94a41c095d..000000000000
Binary files a/static/img/survey_db_powerbi.png and /dev/null differ
diff --git a/static/img/survey_db_powerbi.webp b/static/img/survey_db_powerbi.webp
new file mode 100644
index 000000000000..baf078abc25a
Binary files /dev/null and b/static/img/survey_db_powerbi.webp differ
diff --git a/static/img/survey_db_tableau.png b/static/img/survey_db_tableau.png
deleted file mode 100644
index a0e5aa26ef43..000000000000
Binary files a/static/img/survey_db_tableau.png and /dev/null differ
diff --git a/static/img/survey_db_tableau.webp b/static/img/survey_db_tableau.webp
new file mode 100644
index 000000000000..d9aba9bfb4cf
Binary files /dev/null and b/static/img/survey_db_tableau.webp differ
diff --git a/static/img/surveycto_webhook_config.png b/static/img/surveycto_webhook_config.png
deleted file mode 100644
index ff43f29a6bc5..000000000000
Binary files a/static/img/surveycto_webhook_config.png and /dev/null differ
diff --git a/static/img/surveycto_webhook_config.webp b/static/img/surveycto_webhook_config.webp
new file mode 100644
index 000000000000..8233a55bf601
Binary files /dev/null and b/static/img/surveycto_webhook_config.webp differ
diff --git a/static/img/swisstph.png b/static/img/swisstph.png
deleted file mode 100644
index c0d5340a5ebe..000000000000
Binary files a/static/img/swisstph.png and /dev/null differ
diff --git a/static/img/swisstph.webp b/static/img/swisstph.webp
new file mode 100644
index 000000000000..a0d36c0edfeb
Binary files /dev/null and b/static/img/swisstph.webp differ
diff --git a/static/img/swisstph1.jpg b/static/img/swisstph1.jpg
deleted file mode 100644
index 65fdd53104df..000000000000
Binary files a/static/img/swisstph1.jpg and /dev/null differ
diff --git a/static/img/swisstph1.webp b/static/img/swisstph1.webp
new file mode 100644
index 000000000000..a587e5cde17f
Binary files /dev/null and b/static/img/swisstph1.webp differ
diff --git a/static/img/sync_to_github.png b/static/img/sync_to_github.png
deleted file mode 100644
index cbe97c1c125e..000000000000
Binary files a/static/img/sync_to_github.png and /dev/null differ
diff --git a/static/img/sync_to_github.webp b/static/img/sync_to_github.webp
new file mode 100644
index 000000000000..f0f58fa2f419
Binary files /dev/null and b/static/img/sync_to_github.webp differ
diff --git a/static/img/syncs1.png b/static/img/syncs1.png
deleted file mode 100644
index 77fc46b35bf2..000000000000
Binary files a/static/img/syncs1.png and /dev/null differ
diff --git a/static/img/syncs1.webp b/static/img/syncs1.webp
new file mode 100644
index 000000000000..40b38a14a80f
Binary files /dev/null and b/static/img/syncs1.webp differ
diff --git a/static/img/syncs2.png b/static/img/syncs2.png
deleted file mode 100644
index 57ec94477668..000000000000
Binary files a/static/img/syncs2.png and /dev/null differ
diff --git a/static/img/syncs2.webp b/static/img/syncs2.webp
new file mode 100644
index 000000000000..475761b9cd80
Binary files /dev/null and b/static/img/syncs2.webp differ
diff --git a/static/img/taylor-enouce.jpeg b/static/img/taylor-enouce.jpeg
deleted file mode 100644
index bf2b3748c6ae..000000000000
Binary files a/static/img/taylor-enouce.jpeg and /dev/null differ
diff --git a/static/img/taylor-enouce.webp b/static/img/taylor-enouce.webp
new file mode 100644
index 000000000000..b94ac5a66949
Binary files /dev/null and b/static/img/taylor-enouce.webp differ
diff --git a/static/img/team-kenya-meal.jpg b/static/img/team-kenya-meal.jpg
deleted file mode 100644
index 875b7c6d7b38..000000000000
Binary files a/static/img/team-kenya-meal.jpg and /dev/null differ
diff --git a/static/img/team-kenya-meal.webp b/static/img/team-kenya-meal.webp
new file mode 100644
index 000000000000..52c9fa6b19f1
Binary files /dev/null and b/static/img/team-kenya-meal.webp differ
diff --git a/static/img/team-kenya.jpg b/static/img/team-kenya.jpg
deleted file mode 100644
index 135bbd099ca8..000000000000
Binary files a/static/img/team-kenya.jpg and /dev/null differ
diff --git a/static/img/team-kenya.webp b/static/img/team-kenya.webp
new file mode 100644
index 000000000000..8b92fe3c0416
Binary files /dev/null and b/static/img/team-kenya.webp differ
diff --git a/static/img/team-no-td.jpg b/static/img/team-no-td.jpg
deleted file mode 100644
index 539a8b7338da..000000000000
Binary files a/static/img/team-no-td.jpg and /dev/null differ
diff --git a/static/img/team-no-td.webp b/static/img/team-no-td.webp
new file mode 100644
index 000000000000..144f456eb279
Binary files /dev/null and b/static/img/team-no-td.webp differ
diff --git a/static/img/teamphoto.jpg b/static/img/teamphoto.jpg
deleted file mode 100644
index 28f11e0cc353..000000000000
Binary files a/static/img/teamphoto.jpg and /dev/null differ
diff --git a/static/img/teamphoto.webp b/static/img/teamphoto.webp
new file mode 100644
index 000000000000..7b68d62cb61a
Binary files /dev/null and b/static/img/teamphoto.webp differ
diff --git a/static/img/technical_example.png b/static/img/technical_example.png
deleted file mode 100644
index fc353c3f1512..000000000000
Binary files a/static/img/technical_example.png and /dev/null differ
diff --git a/static/img/technical_example.webp b/static/img/technical_example.webp
new file mode 100644
index 000000000000..69abed74976c
Binary files /dev/null and b/static/img/technical_example.webp differ
diff --git a/static/img/technology1.jpg b/static/img/technology1.jpg
deleted file mode 100644
index f40ca8fa9293..000000000000
Binary files a/static/img/technology1.jpg and /dev/null differ
diff --git a/static/img/technology1.webp b/static/img/technology1.webp
new file mode 100644
index 000000000000..c74bbd40b0c9
Binary files /dev/null and b/static/img/technology1.webp differ
diff --git a/static/img/techpic.jpg b/static/img/techpic.jpg
deleted file mode 100644
index 33f3af2889f1..000000000000
Binary files a/static/img/techpic.jpg and /dev/null differ
diff --git a/static/img/techpic.webp b/static/img/techpic.webp
new file mode 100644
index 000000000000..3c76b6601443
Binary files /dev/null and b/static/img/techpic.webp differ
diff --git a/static/img/timetriggerunslist.png b/static/img/timetriggerunslist.png
deleted file mode 100644
index 40a924f35e8f..000000000000
Binary files a/static/img/timetriggerunslist.png and /dev/null differ
diff --git a/static/img/timetriggerunslist.webp b/static/img/timetriggerunslist.webp
new file mode 100644
index 000000000000..342eaa7fb246
Binary files /dev/null and b/static/img/timetriggerunslist.webp differ
diff --git a/static/img/trackedEntities.jpg b/static/img/trackedEntities.jpg
deleted file mode 100644
index 1cda9c8e439a..000000000000
Binary files a/static/img/trackedEntities.jpg and /dev/null differ
diff --git a/static/img/trackedEntities.webp b/static/img/trackedEntities.webp
new file mode 100644
index 000000000000..c823999e7ac1
Binary files /dev/null and b/static/img/trackedEntities.webp differ
diff --git a/static/img/unicef-cambodia-cc-daravatey-seng.jpg b/static/img/unicef-cambodia-cc-daravatey-seng.jpg
deleted file mode 100644
index 47a99f431b67..000000000000
Binary files a/static/img/unicef-cambodia-cc-daravatey-seng.jpg and /dev/null differ
diff --git a/static/img/unicef-cambodia-cc-daravatey-seng.webp b/static/img/unicef-cambodia-cc-daravatey-seng.webp
new file mode 100644
index 000000000000..792a02af0236
Binary files /dev/null and b/static/img/unicef-cambodia-cc-daravatey-seng.webp differ
diff --git a/static/img/usage.png b/static/img/usage.png
deleted file mode 100644
index 81450c5840e2..000000000000
Binary files a/static/img/usage.png and /dev/null differ
diff --git a/static/img/usage.webp b/static/img/usage.webp
new file mode 100644
index 000000000000..3bdb6929c143
Binary files /dev/null and b/static/img/usage.webp differ
diff --git a/static/img/usethistemplate.png b/static/img/usethistemplate.png
deleted file mode 100644
index cf682b87a22d..000000000000
Binary files a/static/img/usethistemplate.png and /dev/null differ
diff --git a/static/img/usethistemplate.webp b/static/img/usethistemplate.webp
new file mode 100644
index 000000000000..1d54811ce020
Binary files /dev/null and b/static/img/usethistemplate.webp differ
diff --git a/static/img/v1_unified_inbox.webp b/static/img/v1_unified_inbox.webp
new file mode 100644
index 000000000000..0215980b60c4
Binary files /dev/null and b/static/img/v1_unified_inbox.webp differ
diff --git a/static/img/v2_webhook_trigger.webp b/static/img/v2_webhook_trigger.webp
new file mode 100644
index 000000000000..674c215b062d
Binary files /dev/null and b/static/img/v2_webhook_trigger.webp differ
diff --git a/static/img/via-trigger-panel.webp b/static/img/via-trigger-panel.webp
new file mode 100644
index 000000000000..829c353449a3
Binary files /dev/null and b/static/img/via-trigger-panel.webp differ
diff --git a/static/img/vscode-settings.png b/static/img/vscode-settings.png
deleted file mode 100644
index 89c46537296f..000000000000
Binary files a/static/img/vscode-settings.png and /dev/null differ
diff --git a/static/img/vscode-settings.webp b/static/img/vscode-settings.webp
new file mode 100644
index 000000000000..b34226e01136
Binary files /dev/null and b/static/img/vscode-settings.webp differ
diff --git a/static/img/webhook_trigger.png b/static/img/webhook_trigger.png
deleted file mode 100644
index b1a070a07c58..000000000000
Binary files a/static/img/webhook_trigger.png and /dev/null differ
diff --git a/static/img/webhook_trigger.webp b/static/img/webhook_trigger.webp
new file mode 100644
index 000000000000..bec6a73c6157
Binary files /dev/null and b/static/img/webhook_trigger.webp differ
diff --git a/static/img/webhooks1.png b/static/img/webhooks1.png
deleted file mode 100644
index e1ca81d07588..000000000000
Binary files a/static/img/webhooks1.png and /dev/null differ
diff --git a/static/img/webhooks1.webp b/static/img/webhooks1.webp
new file mode 100644
index 000000000000..c5ffb776a706
Binary files /dev/null and b/static/img/webhooks1.webp differ
diff --git a/static/img/work_order_shot.png b/static/img/work_order_shot.png
deleted file mode 100644
index bb8f02ad5497..000000000000
Binary files a/static/img/work_order_shot.png and /dev/null differ
diff --git a/static/img/work_order_shot.webp b/static/img/work_order_shot.webp
new file mode 100644
index 000000000000..cb779a97227d
Binary files /dev/null and b/static/img/work_order_shot.webp differ
diff --git a/static/img/workflow-ocl-example.png b/static/img/workflow-ocl-example.png
deleted file mode 100644
index 428c6c72640d..000000000000
Binary files a/static/img/workflow-ocl-example.png and /dev/null differ
diff --git a/static/img/workflow-ocl-example.webp b/static/img/workflow-ocl-example.webp
new file mode 100644
index 000000000000..7556b1a3117b
Binary files /dev/null and b/static/img/workflow-ocl-example.webp differ
diff --git a/static/img/workflow.png b/static/img/workflow.png
deleted file mode 100644
index 5e2488413a89..000000000000
Binary files a/static/img/workflow.png and /dev/null differ
diff --git a/static/img/workflow.webp b/static/img/workflow.webp
new file mode 100644
index 000000000000..2963c08e5827
Binary files /dev/null and b/static/img/workflow.webp differ
diff --git a/static/img/workflow_automation.png b/static/img/workflow_automation.png
deleted file mode 100644
index cc6266657830..000000000000
Binary files a/static/img/workflow_automation.png and /dev/null differ
diff --git a/static/img/workflow_automation.webp b/static/img/workflow_automation.webp
new file mode 100644
index 000000000000..c24ad24a2c63
Binary files /dev/null and b/static/img/workflow_automation.webp differ
diff --git a/static/img/workflow_canvas_toggle.webp b/static/img/workflow_canvas_toggle.webp
new file mode 100644
index 000000000000..03ef4dfe0cc1
Binary files /dev/null and b/static/img/workflow_canvas_toggle.webp differ
diff --git a/static/img/workflow_fhir_fhir.png b/static/img/workflow_fhir_fhir.png
deleted file mode 100644
index 1aff55357a24..000000000000
Binary files a/static/img/workflow_fhir_fhir.png and /dev/null differ
diff --git a/static/img/workflow_fhir_fhir.webp b/static/img/workflow_fhir_fhir.webp
new file mode 100644
index 000000000000..c34c2fdbd748
Binary files /dev/null and b/static/img/workflow_fhir_fhir.webp differ
diff --git a/static/img/workflow_list_toggle.webp b/static/img/workflow_list_toggle.webp
new file mode 100644
index 000000000000..a62688ef4372
Binary files /dev/null and b/static/img/workflow_list_toggle.webp differ
diff --git a/static/img/workflow_nonfhir_fhir.png b/static/img/workflow_nonfhir_fhir.png
deleted file mode 100644
index dd16f62debe1..000000000000
Binary files a/static/img/workflow_nonfhir_fhir.png and /dev/null differ
diff --git a/static/img/workflow_nonfhir_fhir.webp b/static/img/workflow_nonfhir_fhir.webp
new file mode 100644
index 000000000000..4ca75362fb49
Binary files /dev/null and b/static/img/workflow_nonfhir_fhir.webp differ
diff --git a/static/img/workflow_specs.png b/static/img/workflow_specs.png
deleted file mode 100644
index ed2631115164..000000000000
Binary files a/static/img/workflow_specs.png and /dev/null differ
diff --git a/static/img/workflow_specs.webp b/static/img/workflow_specs.webp
new file mode 100644
index 000000000000..7f4a5390a182
Binary files /dev/null and b/static/img/workflow_specs.webp differ
diff --git a/static/img/workflows-v-adaptors.png b/static/img/workflows-v-adaptors.png
deleted file mode 100644
index eaccacbbc1c0..000000000000
Binary files a/static/img/workflows-v-adaptors.png and /dev/null differ
diff --git a/static/img/workflows-v-adaptors.webp b/static/img/workflows-v-adaptors.webp
new file mode 100644
index 000000000000..7fa9d5b4c141
Binary files /dev/null and b/static/img/workflows-v-adaptors.webp differ
diff --git a/static/img/x-runner.jpg b/static/img/x-runner.jpg
deleted file mode 100644
index c27cb12157eb..000000000000
Binary files a/static/img/x-runner.jpg and /dev/null differ
diff --git a/static/img/x-runner.webp b/static/img/x-runner.webp
new file mode 100644
index 000000000000..ce52a7da26f9
Binary files /dev/null and b/static/img/x-runner.webp differ
diff --git a/static/img/zero-persistence.webp b/static/img/zero-persistence.webp
new file mode 100644
index 000000000000..2083d1ed6f97
Binary files /dev/null and b/static/img/zero-persistence.webp differ
diff --git a/static/img/zero_persistence_architecture.png b/static/img/zero_persistence_architecture.png
deleted file mode 100644
index 83af4250ac6b..000000000000
Binary files a/static/img/zero_persistence_architecture.png and /dev/null differ
diff --git a/static/img/zero_persistence_architecture.webp b/static/img/zero_persistence_architecture.webp
new file mode 100644
index 000000000000..eb97ce5d4275
Binary files /dev/null and b/static/img/zero_persistence_architecture.webp differ
diff --git a/versioned_docs/version-legacy/about-lightning.md b/versioned_docs/version-legacy/about-lightning.md
index e9f57b87ba9c..b925e24901e0 100644
--- a/versioned_docs/version-legacy/about-lightning.md
+++ b/versioned_docs/version-legacy/about-lightning.md
@@ -24,7 +24,7 @@ comes with a user-friendly, low-code interface with the full functionality
needed for organizations to build, run and audit their workflows all in one
place.
-
+
### Build
@@ -34,7 +34,7 @@ non-technical users, bridging the gap between the IT specialists that build out
automations and program managers that are the real business/ program experts on
the processes that need automating.
-
+
### Audit
@@ -45,7 +45,7 @@ managing their savings to make sure they can afford the next harvest. Lightning
provides users with a dashboard that allows them to monitor the health of their
integrations to make sure no request goes unprocessed.
-
+
## Features
@@ -334,7 +334,7 @@ comments are welcome there. If you would like to become a beta user or learn
more about Lightning, book in a call with our product manager here:
https://calendly.com/amber-openfn/short-call.
-
+
## Lightning FAQ
diff --git a/versioned_docs/version-legacy/build/example-build.md b/versioned_docs/version-legacy/build/example-build.md
index f99b6c58388e..a8f0f94d386e 100644
--- a/versioned_docs/version-legacy/build/example-build.md
+++ b/versioned_docs/version-legacy/build/example-build.md
@@ -39,42 +39,42 @@ you'll see that a sample project has been created for you.
Create a new project called ‘Kobo case registrations’ by clicking on the blue +
icon at the bottom right hand corner of your dashboard.
-
+
When you click 'View' to enter your project space, you'll be taken to your
**inbox**. This is where you will receive **messages** - the data that gets sent
from your source system to OpenFn. Copy your **inbox url** to configure
KoboToolbox to send data to it.
-
+
[Log into](https://kf.kobotoolbox.org/accounts/login/#/) our KoboToolbox demo
account with _username: openfn_demo and password: openfn_demo_. Select the form
you’d like to connect (if using our demo account this will be 'COVID 19 case
registration') and go to Settings -> REST services -> Register a new service.
-
+
Set the service name to OpenFn and the URL to your project inbox url.
-
+
Your form should now be configured to send data to your OpenFn project inbox
whenever a response is submitted. We can test this out by submitting some form
responses at Form -> Open.
-
+
Return to your project inbox. You should see a new message there, which contains
the data submitted in the KoboToolbox form response.
-
+
If you click on the message, and open up the **message body** you’ll see the
data that you submitted to the form. To view the entire message, open it in full
screen.
-
+
Once you can see the entire message, you need to identify a data point that will
be the same for every submission. In this case, we know that all of our messages
@@ -82,7 +82,7 @@ will have the same form ID. Save the snippet you have identified
(`"\_xform_id_string": "aDReHdA7UuNBYsiCXQBr43"`), you'll need it later to
create your trigger.
-
+
## 3. Create credentials to connect your destination system
@@ -93,14 +93,14 @@ sheet.
Head to the credentials section of your dashboard, and once again click the
blue + sign to create new credentials.
-
+
You’ll see various apps you recognise - these are all of the systems that we can
handle credentials for. Select the `Sheets` one, and log into your google
account when you get the pop up window. You’ll get a confirmation message. Close
the window and give your new project access to these credentials.
-
+
You’ve now created credentials that will allow you to perform operations in
google sheets from within your job.
@@ -115,7 +115,7 @@ where it should go.
Navigate to the jobs section in your dashboard, then click the + icon to create
a new job.
-
+
Give the job a name (we’ll make ours “Kobo to sheets”).
@@ -135,14 +135,14 @@ message body sent by each submitted form response to your inbox.
This message filter will trigger your job whenever a message which includes the
snippet comes into your inbox.
-
+
Save your trigger. You should see a confirmation message “Found x matching
messages”. To see the data from your last message inside the
-[initial state](../jobs/state.md), drag the
+[initial state](/documentation/legacy/jobs/state), drag the
**Expression** panel to the right.
-
+
### 4.2 Select an API adaptor
@@ -163,7 +163,7 @@ Open up the inline documentation for the adaptor to see the available functions.
Copy the appendValues function, then paste it into your Expression editor. It
should look something like this.
-
+
### 4.4 Edit the function in your expression editor
@@ -175,7 +175,7 @@ want to send.
First, get your spreadsheet ID from the URL of your google sheet (between `d/`
and `/edit`).
-
+
Copy and paste the ID into your `appendValues` operation to replace the
placeholder value for `spreadsheetId`. This ensures your values get appended to
@@ -188,7 +188,7 @@ state window and paste it to replace the placeholder text ('From expression')
inside `values: []`. Repeat this for the following values, and remove line 7 as
this would add a second row to your sheet.
-
+
Your operation should now look like this:
@@ -210,7 +210,7 @@ Click `Save and run` to get a ‘Success!’ response in the `run logs` and see
the data entries between the square brackets [ ] have been added to your google
sheet.
-
+
## 5. Set autoprocess to true
@@ -218,7 +218,7 @@ You have now written and tested your job. In order to run your job automatically
every time a message matches the trigger inclusion criteria, turn on
auto-process.
-
+
You're all set! Try out your job by submitting another form response to see the
data automatically populate your google sheet.
diff --git a/versioned_docs/version-legacy/build/lightning-quick-start.md b/versioned_docs/version-legacy/build/lightning-quick-start.md
index 2c372d2b892d..a39e56cb1269 100644
--- a/versioned_docs/version-legacy/build/lightning-quick-start.md
+++ b/versioned_docs/version-legacy/build/lightning-quick-start.md
@@ -19,7 +19,7 @@ to your inbox to confirm your email.
Click on the 'sample workflow' created for you on registration.
-
+
:::tip
@@ -37,7 +37,7 @@ taking a patient’s name and age and:
2. converting it to the same format as DHIS2;
3. uploading it to DHIS2.
-
+
It is made up of 3 _jobs_.
@@ -55,7 +55,7 @@ Click on Job 3 to view more details about it in the setup and editor tab.
The SETUP TAB is where you define the when, where and how of your job.
-
+
**When: trigger**
@@ -110,7 +110,7 @@ job during previous runs.
In job 3, we'll be using the data values that are in `names` which are
`"Wycliffe"` and `"Orao"` in this example. Can you see them?
-
+
:::tip
@@ -128,11 +128,11 @@ state (which contains your input) to use.
When you need to use data that comes from your webhook trigger (data sent from
your external system), cron trigger, or a previous job you can find it in
-`state`. Learn more [here](../jobs/state.md/).
+`state`. Learn more [here](/documentation/legacy/jobs/state).
:::
-
+
In this job, we're using the `names` data from state (which we saw in the Input
tab).
@@ -146,7 +146,7 @@ It gets added from the adaptor documentation _below_ the editor as an example
operation, and is then configured to use specific values from the state input
data. (see image below for details)
-
+
## 3. Run the sample workflow
@@ -176,7 +176,7 @@ data below into the `custom input`, then click `run`.
}
```
-
+
You should now be able to
[see your request on the history page](#4-check-your-request-got-processed-correctly).
@@ -232,18 +232,18 @@ account with _username: openfn_demo and password: openfn_demo_. Select the form
you’d like to connect ('Lightning sample workflow') and go to Settings -> REST
services -> Register a new service.
-
+
Set the service name to OpenFn and the URL to the webhook URL (you can copy is
from the first node on your workflow).
-
+
Your form should now be configured to send data to the webhook trigger for your
first job whenever a response is submitted. We can test this out by submitting
some form responses at Form -> Open.
-
+
Once you've made a form submission, you should be able to
[see your request on the history page](#4-check-your-request-got-processed-correctly).
@@ -261,11 +261,11 @@ Now that you have run your workflow, head to the history page to see the work
order. You'll see it has a status of 'Success' which means it got processed
correctly.
-
+
Click on the chevron next to the status to expand it and see each job run.
-
+
## 5. Make a run that fails, then edit the job and rerun it to make it succeed
@@ -284,7 +284,7 @@ old using the data below.
Head to the history page and see that the work order has a status of 'Failure'.
This is because the patient is **not** older than 18 months.
-
+
Let's say we made a mistake and _actually_ wanted to register any patient that
is _**both**_ 18 months old _**and**_ above. We want to edit the job logic and
@@ -312,19 +312,19 @@ to find it.
Expand the work order, and click the 'rerun' button next to the first job run.
-
+
You'll see a new **attempt** created in the same work order, which now succeeds.
The work order status also gets updated to the status of the last attempt to
show 'Success'.
-
+
Rerun the same work order, this time from 'Job 3 - Upload to DHIS2'. You'll see
the runs for Job 1 and 2 get copied over to the new attempt, so that their
output can be used for the input of Job 3.
-
+
:::tip Note
diff --git a/versioned_docs/version-legacy/build/triggers.md b/versioned_docs/version-legacy/build/triggers.md
index d6d2db598c69..e30f72e0d7b8 100644
--- a/versioned_docs/version-legacy/build/triggers.md
+++ b/versioned_docs/version-legacy/build/triggers.md
@@ -77,9 +77,9 @@ This way you don’t have to wait for the timer to expire before testing! Simply
click the process/ “play” button now available via the Job, Run, and Activity
History pages.
-
+
-
+
#### Keeping a cursor in `state` for timer Jobs
diff --git a/versioned_docs/version-legacy/design/design-quickstart.md b/versioned_docs/version-legacy/design/design-quickstart.md
index 98b6b51c417c..43eb824000c1 100644
--- a/versioned_docs/version-legacy/design/design-quickstart.md
+++ b/versioned_docs/version-legacy/design/design-quickstart.md
@@ -109,7 +109,7 @@ See the below example BPMN diagram for the user story:
> contacts in my SMS campaign configured on RapidPro to send them automated
> alerts and program updates.
-
+
## 3. Map data elements to be exchanged
diff --git a/versioned_docs/version-legacy/design/when-to-integrate.md b/versioned_docs/version-legacy/design/when-to-integrate.md
deleted file mode 100644
index 2055f4d64504..000000000000
--- a/versioned_docs/version-legacy/design/when-to-integrate.md
+++ /dev/null
@@ -1,8 +0,0 @@
----
-Title: When to Integrate
----
-
-This article is a stub... it's coming soon.
-
-In the meantime, check out Aleksa Krolls'
-[Three Questions To Ask](articles/2020/06/24/three-questions-to-ask).
diff --git a/versioned_docs/version-legacy/devtools/home.md b/versioned_docs/version-legacy/devtools/home.md
index a06ca9e0ebeb..0c89f82c42b8 100644
--- a/versioned_docs/version-legacy/devtools/home.md
+++ b/versioned_docs/version-legacy/devtools/home.md
@@ -5,8 +5,7 @@ title: Devtools
:::caution Devtools is deprecated
Please note that [OpenFn/devtools](https://github.com/OpenFn/devtools) are being
-deprecated and replaced by [OpenFn/cli](../cli.md). Learn more about
-CLI
+deprecated and replaced by [OpenFn/cli](../cli.md). Learn more about CLI
[github.com/OpenFn/cli/](https://github.com/OpenFn/kit/tree/main/packages/cli)
:::
@@ -22,9 +21,10 @@ The [Devtools](https://github.com/OpenFn/devtools) repo is a collection of bash
and Node scripts, as well as a _suggested_ (but not necessary) directory
structure for working with OpenFn jobs and adaptors.
-To run OpenFn jobs locally, you only need [Core](https://github.com/OpenFn/core) and at
-least one adaptor, e.g. [language-http](https://github.com/OpenFn/language-http)
-and you may prefer to install core globally via `npm install -g @openfn/core`
+To run OpenFn jobs locally, you only need [Core](https://github.com/OpenFn/core)
+and at least one adaptor, e.g.
+[language-http](https://github.com/OpenFn/language-http) and you may prefer to
+install core globally via `npm install -g @openfn/core`
:::
@@ -55,7 +55,7 @@ import CodeBlock from '@theme/CodeBlock';
- git clone https://github.com/OpenFn/devtools.git{'\n'}
+ git clone https:\/\/github.com/OpenFn/devtools.git{'\n'}
cd devtools{'\n'}
./install.sh https
@@ -91,7 +91,7 @@ Execute takes:
```mdx-code-block
import ReactPlayer from 'react-player';
-
+
```
### Install a specific adaptor version
@@ -323,7 +323,7 @@ import TabItem from '@theme/TabItem';
>
- ssh-add
+ ssh-add <path-to-your-ssh-file>
@@ -339,7 +339,7 @@ In VSCode, go to `Settings`, and in the search bar, type
`terminal.integrated.inherit`. You should see the option in the image below and
check it if it's unchecked.
-
+
### GitHub token sharing
diff --git a/versioned_docs/version-legacy/getting-started/commcare-project-walkthrough.md b/versioned_docs/version-legacy/getting-started/commcare-project-walkthrough.md
index 960d14aac5f4..d9d5042c9855 100644
--- a/versioned_docs/version-legacy/getting-started/commcare-project-walkthrough.md
+++ b/versioned_docs/version-legacy/getting-started/commcare-project-walkthrough.md
@@ -9,8 +9,8 @@ title:
minute!)
- You have checked out our glossary and have an understanding of basic OpenFn
and API terminology. Check out the pages below to get started
- - [OpenFn Concepts](../getting-started/terminology.md)
- - [A glossary for data integration](../getting-started/glossary.md)
+ - [OpenFn Concepts](/documentation/legacy/getting-started/terminology)
+ - [A glossary for data integration](/documentation/legacy/getting-started/glossary)
- You have a CommCare application with at least one form configured. This is
your source system.
- You have a PostgreSQL database configured. This is your destination system.
@@ -23,7 +23,7 @@ also follow along with the prebuilt solution. Follow along at the links below:**
- Username: testuser
- Password: 123
-
+
3. [OpenFn project](https://www.openfn.org/projects/commcare-demo/jobs)
4. [Public report that shows records in the PostgreSQL database](https://analytics.openfn.org/public/question/095449a9-5696-463c-a4fb-24614c9f08a5)
@@ -45,7 +45,7 @@ data on patients registered!
:::
-
+
**This integration can be broken up into two parts:**
@@ -70,7 +70,7 @@ inbox. After data forwarding is set up, it happens automatically, **_in
real-time for all forms and cases_**. Learn more about configuring a webhook
[here](/adaptors/commcare#webhook-forward-cases-andor-forms-from-commcare-to-openfn-using-rest-service).
-
+
### Option 2: Extracting Commcare data via the REST API
@@ -110,7 +110,7 @@ In the credential `JSON Configuration`, add your credential as follows:
:::tip
-Check out [this](../getting-started/terminology.md/#inbox) docs page on
+Check out [this](/documentation/legacy/getting-started/terminology#inbox) docs page on
how to find your OpenFn inbox URL to fill in the configuration above.
:::
@@ -171,15 +171,13 @@ get(
6. **Once you are finished configuring and writing your job, save and run it!**
-
-
7. **Check out the `Activity History` tab to see if your run succeeded.** If it
succeeded, you should see:
- Successful run log (look for the green!)
- New Messages in your `Inbox` containing data for any forms submitted in the
time frame specified in your query.
-
+
:::info
@@ -217,7 +215,7 @@ there are 2:**
for how to create your own `mapping specification document` to map data
elements to be exchanged.
-
+
1. **Create a new message filter trigger, to run our second job for every new
patient record received in the OpenFn inbox.** Learn more about message
@@ -273,8 +271,6 @@ to be exchanged.
1. **Save and turn on the job**
-
-
## Time to test!
1. Submit a form in CommCare
@@ -287,11 +283,11 @@ to be exchanged.
5. Check out the `Activity History` and ensure that both runs passed (look for
the green checks in the `Status/Action` column).
-
+
6. **Finally, refresh your database and check out the new submission data!**
-
+
While this guide is specifically for PostgreSQL databases, you can generally
follow these same steps for other database types (e.g., MS SQL or MySQL)—simply
diff --git a/versioned_docs/version-legacy/getting-started/integration-toolkit.md b/versioned_docs/version-legacy/getting-started/integration-toolkit.md
index eb6496d3e968..018c3f8531e0 100644
--- a/versioned_docs/version-legacy/getting-started/integration-toolkit.md
+++ b/versioned_docs/version-legacy/getting-started/integration-toolkit.md
@@ -15,7 +15,7 @@ in integration and interoperability projects. The Toolkit is both a recognized
[Digital Public Good](https://digitalpublicgoods.net/) ("DPG") and a
[Digital Square Global Good](https://digitalsquare.org/digital-health-global-goods).
-
+
The Toolkit provides a suite of software tools and documentation to help users
design, build, and automate integrations.
@@ -31,7 +31,7 @@ deployment pathway for the Integration Toolkit) and back again (see below) but
to really understand the toolkit you've got to first understand Open Function
Group and `platform`, the enterprise iPaaS.
-
+
Open Function Group has been building free and open source software (FOSS) for
data integration projects in the health, humanitarian, and international
@@ -101,7 +101,7 @@ You can read all about it [here](../about-lightning.md)!
## Architecture for implementation
-
+
## Open Source Steering Committee (OSSC)
diff --git a/versioned_docs/version-legacy/intro/home.md b/versioned_docs/version-legacy/intro/home.md
index 051a39af11a5..59794bc9e14e 100644
--- a/versioned_docs/version-legacy/intro/home.md
+++ b/versioned_docs/version-legacy/intro/home.md
@@ -47,7 +47,7 @@ users the freedom to switch between any and all of the OpenFn products.
All OpenFn products, other than the OpenFn iPaaS are part of the free and
open-source
-[**OpenFn Integration Toolkit**](../getting-started/integration-toolkit.md).
+[**OpenFn Integration Toolkit**](/documentation/legacy/getting-started/integration-toolkit).
This Toolkit is a **Digital Public Good** (a "DPG") recognized in the
[DPG Registry](https://digitalpublicgoods.net/registry/) and Digital Square's
[Global Goods Guidebook](https://digitalsquare.org/resourcesrepository/global-goods-guidebook).
diff --git a/versioned_docs/version-legacy/jobs/editing_locally.md b/versioned_docs/version-legacy/jobs/editing_locally.md
index cd7faeeb33af..19010097452d 100644
--- a/versioned_docs/version-legacy/jobs/editing_locally.md
+++ b/versioned_docs/version-legacy/jobs/editing_locally.md
@@ -6,7 +6,7 @@ To edit jobs, instead of using the OpenFn
[Job Studio](./job-studio.md) you can also use your favorite text
editor and make changes offline, committing and pushing to GitHub to deploy to
your OpenFn using the
-[version control feature](../manage/platform-mgmt.md/#github-version-control).
+[version control feature](/documentation/legacy/manage/platform-mgmt#github-version-control).
First, make sure that version control is set up for your project and the job in
question. When that's all done, follow the steps below:
@@ -17,7 +17,7 @@ question. When that's all done, follow the steps below:
2. Clone the repo from GitHub. Depending on how you're connecting, grab the
HTTPS or SSH URL of the repository.
-
+
:::tip
@@ -43,14 +43,14 @@ for more info.)
6. To edit your jobs, use a code editor such as
[Visual Studio Code](https://code.visualstudio.com/download).
-
+
7. Make sure you install the
[Prettier VSCode Extension](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode)
and set is as default formatter in Settings as seen below. This will apply
the correct code formatting to the files you change.
-
+
8. Once you're done, you can check which files you changed with `git status`.
diff --git a/versioned_docs/version-legacy/jobs/job-design-intro.md b/versioned_docs/version-legacy/jobs/job-design-intro.md
index 6825cd109916..bb5d6b6207e0 100644
--- a/versioned_docs/version-legacy/jobs/job-design-intro.md
+++ b/versioned_docs/version-legacy/jobs/job-design-intro.md
@@ -24,7 +24,7 @@ Step 2: Map your data elements
1. Export the metadata of your form (input) & destination DB (output).
2. Paste the metadata into an Excel spreadsheet to create a mapping sheet:
-
+
3. Map data elements & define rules for data cleaning and transformation a. How
should the data collected be translated into your destination system’s data
diff --git a/versioned_docs/version-legacy/jobs/multiple-operations.md b/versioned_docs/version-legacy/jobs/multiple-operations.md
index 054caa53bab1..90dafe92147d 100644
--- a/versioned_docs/version-legacy/jobs/multiple-operations.md
+++ b/versioned_docs/version-legacy/jobs/multiple-operations.md
@@ -26,7 +26,7 @@ You can do many things in sequence with OpenFn, whether using `core`,
- You update a `cursor` in a series of operations that involve `GET` and `POST`.
When the `POST` fails, you don't want to update the `cursor` for the
subsequent job run which contains the `GET`.
-- Your operations don't take too long (<100s in total for `platform`) and you
+- Your operations don't take too long (<100s in total for `platform`) and you
want to reduce the number of executions.
### Reasons to post back to the inbox
diff --git a/versioned_docs/version-legacy/jobs/working_with_branches.md b/versioned_docs/version-legacy/jobs/working_with_branches.md
index c2df368f1ed2..87885635f64d 100644
--- a/versioned_docs/version-legacy/jobs/working_with_branches.md
+++ b/versioned_docs/version-legacy/jobs/working_with_branches.md
@@ -42,14 +42,14 @@ repo to your local folder.
6. On GitHub, you can create a Pull Request to get your changes reviewed and
approved.
- 
+ 
- 
+ 
7. As you keep working with branches, make sure you check which branch you're on
with `git status`.
-
+
8. To keep your local copy up to date with the remote repo, switch to `main`
with `git checkout main` and hit `git pull` to pull any changes.
diff --git a/versioned_docs/version-legacy/manage/platform-mgmt.md b/versioned_docs/version-legacy/manage/platform-mgmt.md
index 491565ed8c06..f6bca49377b0 100644
--- a/versioned_docs/version-legacy/manage/platform-mgmt.md
+++ b/versioned_docs/version-legacy/manage/platform-mgmt.md
@@ -198,14 +198,14 @@ The **four configuration steps** include giving the job a name, defining what
[authentication](../build/credentials.md) details.
The **expression editor** is the area where you write your
-[job expression](../build/jobs.md/#composing-job-expressions). Fill-in
+[job expression](../build/jobs.md#composing-job-expressions). Fill-in
all the details, and click on the **Save** icon in the top-right corner to save
your job's configuration changes.
#### Inspecting job's initial state
This feature allows you to view the
-[initial state](../jobs/state.md/#initial-state) of a selected job.
+[initial state](/documentation/legacy/jobs/state#initial-state) of a selected job.
Note that this feature is currently only available for
[message-triggered jobs](../build/triggers.md#message-filter-triggers).
@@ -518,13 +518,13 @@ runs fail due to an error message.
for reprocessing and gives you an approximate number of runs that will be
reprocessed.
-
+
3. Click "Reprocess" when you're happy with the query. You'll get feedback on
the number of runs enqueued within seconds, and you should see your project
queue fill up then empty over time as the batch is processed.
-
+
:::info
@@ -554,12 +554,12 @@ You can download and review OpenFn runs data by exporting to a CSV file.
2. Click the **Export as CSV** button to review and confirm the desired export.
-
+
3. Click the "Export" button to submit the request. A link to download the file
will be sent to your email address shortly.
-
+
## Inbox
@@ -588,7 +588,7 @@ by:
custom date range yourself. Note that the default inbox view shows “Last 30
Days”.
-
+
### Bulk reprocess messages
@@ -604,13 +604,13 @@ destination application, then this feature will help you do so more quickly!
for reprocessing and gives you an approximate number of messages that will be
reprocessed.
-
+
3. Click "Reprocess" when you're happy with the query. You'll get feedback on
the number of messages enqueued within seconds, and you should see your
project queue fill up then empty over time as the batch is processed.
-
+
#### Note when bulk reprocessing messages
@@ -622,7 +622,7 @@ destination application, then this feature will help you do so more quickly!
- Remember that OpenFn plans are run-based, and you can monitor usage in
**Project Settings** to ensure that you don’t hit any run limits when bulk
- reprocessing! 
+ reprocessing! 
### Export messages to CSV
@@ -633,12 +633,12 @@ You can now download and review OpenFn message data by exporting to a CSV file.
2. Click the **Export as CSV** button to review and confirm the desired export.
-
+
3. Click the "Export" button to submit the request. A link to download the file
will be sent to your email address shortly.
-
+
## Search Console
@@ -728,8 +728,6 @@ steps to reset it:
5. Enter your recovery token and a new password into the OpenFn "Reset Password"
page.
-
-
## Project settings
This section of OpenFn platform allows you to view and update the project
@@ -915,7 +913,7 @@ To explore these features, on the left hand navigation ribbon click on the
_Please refer to the screenshot below for help navigating the functionality of
this page._
-
+
### User Access
@@ -958,7 +956,7 @@ and Basic Auth types, which will prompt administrators to either generate an API
token or to setup a username:password credential. Once this inbox authentication
is configured, any HTTP requests made to the OpenFn Inbox URL must include
either this `x-api-key` token or username:password in the request header.
-
+
#### Rotating auth methods
diff --git a/versioned_docs/version-legacy/manage/troubleshooting-tips-on-platform.md b/versioned_docs/version-legacy/manage/troubleshooting-tips-on-platform.md
index 635c7309bbd7..bec58f27b96d 100644
--- a/versioned_docs/version-legacy/manage/troubleshooting-tips-on-platform.md
+++ b/versioned_docs/version-legacy/manage/troubleshooting-tips-on-platform.md
@@ -12,7 +12,7 @@ Currently, this section is specific to **OpenFn/platform**.
## Runs
One of the most helpful pages for troubleshooting on OpenFn platform is the
-[Activity History](../getting-started/terminology.md#activity-history).
+[Activity History](/documentation/legacy/getting-started/terminology#activity-history).
This pages provides a list of all of the runs executed in a project and always
marks any failed runs red or yellow. Project administrators can troubleshoot
errors by clicking into the run to review the run details. Keep reading for all
@@ -95,15 +95,15 @@ Leverage the various search functionalities in OpenFn to find the right messages
and runs to support your troubleshooting. You can search in the Inbox, Activity
History, and Search Console.
-1. **[Inbox](../manage/platform-mgmt.md#inbox)** - The inbox contains
+1. **[Inbox](/documentation/legacy/manage/platform-mgmt#inbox)** - The inbox contains
all the messages that have been sent to your project. Search your project
inbox for messages that contain a specific body text. You can also filter
these messages by date, run status and trigger. You can even use filtering to
see all the messages whose last run failed, so you can get to
troubleshooting! Learn more about inbox filtering
- [here](../manage/platform-mgmt.md#inbox).
+ [here](/documentation/legacy/manage/platform-mgmt#inbox).
-2. **[Activity history](../getting-started/terminology.md#activity-history)** -
+2. **[Activity history](/documentation/legacy/getting-started/terminology#activity-history)** -
As discussed above, the activity history records all runs for the current
project. It has similar search and filtering capabilities as the inbox except
it doesn't require valid JSON in the search box. Use the search in activity
diff --git a/versioned_docs/version-legacy/portability-versions.md b/versioned_docs/version-legacy/portability-versions.md
index bc5192cfb980..dcfd586b5c27 100644
--- a/versioned_docs/version-legacy/portability-versions.md
+++ b/versioned_docs/version-legacy/portability-versions.md
@@ -10,7 +10,7 @@ be used to import or export projects between OpenFn/platform and OpenFn/engine.
The portability specification v4 defines how entire projects (groups of
workflows with their associated triggers, credentials and jobs) can be
represented as code. This specification has been written for
-[Lightning](./getting-started/integration-toolkit.md/#lightning-coming-soon),
+[Lightning](/documentation/legacy/getting-started/integration-toolkit#lightning-coming-soon),
the fully open source webb app which extends the OpenFn DPG. It aims to (a)
improve developer experience, allowing them to build and test workflows locally;
(b) enable version control and an audit trail of project changes; and (c) enable
diff --git a/versioned_docs/version-legacy/source-apps.md b/versioned_docs/version-legacy/source-apps.md
index 5895f4afd0a4..1d94ba6434ec 100644
--- a/versioned_docs/version-legacy/source-apps.md
+++ b/versioned_docs/version-legacy/source-apps.md
@@ -38,7 +38,7 @@ probably want a feature in your user-interface that allows them to turn on and
off these various services. See the below example from Kobo Toolbox (left) and
CommCare (right).
-
+
## When to send?
diff --git a/versioned_docs/version-legacy/standards/openhie.md b/versioned_docs/version-legacy/standards/openhie.md
index 92e6926f12ed..f81c912eee38 100644
--- a/versioned_docs/version-legacy/standards/openhie.md
+++ b/versioned_docs/version-legacy/standards/openhie.md
@@ -33,7 +33,7 @@ used as a fully OpenHIE-compliant **_interoperability layer _**because it does
not leverage the IHE ATNA profile (see
[requirement IOL-WF1](https://guides.ohie.org/arch-spec/openhie-component-specifications-1/openhie-interoperability-layer-iol#openhie-iol-workflow-requirements)).
-
+
_For an overview of OpenFn Lightning and how it fits into OpenHIE, see our
[introduction for the OpenHIE showcase](https://www.youtube.com/watch?v=PTRRZBYtqyc)_
@@ -78,4 +78,4 @@ leveraging OpenFn’s out-of-box DHIS2 adaptor and reusable workflow templates t
quickly develop automation that reformats data received from CommCare and maps
it to the DHIS2 data model.
-
+
diff --git a/versioned_docs/version-legacy/style-guide.md b/versioned_docs/version-legacy/style-guide.md
deleted file mode 100644
index cffd4830b3d8..000000000000
--- a/versioned_docs/version-legacy/style-guide.md
+++ /dev/null
@@ -1,297 +0,0 @@
----
-id: style-guide
-title: Style Guide
-sidebar_label: Style Guide
-slug: /style-guide
----
-
-You can write content using
-[GitHub-flavored Markdown syntax](https://github.github.com/gfm/).
-
-:::tip
-
-We use a `.prettierrc` file to enforce standard styles via the "Prettier" code
-formatter. If you are using VsCode, you can install prettier via
-https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode
-
-Make sure to format you work before opening a PR.
-
-:::
-
-## Markdown Syntax
-
-To serve as an example page when styling markdown based Docusaurus sites.
-
-## Headers
-
-# H1 - Create the best documentation
-
-## H2 - Create the best documentation
-
-### H3 - Create the best documentation
-
-#### H4 - Create the best documentation
-
-##### H5 - Create the best documentation
-
-###### H6 - Create the best documentation
-
----
-
-## Emphasis
-
-Emphasis, aka italics, with _asterisks_ or _underscores_.
-
-Strong emphasis, aka bold, with **asterisks** or **underscores**.
-
-Combined emphasis with **asterisks and _underscores_**.
-
-Strikethrough uses two tildes. ~~Scratch this.~~
-
----
-
-## Lists
-
-1. First ordered list item
-1. Another item
- - Unordered sub-list.
-1. Actual numbers don't matter, just that it's a number
- 1. Ordered sub-list
-1. And another item.
-
-- Unordered list can use asterisks
-
-* Or minuses
-
-- Or pluses
-
----
-
-## Links
-
-[I'm an inline-style link](https://www.google.com/)
-
-[I'm an inline-style link with title](https://www.google.com/ "Google's Homepage")
-
-[I'm a reference-style link][arbitrary case-insensitive reference text]
-
-[You can use numbers for reference-style link definitions][1]
-
-Or leave it empty and use the [link text itself].
-
-URLs and URLs in angle brackets will automatically get turned into links.
-http://www.example.com/ or and sometimes example.com
-(but not on GitHub, for example).
-
-Some text to show that the reference links can follow later.
-
-[arbitrary case-insensitive reference text]: https://www.mozilla.org/
-[1]: http://slashdot.org/
-[link text itself]: http://www.reddit.com/
-
----
-
-## Images
-
-Here's our logo (hover to see the title text):
-
-Inline-style:
-
-
-Reference-style: ![alt text][logo]
-
-
-[logo]: https://github.com/adam-p/markdown-here/raw/master/src/common/images/icon48.png
- 'Logo Title Text 2'
-
-
-Images from any folder can be used by providing path to file. Path should be
-relative to markdown file.
-
-
-
-### Image sizing/styling
-
-Images can be sized using inline HTML.
-
-
-
----
-
-## Gifs
-
-Gifs are helpful for demonstrating short sequences of user behaviour.
-
-
-
-There are many tools that will help you create GIFs:
-
-- [Peek](https://github.com/phw/peek)
-- [Capture to a Gif](https://chrome.google.com/webstore/detail/capture-to-a-gif/eapecadlmfblmnfnojebefkbginhggeh)
-- [Chrome Capture](https://chrome.google.com/webstore/detail/chrome-capture-screenshot/ggaabchcecdbomdcnbahdfddfikjmphe)
-
-:::note
-
-Please note that if you're using an animated "cursor dot" and "show/click
-animation", the hex code we use is **#B53F48**.
-
-:::
-
----
-
-## Code
-
-```javascript
-var s = 'JavaScript syntax highlighting';
-alert(s);
-```
-
-```python
-s = "Python syntax highlighting"
-print(s)
-```
-
-```
-No language indicated, so no syntax highlighting.
-But let's throw in a tag.
-```
-
-```js {2}
-function highlightMe() {
- console.log('This line can be highlighted!');
-}
-```
-
----
-
-## Tables
-
-Colons can be used to align columns.
-
-| Tables | Are | Cool |
-| ------------- | :-----------: | -----: |
-| col 3 is | right-aligned | \$1600 |
-| col 2 is | centered | \$12 |
-| zebra stripes | are neat | \$1 |
-
-There must be at least 3 dashes separating each header cell. The outer pipes (|)
-are optional, and you don't need to make the raw Markdown line up prettily. You
-can also use inline Markdown.
-
-| Markdown | Less | Pretty |
-| -------- | --------- | ---------- |
-| _Still_ | `renders` | **nicely** |
-| 1 | 2 | 3 |
-
----
-
-## Blockquotes
-
-> Blockquotes are very handy in email to emulate reply text. This line is part
-> of the same quote.
-
-Quote break.
-
-> This is a very long line that will still be quoted properly when it wraps. Oh
-> boy let's keep writing to make sure this is long enough to actually wrap for
-> everyone. Oh, you can _put_ **Markdown** into a blockquote.
-
----
-
-## Inline HTML
-
-