[QA-G11] OpenSearch Migration — Phase 3 End-to-End Content Validation

[fabrizzio-dotCMS]

Parent Epic

#35476 — QA EPIC: OpenSearch Migration

Unblocked by

PR #35609 — Vendor-neutral SearchAPI and phase-aware router

---

What are we testing?

In Phase 3, dotCMS writes and reads content exclusively through OpenSearch. Elasticsearch is fully decommissioned — no writes go there anymore.

The goal of this issue is to verify that a human tester, using the regular dotCMS admin tools, can:

1. Create, edit, and publish content normally.
2. Find that content through the standard search and query tools.
3. Confirm — via OpenSearch Dashboards — that the document was written only in OpenSearch, not in Elasticsearch.

⚠️ Use an isolated QA environment only. Phase 3 is not validated for production scale. Do not run these tests on any customer-facing instance.

---

Prerequisites

1. Start the migration stack

docker compose -f docker/docker-compose-examples/os-migration/docker-compose.yml up -d

Wait until both clusters are healthy:

curl -s http://localhost:9200/_cluster/health | jq .status   # should be "green" or "yellow"
curl -s http://localhost:9201/_cluster/health | jq .status   # should be "green" or "yellow"

2. Configure dotCMS for Phase 3

Add these lines to dotmarketing-config.properties and restart dotCMS:

FEATURE_FLAG_OPEN_SEARCH_PHASE=3
OS_ENDPOINTS=http://localhost:9201
OS_AUTH_BASIC_USER=admin
OS_AUTH_BASIC_PASSWORD=admin
OS_TLS_ENABLED=false

3. Confirm Phase 3 is active

After dotCMS starts, check the startup log for this line:

Migration Phase: PHASE_3_OPENSEARCH_ONLY

---

How to confirm a document was indexed in OpenSearch (not Elasticsearch)

After any content action below, open OpenSearch Dashboards (http://localhost:5602) and run this query in Dev Tools:

GET /cluster_<id>.working_<timestamp>/_search
{
  "query": { "term": { "identifier": "<your-content-identifier>" } }
}

Then open Kibana (http://localhost:5601) and run the same query against the ES index. The expected result is: found in OS, not found in ES.

---

Test Scenarios

F-3 — Content Search portlet finds freshly published content (data from OS)

Goal: Verify the Content Search portlet returns results that come from OpenSearch in Phase 3.

Step	Action	What to check
1	Log in to dotCMS Admin. Go to Content > Content Search.	Portlet loads without error.
2	Create a new piece of content (e.g. a Blog post or Generic Content). Give it a unique title like Phase3-Test-<date>. Save and Publish it.	Content saved and published successfully. No error message.
3	In the Content Search portlet, search for the title you just used.	Your content item appears in the results list.
4	In OpenSearch Dashboards, search for the identifier of that content item.	Document found in the OS working index.
5	In Kibana, search for the same identifier.	Document not found in the ES index (ES is decommissioned in Phase 3).

---

F-4 — Site Browser displays content indexed in OS

Goal: Verify the Site Browser loads and displays pages and content using the OS index.

Step	Action	What to check
1	Go to Content > Site Browser. Navigate to any folder that contains pages or content.	Folder tree and file list loads without error.
2	Open a page or content item from the browser.	Page/content opens and displays correctly. No missing content or broken references.
3	Create a new HTML page inside a folder. Save and Publish it.	Page saved and published successfully.
4	Verify the page appears in the Site Browser immediately after publishing.	Page is visible in the folder listing.
5	In OpenSearch Dashboards, search for the page's identifier.	Document found in the OS live and working indices.
6	In Kibana, search for the same identifier.	Document not found in ES.

---

F-5 — Query Tool returns results sourced from OpenSearch

Goal: Verify the Query Tool executes searches against OS and returns the expected results.

Step	Action	What to check
1	Go to Admin > System > Query Tool (or navigate to /dotAdmin/#/query-tool).	Query Tool loads without error.
2	Run a basic Lucene query such as +basetype:5 (Pages) or +contenttype:Blog.	Results list loads. At least one result is returned.
3	Note the identifier of one result. Open OpenSearch Dashboards and search for that identifier.	Document found in the OS index.
4	Optionally toggle between Live and Working in the Query Tool.	Both modes return results without error.
5	Open Kibana and run the same query.	Result counts in Kibana are equal to or less than OS (ES index may be stale or empty after Phase 3 activation). Document writes after Phase 3 activation should appear only in OS.

---

F-6 — GraphQL queries return content from OS index

Goal: Verify the GraphQL API serves content sourced from OpenSearch.

Step	Action	What to check
1	Open the GraphQL playground at http://localhost:8082/api/v1/graphql.	Playground loads.
2	Run a basic query to list content, for example:

{
  search(query: "+basetype:5", limit: 5) {
    contentlets {
      identifier
      title
    }
  }
}

| | Expected: JSON response with at least one contentlet. No error or empty result. |
| 3 | Note an identifier from the response. Open OpenSearch Dashboards and search for it. | Document found in the OS index. |
| 4 | Publish a new piece of content. Wait a few seconds. Re-run the GraphQL query. | Newly published content appears in the GraphQL response. |
| 5 | In Kibana, confirm the newly created content is not in the ES index. | Document absent from ES — confirming write goes to OS only. |

---

F-7 — Velocity templates render content from OS index

Goal: Verify that $ESContent.search() and $dotcontent in Velocity templates return content sourced from OS.

Step	Action	What to check
1	Find or create a page that uses a Velocity template containing $ESContent.search(query) or $dotcontent.find(identifier).	Page exists and is accessible.
2	Request the page in a browser or via GET http://localhost:8082/your-page-url.	Page renders without error. No stack trace. Content appears in the rendered HTML.
3	Take note of one content item returned in the template. Look it up in OpenSearch Dashboards.	Document found in OS index.
4	Edit one of the content items used in that template. Change a visible field (e.g. title). Save and Publish.	Content saved and published.
5	Reload the page.	Updated content appears in the rendered page.
6	In OpenSearch Dashboards, search for the updated document.	Updated field value visible in the OS document.
7	In Kibana, check the same document.	ES document is stale or absent — no write happened to ES in Phase 3.

---

F-8 — OS unavailable: search tools show a clear error (no silent failure)

Goal: Confirm that when OpenSearch is down in Phase 3, the user sees a clear error message — not a blank result or a silent fallback to Elasticsearch.

Step	Action	What to check
1	Stop OpenSearch: docker compose stop opensearch.	OS is now unavailable. ES is still running.
2	In Content Search, run any query.	An error is displayed to the user (toast message, error panel, or HTTP 500). No empty result set pretending everything is fine.
3	In the Query Tool, run a query.	Same visible error. No results from ES appearing as if from OS.
4	Check dotCMS logs for the error.	Log shows the OS connection failure. No log line showing a fallback read from ES.
5	Restart OpenSearch: docker compose start opensearch. Wait ~10 seconds. Re-run the queries.	Results return normally.

---

Acceptance Criteria

• F-3: Content published in Phase 3 appears in Content Search portlet; found in OS index, absent in ES
• F-4: Site Browser displays published content; document confirmed in OS index only
• F-5: Query Tool returns results from OS; new content written after Phase 3 activation is absent from ES
• F-6: GraphQL returns content from OS; newly published content appears in GraphQL and is absent from ES
• F-7: Velocity template renders content from OS; edited content reflects in page and in OS index; ES document is stale or absent
• F-8: When OS is down, all search tools show a clear error — no silent empty result and no fallback to ES