fix(data-imports): don't fail imports on a failed row count refresh

[posthog]

Problem

Warehouse imports were failing after the data had already landed, and the error told customers to check their bucket credentials.

Two things go wrong on the same path. After a sync writes its data, _validate_and_update refreshes the row count for cumulative schemas by calling DataWarehouseTable.get_count(). That count runs chdb first (30s hard timeout), then falls back to the ClickHouse cluster over the Delta parquet glob with no explicit structure, so ClickHouse infers the schema across files. When chunks disagree on a column's physical type, a plain SELECT count() fails with Cannot convert string '...' to type Date.

1. The exception propagates and fails the whole import job, even though the sync itself succeeded.
2. _safe_expose_ch_error finds no matching entry in ExtractErrors, so it falls through to the catch-all: "check that the files URL pattern, file format, and credentials are correct". Credentials are the one thing that isn't wrong.

Both legs of the fallback are failing in the wild - some occurrences hit the chdb timeout, others the ClickHouse type mismatch - across several teams over the last month.

Changes

• The cumulative row-count refresh is now non-fatal. If the count query fails, the previous row_count is kept and a warning is logged instead of sinking the job. Applied to both validate_schema_and_update_table and register_cdc_companion_table, which had the same shape.
• Added a type-mismatch entry to ExtractErrors so the surfaced message names the real cause (files disagreeing on a column's type) and suggests a resync.

Note

Passing an explicit structure to build_function_call from get_count() would fix the inference fragility at the root. That's a bigger change and worth weighing separately, so it isn't here.

How did you test this code?

Automated tests only, all run locally and passing:

• products/warehouse_sources/backend/tests/test_table.py - 12 passed, including the new case asserting the real production message maps to the type-mismatch copy instead of the credentials catch-all. Regression it catches: dropping the ExtractErrors entry silently sends users back to checking credentials that were never the problem.
• products/warehouse_sources/backend/temporal/data_imports/pipelines/test_pipeline_sync.py - 16 passed, including the new TestValidateSchemaAndUpdateTable, which covers the failing count (job completes, previous row_count kept, new queryable_folder still saved) and the happy path (count refreshed from the table, not taken from the run's delta). Regressions they catch: a count query raising again taking down the whole activity, or the new guard swallowing a successful refresh.

No manual testing - I didn't reproduce a real S3 sync. Note that the Django test job is skipped on draft PRs, so these hadn't run in CI at the time of writing.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

No user-facing docs cover this error string.

🤖 Agent context

Autonomy: Fully autonomous

Written by Claude (Claude Code) from a PostHog inbox report. Skills invoked: /writing-user-facing-copy for the new error message, /writing-tests for the test decisions.

A couple of decisions worth flagging for review:

• The ExtractErrors key is the substring Cannot convert string. It's broad on its own, but _safe_expose_ch_error is only reached from the S3 table-function paths (get_count, get_columns, column validation), where that message means the files disagree on a type. Happy to make it narrower if you'd rather anchor on While executing ReadFromObjectStorage.
• The count is called with safe_expose_ch_error=False inside the guard, so the raw ClickHouse error lands in the log rather than the friendly-but-now-irrelevant rewrite.
• I did not add a test through the existing _register_companion_sync helper in test_pipeline_sync.py - it's a hand-copied duplicate of the production function, so a test against it would prove nothing about the real code path. The new tests call the real validate_schema_and_update_table.

---

Created with PostHog Desktop from this inbox report.

[trunk-io]

Merging to master in this repository is managed by Trunk.

• To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

[scheduled-actions-posthog]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.