MSSQL connector does not report empty streams in streamStats (unlike PostgreSQL)

[SivaramaKrishnan (sivaramakrishnan-stratforge)]

Environment

• Airbyte Platform: 0.58.0
• Deployment: OSS (Docker, self-hosted)
• Source (MSSQL): airbyte/source-mssql:4.0.15
• Source (PostgreSQL for comparison): airbyte/source-postgres:3.8.1
• Destination: airbyte/destination-clickhouse:1.0.0
• Sync Mode: Full Refresh | Overwrite

Observed Behavior

I'm seeing inconsistent behavior between the PostgreSQL and MSSQL source connectors when syncing empty tables.

PostgreSQL (source-postgres:3.8.1)

If a table contains 0 rows, the stream is still included in the job's streamStats with explicit zero values.

Example:

{
  "streamName": "sync_2",
  "stats": {
    "recordsEmitted": 0,
    "bytesEmitted": 0,
    "recordsCommitted": 0
  }
}

The logs also show that the connector inspects the table, determines that it is empty, and still emits the appropriate checkpoint/state messages.

MSSQL (source-mssql:4.0.15)

For the same scenario:

• Sync Mode: Full Refresh | Overwrite
• Empty table (0 rows)
• Sync completes successfully without any errors

The empty stream is completely absent from streamStats.

There is:

• no error,
• no failed stream,
• no zero-value statistics,
• and no indication that the stream was processed.

Only the non-empty tables appear in the job response.

Reproduction

Database contains three tables:

• table_1 → ~50k rows
• table_2 → ~50k rows
• table_3 → 0 rows

Run a Full Refresh | Overwrite sync to ClickHouse.

Result

PostgreSQL source

streamStats contains all three streams, including the empty one with zero values.

MSSQL source

streamStats only contains the two non-empty streams. The empty table is omitted entirely.

Questions

1. Is omitting empty streams from streamStats the expected behavior for the MSSQL connector?

2. Is this because source-mssql still relies on the older JDBC/AbstractJdbcSource implementation, while newer connectors such as PostgreSQL have migrated to the newer CDK architecture that reports statistics for every processed stream, including empty ones?

3. Is there an existing GitHub issue or roadmap item for migrating the MSSQL connector to the newer CDK implementation, or otherwise making streamStats behavior consistent across connectors?

Why this matters

I'm building sync-completion notifications that report the outcome of every synced table.

With the PostgreSQL connector, I can reliably identify tables that were processed but contained no data by checking for a streamStats entry with zero values.

With the MSSQL connector, there is no equivalent signal because empty streams are omitted entirely. This makes it impossible to distinguish between:

• a table that was successfully processed but contained no records, and
• a table that never appeared in the job statistics.

Is there a recommended way to detect successfully processed empty streams for the MSSQL connector, or is this currently a known limitation?

[Airbyte Support Bot (airbyte-support-bot)]

Hi SivaramaKrishnan (@sivaramakrishnan-stratforge), thank you for the detailed report — the side-by-side comparison between the Postgres and MSSQL sources, with versions and a clear reproduction, is very helpful.

We're looking into the difference in streamStats reporting for zero-row streams and have escalated it to the team that owns the database sources for investigation: airbytehq/oncall#13252. We can't confirm at this stage whether the current MSSQL behavior is intended or what the recommended detection mechanism is — the team will assess that.

A couple of things that would help the investigation, if you're able to add them:

• Whether you see the same omission on an Incremental sync of an empty table, or only on Full Refresh | Overwrite.
• The relevant excerpt of the source connector logs for the empty stream from an MSSQL sync — specifically any stream status or state lines mentioning that stream. Please mask or remove any sensitive information (API keys, passwords, tokens, connection strings, hostnames, or table/column data you'd rather not share) before pasting, and paste the excerpt inline in a fenced code block rather than linking to it.
• Whether the empty table is still created/emptied correctly in ClickHouse despite being missing from streamStats.

Interim workaround for your notification use case

Until the reporting behavior is settled, one approach is to treat the configured catalog as the source of truth rather than streamStats: take the set of enabled streams for the connection (available from the connection's configured catalog via the API), and treat any enabled stream that has no streamStats entry in a successful job as "processed with zero records." That inference is only safe for jobs whose overall status is succeeded, since a failed or cancelled job can omit streams for unrelated reasons.

This is a workaround for the asymmetry you've described, not a recommendation about the intended behavior.

On your third question, we didn't find an existing public issue tracking streamStats consistency for empty streams across database sources, which is why we've opened the internal item above.

---

Need more help? Join Airbyte Community Slack for peer support, or if you're a Cloud customer, open a support ticket referencing this URL.

[SivaramaKrishnan (sivaramakrishnan-stratforge)]

Hi, thanks for the detailed response and for escalating this to the database sources team.

For the additional points:

• Incremental | Append: We do see the empty stream in streamStats when the source table has zero rows. The stream is also processed correctly in the destination.
• Full Refresh | Overwrite: This is where the issue occurs. For an empty MSSQL table, the stream is missing from streamStats from aribyte api, even though the destination table is correctly emptied/replaced as expected.
• Destination behavior: The ClickHouse destination is updated correctly in both cases. For Full Refresh | Overwrite, the destination table is emptied as expected despite the stream being absent from streamStats.

So, based on our testing, the issue appears to be specific to MSSQL + Full Refresh | Overwrite for empty source tables, rather than empty streams in general.

[SivaramaKrishnan (sivaramakrishnan-stratforge)]

Hi Airbyte Support Bot (@airbyte-support-bot) any update on this?