Skip to content

fix: require stream_count >= 1 on DynamicStreamCheckConfig#992

Merged
Anatolii Yatsuk (tolik0) merged 5 commits intomainfrom
devin/1776453297-fix-stream-count-zero-check-all
Apr 21, 2026
Merged

fix: require stream_count >= 1 on DynamicStreamCheckConfig#992
Anatolii Yatsuk (tolik0) merged 5 commits intomainfrom
devin/1776453297-fix-stream-count-zero-check-all

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot commented Apr 17, 2026

Summary

Clarifies and tightens the semantics of stream_count on DynamicStreamCheckConfig used by CheckStream for dynamic streams:

  • stream_count unset (default) — all generated streams are checked.
  • stream_count >= 1 — the first N generated streams are checked (capped at the number of available streams).
  • stream_count < 1 — rejected at validation time via minimum: 1 / ge=1.

Previously stream_count defaulted to 0 and the runtime treated <= 0 as "check all", which conflated "unset" with "zero" and accepted nonsensical negative values. Now stream_count is optional; when set, it must be a positive integer.

Changes

  • airbyte_cdk/sources/declarative/declarative_component_schema.yaml: drop default: 0, add minimum: 1, update description.
  • airbyte_cdk/sources/declarative/models/declarative_component_schema.py: stream_count: Optional[int] = None, ge=1.
  • airbyte_cdk/sources/declarative/checks/check_stream.py:
    • DynamicStreamCheckConfig.stream_count is now Optional[int] = None.
    • _check_generated_streams_availability checks all streams when max_count is None, otherwise slices to max_count.
  • airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py: pass stream_count through as-is so None is preserved.

Review & Testing Checklist for Human

  • Confirm the new semantics (None = check all, N >= 1 = check first N, < 1 rejected) match the intended behavior for DynamicStreamCheckConfig.stream_count.
  • Verify this is not a breaking change for existing manifests — any manifest that explicitly set stream_count: 0 or a negative value will now fail validation. Those values were accepted before even though they had no sensible meaning.
  • Spot-check a manifest that relies on the default behavior to confirm all generated streams are still being checked during check.

Test plan

Run the declarative checks unit tests locally:

poetry run pytest unit_tests/sources/declarative/checks/test_check_stream.py -v

All tests should pass, including the new test_stream_count_unset_checks_all_streams, test_stream_count_unset_failed, and test_check_stream_non_positive_stream_count[zero|negative] cases.

Notes

  • DynamicStreamCheckConfig.stream_count becoming Optional[int] with default None is technically a type-level breaking change for any external Python caller constructing the dataclass with a positional int. In practice the component is built via the factory from the declarative manifest, so this should be safe.
  • Three failing tests seen locally under test_concurrent_declarative_source.py (test_read_with_concurrent_and_synchronous_streams_with_concurrent_state, test_read_with_concurrent_and_synchronous_streams_with_sequential_state, test_read_concurrent_skip_streams_not_in_catalog) also fail on main without these changes and are unrelated to stream_count.

Link to Devin session: https://app.devin.ai/sessions/ec09e8ac822145cdb110cd25dce60013

…kConfig

Previously, DynamicStreamCheckConfig.stream_count defaulted to 0, which
caused _check_generated_streams_availability to slice generated_streams[:0],
resulting in zero streams being checked. This meant connectors using
DynamicStreamCheckConfig without explicitly setting stream_count would
silently skip all dynamic stream availability checks.

Now, stream_count <= 0 means 'check all generated streams', which is the
expected behavior when no limit is specified.

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1776453297-fix-stream-count-zero-check-all#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1776453297-fix-stream-count-zero-check-all

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 17, 2026

PyTest Results (Fast)

4 022 tests  +4   4 011 ✅ +4   7m 15s ⏱️ -25s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 144386e. ± Comparison against base commit 1256a1f.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 17, 2026

PyTest Results (Full)

4 025 tests  +4   4 013 ✅ +4   11m 33s ⏱️ ±0s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 144386e. ± Comparison against base commit 1256a1f.

♻️ This comment has been updated with latest results.

Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
@tolik0
Copy link
Copy Markdown
Contributor

Anatolii Yatsuk (tolik0) commented Apr 21, 2026

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/24719746821

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

Devin is currently unreachable - the session may have died.

Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
@devin-ai-integration devin-ai-integration Bot changed the title fix: treat stream_count=0 as 'check all streams' in DynamicStreamCheckConfig fix: clarify stream_count semantics on DynamicStreamCheckConfig Apr 21, 2026
Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
@devin-ai-integration devin-ai-integration Bot changed the title fix: clarify stream_count semantics on DynamicStreamCheckConfig fix: require stream_count >= 1 on DynamicStreamCheckConfig Apr 21, 2026
Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
@tolik0
Copy link
Copy Markdown
Contributor

Anatolii Yatsuk (tolik0) commented Apr 21, 2026

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/24727032304

@tolik0 Anatolii Yatsuk (tolik0) merged commit fd553bd into main Apr 21, 2026
31 of 32 checks passed
@tolik0 Anatolii Yatsuk (tolik0) deleted the devin/1776453297-fix-stream-count-zero-check-all branch April 21, 2026 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants