feat(core): Improve memory allocations

[iambriccardo]

This PR improves memory allocations in the core of ETL by removing the collection of the vector, failing as soon as an error occurs while processing table rows, and ensuring table schemas are not cloned.

In addition, it adds better handling of tests during partial CI runs.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

---

ℹ️ Recent review info

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Cache: Disabled due to Reviews > Disable Cache setting

Disabled knowledge base sources:

• Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between f8123eb and fcd08b3.

📒 Files selected for processing (12)

• .github/workflows/ci.yml
• etl-api/tests/validators.rs
• etl-destinations/src/bigquery/core.rs
• etl-destinations/src/bigquery/test_utils.rs
• etl-destinations/tests/bigquery_pipeline.rs
• etl/src/concurrency/stream.rs
• etl/src/replication/table_sync.rs
• etl/src/store/both/memory.rs
• etl/src/store/both/postgres.rs
• etl/src/store/schema/base.rs
• etl/src/test_utils/notifying_store.rs
• etl/src/workers/table_sync_copy.rs

💤 Files with no reviewable changes (1)

• .github/workflows/ci.yml

---

📝 Walkthrough

Summary by CodeRabbit

Release Notes

• Tests

  • BigQuery-related tests now gracefully skip when required environment variables are unavailable, reducing spurious test failures.

• Performance

  • Improved string handling efficiency in BigQuery table operations through optimized conversion caching.

• Improvements

  • Enhanced error handling in stream batching operations for better robustness and reliability.

Walkthrough

This pull request introduces error-aware streaming with conditional test execution and schema storage refactoring. Key changes include: renaming BatchBackpressureStream to TryBatchBackpressureStream to handle Result types with immediate error propagation; updating the SchemaStore trait and implementations to return Arc<TableSchema> instead of unit type; adding environment variable guards (skip_if_missing_bigquery_env_vars) to skip BigQuery tests when credentials are absent; propagating Arc-wrapped schema and publication name types through table copy functions; caching string conversions in BigQuery operations; and consolidating CI test execution to run across the entire workspace without excluding the etl-destinations crate.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coveralls]

coverage: 68.097% (-0.2%) from 68.264%
when pulling fcd08b3 on improve-allocations
into f8123eb on main.