Closed
Conversation
* Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com>
apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not
* Record when job is queueud in scheduler metrics * add additional buckets for exec times
* configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Prepare 0.10.0 Release (apache#522) * bump version * CHANGELOG * Ballista gets a docker image!!! (apache#521) * Ballista gets a docker image!!! * Enable flight sql * Allow executing startup script * Allow executing executables * Clippy * Remove capture group (apache#527) * fix python build in CI (apache#528) * fix python build in CI * save progress * use same min rust version in all crates * fix * use image from pyo3 * use newer image from pyo3 * do not require protoc * wheels now generated * rat - exclude generated file * Update docs for simplified instructions (apache#532) * Update docs for simplified instructions * Fix whoopsie * Update docs/source/user-guide/flightsql.md Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: Andy Grove <andygrove73@gmail.com> * remove --locked (apache#533) * Bump actions/labeler from 4.0.2 to 4.1.0 (apache#525) * Provide a memory StateBackendClient (apache#523) * Rename StateBackend::Standalone to StateBackend:Sled * Copy utility files from sled crate since they cannot be used directly * Provide a memory StateBackendClient * Fix dashmap deadlock issue * Fix for the comments Co-authored-by: yangzhong <yangzhong@ebay.com> * only build docker images on rc tags (apache#535) * docs: fix style in the Helm readme (apache#551) * Fix Helm chart's image format (apache#550) * Update datafusion requirement from 14.0.0 to 15.0.0 (apache#552) * Update datafusion requirement from 14.0.0 to 15.0.0 * Fix UT * Fix python * Fix python * Fix Python Co-authored-by: yangzhong <yangzhong@ebay.com> * Make it concurrently to launch tasks to executors (apache#557) * Make it concurrently to launch tasks to executors * Refine for comments Co-authored-by: yangzhong <yangzhong@ebay.com> * fix(ui): fix last seen (apache#562) * Support Alibaba Cloud OSS with ObjectStore (apache#567) * Fix cargo clippy (apache#571) Co-authored-by: yangzhong <yangzhong@ebay.com> * Super minor spelling error (apache#573) * Update env_logger requirement from 0.9 to 0.10 (apache#539) Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version. - [Release notes](https://github.com/rust-cli/env_logger/releases) - [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md) - [Commits](rust-cli/env_logger@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: env_logger dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update graphviz-rust requirement from 0.4.0 to 0.5.0 (apache#574) Updates the requirements on [graphviz-rust](https://github.com/besok/graphviz-rust) to permit the latest version. - [Release notes](https://github.com/besok/graphviz-rust/releases) - [Changelog](https://github.com/besok/graphviz-rust/blob/master/CHANGELOG.md) - [Commits](https://github.com/besok/graphviz-rust/commits) --- updated-dependencies: - dependency-name: graphviz-rust dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated readme to contain correct versions of dependencies. (apache#580) * Fix benchmark image link (apache#596) * Add support for Azure (apache#599) * Remove outdated script and use evergreen version of rust (apache#597) * Remove outdated script and use evergreen version of rust * Use debian protobuf * Customize session builder * Add setter for executor slots policy * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * Add ClusterState trait * Expose active job count * Remove println * Resubmit jobs when no resources available for scheduling * Make parse_physical_expr public * Reduce log spam * Fix job submitted metric by ignoring resubmissions * Record when job is queued in scheduler metrics (#28) * Record when job is queueud in scheduler metrics * add additional buckets for exec times * fmt * clippy * tomlfmt Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yahoNanJing <90197956+yahoNanJing@users.noreply.github.com> Co-authored-by: yangzhong <yangzhong@ebay.com> Co-authored-by: Xin Hao <haoxinst@gmail.com> Co-authored-by: Duyet Le <5009534+duyet@users.noreply.github.com> Co-authored-by: r.4ntix <antix.blue@antix.blue> Co-authored-by: Jeremy Dyer <jdye64@gmail.com> Co-authored-by: Sai Krishna Reddy Lakkam <86965352+saikrishna1-bidgely@users.noreply.github.com> Co-authored-by: Aidan Kovacic <95648995+aidankovacic-8451@users.noreply.github.com> Co-authored-by: Dan Harris <dan@thinkharder.dev> Co-authored-by: Dan Harris <1327726+thinkharderdev@users.noreply.github.com> Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Dan Harris <dan@coralogix.com>
* configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Prepare 0.10.0 Release (apache#522) * bump version * CHANGELOG * Ballista gets a docker image!!! (apache#521) * Ballista gets a docker image!!! * Enable flight sql * Allow executing startup script * Allow executing executables * Clippy * Remove capture group (apache#527) * fix python build in CI (apache#528) * fix python build in CI * save progress * use same min rust version in all crates * fix * use image from pyo3 * use newer image from pyo3 * do not require protoc * wheels now generated * rat - exclude generated file * Update docs for simplified instructions (apache#532) * Update docs for simplified instructions * Fix whoopsie * Update docs/source/user-guide/flightsql.md Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: Andy Grove <andygrove73@gmail.com> * remove --locked (apache#533) * Bump actions/labeler from 4.0.2 to 4.1.0 (apache#525) * Provide a memory StateBackendClient (apache#523) * Rename StateBackend::Standalone to StateBackend:Sled * Copy utility files from sled crate since they cannot be used directly * Provide a memory StateBackendClient * Fix dashmap deadlock issue * Fix for the comments Co-authored-by: yangzhong <yangzhong@ebay.com> * only build docker images on rc tags (apache#535) * docs: fix style in the Helm readme (apache#551) * Fix Helm chart's image format (apache#550) * Update datafusion requirement from 14.0.0 to 15.0.0 (apache#552) * Update datafusion requirement from 14.0.0 to 15.0.0 * Fix UT * Fix python * Fix python * Fix Python Co-authored-by: yangzhong <yangzhong@ebay.com> * Make it concurrently to launch tasks to executors (apache#557) * Make it concurrently to launch tasks to executors * Refine for comments Co-authored-by: yangzhong <yangzhong@ebay.com> * fix(ui): fix last seen (apache#562) * Support Alibaba Cloud OSS with ObjectStore (apache#567) * Fix cargo clippy (apache#571) Co-authored-by: yangzhong <yangzhong@ebay.com> * Super minor spelling error (apache#573) * Update env_logger requirement from 0.9 to 0.10 (apache#539) Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version. - [Release notes](https://github.com/rust-cli/env_logger/releases) - [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md) - [Commits](rust-cli/env_logger@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: env_logger dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update graphviz-rust requirement from 0.4.0 to 0.5.0 (apache#574) Updates the requirements on [graphviz-rust](https://github.com/besok/graphviz-rust) to permit the latest version. - [Release notes](https://github.com/besok/graphviz-rust/releases) - [Changelog](https://github.com/besok/graphviz-rust/blob/master/CHANGELOG.md) - [Commits](https://github.com/besok/graphviz-rust/commits) --- updated-dependencies: - dependency-name: graphviz-rust dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated readme to contain correct versions of dependencies. (apache#580) * Fix benchmark image link (apache#596) * Add support for Azure (apache#599) * Remove outdated script and use evergreen version of rust (apache#597) * Remove outdated script and use evergreen version of rust * Use debian protobuf * feat: update script such that ballista-cli image is built as well (apache#601) * Fix Cargo.toml format issue (apache#616) * Refactor executor main (apache#614) * Refactor executor main * copy all configs * toml fmt * Refactor scheduler main (apache#615) * refactor scheduler main * toml fmt * Python: add method to get explain output as a string (apache#593) * Update contributor guide (apache#617) * Cluster state refactor part 1 (apache#560) * Customize session builder * Add setter for executor slots policy * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Add ClusterState trait * Refactor slightly for clarity * Revert "Use node-level local limit (#20)" This reverts commit ff96bcd. * Revert "Public method for stage metrics" This reverts commit a802315. * Revert "Public method for getting execution graph" This reverts commit 490bda5. * Revert "Add public methods to SchedulerServer" This reverts commit 5ad27c0. * Revert "Add queued and completed timestamps to successful job status" This reverts commit c615fce. * Revert "Construct Executor with functions" This reverts commit 24d4830. * Always forget the apache header Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> * replace master with main (apache#621) * implement new release process (apache#623) * add docs on who can release (apache#632) * Upgrade to DataFusion 16 (again) (apache#636) * Update datafusion dependency to the latest version (apache#612) * Update datafusion dependency to the latest version * Fix python * Skip ut of test_window_lead due to apache/datafusion-python#135 * Fix clippy --------- Co-authored-by: yangzhong <yangzhong@ebay.com> * Upgrade to DataFusion 17 (apache#639) * Upgrade to DF 17 * Restore original error handling functionality * Customize session builder * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * Add ClusterState trait * Expose active job count * Remove println * Resubmit jobs when no resources available for scheduling * Make parse_physical_expr public * Reduce log spam * Fix job submitted metric by ignoring resubmissions * Record when job is queued in scheduler metrics (#28) * Record when job is queueud in scheduler metrics * add additional buckets for exec times * Upstream rebase (#29) * configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Prepare 0.10.0 Release (apache#522) * bump version * CHANGELOG * Ballista gets a docker image!!! (apache#521) * Ballista gets a docker image!!! * Enable flight sql * Allow executing startup script * Allow executing executables * Clippy * Remove capture group (apache#527) * fix python build in CI (apache#528) * fix python build in CI * save progress * use same min rust version in all crates * fix * use image from pyo3 * use newer image from pyo3 * do not require protoc * wheels now generated * rat - exclude generated file * Update docs for simplified instructions (apache#532) * Update docs for simplified instructions * Fix whoopsie * Update docs/source/user-guide/flightsql.md Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: Andy Grove <andygrove73@gmail.com> * remove --locked (apache#533) * Bump actions/labeler from 4.0.2 to 4.1.0 (apache#525) * Provide a memory StateBackendClient (apache#523) * Rename StateBackend::Standalone to StateBackend:Sled * Copy utility files from sled crate since they cannot be used directly * Provide a memory StateBackendClient * Fix dashmap deadlock issue * Fix for the comments Co-authored-by: yangzhong <yangzhong@ebay.com> * only build docker images on rc tags (apache#535) * docs: fix style in the Helm readme (apache#551) * Fix Helm chart's image format (apache#550) * Update datafusion requirement from 14.0.0 to 15.0.0 (apache#552) * Update datafusion requirement from 14.0.0 to 15.0.0 * Fix UT * Fix python * Fix python * Fix Python Co-authored-by: yangzhong <yangzhong@ebay.com> * Make it concurrently to launch tasks to executors (apache#557) * Make it concurrently to launch tasks to executors * Refine for comments Co-authored-by: yangzhong <yangzhong@ebay.com> * fix(ui): fix last seen (apache#562) * Support Alibaba Cloud OSS with ObjectStore (apache#567) * Fix cargo clippy (apache#571) Co-authored-by: yangzhong <yangzhong@ebay.com> * Super minor spelling error (apache#573) * Update env_logger requirement from 0.9 to 0.10 (apache#539) Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version. - [Release notes](https://github.com/rust-cli/env_logger/releases) - [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md) - [Commits](rust-cli/env_logger@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: env_logger dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update graphviz-rust requirement from 0.4.0 to 0.5.0 (apache#574) Updates the requirements on [graphviz-rust](https://github.com/besok/graphviz-rust) to permit the latest version. - [Release notes](https://github.com/besok/graphviz-rust/releases) - [Changelog](https://github.com/besok/graphviz-rust/blob/master/CHANGELOG.md) - [Commits](https://github.com/besok/graphviz-rust/commits) --- updated-dependencies: - dependency-name: graphviz-rust dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated readme to contain correct versions of dependencies. (apache#580) * Fix benchmark image link (apache#596) * Add support for Azure (apache#599) * Remove outdated script and use evergreen version of rust (apache#597) * Remove outdated script and use evergreen version of rust * Use debian protobuf * Customize session builder * Add setter for executor slots policy * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * Add ClusterState trait * Expose active job count * Remove println * Resubmit jobs when no resources available for scheduling * Make parse_physical_expr public * Reduce log spam * Fix job submitted metric by ignoring resubmissions * Record when job is queued in scheduler metrics (#28) * Record when job is queueud in scheduler metrics * add additional buckets for exec times * fmt * clippy * tomlfmt Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yahoNanJing <90197956+yahoNanJing@users.noreply.github.com> Co-authored-by: yangzhong <yangzhong@ebay.com> Co-authored-by: Xin Hao <haoxinst@gmail.com> Co-authored-by: Duyet Le <5009534+duyet@users.noreply.github.com> Co-authored-by: r.4ntix <antix.blue@antix.blue> Co-authored-by: Jeremy Dyer <jdye64@gmail.com> Co-authored-by: Sai Krishna Reddy Lakkam <86965352+saikrishna1-bidgely@users.noreply.github.com> Co-authored-by: Aidan Kovacic <95648995+aidankovacic-8451@users.noreply.github.com> Co-authored-by: Dan Harris <dan@thinkharder.dev> Co-authored-by: Dan Harris <1327726+thinkharderdev@users.noreply.github.com> Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Dan Harris <dan@coralogix.com> * Post merge update * update message formatting * post merge update * another post-merge updates * update github actions * clippy * update script * fmt --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yahoNanJing <90197956+yahoNanJing@users.noreply.github.com> Co-authored-by: yangzhong <yangzhong@ebay.com> Co-authored-by: Xin Hao <haoxinst@gmail.com> Co-authored-by: Duyet Le <5009534+duyet@users.noreply.github.com> Co-authored-by: r.4ntix <antix.blue@antix.blue> Co-authored-by: Jeremy Dyer <jdye64@gmail.com> Co-authored-by: Sai Krishna Reddy Lakkam <86965352+saikrishna1-bidgely@users.noreply.github.com> Co-authored-by: Aidan Kovacic <95648995+aidankovacic-8451@users.noreply.github.com> Co-authored-by: Tim Van Wassenhove <tim@timvw.be> Co-authored-by: Dan Harris <1327726+thinkharderdev@users.noreply.github.com> Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Brent Gardner <bgardner@squarelabs.net> Co-authored-by: Dan Harris <dan@thinkharder.dev> Co-authored-by: Dan Harris <dan@coralogix.com>
* configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Prepare 0.10.0 Release (apache#522) * bump version * CHANGELOG * Ballista gets a docker image!!! (apache#521) * Ballista gets a docker image!!! * Enable flight sql * Allow executing startup script * Allow executing executables * Clippy * Remove capture group (apache#527) * fix python build in CI (apache#528) * fix python build in CI * save progress * use same min rust version in all crates * fix * use image from pyo3 * use newer image from pyo3 * do not require protoc * wheels now generated * rat - exclude generated file * Update docs for simplified instructions (apache#532) * Update docs for simplified instructions * Fix whoopsie * Update docs/source/user-guide/flightsql.md Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: Andy Grove <andygrove73@gmail.com> * remove --locked (apache#533) * Bump actions/labeler from 4.0.2 to 4.1.0 (apache#525) * Provide a memory StateBackendClient (apache#523) * Rename StateBackend::Standalone to StateBackend:Sled * Copy utility files from sled crate since they cannot be used directly * Provide a memory StateBackendClient * Fix dashmap deadlock issue * Fix for the comments Co-authored-by: yangzhong <yangzhong@ebay.com> * only build docker images on rc tags (apache#535) * docs: fix style in the Helm readme (apache#551) * Fix Helm chart's image format (apache#550) * Update datafusion requirement from 14.0.0 to 15.0.0 (apache#552) * Update datafusion requirement from 14.0.0 to 15.0.0 * Fix UT * Fix python * Fix python * Fix Python Co-authored-by: yangzhong <yangzhong@ebay.com> * Make it concurrently to launch tasks to executors (apache#557) * Make it concurrently to launch tasks to executors * Refine for comments Co-authored-by: yangzhong <yangzhong@ebay.com> * fix(ui): fix last seen (apache#562) * Support Alibaba Cloud OSS with ObjectStore (apache#567) * Fix cargo clippy (apache#571) Co-authored-by: yangzhong <yangzhong@ebay.com> * Super minor spelling error (apache#573) * Update env_logger requirement from 0.9 to 0.10 (apache#539) Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version. - [Release notes](https://github.com/rust-cli/env_logger/releases) - [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md) - [Commits](rust-cli/env_logger@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: env_logger dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update graphviz-rust requirement from 0.4.0 to 0.5.0 (apache#574) Updates the requirements on [graphviz-rust](https://github.com/besok/graphviz-rust) to permit the latest version. - [Release notes](https://github.com/besok/graphviz-rust/releases) - [Changelog](https://github.com/besok/graphviz-rust/blob/master/CHANGELOG.md) - [Commits](https://github.com/besok/graphviz-rust/commits) --- updated-dependencies: - dependency-name: graphviz-rust dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated readme to contain correct versions of dependencies. (apache#580) * Fix benchmark image link (apache#596) * Add support for Azure (apache#599) * Remove outdated script and use evergreen version of rust (apache#597) * Remove outdated script and use evergreen version of rust * Use debian protobuf * feat: update script such that ballista-cli image is built as well (apache#601) * Fix Cargo.toml format issue (apache#616) * Refactor executor main (apache#614) * Refactor executor main * copy all configs * toml fmt * Refactor scheduler main (apache#615) * refactor scheduler main * toml fmt * Python: add method to get explain output as a string (apache#593) * Update contributor guide (apache#617) * Cluster state refactor part 1 (apache#560) * Customize session builder * Add setter for executor slots policy * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Add ClusterState trait * Refactor slightly for clarity * Revert "Use node-level local limit (#20)" This reverts commit ff96bcd. * Revert "Public method for stage metrics" This reverts commit a802315. * Revert "Public method for getting execution graph" This reverts commit 490bda5. * Revert "Add public methods to SchedulerServer" This reverts commit 5ad27c0. * Revert "Add queued and completed timestamps to successful job status" This reverts commit c615fce. * Revert "Construct Executor with functions" This reverts commit 24d4830. * Always forget the apache header Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> * replace master with main (apache#621) * implement new release process (apache#623) * add docs on who can release (apache#632) * Upgrade to DataFusion 16 (again) (apache#636) * Update datafusion dependency to the latest version (apache#612) * Update datafusion dependency to the latest version * Fix python * Skip ut of test_window_lead due to apache/datafusion-python#135 * Fix clippy --------- Co-authored-by: yangzhong <yangzhong@ebay.com> * Upgrade to DataFusion 17 (apache#639) * Upgrade to DF 17 * Restore original error handling functionality * check in benchmark image (apache#647) * Remove `python` dir & python-related workflows (apache#654) * refactor: remove python dir & python-related workflows * remove brackets * Handle job resubmission (apache#586) * Handle job resubmission * Make resubmission configurable and add test * Fix debug log * Add executor self-registration mechanism in the heartbeat service (apache#649) Co-authored-by: yangzhong <yangzhong@ebay.com> * Cluster state refactor Part 2 (apache#658) * Customize session builder * Add setter for executor slots policy * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Add ClusterState trait * Refactor slightly for clarity * Revert "Use node-level local limit (#20)" This reverts commit ff96bcd. * Revert "Public method for stage metrics" This reverts commit a802315. * Revert "Public method for getting execution graph" This reverts commit 490bda5. * Revert "Add public methods to SchedulerServer" This reverts commit 5ad27c0. * Revert "Add queued and completed timestamps to successful job status" This reverts commit c615fce. * Revert "Construct Executor with functions" This reverts commit 24d4830. * Always forget the apache header * WIP * Implement JobState * Tests and fixes * do not hold ref across await point * Fix clippy warnings * Fix tomlfmt github action * uncomment test --------- Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> * Upgrade to DataFusion 18.0.0-rc1 (apache#664) * Add executor terminating status for graceful shutdown * Remove empty file * Minor refactor to reduce duplicate code (apache#659) * move test_util to ballista-examples package (apache#661) * Upgrade to DataFusion 18 (apache#668) * Enable physical plan round-trip tests (apache#666) * Customize session builder * Construct Executor with functions * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * Add ClusterState trait * Expose active job count * Make parse_physical_expr public * Fix job submitted metric by ignoring resubmissions * Record when job is queued in scheduler metrics (#28) * Record when job is queueud in scheduler metrics * add additional buckets for exec times * Upstream rebase (#29) * configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Prepare 0.10.0 Release (apache#522) * bump version * CHANGELOG * Ballista gets a docker image!!! (apache#521) * Ballista gets a docker image!!! * Enable flight sql * Allow executing startup script * Allow executing executables * Clippy * Remove capture group (apache#527) * fix python build in CI (apache#528) * fix python build in CI * save progress * use same min rust version in all crates * fix * use image from pyo3 * use newer image from pyo3 * do not require protoc * wheels now generated * rat - exclude generated file * Update docs for simplified instructions (apache#532) * Update docs for simplified instructions * Fix whoopsie * Update docs/source/user-guide/flightsql.md Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: Andy Grove <andygrove73@gmail.com> * remove --locked (apache#533) * Bump actions/labeler from 4.0.2 to 4.1.0 (apache#525) * Provide a memory StateBackendClient (apache#523) * Rename StateBackend::Standalone to StateBackend:Sled * Copy utility files from sled crate since they cannot be used directly * Provide a memory StateBackendClient * Fix dashmap deadlock issue * Fix for the comments Co-authored-by: yangzhong <yangzhong@ebay.com> * only build docker images on rc tags (apache#535) * docs: fix style in the Helm readme (apache#551) * Fix Helm chart's image format (apache#550) * Update datafusion requirement from 14.0.0 to 15.0.0 (apache#552) * Update datafusion requirement from 14.0.0 to 15.0.0 * Fix UT * Fix python * Fix python * Fix Python Co-authored-by: yangzhong <yangzhong@ebay.com> * Make it concurrently to launch tasks to executors (apache#557) * Make it concurrently to launch tasks to executors * Refine for comments Co-authored-by: yangzhong <yangzhong@ebay.com> * fix(ui): fix last seen (apache#562) * Support Alibaba Cloud OSS with ObjectStore (apache#567) * Fix cargo clippy (apache#571) Co-authored-by: yangzhong <yangzhong@ebay.com> * Super minor spelling error (apache#573) * Update env_logger requirement from 0.9 to 0.10 (apache#539) Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version. - [Release notes](https://github.com/rust-cli/env_logger/releases) - [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md) - [Commits](rust-cli/env_logger@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: env_logger dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update graphviz-rust requirement from 0.4.0 to 0.5.0 (apache#574) Updates the requirements on [graphviz-rust](https://github.com/besok/graphviz-rust) to permit the latest version. - [Release notes](https://github.com/besok/graphviz-rust/releases) - [Changelog](https://github.com/besok/graphviz-rust/blob/master/CHANGELOG.md) - [Commits](https://github.com/besok/graphviz-rust/commits) --- updated-dependencies: - dependency-name: graphviz-rust dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated readme to contain correct versions of dependencies. (apache#580) * Fix benchmark image link (apache#596) * Add support for Azure (apache#599) * Remove outdated script and use evergreen version of rust (apache#597) * Remove outdated script and use evergreen version of rust * Use debian protobuf * Customize session builder * Add setter for executor slots policy * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * Add ClusterState trait * Expose active job count * Remove println * Resubmit jobs when no resources available for scheduling * Make parse_physical_expr public * Reduce log spam * Fix job submitted metric by ignoring resubmissions * Record when job is queued in scheduler metrics (#28) * Record when job is queueud in scheduler metrics * add additional buckets for exec times * fmt * clippy * tomlfmt Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yahoNanJing <90197956+yahoNanJing@users.noreply.github.com> Co-authored-by: yangzhong <yangzhong@ebay.com> Co-authored-by: Xin Hao <haoxinst@gmail.com> Co-authored-by: Duyet Le <5009534+duyet@users.noreply.github.com> Co-authored-by: r.4ntix <antix.blue@antix.blue> Co-authored-by: Jeremy Dyer <jdye64@gmail.com> Co-authored-by: Sai Krishna Reddy Lakkam <86965352+saikrishna1-bidgely@users.noreply.github.com> Co-authored-by: Aidan Kovacic <95648995+aidankovacic-8451@users.noreply.github.com> Co-authored-by: Dan Harris <dan@thinkharder.dev> Co-authored-by: Dan Harris <1327726+thinkharderdev@users.noreply.github.com> Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Dan Harris <dan@coralogix.com> * Update from upstream (#30) * configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Prepare 0.10.0 Release (apache#522) * bump version * CHANGELOG * Ballista gets a docker image!!! (apache#521) * Ballista gets a docker image!!! * Enable flight sql * Allow executing startup script * Allow executing executables * Clippy * Remove capture group (apache#527) * fix python build in CI (apache#528) * fix python build in CI * save progress * use same min rust version in all crates * fix * use image from pyo3 * use newer image from pyo3 * do not require protoc * wheels now generated * rat - exclude generated file * Update docs for simplified instructions (apache#532) * Update docs for simplified instructions * Fix whoopsie * Update docs/source/user-guide/flightsql.md Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: Andy Grove <andygrove73@gmail.com> * remove --locked (apache#533) * Bump actions/labeler from 4.0.2 to 4.1.0 (apache#525) * Provide a memory StateBackendClient (apache#523) * Rename StateBackend::Standalone to StateBackend:Sled * Copy utility files from sled crate since they cannot be used directly * Provide a memory StateBackendClient * Fix dashmap deadlock issue * Fix for the comments Co-authored-by: yangzhong <yangzhong@ebay.com> * only build docker images on rc tags (apache#535) * docs: fix style in the Helm readme (apache#551) * Fix Helm chart's image format (apache#550) * Update datafusion requirement from 14.0.0 to 15.0.0 (apache#552) * Update datafusion requirement from 14.0.0 to 15.0.0 * Fix UT * Fix python * Fix python * Fix Python Co-authored-by: yangzhong <yangzhong@ebay.com> * Make it concurrently to launch tasks to executors (apache#557) * Make it concurrently to launch tasks to executors * Refine for comments Co-authored-by: yangzhong <yangzhong@ebay.com> * fix(ui): fix last seen (apache#562) * Support Alibaba Cloud OSS with ObjectStore (apache#567) * Fix cargo clippy (apache#571) Co-authored-by: yangzhong <yangzhong@ebay.com> * Super minor spelling error (apache#573) * Update env_logger requirement from 0.9 to 0.10 (apache#539) Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version. - [Release notes](https://github.com/rust-cli/env_logger/releases) - [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md) - [Commits](rust-cli/env_logger@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: env_logger dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update graphviz-rust requirement from 0.4.0 to 0.5.0 (apache#574) Updates the requirements on [graphviz-rust](https://github.com/besok/graphviz-rust) to permit the latest version. - [Release notes](https://github.com/besok/graphviz-rust/releases) - [Changelog](https://github.com/besok/graphviz-rust/blob/master/CHANGELOG.md) - [Commits](https://github.com/besok/graphviz-rust/commits) --- updated-dependencies: - dependency-name: graphviz-rust dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated readme to contain correct versions of dependencies. (apache#580) * Fix benchmark image link (apache#596) * Add support for Azure (apache#599) * Remove outdated script and use evergreen version of rust (apache#597) * Remove outdated script and use evergreen version of rust * Use debian protobuf * feat: update script such that ballista-cli image is built as well (apache#601) * Fix Cargo.toml format issue (apache#616) * Refactor executor main (apache#614) * Refactor executor main * copy all configs * toml fmt * Refactor scheduler main (apache#615) * refactor scheduler main * toml fmt * Python: add method to get explain output as a string (apache#593) * Update contributor guide (apache#617) * Cluster state refactor part 1 (apache#560) * Customize session builder * Add setter for executor slots policy * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Add ClusterState trait * Refactor slightly for clarity * Revert "Use node-level local limit (#20)" This reverts commit ff96bcd. * Revert "Public method for stage metrics" This reverts commit a802315. * Revert "Public method for getting execution graph" This reverts commit 490bda5. * Revert "Add public methods to SchedulerServer" This reverts commit 5ad27c0. * Revert "Add queued and completed timestamps to successful job status" This reverts commit c615fce. * Revert "Construct Executor with functions" This reverts commit 24d4830. * Always forget the apache header Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> * replace master with main (apache#621) * implement new release process (apache#623) * add docs on who can release (apache#632) * Upgrade to DataFusion 16 (again) (apache#636) * Update datafusion dependency to the latest version (apache#612) * Update datafusion dependency to the latest version * Fix python * Skip ut of test_window_lead due to apache/datafusion-python#135 * Fix clippy --------- Co-authored-by: yangzhong <yangzhong@ebay.com> * Upgrade to DataFusion 17 (apache#639) * Upgrade to DF 17 * Restore original error handling functionality * Customize session builder * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * Add ClusterState trait * Expose active job count * Remove println * Resubmit jobs when no resources available for scheduling * Make parse_physical_expr public * Reduce log spam * Fix job submitted metric by ignoring resubmissions * Record when job is queued in scheduler metrics (#28) * Record when job is queueud in scheduler metrics * add additional buckets for exec times * Upstream rebase (#29) * configure_me_codegen retroactively reserved on our `bind_host` parame… (apache#520) * configure_me_codegen retroactively reserved on our `bind_host` parameter name * Add label and pray * Add more labels why not * Prepare 0.10.0 Release (apache#522) * bump version * CHANGELOG * Ballista gets a docker image!!! (apache#521) * Ballista gets a docker image!!! * Enable flight sql * Allow executing startup script * Allow executing executables * Clippy * Remove capture group (apache#527) * fix python build in CI (apache#528) * fix python build in CI * save progress * use same min rust version in all crates * fix * use image from pyo3 * use newer image from pyo3 * do not require protoc * wheels now generated * rat - exclude generated file * Update docs for simplified instructions (apache#532) * Update docs for simplified instructions * Fix whoopsie * Update docs/source/user-guide/flightsql.md Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: Andy Grove <andygrove73@gmail.com> * remove --locked (apache#533) * Bump actions/labeler from 4.0.2 to 4.1.0 (apache#525) * Provide a memory StateBackendClient (apache#523) * Rename StateBackend::Standalone to StateBackend:Sled * Copy utility files from sled crate since they cannot be used directly * Provide a memory StateBackendClient * Fix dashmap deadlock issue * Fix for the comments Co-authored-by: yangzhong <yangzhong@ebay.com> * only build docker images on rc tags (apache#535) * docs: fix style in the Helm readme (apache#551) * Fix Helm chart's image format (apache#550) * Update datafusion requirement from 14.0.0 to 15.0.0 (apache#552) * Update datafusion requirement from 14.0.0 to 15.0.0 * Fix UT * Fix python * Fix python * Fix Python Co-authored-by: yangzhong <yangzhong@ebay.com> * Make it concurrently to launch tasks to executors (apache#557) * Make it concurrently to launch tasks to executors * Refine for comments Co-authored-by: yangzhong <yangzhong@ebay.com> * fix(ui): fix last seen (apache#562) * Support Alibaba Cloud OSS with ObjectStore (apache#567) * Fix cargo clippy (apache#571) Co-authored-by: yangzhong <yangzhong@ebay.com> * Super minor spelling error (apache#573) * Update env_logger requirement from 0.9 to 0.10 (apache#539) Updates the requirements on [env_logger](https://github.com/rust-cli/env_logger) to permit the latest version. - [Release notes](https://github.com/rust-cli/env_logger/releases) - [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md) - [Commits](rust-cli/env_logger@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: env_logger dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update graphviz-rust requirement from 0.4.0 to 0.5.0 (apache#574) Updates the requirements on [graphviz-rust](https://github.com/besok/graphviz-rust) to permit the latest version. - [Release notes](https://github.com/besok/graphviz-rust/releases) - [Changelog](https://github.com/besok/graphviz-rust/blob/master/CHANGELOG.md) - [Commits](https://github.com/besok/graphviz-rust/commits) --- updated-dependencies: - dependency-name: graphviz-rust dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updated readme to contain correct versions of dependencies. (apache#580) * Fix benchmark image link (apache#596) * Add support for Azure (apache#599) * Remove outdated script and use evergreen version of rust (apache#597) * Remove outdated script and use evergreen version of rust * Use debian protobuf * Customize session builder * Add setter for executor slots policy * Construct Executor with functions * Add queued and completed timestamps to successful job status * Add public methods to SchedulerServer * Public method for getting execution graph * Public method for stage metrics * Use node-level local limit (#20) * Use node-level local limit * serialize limit in shuffle writer * Revert "Merge pull request #19 from coralogix/sc-5792" This reverts commit 08140ef, reversing changes made to a7f1384. * add log * make sure we don't forget limit for shuffle writer * update accum correctly and try to break early * Check local limit accumulator before polling for more data * fix build Co-authored-by: Martins Purins <martins.purins@coralogix.com> * Add ClusterState trait * Expose active job count * Remove println * Resubmit jobs when no resources available for scheduling * Make parse_physical_expr public * Reduce log spam * Fix job submitted metric by ignoring resubmissions * Record when job is queued in scheduler metrics (#28) * Record when job is queueud in scheduler metrics * add additional buckets for exec times * fmt * clippy * tomlfmt Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yahoNanJing <90197956+yahoNanJing@users.noreply.github.com> Co-authored-by: yangzhong <yangzhong@ebay.com> Co-authored-by: Xin Hao <haoxinst@gmail.com> Co-authored-by: Duyet Le <5009534+duyet@users.noreply.github.com> Co-authored-by: r.4ntix <antix.blue@antix.blue> Co-authored-by: Jeremy Dyer <jdye64@gmail.com> Co-authored-by: Sai Krishna Reddy Lakkam <86965352+saikrishna1-bidgely@users.noreply.github.com> Co-authored-by: Aidan Kovacic <95648995+aidankovacic-8451@users.noreply.github.com> Co-authored-by: Dan Harris <dan@thinkharder.dev> Co-authored-by: Dan Harris <1327726+thinkharderdev@users.noreply.github.com> Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Dan Harris <dan@coralogix.com> * Post merge update * update message formatting * post merge update * another post-merge updates * update github actions * clippy * update script * fmt --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yahoNanJing <90197956+yahoNanJing@users.noreply.github.com> Co-authored-by: yangzhong <yangzhong@ebay.com> Co-authored-by: Xin Hao <haoxinst@gmail.com> Co-authored-by: Duyet Le <5009534+duyet@users.noreply.github.com> Co-authored-by: r.4ntix <antix.blue@antix.blue> Co-authored-by: Jeremy Dyer <jdye64@gmail.com> Co-authored-by: Sai Krishna Reddy Lakkam <86965352+saikrishna1-bidgely@users.noreply.github.com> Co-authored-by: Aidan Kovacic <95648995+aidankovacic-8451@users.noreply.github.com> Co-authored-by: Tim Van Wassenhove <tim@timvw.be> Co-authored-by: Dan Harris <1327726+thinkharderdev@users.noreply.github.com> Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Brent Gardner <bgardner@squarelabs.net> Co-authored-by: Dan Harris <dan@thinkharder.dev> Co-authored-by: Dan Harris <dan@coralogix.com> * post merge fixes * fix branch naming in github actions * cleanup * fmt * update imports --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Brent Gardner <brent.gardner@spaceandtime.io> Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: yahoNanJing <90197956+yahoNanJing@users.noreply.github.com> Co-authored-by: yangzhong <yangzhong@ebay.com> Co-authored-by: Xin Hao <haoxinst@gmail.com> Co-authored-by: Duyet Le <5009534+duyet@users.noreply.github.com> Co-authored-by: r.4ntix <antix.blue@antix.blue> Co-authored-by: Jeremy Dyer <jdye64@gmail.com> Co-authored-by: Sai Krishna Reddy Lakkam <86965352+saikrishna1-bidgely@users.noreply.github.com> Co-authored-by: Aidan Kovacic <95648995+aidankovacic-8451@users.noreply.github.com> Co-authored-by: Tim Van Wassenhove <tim@timvw.be> Co-authored-by: Dan Harris <1327726+thinkharderdev@users.noreply.github.com> Co-authored-by: Martins Purins <martins.purins@coralogix.com> Co-authored-by: Brent Gardner <bgardner@squarelabs.net> Co-authored-by: Ian Alexander Joiner <14581281+iajoiner@users.noreply.github.com> Co-authored-by: Dan Harris <dan@thinkharder.dev> Co-authored-by: jiangzhx <jiangzhx@gmail.com> Co-authored-by: Dan Harris <dan@coralogix.com>
Co-authored-by: Daniël Heres <daniel.heres@coralogix.com>
* Downgrade expected error to warning * add context
* Mark as pub * Fmt --------- Co-authored-by: Daniël Heres <daniel.heres@coralogix.com>
Co-authored-by: Daniël Heres <daniel.heres@coralogix.com>
Co-authored-by: Daniël Heres <daniel.heres@coralogix.com>
* serialize configoptions and pass them to executor and allow extensions for TaskContext * use ConfigOptions::with_extensions
* Several improvements to circuit breaker * Fix test code * Add cache update code * Fix clippy warnings * Skip sending empty circuit breaker updates * More improvements * Improvements * Add another log message * Add log message, fix clippy warnings * Slightly improve logic * Improve * Fix test * Send empty circuit breaker updates, ignoring zero delta messages * Fix potential race condition * Fix partition states overriding eachother * Fix bug in execution_graph trip_stage * Small improvement
* Noop circuit breaker in case of failed registration * Fix clippy
* Add metrics to circuit breaker client * Add another metric
* Attempt to fix growth of circuit breaker client state * Add another gauge * Reduce number of times state_per_stage is accessed * Use IntGauge
* Upgrade datafusion to 30.0.0 (apache#866) * Fix * Cherry-pick * Cherry-pick * Cherry-pick * Cherry-pick * Patch versions * Fmt * Clippy * Toml * Fixes * Fmt --------- Co-authored-by: r.4ntix <r.4ntix@gmail.com> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* Improve circuit breaker client: Per-scheduler batching * Slight improvement and fix * Revert one change
* Add metric for task idle time * clippy and toml format * clippy
* Better load average metric for executors * Do not overwrite the same metrics
github-actions
bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #.
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?