Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added process.Background() and process.Forwarded() #804

Merged
merged 10 commits into from
Sep 27, 2023
Merged

Added process.Background() and process.Forwarded() #804

merged 10 commits into from
Sep 27, 2023

Conversation

nfx
Copy link
Contributor

@nfx nfx commented Sep 25, 2023

Changes

This PR adds higher-level wrappers for calling subprocesses. One of the steps to get #637 in, as previously discussed.

The reason to add process.Forwarded() is to proxy Python's input() calls from a child process seamlessly. Another use-case is plugging in less as a pager for the list results.

Tests

make test

This PR adds higher-level wrappers for calling subprocesses
@nfx nfx requested a review from pietern September 25, 2023 22:35
@nfx nfx marked this pull request as ready for review September 25, 2023 22:36
Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like centralizing this, as long as it is flexible enough that we don't have to use a mix of libs/process and os/exec.

libs/git/clone.go Show resolved Hide resolved
libs/process/background.go Outdated Show resolved Hide resolved
libs/process/background.go Outdated Show resolved Hide resolved
libs/process/background.go Outdated Show resolved Hide resolved
libs/process/background.go Outdated Show resolved Hide resolved
libs/process/forwarded.go Outdated Show resolved Hide resolved
libs/process/forwarded.go Outdated Show resolved Hide resolved
libs/process/forwarded.go Outdated Show resolved Hide resolved
libs/process/forwarded_test.go Outdated Show resolved Hide resolved
libs/process/opts.go Outdated Show resolved Hide resolved
@nfx nfx requested a review from pietern September 26, 2023 15:20
libs/env/context.go Outdated Show resolved Hide resolved
libs/env/context.go Show resolved Hide resolved
libs/env/context_test.go Show resolved Hide resolved
libs/process/background.go Outdated Show resolved Hide resolved
libs/process/forwarded.go Outdated Show resolved Hide resolved
libs/process/opts.go Outdated Show resolved Hide resolved
python/runner_test.go Outdated Show resolved Hide resolved
@nfx nfx requested a review from pietern September 26, 2023 18:36
@nfx nfx enabled auto-merge September 26, 2023 18:37
Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Windows test failures are caused by:

  • \r\n line feed
  • SYSTEMROOT env var being set

Everything else LGTM.

@nfx nfx added this pull request to the merge queue Sep 27, 2023
Merged via the queue into main with commit 7171874 Sep 27, 2023
4 checks passed
@nfx nfx deleted the libs/process branch September 27, 2023 09:11
andrewnester added a commit that referenced this pull request Sep 27, 2023
Bundles:
 * Enable target overrides for pipeline clusters ([#792](#792)).
 * Add support for regex patterns in template schema ([#768](#768)).
 * Make the default `databricks bundle init` template more self-explanatory ([#796](#796)).
 * Make a notebook wrapper for Python wheel tasks optional ([#797](#797)).
 * Added a warning when Python wheel wrapper needs to be used ([#807](#807)).

Internal:
 * Added `process.Background()` and `process.Forwarded()` ([#804](#804)).

Dependency updates:
 * Bump golang.org/x/term from 0.11.0 to 0.12.0 ([#798](#798)).
 * Bump github.com/hashicorp/terraform-exec from 0.18.1 to 0.19.0 ([#801](#801)).
 * Bump golang.org/x/oauth2 from 0.11.0 to 0.12.0 ([#802](#802)).
@andrewnester andrewnester mentioned this pull request Sep 27, 2023
github-merge-queue bot pushed a commit that referenced this pull request Sep 27, 2023
Bundles:
* Enable target overrides for pipeline clusters
([#792](#792)).
* Add support for regex patterns in template schema
([#768](#768)).
* Make the default `databricks bundle init` template more
self-explanatory ([#796](#796)).
* Make a notebook wrapper for Python wheel tasks optional
([#797](#797)).
* Added a warning when Python wheel wrapper needs to be used
([#807](#807)).

Internal:
* Added `process.Background()` and `process.Forwarded()`
([#804](#804)).

Dependency updates:
* Bump golang.org/x/term from 0.11.0 to 0.12.0
([#798](#798)).
* Bump github.com/hashicorp/terraform-exec from 0.18.1 to 0.19.0
([#801](#801)).
* Bump golang.org/x/oauth2 from 0.11.0 to 0.12.0
([#802](#802)).
github-merge-queue bot pushed a commit that referenced this pull request Oct 3, 2023
This PR adds a few utilities related to Python interpreter detection:

- `python.DetectInterpreters` to detect all Python versions available in
`$PATH` by executing every matched binary name with `--version` flag.
- `python.DetectVirtualEnvPath` to detect if there's any child virtual
environment in `src` directory
- `python.DetectExecutable` to detect if there's python3 installed
either by `which python3` command or by calling
`python.DetectInterpreters().AtLeast("v3.8")`

To be merged after #804, as one of
the steps to get #637 in, as
previously discussed.
hectorcast-db pushed a commit that referenced this pull request Oct 13, 2023
## Changes
This PR adds higher-level wrappers for calling subprocesses. One of the
steps to get #637 in, as
previously discussed.

The reason to add `process.Forwarded()` is to proxy Python's `input()`
calls from a child process seamlessly. Another use-case is plugging in
`less` as a pager for the list results.

## Tests
`make test`
hectorcast-db pushed a commit that referenced this pull request Oct 13, 2023
Bundles:
* Enable target overrides for pipeline clusters
([#792](#792)).
* Add support for regex patterns in template schema
([#768](#768)).
* Make the default `databricks bundle init` template more
self-explanatory ([#796](#796)).
* Make a notebook wrapper for Python wheel tasks optional
([#797](#797)).
* Added a warning when Python wheel wrapper needs to be used
([#807](#807)).

Internal:
* Added `process.Background()` and `process.Forwarded()`
([#804](#804)).

Dependency updates:
* Bump golang.org/x/term from 0.11.0 to 0.12.0
([#798](#798)).
* Bump github.com/hashicorp/terraform-exec from 0.18.1 to 0.19.0
([#801](#801)).
* Bump golang.org/x/oauth2 from 0.11.0 to 0.12.0
([#802](#802)).
hectorcast-db pushed a commit that referenced this pull request Oct 13, 2023
This PR adds a few utilities related to Python interpreter detection:

- `python.DetectInterpreters` to detect all Python versions available in
`$PATH` by executing every matched binary name with `--version` flag.
- `python.DetectVirtualEnvPath` to detect if there's any child virtual
environment in `src` directory
- `python.DetectExecutable` to detect if there's python3 installed
either by `which python3` command or by calling
`python.DetectInterpreters().AtLeast("v3.8")`

To be merged after #804, as one of
the steps to get #637 in, as
previously discussed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants