New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows: break the dependency on Bash #4319

Open
laszlocsomor opened this Issue Dec 18, 2017 · 4 comments

Comments

Projects
None yet
4 participants
@laszlocsomor
Copy link
Contributor

laszlocsomor commented Dec 18, 2017

Bazel should not require the presence of Bash, nor the presence of the BAZEL_SH envvar.

Bash should be just another tool(chain) like Python.

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 9, 2018

bash: Add a toolchain for local Bash.
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.

Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
--shell_executable flag.

See bazelbuild#4319

Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 12, 2018

bash: Add a toolchain for local Bash.
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.

Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
--shell_executable flag.

See bazelbuild#4319

Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 12, 2018

bash: Add a toolchain for local Bash.
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.

Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
--shell_executable flag.

See bazelbuild#4319

Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 12, 2018

bash: Add a toolchain for local Bash.
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.

Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
--shell_executable flag.

See bazelbuild#4319

Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 12, 2018

bash: Add a toolchain for local Bash.
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.

Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
--shell_executable flag.

See bazelbuild#4319

Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 13, 2018

bash: Add a toolchain for local Bash.
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.

Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
--shell_executable flag.

See bazelbuild#4319

Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 13, 2018

bash: Add a toolchain for local Bash.
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.

Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
--shell_executable flag.

See bazelbuild#4319

Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

bazel-io pushed a commit that referenced this issue Apr 16, 2018

bash: Add a toolchain for local Bash.
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.

Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
`--shell_executable` flag.

See #4319

Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9

Closes #4980.

Change-Id: Ic2406a4da260b284e15852070d58472ca18340af
PiperOrigin-RevId: 193022708

bazel-io pushed a commit that referenced this issue Apr 23, 2018

SpawnAction.setShellCommand: expect shell path
SpawnAction.setShellCommand(String) now expects
the shell interpreter's path as an argument. This
change enables two things:
- rules can report an error if the shell is
  missing
- SpawnAction no longer has to know about a
  default shell

The new ShToolchain class will later also be
responsible for retrieving the active shell
toolchain (added in 81ed3ad).

This change brings Bazel closer to not depend on
the shell unless it has to (e.g. to run shell
scripts).

See #4319

RELNOTES: none
PiperOrigin-RevId: 193885943

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 23, 2018

windows,client: fix error reporting
Fix error reporting in the path conversion methods
of the Bazel client. Previously the error
reporting logic used GetLastErrorString in places
where it was not appropriate (i.e. it was not a
failed Windows API call that caused an error).

This cleanup prepares removing the concept of the
MSYS root from the Bazel client, since MSYS paths
are no longer supported and we want to cut Bazel's
dependency on Bash (thus MSYS) completely.

See bazelbuild#4319

Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 23, 2018

windows,client: fix error reporting
Fix error reporting in the path conversion methods
of the Bazel client. Previously the error
reporting logic used GetLastErrorString in places
where it was not appropriate (i.e. it was not a
failed Windows API call that caused an error).

This cleanup prepares removing the concept of the
MSYS root from the Bazel client, since MSYS paths
are no longer supported and we want to cut Bazel's
dependency on Bash (thus MSYS) completely.

See bazelbuild#4319

Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 23, 2018

windows,client: fix error reporting
Fix error reporting in the path conversion methods
of the Bazel client. Previously the error
reporting logic used GetLastErrorString in places
where it was not appropriate (i.e. it was not a
failed Windows API call that caused an error).

This cleanup prepares removing the concept of the
MSYS root from the Bazel client, since MSYS paths
are no longer supported and we want to cut Bazel's
dependency on Bash (thus MSYS) completely.

See bazelbuild#4319

Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 23, 2018

windows,client: fix error reporting
Fix error reporting in the path conversion methods
of the Bazel client. Previously the error
reporting logic used GetLastErrorString in places
where it was not appropriate (i.e. it was not a
failed Windows API call that caused an error).

This cleanup prepares removing the concept of the
MSYS root from the Bazel client, since MSYS paths
are no longer supported and we want to cut Bazel's
dependency on Bash (thus MSYS) completely.

See bazelbuild#4319

Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088

bazel-io pushed a commit that referenced this issue Apr 24, 2018

windows,client: fix error reporting
Fix error reporting in the path conversion methods
of the Bazel client. Previously the error
reporting logic used GetLastErrorString in places
where it was not appropriate (i.e. it was not a
failed Windows API call that caused an error).

This cleanup prepares removing the concept of the
MSYS root from the Bazel client, since MSYS paths
are no longer supported and we want to cut Bazel's
dependency on Bash (thus MSYS) completely.

See #4319

Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088

Closes #5072.

Change-Id: Ie50a20e0ee0c572592f637340a2f2948c7f53088
PiperOrigin-RevId: 194052665

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 25, 2018

windows,client: no longer support Unix-style paths
The Bazel client no longer supports MSYS paths.
The only exception is "/dev/null" which the client
treats as "NUL".

After this change you can no longer pass MSYS
paths as Bazel flag values on Windows.

See bazelbuild#4319

Change-Id: I39d81843015c5a4014dd5953bac2e1c29dcd5bed

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue Apr 25, 2018

windows,client: no longer support Unix-style paths
The Bazel client no longer supports MSYS paths.
The only exception is "/dev/null" which the client
treats as "NUL".

After this change you can no longer pass MSYS
paths as Bazel flag values on Windows.

See bazelbuild#4319

Change-Id: I39d81843015c5a4014dd5953bac2e1c29dcd5bed

bazel-io pushed a commit that referenced this issue Apr 26, 2018

windows,client: no longer support Unix-style paths
The Bazel client no longer supports MSYS paths.
The only exception is "/dev/null" which the client
treats as "NUL".

After this change you can no longer pass MSYS
paths as Bazel flag values on Windows.

See #4319

Change-Id: I39d81843015c5a4014dd5953bac2e1c29dcd5bed
PiperOrigin-RevId: 194372504

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue May 25, 2018

shell toolchain: rules and aspects now require it
Skylark rules, aspects, and native rules that need
a shell toolchain, now declare that they require
this toolchain.

See bazelbuild#4319

Change-Id: I7c18fc7b3b81cc18a0c43d3bd25e17797b5ecbd5

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue May 28, 2018

shell toolchain: genrule now uses it
genrule() now uses the shell toolchain (as well as
the ShellConfiguration config fragment) to look up
the shell interpreter's path.

In doing so, genrule() is now able to use the
shell interpreter registered as a toolchain, even
if there's no `--shell_executable` flag specified
(or its value is empty).

Subsequent commits will migrate more and more
rules to depend on the shell toolchain.

This commit takes us closer to:

1. be able to detect the local shell interpreter's
   location, especially when it's not the default
   /bin/bash

2. be able to define different shell toolchains
   (different interpreter paths) for remote builds

3. gracefully fail the build if the machine has no
   shell installed but the action graph included a
   shell action

See bazelbuild#4319

Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue May 28, 2018

shell toolchain: genrule now uses it
genrule() now uses the shell toolchain (as well as
the ShellConfiguration config fragment) to look up
the shell interpreter's path.

In doing so, genrule() is now able to use the
shell interpreter registered as a toolchain, even
if there's no `--shell_executable` flag specified
(or its value is empty).

Subsequent commits will migrate more and more
rules to depend on the shell toolchain.

This commit takes us closer to:

1. be able to detect the local shell interpreter's
   location, especially when it's not the default
   /bin/bash

2. be able to define different shell toolchains
   (different interpreter paths) for remote builds

3. gracefully fail the build if the machine has no
   shell installed but the action graph included a
   shell action

See bazelbuild#4319

Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue May 28, 2018

shell toolchain: genrule now uses it
genrule() now uses the shell toolchain (as well as
the ShellConfiguration config fragment) to look up
the shell interpreter's path.

Also the CommandHelper no longer looks up the
shell interpreter's path itself. Instead its ctor
takes the path as an argument. This allows
supporting rules that already use the shell
toolchain and rules that still only consider the
configuration fragment.

With these changes genrule() is now able to use
the shell interpreter registered as a toolchain,
even if there's no `--shell_executable` flag
specified (or its value is empty).

Subsequent commits will migrate more and more
rules to depend on the shell toolchain.

This commit takes us closer to:

1. be able to detect the local shell interpreter's
   location, especially when it's not the default
   /bin/bash

2. be able to define different shell toolchains
   (different interpreter paths) for remote builds

3. gracefully fail the build if the machine has no
   shell installed but the action graph included a
   shell action

See bazelbuild#4319

Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue May 28, 2018

CommandHelper: do not look up shell's path
The CommandHelper no longer looks up the shell
interpreter's path itself. Instead its ctor takes
the path as an argument.

This change will allow incrementally migrating
rules that use CommandHelper to start depending on
the shell toolchain.

Shell-using rules today only use the
ShellConfiguration config fragment to look up the
shell's path. In the future, more and more rules
will also retrieve the active shell toolchain, and
be able to:

1. use the auto-detected local shell interpreter's
   location, especially when it's not the default
   /bin/bash

2. use define different shell toolchains
  (different interpreter paths) for remote builds

3. gracefully fail the build if the machine has no
   shell installed but the action graph included a
   shell action

See bazelbuild#4319

Change-Id: I4da4e77e7d1fe57e8e4f5eb8820d03a840915e20

bazel-io pushed a commit that referenced this issue May 28, 2018

CommandHelper: do not look up shell's path
The CommandHelper no longer looks up the shell
interpreter's path itself. Instead its ctor takes
the path as an argument.

This change will allow incrementally migrating
rules that use CommandHelper to start depending on
the shell toolchain.

Shell-using rules today only use the
ShellConfiguration config fragment to look up the
shell's path. In the future, more and more rules
will also retrieve the active shell toolchain, and
be able to:

1. use the auto-detected local shell interpreter's
   location, especially when it's not the default
   /bin/bash

2. use define different shell toolchains
  (different interpreter paths) for remote builds

3. gracefully fail the build if the machine has no
   shell installed but the action graph included a
   shell action

See #4319

Change-Id: I4da4e77e7d1fe57e8e4f5eb8820d03a840915e20

Closes #5283.

Change-Id: I4da4e77e7d1fe57e8e4f5eb8820d03a840915e20
PiperOrigin-RevId: 198298315

laszlocsomor added a commit to laszlocsomor/bazel that referenced this issue May 29, 2018

shell toolchain: genrule now uses it
genrule() now uses the shell toolchain (as well as
the ShellConfiguration config fragment) to look up
the shell interpreter's path.

Also the CommandHelper no longer looks up the
shell interpreter's path itself. Instead its ctor
takes the path as an argument. This allows
supporting rules that already use the shell
toolchain and rules that still only consider the
configuration fragment.

With these changes genrule() is now able to use
the shell interpreter registered as a toolchain,
even if there's no `--shell_executable` flag
specified (or its value is empty).

Subsequent commits will migrate more and more
rules to depend on the shell toolchain.

This commit takes us closer to:

1. be able to detect the local shell interpreter's
   location, especially when it's not the default
   /bin/bash

2. be able to define different shell toolchains
   (different interpreter paths) for remote builds

3. gracefully fail the build if the machine has no
   shell installed but the action graph included a
   shell action

See bazelbuild#4319

Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7

Closes bazelbuild#5282.

Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7
PiperOrigin-RevId: 198288778

bazel-io pushed a commit that referenced this issue May 29, 2018

shell toolchain: genrule now uses it
genrule() now uses the shell toolchain (as well as
the ShellConfiguration config fragment) to look up
the shell interpreter's path.

Also the CommandHelper no longer looks up the
shell interpreter's path itself. Instead its ctor
takes the path as an argument. This allows
supporting rules that already use the shell
toolchain and rules that still only consider the
configuration fragment.

With these changes genrule() is now able to use
the shell interpreter registered as a toolchain,
even if there's no `--shell_executable` flag
specified (or its value is empty).

Subsequent commits will migrate more and more
rules to depend on the shell toolchain.

This commit takes us closer to:

1. be able to detect the local shell interpreter's
   location, especially when it's not the default
   /bin/bash

2. be able to define different shell toolchains
   (different interpreter paths) for remote builds

3. gracefully fail the build if the machine has no
   shell installed but the action graph included a
   shell action

See #4319

Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7

Closes #5282.

Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7
PiperOrigin-RevId: 198394021

bazel-io pushed a commit that referenced this issue May 30, 2018

Automated rollback of commit 7fcbc8f.
*** Reason for rollback ***

According to a post-submit regression test, this
commit increased Bazel's memory usage slightly
beyond some threshold. That event prompted me to
consider and question the necessity of this
commit.

My goal is to make Bazel work on Windows without
requiring MSYS, failing the build only if an
action is a shell action. Toolchainifying the
shell is related to, but not required by this
effort. What is required is to fail the build when
Bazel tries to create a shell action but the shell
is missing, and we have that already with
ShToolchain.getPath (which only considers the
ShellConfiguration fragment).

What's missing for my goal is to reimplement
hardcoded shell actions
(i.e. SpawnAction.builder().setShellCommand(...))
without using the shell where possible.

*** Original change description ***

shell toolchain: genrule now uses it

genrule() now uses the shell toolchain (as well as
the ShellConfiguration config fragment) to look up
the shell interpreter's path.

Also the CommandHelper no longer looks up the
shell interpreter's path itself. Instead its ctor
takes the path as an argument. This allows
supporting rules that already use the shell
toolchain and rules that still only consider the
configuration fragment.

With these changes genrule() is now able to use
the shell interpreter registered as a toolchain,
even if there's no `--shell_executable` flag
specified (or its value is empty).

Subsequent commits will migrate more and more
rules to depend on the shell toolchain.

This commit takes us closer to:

1. be able to detect the local shell interpreter's
   location, especially when it's not the default
   /bin/bash

2. be able to define different shell toolchains
   (different interpreter paths) for remote builds

3. gracefully fail the build if the machine has no
   shell installed but the action graph included a
   shell action

See #4319

Change-Id: I0674c7e2d5917643d87b48b19a1cb43e606ad2f7

Closes #5282.

***

RELNOTES: none
PiperOrigin-RevId: 198527351

@laszlocsomor laszlocsomor referenced this issue Jul 3, 2018

Open

Windows: Bash-less test execution #5508

3 of 6 tasks complete
@EdSchouten

This comment has been minimized.

Copy link
Contributor

EdSchouten commented Oct 7, 2018

Hi László,

Thanks for working on this kind of stuff. I managed to get Bazel working somewhat all right on Windows, performing build actions on a Linux-based build cluster. Getting Bazel to work with fewer local dependencies (e.g., msys2) would be very nice to have for us.

Based on your design document, I was wondering the following. Has there ever been any thought of tackling this problem by -- hold your breath -- deprecating features such as genrule() and ctx.actions.run_shell() entirely? Grepping for these functions in the Bazel codebase and some of the commonly used set of rules_*, I see that they are very often used to accomplish the following:

  • To run commands with certain environment variables set. This may also be accomplished using ctx.actions.run(env=...).
  • To run commands in different working directories. Though this is not configurable through ctx.actions.run(), the Remote Execution v2 protocol does support it. Maybe ctx.actions.run() could expose that functionality as well?
  • To invoke basic file manipulation shell commands (cp, sort, grep, sed).

The problem with the last use case is that genrule() effectively makes invoking shell commands so easy, that people tend to copy these declarations around. It seems to stimulate copy and paste programming. My guess is that if genrule() didn't exist and people always had to use the more verbose rule() function, someone would eventually stand up and create a rules_file_manipulation library to abstract away from all of those shell commands. Porting such a library to Windows would be a one-time effort.

One could also argue that using genrule() to create a byte-for-byte copy of a file shows the lack of an alias()-like function for output files. With Bazel's data model, both output files could refer to the same object in the build graph.

My feeling is that as long as Bazel's core language provides features for directly invoking UNIX shell commands, non-UNIX operating systems will always remain second class systems in terms of support. Somewhat related to this, removing genrule() is also a good step towards phasing out bazel-genfiles, which apparently is deprecated already.

Edit: Skylib may be gaining a concat function at some point. It may be a good idea to add bindings for basic file manipulation there as well?

@laszlocsomor

This comment has been minimized.

Copy link
Contributor

laszlocsomor commented Oct 11, 2018

Hi Ed,

Thanks for your insightful comments!

Indeed genrule/run_shell are solid sources or portability problems, however I believe deprecating them is currently infeasible. (I'm not attempting to speculate about future feasibility.) A more realistic-looking approach is to implement common functionality in Starlark and to expose them through repository_ctx/ctx. To the common tasks you mentioned, let me add rm, zip/unzip, tar/untar, and diff. The list could be even longer.

Skylib may not be the right place to add such functionality, as it'd introduce dependency on binutils / cmd.exe. Bazel's Starlark API seems like a better place.

laszlocsomor added a commit to laszlocsomor/bazel-skylib that referenced this issue Nov 30, 2018

maprule: add implementation and tests
maprule() is an improved version of
native.genrule(), with the following advantages:

- maprule can process source files in parallel,
  creating separate actions for each of them

- maprule does not require declaring all output
  files; it just requires declaring output
  templates, specifying the per-file outputs for
  each source

- maprule supports specifying the command's
  language, allowing Bash or cmd.exe syntax

- maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting (especially with cmd.exe
  syntax, where passing empty arguments is
  impossible); these paths can be passed as
  envvars instead  -- TODO: this is just wishful

- maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths to the command

See bazelbuild/bazel#4319

laszlocsomor added a commit to laszlocsomor/bazel-skylib that referenced this issue Dec 3, 2018

maprule: add implementation and tests
maprule() is an improved version of
native.genrule(), with the following advantages:

- maprule can process source files in parallel,
  creating separate actions for each of them

- maprule does not require declaring all output
  files; it just requires declaring output
  templates, specifying the per-file outputs for
  each source

- maprule supports specifying the command's
  language, allowing Bash or cmd.exe syntax

- maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting (especially with cmd.exe
  syntax, where passing empty arguments is
  impossible); these paths can be passed as
  envvars instead  -- TODO: this is just wishful

- maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths to the command

See bazelbuild/bazel#4319

laszlocsomor added a commit to laszlocsomor/bazel-skylib that referenced this issue Dec 3, 2018

maprule: add implementation and tests
maprule() is an improved version of
native.genrule(), with the following advantages:

- Maprule can process source files in parallel,
  creating separate actions for each of them.

- Maprule does not require declaring all output
  files. Instead you declare templates for the
  output files produces for each source. Therefore
  N source files and M templates produce N*M
  outputs.

- Maprule supports specifying the command's
  language, allowing Bash or cmd.exe syntax.

- Maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting. (Particluarly in case
  of cmd.exe passing empty arguments is
  impossible). These paths can be passed as
  envvars instead.

- Maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths of labels in "tools" or "srcs" (the
  shared sources available for all actions) to the
  command.

See bazelbuild/bazel#4319

laszlocsomor added a commit to laszlocsomor/bazel-skylib that referenced this issue Dec 5, 2018

maprule: add implementation and tests
maprule() is an improved version of
native.genrule(), with the following advantages:

- Maprule can process source files in parallel,
  creating separate actions for each of them.

- Maprule does not require declaring all output
  files. Instead you declare templates for the
  output files produces for each source. Therefore
  N source files and M templates produce N*M
  outputs.

- Maprule supports both Bash or cmd.exe syntax for
  its commands via the specialized rules
  bash_maprule and cmd_maprule.

- Maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting. (Particluarly in case
  of cmd.exe passing empty arguments is
  impossible). These paths can be passed as
  envvars instead.

- Maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths of labels in "tools" or "srcs" (the
  shared sources available for all actions) to the
  command.

See bazelbuild/bazel#4319

laszlocsomor added a commit to laszlocsomor/bazel-skylib that referenced this issue Dec 5, 2018

maprule: add implementation and tests
maprule() is an improved version of
native.genrule(), with the following advantages:

- Maprule can process source files in parallel,
  creating separate actions for each of them.

- Maprule does not require declaring all output
  files. Instead you declare templates for the
  output files produces for each source. Therefore
  N source files and M templates produce N*M
  outputs.

- Maprule supports both Bash or cmd.exe syntax for
  its commands via the specialized rules
  bash_maprule and cmd_maprule.

- Maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting. (Particluarly in case
  of cmd.exe passing empty arguments is
  impossible). These paths can be passed as
  envvars instead.

- Maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths of labels in "tools" or "srcs" (the
  shared sources available for all actions) to the
  command.

See bazelbuild/bazel#4319

laszlocsomor added a commit to laszlocsomor/bazel-skylib that referenced this issue Dec 6, 2018

maprule: add implementation and tests
maprule() is an improved version of
native.genrule(), with the following advantages:

- Maprule can process source files in parallel,
  creating separate actions for each of them.

- Maprule does not require declaring all output
  files. Instead you declare templates for the
  output files produces for each source. Therefore
  N source files and M templates produce N*M
  outputs.

- Maprule supports both Bash or cmd.exe syntax for
  its commands via the specialized rules
  bash_maprule and cmd_maprule.

- Maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting. (Particluarly in case
  of cmd.exe passing empty arguments is
  impossible). These paths can be passed as
  envvars instead.

- Maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths of labels in "tools" or "srcs" (the
  shared sources available for all actions) to the
  command.

See bazelbuild/bazel#4319

laszlocsomor added a commit to laszlocsomor/bazel-skylib that referenced this issue Dec 6, 2018

maprule: add implementation and tests
maprule() is an improved version of
native.genrule(), with the following advantages:

- Maprule can process source files in parallel,
  creating separate actions for each of them.

- Maprule does not require declaring all output
  files. Instead you declare templates for the
  output files produces for each source. Therefore
  N source files and M templates produce N*M
  outputs.

- Maprule supports both Bash or cmd.exe syntax for
  its commands via the specialized rules
  bash_maprule and cmd_maprule.

- Maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting. (Particluarly in case
  of cmd.exe passing empty arguments is
  impossible). These paths can be passed as
  envvars instead.

- Maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths of labels in "tools" or "srcs" (the
  shared sources available for all actions) to the
  command.

See bazelbuild/bazel#4319

laszlocsomor added a commit to laszlocsomor/bazel-skylib that referenced this issue Jan 3, 2019

maprule: add implementation and tests
maprule() is an improved version of
native.genrule(), with the following advantages:

- Maprule can process source files in parallel,
  creating separate actions for each of them.

- Maprule does not require declaring all output
  files. Instead you declare templates for the
  output files produces for each source. Therefore
  N source files and M templates produce N*M
  outputs.

- Maprule supports both Bash or cmd.exe syntax for
  its commands via the specialized rules
  bash_maprule and cmd_maprule.

- Maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting. (Particluarly in case
  of cmd.exe passing empty arguments is
  impossible). These paths can be passed as
  envvars instead.

- Maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths of labels in "tools" or "srcs" (the
  shared sources available for all actions) to the
  command.

See bazelbuild/bazel#4319

laszlocsomor added a commit to bazelbuild/bazel-skylib that referenced this issue Jan 8, 2019

maprule: an improved version of genrule() (#86)
maprule() is an improved version of
native.genrule(), with the following advantages:

- Maprule can process source files in parallel,
  creating separate actions for each of them.

- Maprule does not require declaring all output
  files. Instead you declare templates for the
  output files yielded for each source. Therefore
  N source files and M templates yield N*M
  outputs.

- Maprule supports both Bash and cmd.exe syntax
  for its commands via the specialized rules
  bash_maprule and cmd_maprule.

- Maprule's cmd attribute does deliberately not
  support $(location) expression nor Make
  Variables, in order to avoid issues and
  challenges with quoting. (In case of cmd.exe
  passing empty arguments is impossible). These
  paths can be passed as envvars instead.

- Maprule's add_env attribute does support
  $(location) expressions (and some extra
  placeholders) and is the idiomatic way to pass
  execpaths of labels in "tools" or "srcs" (the
  shared sources available for all actions) to the
  command.

See bazelbuild/bazel#4319
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment