Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature request: repository_ctx.copy #11858

Open
guibou opened this issue Jul 28, 2020 · 7 comments
Open

feature request: repository_ctx.copy #11858

guibou opened this issue Jul 28, 2020 · 7 comments
Labels
not stale Issues or PRs that are inactive but not considered stale P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request

Comments

@guibou
Copy link
Contributor

guibou commented Jul 28, 2020

Description of the problem / feature request:

I'd like a repository_ctx.copy function, to copy file. I'm currently using repository_ctx.template, but that's an abuse of the API and it does not work with binariy files: #11857

Feature requests: what underlying problem are you trying to solve with this feature?

Coping file in repository rules. I can use execute(["cp", ...), but a copy api can be more convenient and more robust as it does not depend on the availability of a cp command.

@oquenchil oquenchil added team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website type: feature request untriaged labels Jul 28, 2020
@alandonovan
Copy link
Contributor

The Java implementation of Starlark has no datatype capable of holding byte strings, so template will indeed not work for binary file copies. (Technically strings can hold bytes, but only because of Bazel's regrettable use of Latin-1 to interpret file names and file contents, which means that bytes in file names and BUILD/.bzl files are internally represented as the chars of a Java string. This is a deep and pervasive bug that requires a deep and pervasive fix.)

A copy function seems reasonable, but I have very little experience with the repository_rule API; someone more knowledgeable should review it. Tony? @aiuto

@mai93
Copy link
Contributor

mai93 commented Sep 7, 2020

@guibou, as a workaround for the lack of the copy API, you can use the following to copy binary files without corrupting them.

ctx.file(TARGET_FILENAME, ctx.read(SOURCE_FILENAME), executable=True, legacy_utf8=False)

However, it may not be efficient in case of very large files.

@philwo philwo added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels Nov 25, 2020
@crncnnr
Copy link

crncnnr commented Mar 12, 2021

I would also appreciate if this feature could be added, it would be a better way of copying binary files.

@guibou
Copy link
Contributor Author

guibou commented Mar 21, 2021

@mai93 Thank you for your workaround. Is there a way to "copy" the executable flag of the source file, instead of setting it unconditionally at True as your example does?

guibou added a commit to tweag/rules_nixpkgs that referenced this issue Mar 21, 2021
This closes #136

`repository_ctx` is meant to work on "text" files and does text encoding
and decoding, which breaks in the context of a binary file.

There is no `cp` API in repository_ctx, as suggested in
bazelbuild/bazel#11858, but
bazelbuild/bazel#11857 suggested to use this
approach with `repository_ctx.file`.
@guibou
Copy link
Contributor Author

guibou commented Mar 21, 2021

I've found how to copy the executable bit, see https://github.com/tweag/rules_nixpkgs/pull/161/files

@philwo philwo removed the team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website label Nov 29, 2021
@github-actions
Copy link

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

@github-actions github-actions bot added the stale Issues or PRs that are stale (no activity for 30 days) label May 24, 2023
@fmeum
Copy link
Collaborator

fmeum commented May 24, 2023

@bazelbuild/triage not stale

@sgowroji sgowroji added not stale Issues or PRs that are inactive but not considered stale and removed stale Issues or PRs that are stale (no activity for 30 days) labels May 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
not stale Issues or PRs that are inactive but not considered stale P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Projects
None yet
Development

No branches or pull requests

8 participants