Feature: java_library.resource_add_prefix

[BlackIsTheNewBlack]

Description of the feature request:

java_library has a resource_strip_prefix property which removes a path prefix. Please add the complementary property "resource_add_prefix" which would place files in a different folder.

Feature requests: what underlying problem are you trying to solve with this feature?

Many times resources need to be placed in specially named folders in order to work. For example, Java ServiceLoader resources have to be placed under META-INF/services. Ideally, in the source folders the resource file would be placed right near the {Service}.java it describes. Currently that is not possible, it has to be either placed in a META-INF/services subfolder (which increases the complexity of the directory structure) or in a dedicated resources/META-INF/services folder which aggreggates all service resource, which breaks modularity.

Without resource_add_prefix:
Service.java
META-INF/
services/
com.test.Service

With proposed resource_add_prefix='META-INF/services':
Service.java
com.test.Service

What operating system are you running Bazel on?

Windows 10

What's the output of bazel info release?

release 19.1

Have you found anything relevant by searching the web?

Nothing

[cushon]

ServiceLoader resources have to be placed under META-INF/services

Consider using AutoService for this. It's an annotation processor that generates the META-INF/services files for you.

Currently that is not possible

It's not possible using attributes on java_library, but if it's necessary to have full control over the resource layout you can generate a separate .jar, add it to the build using java_import, and then add the java_import to the deps or runtime_deps of the original java_library.

[ittaiz]

Also another option is to use a genrule with the zipper from bazel_tools

…

On Fri, 16 Nov 2018 at 2:57 Liam Miller-Cushon ***@***.***> wrote: ServiceLoader resources have to be placed under META-INF/services Consider using AutoService <https://github.com/google/auto/tree/master/service#autoservice> for this. It's an annotation processor that generates the META-INF/services files for you. Currently that is not possible It's not possible using attributes on java_library, but if it's necessary to have full control over the resource layout you can generate a separate .jar, add it to the build using java_import, and then add the java_import to the deps or runtime_deps of the original java_library. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#6683 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF__O0vujp66G1vZadIv7OSpH1PNQks5uvg1ygaJpZM4YfYsb> .

[BlackIsTheNewBlack]

@ittaiz
Thank you for the tip. Please notice that the zipper is not documented and requires two extra rules to manage. The same argument could be made for strip_resource_prefix, why have it when you can just merge an external archive?

@cushon
Actually I did not need it for ServiceLoader, it was just an example, I need it for a Vaadin web app, which actually requires 3 extra directory levels (META-INF/resources/frontend) and webjars (META-INF/resources/webjars/iron-...) . IMHO the java_library rule should be self contained if possible, strip_resource_prefix is half usable without its complementary resource_add_prefix.

Thank you for your time.

[cushon]

The same argument could be made for strip_resource_prefix, why have it when you can just merge an external archive?

I think it's worth considering deprecating resource_strip_prefix. @lberki do you remember the motivation for beccb9b? I wonder if it was to work around issues like #6353?

IMHO the java_library rule should be self contained if possible

I don't see this is a goal. I'd rather have slightly more rules that solve specific problems than having java_library try to do everything.

[ittaiz]

@rarsel, Did not mean to say your request isn’t valid just to offer a workaround. @cushon, Aren’t you concerned about the additional IO? One needs to unzip and zip. Maybe I’m zooming in too much.

…

On Mon, 19 Nov 2018 at 19:17 Liam Miller-Cushon ***@***.***> wrote: The same argument could be made for strip_resource_prefix, why have it when you can just merge an external archive? I think it's worth considering deprecating resource_strip_prefix <https://docs.bazel.build/versions/master/be/java.html#java_binary.resource_strip_prefix>. @lberki <https://github.com/lberki> do you remember the motivation for beccb9b <beccb9b>? I wonder if it was to work around issues like #6353 <#6353>? IMHO the java_library rule should be self contained if possible I don't see this is a goal. I'd rather have slightly more rules that solve specific problems than having java_library try to do everything. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6683 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIFzSs3lspql7_P8nA17DeSTgoeFsZks5uwueRgaJpZM4YfYsb> .

[BlackIsTheNewBlack]

Hello,

I think it's worth considering deprecating resource_strip_prefix.

Please do not deprecate resource_strip_prefix unless you provide a mechanism for controlling the paths layouts in the resulting jar, which should at least be able to... strip the prefix. The layout of resources in the JARs has to be separated from the logical layout of source files and resource file, because they are guided by different requirements.

IMHO the java_library rule should be self contained if possible
I don't see this is a goal. I'd rather have slightly more rules that solve specific problems than having java_library try to do everything.

How about this:
Explicitely map the layout of resources with a map of labels and some regex magic using capturing groups:

$STRIP='//some/embedding/root'

META_ROOT = 'META-INF/resources/frontend'

resources = glob(
{
"$STRIP/(**)/(*).properties.in": i18n("$META_ROOT/$1/$LANG/$2.properties"),
} )

In this example, the resource prefix is stripped, the destination root is changed, same for file extension, and it is also using custom logic for the localization language.

This approach can replace java_library.resources, resource_strip_prefix and classpath_resources properties, which would become obsolete, along with their limitations such as "This heuristic cannot be overridden.". #6353 can also be closed because the system wide configuration can be replaced with a specific map and some function calls.

In order to address the IO concern raised by @ittaiz, please consider adding a rule equivalent with rsync which would operate directly on such a map of source files to destination files, based in a dedicated directory instead of a zip.

[cushon]

Aren’t you concerned about the additional IO? One needs to unzip and zip.

The implementation of java_library.resources creates a separate action to add the resources to the output jar, to avoid re-running the compilation action if only the resources change. Creating a completely separate jar for the resources should be marginally less expensive than merging the resources into the library's output jar. And either way, creating the zip is almost free compared to the cost of compilation.

a mechanism for controlling the paths layouts in the resulting jar

I think the approach described earlier in the bug provides full control over the resource layout? #6683 (comment)

[BlackIsTheNewBlack]

I think the approach described earlier in the bug provides full control over the resource layout?

In my opinion users of Bazel should not be expected to program the build system in order to achieve common results. Composing sets of build outputs into a precise dir layout is a generic operation which deserves its own rule. It would greatly simplify the treatment of jar resources and web site builds in general.

My previous comment (#6683 (comment)) was to describe how such a layout mapping rule should look to the user. It is just a map of search,replace pairs, ie a function from the old path to the new path. Should the java_library rule depend on such a file_mapping rule it could replace the resource_strip_prefix and the proposed resource_add_prefix.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 2+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[github-actions]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please reach out to the triage team (@bazelbuild/triage). Thanks!