Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resource filtering behaviour and documentation could be improved #2261

Open
bric3 opened this issue Jun 8, 2017 · 1 comment
Open

Resource filtering behaviour and documentation could be improved #2261

bric3 opened this issue Jun 8, 2017 · 1 comment
Labels
a:feature A new functionality in:resource-handler resources files

Comments

@bric3
Copy link
Contributor

bric3 commented Jun 8, 2017

Gradle will fail to expand ant / maven / spring properties, with obscure reason when you discover Gradle after 10 years of usage with properties in this style ${x.y.z}.

Expected Behavior

Given files with dotted properties surrounded with ${ }

whatever.prop.sub=${integration.prop.sub.x.y.z}

and filtering task like

processTestResources {
    expand 'integration.prop.sub.x.y.z': 'blue'
}

and running the command

gradle clean processTestResources

I expect either

  • gradle to fail, but
    • to explain what was wrong the property that could not be filtered
    • and to suggest valid properties in the filtered file and in the build script
  • gradle to try other expansion mechanism than the groovy one ; yet it's probably good to make these expansion mechanism configurable, and in this case fail the build but suggest the other expansion mechanism

Current Behavior

Today such build fail, saying something is wrong

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':processTestResources'.
> Could not copy file '<edited>/test.properties' to '<edited>/test.properties'.

but you have to dig further in gradle to understand what went wrong by adding --stacktrace

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':processTestResources'.
> Could not copy file '/Users/brice/work/renault/xpack-oauth-realm/src/test/resources/test.properties' to '/Users/brice/work/renault/xpack-oauth-realm/build/resources/test/test.properties'.

* Try:
Run with --info or --debug option to get more log output.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':processTestResources'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:98)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:68)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:88)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.execute(DefaultTaskGraphExecuter.java:236)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.execute(DefaultTaskGraphExecuter.java:228)
        at org.gradle.internal.Transformers$4.transform(Transformers.java:169)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:106)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:61)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:228)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:215)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:77)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:58)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:32)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:113)
        at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
        at org.gradle.execution.DefaultBuildExecuter.access$000(DefaultBuildExecuter.java:23)
        at org.gradle.execution.DefaultBuildExecuter$1.proceed(DefaultBuildExecuter.java:43)
        at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:30)
        at org.gradle.initialization.DefaultGradleLauncher$RunTasksAction.execute(DefaultGradleLauncher.java:230)
        at org.gradle.initialization.DefaultGradleLauncher$RunTasksAction.execute(DefaultGradleLauncher.java:227)
        at org.gradle.internal.Transformers$4.transform(Transformers.java:169)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:106)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:56)
        at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:161)
        at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:112)
        at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:95)
        at org.gradle.launcher.exec.GradleBuildController.run(GradleBuildController.java:66)
        at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
        at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:41)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
        at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:75)
        at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:49)
        at org.gradle.tooling.internal.provider.ServicesSetupBuildActionExecuter.execute(ServicesSetupBuildActionExecuter.java:49)
        at org.gradle.tooling.internal.provider.ServicesSetupBuildActionExecuter.execute(ServicesSetupBuildActionExecuter.java:31)
        at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:67)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:37)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:26)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:34)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:74)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:72)
        at org.gradle.util.Swapper.swap(Swapper.java:38)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:72)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.LogAndCheckHealth.execute(LogAndCheckHealth.java:55)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:72)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
        at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:297)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:46)
Caused by: org.gradle.api.GradleException: Could not copy file '/Users/brice/work/renault/xpack-oauth-realm/src/test/resources/test.properties' to '/Users/brice/work/renault/xpack-oauth-realm/build/resources/test/test.properties'.
        at org.gradle.api.internal.file.AbstractFileTreeElement.copyTo(AbstractFileTreeElement.java:79)
        at org.gradle.api.internal.file.copy.DefaultFileCopyDetails.copyTo(DefaultFileCopyDetails.java:104)
        at org.gradle.api.internal.file.copy.FileCopyAction$FileCopyDetailsInternalAction.processFile(FileCopyAction.java:44)
        at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator$1$1.processFile(NormalizingCopyActionDecorator.java:66)
        at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator$1$1.processFile(DuplicateHandlingCopyActionDecorator.java:60)
        at org.gradle.api.internal.file.copy.CopyFileVisitorImpl.processFile(CopyFileVisitorImpl.java:62)
        at org.gradle.api.internal.file.copy.CopyFileVisitorImpl.visitFile(CopyFileVisitorImpl.java:46)
        at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:85)
        at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:58)
        at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker.walkDir(Jdk7DirectoryWalker.java:58)
        at org.gradle.api.internal.file.collections.DirectoryFileTree.walkDir(DirectoryFileTree.java:171)
        at org.gradle.api.internal.file.collections.DirectoryFileTree.visitFrom(DirectoryFileTree.java:149)
        at org.gradle.api.internal.file.collections.DirectoryFileTree.visit(DirectoryFileTree.java:134)
        at org.gradle.api.internal.file.collections.FileTreeAdapter.visit(FileTreeAdapter.java:110)
        at org.gradle.api.internal.file.CompositeFileTree.visit(CompositeFileTree.java:87)
        at org.gradle.api.internal.file.copy.CopySpecActionImpl.execute(CopySpecActionImpl.java:39)
        at org.gradle.api.internal.file.copy.CopySpecActionImpl.execute(CopySpecActionImpl.java:24)
        at org.gradle.api.internal.file.copy.DefaultCopySpec$DefaultCopySpecResolver.walk(DefaultCopySpec.java:648)
        at org.gradle.api.internal.file.copy.DefaultCopySpec$DefaultCopySpecResolver.walk(DefaultCopySpec.java:650)
        at org.gradle.api.internal.file.copy.DefaultCopySpec.walk(DefaultCopySpec.java:458)
        at org.gradle.api.internal.file.copy.DelegatingCopySpecInternal.walk(DelegatingCopySpecInternal.java:236)
        at org.gradle.api.internal.file.copy.CopySpecBackedCopyActionProcessingStream.process(CopySpecBackedCopyActionProcessingStream.java:38)
        at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator$1.process(DuplicateHandlingCopyActionDecorator.java:44)
        at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator$1.process(NormalizingCopyActionDecorator.java:57)
        at org.gradle.api.internal.file.copy.FileCopyAction.execute(FileCopyAction.java:35)
        at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator.execute(NormalizingCopyActionDecorator.java:53)
        at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator.execute(DuplicateHandlingCopyActionDecorator.java:42)
        at org.gradle.api.internal.file.copy.CopyActionExecuter.execute(CopyActionExecuter.java:40)
        at org.gradle.api.tasks.AbstractCopyTask.copy(AbstractCopyTask.java:179)
        at org.gradle.language.jvm.tasks.ProcessResources.copy(ProcessResources.java:35)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.doExecute(DefaultTaskClassInfoStore.java:141)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:134)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:123)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:692)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:675)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.execute(ExecuteActionsTaskExecuter.java:115)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.execute(ExecuteActionsTaskExecuter.java:109)
        at org.gradle.internal.Transformers$4.transform(Transformers.java:169)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:106)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:56)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:109)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:90)
        ... 70 more
Caused by: groovy.lang.MissingPropertyException: No such property: integ for class: SimpleTemplateScript25
        at SimpleTemplateScript25.run(SimpleTemplateScript25.groovy:1)
        at org.gradle.api.internal.file.copy.FilterChain$3.transform(FilterChain.java:121)
        at org.gradle.api.internal.file.copy.FilterChain$3.transform(FilterChain.java:110)
        at org.gradle.api.internal.ChainingTransformer.transform(ChainingTransformer.java:37)
        at org.gradle.api.internal.file.copy.FilterChain.transform(FilterChain.java:56)
        at org.gradle.api.internal.file.copy.FilterChain.transform(FilterChain.java:64)
        at org.gradle.api.internal.file.copy.DefaultFileCopyDetails.open(DefaultFileCopyDetails.java:88)
        at org.gradle.api.internal.file.AbstractFileTreeElement.copyTo(AbstractFileTreeElement.java:56)
        at org.gradle.api.internal.file.copy.DefaultFileCopyDetails.copyTo(DefaultFileCopyDetails.java:96)
        at org.gradle.api.internal.file.AbstractFileTreeElement.copyFile(AbstractFileTreeElement.java:93)
        at org.gradle.api.internal.file.AbstractFileTreeElement.copyTo(AbstractFileTreeElement.java:74)
        ... 112 more

Yet the message hidden in the last cause (MissingPropertyException: No such property: integ for class) is not explicit in how to fix the issue. You have to understand / know the groovy engine for that.

The documentation in both the migrating guide and in the reference are not very clear, and doesn't give examples of both the file being filtered and the build script.

Context

=> Porting a maven/ant project with heavy use of resource filtering to Gradle.
I am using Gradle for years, but with regular incompatibilities and with the prominence of maven project still out there, I still consider myself a noob on Gradle, so I'm regularly following migration guide and documentation.

I expected Gradle to support these kind of defacto standard from the start, having to modify many properties file, in the build script, and in the filtered files was not expected. In the end I preferred to rely on the MavenFileringHack of Elasticsearch. On a developer perspective I'm not a huge fan of very long camel cased properties, I find dotted properties much more readable.

Your Environment

  • Build scan URL: Can't publish the build scan, as behind firewall and the project is private.

Here's what I can give

  • Gradle Version : 3.5
  • OS Info : Mac OS X 10.12.5 (x86_64)
  • JAVA_HOME : /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home
@bmuschko bmuschko added a:feature A new functionality in:core DO NOT USE labels Jul 10, 2017
@damarvin
Copy link

We would need this expand dotted property names feature supported to avoid strange workarounds, e.g. WildFly configurations rely on such properties.

@bmuskalla bmuskalla added this to the Unscheduled milestone Jun 4, 2020
@jjohannes jjohannes removed this from the Unscheduled milestone Mar 19, 2021
@jbartok jbartok added in:invoking-gradle Running Executing in:execution-engine incremental, up-to-date, overlapping outputs in:resource-handler resources files and removed in:core DO NOT USE in:execution-engine incremental, up-to-date, overlapping outputs in:invoking-gradle Running Executing labels Dec 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
a:feature A new functionality in:resource-handler resources files
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@eriwen @bmuskalla @bmuschko @bric3 @jjohannes @jbartok @damarvin