Support Directories in SimpleBlobStoreActionCache

[rahul-malik]

Please provide the following information. The more we know about your system and use case, the more easily and likely we can help.

Description of the problem / feature request / question:

I'm trying to use a hazelcast remote rest cache for building an iOS application. Building the sources works correctly but building the entire application fails because it is unable to upload directory artifacts. It hits the below exception in SimpleBlobStoreActionCache:

      if (file.isDirectory()) {
        // TODO(olaola): to implement this for a directory, will need to create or pass a
        // TreeNodeRepository to call uploadTree.
        throw new UnsupportedOperationException("Storing a directory is not yet supported.");
      }

If possible, provide a minimal example to reproduce the problem:

Build an iOS application with image resources using asset catalogs. I can try to get a sample project together if this is difficult to reproduce.

Environment info

• Operating System:
  macOS Sierra 10.12.6

• Bazel version (output of bazel info release):
  release: release 0.7.0

Have you found anything relevant by searching the web?

(e.g. StackOverflow answers,
GitHub issues,
email threads on the bazel-discuss Google group)

Nope

Anything else, information or logs or outputs that would be helpful?

(If they are large, please upload as attachment or provide link).

Stacktrace:

FAILED: Build did NOT complete successfully
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ACTION_EXECUTION:ActionLookupData{actionLookupNode=CONFIGURED_TARGET://iOS:ShareExtension afb54407aa4fd85223f997d8abc68cb4 (115592980 1791500453), actionIndex=0}' (requested by nodes 'iOS/ShareExtension.resources/actool-PartialInfo.plist //iOS:ShareExtension afb54407aa4fd85223f997d8abc68cb4 (376652741 1791500453)')
	at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:476)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:352)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedOperationException: Storing a directory is not yet supported.
	at com.google.devtools.build.lib.remote.SimpleBlobStoreActionCache.upload(SimpleBlobStoreActionCache.java:176)
	at com.google.devtools.build.lib.remote.SimpleBlobStoreActionCache.upload(SimpleBlobStoreActionCache.java:152)
	at com.google.devtools.build.lib.remote.RemoteSpawnRunner.execLocally(RemoteSpawnRunner.java:261)
	at com.google.devtools.build.lib.remote.RemoteSpawnRunner.exec(RemoteSpawnRunner.java:125)
	at com.google.devtools.build.lib.remote.RemoteSpawnStrategy.exec(RemoteSpawnStrategy.java:157)
	at com.google.devtools.build.lib.analysis.actions.SpawnAction.internalExecute(SpawnAction.java:257)
	at com.google.devtools.build.lib.analysis.actions.SpawnAction.execute(SpawnAction.java:264)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeActionTask(SkyframeActionExecutor.java:844)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.prepareScheduleExecuteAndCompleteAction(SkyframeActionExecutor.java:787)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.access$900(SkyframeActionExecutor.java:106)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:655)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:612)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:402)
	at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:440)
	at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:201)
	at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:401)
	... 4 more
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ACTION_EXECUTION:ActionLookupData{actionLookupNode=CONFIGURED_TARGET://iOS:ShareExtension afb54407aa4fd85223f997d8abc68cb4 (115592980 1791500453), actionIndex=0}' (requested by nodes 'iOS/ShareExtension.resources/actool-PartialInfo.plist //iOS:ShareExtension afb54407aa4fd85223f997d8abc68cb4 (376652741 1791500453)')
	at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:476)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:352)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedOperationException: Storing a directory is not yet supported.
	at com.google.devtools.build.lib.remote.SimpleBlobStoreActionCache.upload(SimpleBlobStoreActionCache.java:176)
	at com.google.devtools.build.lib.remote.SimpleBlobStoreActionCache.upload(SimpleBlobStoreActionCache.java:152)
	at com.google.devtools.build.lib.remote.RemoteSpawnRunner.execLocally(RemoteSpawnRunner.java:261)
	at com.google.devtools.build.lib.remote.RemoteSpawnRunner.exec(RemoteSpawnRunner.java:125)
	at com.google.devtools.build.lib.remote.RemoteSpawnStrategy.exec(RemoteSpawnStrategy.java:157)
	at com.google.devtools.build.lib.analysis.actions.SpawnAction.internalExecute(SpawnAction.java:257)
	at com.google.devtools.build.lib.analysis.actions.SpawnAction.execute(SpawnAction.java:264)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeActionTask(SkyframeActionExecutor.java:844)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.prepareScheduleExecuteAndCompleteAction(SkyframeActionExecutor.java:787)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.access$900(SkyframeActionExecutor.java:106)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:655)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:612)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:402)
	at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:440)
	at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:201)
	at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:401)
	... 4 more

[rahul-malik]

@buchgr - Is the comment / TODO up-to-date? It mentions calling uploadTree but no such method exists in the repository.

Also how would you recommend going about fixing this issue?

[mterring]

A quick solution that I'd really like to see made before the next release would be to simply skip directories rather than fail the build.

That can be done by changing UnsupportedOperationException to IOException. I'd be happy to submit a pull request, but given how trivial the change is, it might be quicker to have an existing contributor make it.

[buchgr]

@rahul-malik So this comment seems out of date. It's both the up- and download of output directories that is currently not implemented. To learn about the details, take a look at the Action, ActionResult and Tree and OutputDirectory messages in the remote_execution.proto. The comments explain it pretty well.

To implement the upload you need to

1. Traverse the directory output recursively and create a Tree proto that corresponds to the directory structure.
2. Upload any files in the directory and subdirectories.
3. Upload the just created Tree protobuf.

To implement the download:

1. Fetch the proto messages that are referenced in Action.output_directories
2. This will be the Tree protobufs you uploaded. They describe the structure of the directories. You need to parse them and download each file.

Let me know if you would like to work on this!

@mterring we love contributions! 😃

[mterring]

We've started the CLA process for my company, I'll submit the pull request while that happens

[hchauvin]

@buchgr @rahul-malik This is a blocker for us. I see that tree traversal with digests is readily implemented in com.google.devtools.build.lib.remote.TreeNodeRepository, so I'm taking a stab at it.

[rahul-malik]

@hchauvin - Sounds great! Thanks for taking it on - happy to help if the surface of the change is too large

[buchgr]

Fixed by #4011