Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change workspace prefix not to use task name to avoid failing workspace creation #751

Merged
merged 3 commits into from Jun 15, 2018

Conversation

Projects
None yet
1 participant
@muga
Copy link
Member

muga commented Apr 7, 2018

This PR changes workspace prefix. The name includes task name. If the task name is long, workspace name is also long. Digdag (0.9.24) server cannot create workspace by "File name too long". This error happens when I tried to reproduce #729.

In this PR, workspace prefix uses workflow name combination with projectId, workflowName, sessionId and attemptId instead of task name. Since the workspace name is internal design so, users might not affect this change directly.

2018-04-06 22:24:23.075 +0000 [ERROR] (0142@[1:muga_digdag_issues_729_ng]+muga_digdag_issues_729_ng+repeat^sub+loop-0+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_aaa) io.digdag.core.agent.OperatorManager: Task failed with unexpected error: java.nio.file.FileSystemException: /tmp/digdag-tempdir6619218628117090050/workspace/+muga_digdag_issues_729_ng+repeat^sub+loop-0+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_aaa_7247227962002122889: File name too long
io.digdag.core.TempFileManager$AllocationException: java.nio.file.FileSystemException: /tmp/digdag-tempdir6619218628117090050/workspace/+muga_digdag_issues_729_ng+repeat^sub+loop-0+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_aaa_7247227962002122889: File name too long
        at io.digdag.core.TempFileManager.createTempDir(TempFileManager.java:105)
        at io.digdag.core.agent.ExtractArchiveWorkspaceManager.createNewWorkspace(ExtractArchiveWorkspaceManager.java:89)
        at io.digdag.core.agent.ExtractArchiveWorkspaceManager.lambda$withExtractedArchive$2(ExtractArchiveWorkspaceManager.java:59)
        at io.digdag.util.RetryExecutor.run(RetryExecutor.java:166)
        at io.digdag.util.RetryExecutor.run(RetryExecutor.java:142)
        at io.digdag.core.agent.ExtractArchiveWorkspaceManager.withExtractedArchive(ExtractArchiveWorkspaceManager.java:56)
        at io.digdag.core.agent.OperatorManager.runWithHeartbeat(OperatorManager.java:135)
        at io.digdag.core.agent.OperatorManager.run(OperatorManager.java:119)
        at io.digdag.core.agent.MultiThreadAgent.lambda$null$0(MultiThreadAgent.java:127)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException: /tmp/digdag-tempdir6619218628117090050/workspace/+muga_digdag_issues_729_ng+repeat^sub+loop-0+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_aaa_7247227962002122889: File name too long
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384)
        at java.nio.file.Files.createDirectory(Files.java:674)
        at java.nio.file.TempFileHelper.create(TempFileHelper.java:136)
        at java.nio.file.TempFileHelper.createTempDirectory(TempFileHelper.java:173)
        at java.nio.file.Files.createTempDirectory(Files.java:950)
        at io.digdag.core.TempFileManager.createTempDir(TempFileManager.java:102)
        ... 13 common frames omitted

@muga muga changed the title Change workspace name not to use task name to avoid failing workspace creation Change workspace prefix not to use task name to avoid failing workspace creation Apr 10, 2018

@muga

This comment has been minimized.

Copy link
Member Author

muga commented Apr 10, 2018

workspace prefix was changed with {projectId}_{workflowName}_{sessionId}_{attemptId}.

@muga muga changed the title Change workspace prefix not to use task name to avoid failing workspace creation [wip] Change workspace prefix not to use task name to avoid failing workspace creation Apr 24, 2018

@muga muga changed the title [wip] Change workspace prefix not to use task name to avoid failing workspace creation Change workspace prefix not to use task name to avoid failing workspace creation May 14, 2018

@muga muga added this to the v0.9.26 milestone May 16, 2018

@muga

This comment has been minimized.

Copy link
Member Author

muga commented Jun 15, 2018

Merging.

@muga muga merged commit 9c40a9e into treasure-data:master Jun 15, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.