digdag ui can’t extract well files from GNU tar ball

[RASBORA]

Hi.

I’m using GNU tar ball when pushing project to digdag server.

tar zcvf project-ball.tar.gz conf/ src/ ./*.dig
digdag upload project-ball.tar.gz example-project -e "http://digdag.server.local"

It works well at digdag server, but digdag ui some doesn't work well.
The edit workflow page list up unneeded blank files.

I guess digdag ui is not considering tar type flag.
Accordingly, ProjectArchive class had better exclude not 0 type flag files.
https://github.com/treasure-data/digdag/blob/master/digdag-ui/console.jsx#L2317

[hiroyuki-sato]

Hello, @RASBORA

Thank you for your reporting.
Could you tell me the following?

• OS Version:
• Digdag version:
• Java version:

I would like to try the same steps in my environment.
Do you have any example project file?

[RASBORA]

Hello, @hiroyuki-sato
Thank you for reply.

Environment

My digdag instance is running on docker.

root@a6aca8305f5f:/# java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
root@a6aca8305f5f:/# cat /etc/debian_version
10.3

Example project

Steps are on README.md, I'd like you to read it.
https://github.com/RASBORA/reproduction-digdag-environment

[hiroyuki-sato]

Hello, @RASBORA

Thank you for your reply.
I reproduced and understand the issue.

Digdag UI shows the edit page even if the content is a directory.

I seem that this issue doesn't relate to GNU tar.
I reproduced the same on the macOS and bsdtar 2.8.3

Have you ever tried tar zcvf project-ball.tar.gz wf.dig folder/task.dig?
(select file path instead of the directory path.)
It seems that you select directories instead of the contents of a directory.

Digdag UI needs to ignore if a path is a directory.

@szyn Could you check this when you get a chance?

Reproduce steps

digdag server -m

cd /tmp
git clone https://github.com/RASBORA/reproduction-digdag-environment.git
cd reproduction-digdag-environment/project/
tar zcvf project-ball.tar.gz wf.dig folder/ folder2/
a wf.dig
a folder
a folder/task.dig
a folder2
a folder2/.gitkeep
digdag upload project-ball.tar.gz example-project

Access 'http://localhost:65432/projects/1/edit'

Environment.

tar --version
bsdtar 2.8.3 - libarchive 2.8.3

• bsdtar 2.8.3
• OS: macOS 10.14.6
• Java: 1.8.0_192
• Digdag: 0.9.41

[yoyama]

There is no activity. Close.