Can i change ARCHIVE_TOTAL_SIZE_LIMIT ?

[morihaya]

Hi there.

I got below error when entered digdag push .

error: Status code 400: {"message":"Total size of the archive exceeds limit (2378971 > 2097152 bytes)","status":400}

I think this cause is ARCHIVE_TOTAL_SIZE_LIMIT from that code.

I want set more big size limit than 2MB.
Is it difficult?

[muga]

Thanks. At least we can parameterize those limits instead of hard-coded.

[muga]

#880 could fix this issue and will be released as v0.9.31. https://github.com/treasure-data/digdag/milestone/10

[morihaya]

@muga

I set these params.

# Set max size to over 2MB
max_attempts_page_size = 40971520000
max_sessions_page_size = 40971520000

And i tried this commands. Then same error occurred.

embulk@test-morihaya]$ du -sh
94M     .
embulk@test-morihaya]$ push morihaya
018-12-10 21:35:32 +0900: Digdag v0.9.31
Creating .digdag/tmp/archive-438016822003144009.tar.gz...
  Archiving sql/select.sql
  Archiving include/secret.dig
..
Workflows:
  test-morihaya.dig
error: Status code 400: {"message":"Size of the uploaded archive file exceeds limit (2097152 bytes)","status":400}

Should i set other params?

[morihaya]

I'm sorry but I set the wrong parameters.
I tried these parameters. But a similar error.

api.max_attempts_page_size = 20971520
api.max_sessions_page_size = 20971520

embulk@test-morihaya]$ du -sh
9.4M    .
embulk@test-morihaya]$ push morihaya
....
error: Status code 400: {"message":"Total size of the archive exceeds limit (9172719 > 2097152 bytes)","status":400}

[serihiro]

@muga
Hi, muga-san.

I think the problem reported in this issue does not concern #880 .
#880 parameterized the max limit size of GET API, not POST API.

In order to parameterize the max limit size of Project, ARCHIVE_TOTAL_SIZE_LIMIT in https://github.com/treasure-data/digdag/blob/master/digdag-server/src/main/java/io/digdag/server/rs/ProjectResource.java#L630-L634 is needed to be configurable, like here https://github.com/treasure-data/digdag/blob/master/digdag-server/src/main/java/io/digdag/server/rs/AttemptResource.java#L80

e.g.

ARCHIVE_TOTAL_SIZE_LIMIT = systemConfig.get("api.max_achive_total_size_limit", Integer.class, DEFAULT_ARCHIVE_TOTAL_SIZE_LIMIT);

[morihaya]

Great Thanks to everyone !
I was able to successfully push the 17 MB directory.

I setted below parameter.

api.max_archive_total_size_limit = 20971520

This issue close.