The memory limit was exceeded

[pdxen-jisun]

메모리 에러가 떴는데 어떻게 해결해야 하나요?

>nsml run -d nia_dm_prelim
time="2021/11/21 11:18:52.154" level=info msg=".nsmlignore check - start"
time="2021/11/21 11:18:52.238" level=info msg=".nsmlignore check - done"
time="2021/11/21 11:18:52.330" level=info msg="file integrity check - start"
time="2021/11/21 11:18:52.334" level=info msg="file integrity check - done"
time="2021/11/21 11:18:52.337" level=info msg="LICENSE-THIRD-PARTY.txt 30 KiB - start"
time="2021/11/21 11:18:52.340" level=info msg="LICENSE-THIRD-PARTY.txt 30 KiB - done (1/5 20.00%) (30 KiB/60 KiB 48.85%)"
time="2021/11/21 11:18:52.346" level=info msg="README.hack.txt 557 B - start"
time="2021/11/21 11:18:52.349" level=info msg="README.hack.txt 557 B - done (2/5 40.00%) (30 KiB/60 KiB 49.75%)"
time="2021/11/21 11:18:52.354" level=info msg="README.md 2.0 KiB - start"
time="2021/11/21 11:18:52.356" level=info msg="README.md 2.0 KiB - done (3/5 60.00%) (32 KiB/60 KiB 53.04%)"
time="2021/11/21 11:18:52.361" level=info msg="main.py 28 KiB - start"
time="2021/11/21 11:18:52.364" level=info msg="main.py 28 KiB - done (4/5 80.00%) (60 KiB/60 KiB 99.47%)"
time="2021/11/21 11:18:52.370" level=info msg="setup.py 328 B - start"
time="2021/11/21 11:18:52.372" level=info msg="setup.py 328 B - done (5/5 100.00%) (60 KiB/60 KiB 100.00%)"
...Rejected: The memory limit was exceeded (max: 57GB, usage: 51GB, requested: 25GB)
Since cluster does not have enough computing resources, job has been aborted. Please retry later
Error: No machine available that meets ALL your request such as CPU, RAM, Disk, GPU or its driver version
time="2021/11/21 11:18:53.165" level=fatal msg="Internal server error"

[DatathonInfo]

안녕하세요 참가자님,

팀원 중 한분께서 현재 submit 중이신것 같습니다.

감사합니다.
A.I.D.D. 2021 운영팀 드림