在 go-judge 上启动 MPI 程序

[ConnectionFailedd]

我在尝试使用 Hydro 搭建并行计算课程的 OJ 平台，使用到了 go-judge 作为后台 judge 服务器。然而，我无法正常使用 mpirun -np 8 foo 启动 MPI 程序。
我尝试直接向 localhost:5050/run 地址发送 POST 请求来启动 MPI 程序，得到了这样的输出：

[
    {
        "status": "Nonzero Exit Status",
        "exitStatus": 1,
        "time": 25031000,
        "memory": 7532544,
        "runTime": 27193668,
        "files": {
            "stderr": "[executor_server:00026] opal_ifinit: unable to find network interfaces.\n--------------------------------------------------------------------------\nmpirun has detected an attempt to run as root.\n\nRunning as root is *strongly* discouraged as any mistake (e.g., in\ndefining TMPDIR) or bug can result in catastrophic damage to the OS\nfile system, leaving your system in an unusable state.\n\nWe strongly suggest that you run mpirun as a non-root user.\n\nYou can override this protection by adding the --allow-run-as-root option\nto the cmd line or by setting two environment variables in the following way:\nthe variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this\nprotection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and\nadd one more layer of certainty that you want to do so.\nWe reiterate our advice against doing so - please proceed at your own risk.\n--------------------------------------------------------------------------\n",
            "stdout": ""
        }
    }
]

为方便查看，将 stderr 的部分展开如下：

[executor_server:00026] opal_ifinit: unable to find network interfaces.
--------------------------------------------------------------------------
mpirun has detected an attempt to run as root.

Running as root is *strongly* discouraged as any mistake (e.g., in
defining TMPDIR) or bug can result in catastrophic damage to the OS
file system, leaving your system in an unusable state.

We strongly suggest that you run mpirun as a non-root user.
You can override this protection by adding the --allow-run-as-root option
to the cmd line or by setting two environment variables in the following way:
the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
add one more layer of certainty that you want to do so.
We reiterate our advice against doing so - please proceed at your own risk.
--------------------------------------------------------------------------

根据这段报错的说法，我在启动 MPI 的命令中加入了 --allow-run-as-root 选项，然而，出现了新的问题：

[executor_server:00027] opal_ifinit: unable to find network interfaces.
hwloc/linux: failed to find sysfs cpu topology directory, aborting linux discovery.
[executor_server:00027] *** Process received signal ***
[executor_server:00027] Signal: Segmentation fault (11)
[executor_server:00027] Signal code: Address not mapped (1)
[executor_server:00027] Failing at address: 0x20
[executor_server:00027] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x45320)[0x7be55b445320]
[executor_server:00027] [ 1] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_home_directory+0x28)[0x7be5582e4ac8]
[executor_server:00027] [ 2] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_mca_base_var_cache_files+0x32)[0x7be5582e0d52]
[executor_server:00027] [ 3] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_mca_base_var_init+0x247)[0x7be5582e16a7]
[executor_server:00027] [ 4] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_init_util+0x89)[0x7be5582af609]
[executor_server:00027] [ 5] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_rte_init+0x77)[0x7be5582afc27]
[executor_server:00027] [ 6] /lib/x86_64-linux-gnu/libpmix.so.2(PMIx_server_init+0x30e)[0x7be558274d3e]
[executor_server:00027] [ 7] /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_pmix_ext3x.so(ext3x_server_init+0x251)[0x7be55ad97731]
[executor_server:00027] [ 8] /lib/x86_64-linux-gnu/libopen-rte.so.40(pmix_server_init+0x31f)[0x7be55b81d44f]
[executor_server:00027] [ 9] /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_ess_hnp.so(+0x3f18)[0x7be55b2c1f18]
[executor_server:00027] [10] /lib/x86_64-linux-gnu/libopen-rte.so.40(orte_init+0x2aa)[0x7be55b86d15a]
[executor_server:00027] [11] /lib/x86_64-linux-gnu/libopen-rte.so.40(orte_submit_init+0x911)[0x7be55b8170e1]
[executor_server:00027] [12] /usr/bin/mpirun(+0x11e8)[0x5c6e2379c1e8]
[executor_server:00027] [13] /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7be55b42a1ca]
[executor_server:00027] [14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7be55b42a28b]
[executor_server:00027] [15] /usr/bin/mpirun(+0x1415)[0x5c6e2379c415]
[executor_server:00027] *** End of error message ***

因此，我想要咨询的问题是：

• 我能否在 go-judge 上启动 MPI 程序？
• 如果可以，我应当如何启动？
• 最后，我能否在 go-judge 上不以 root 身份启动 MPI（或别的类型的）程序？

我所使用的 CPU 和操作系统为：

$ cat /etc/os-release 
PRETTY_NAME="Ubuntu Noble Numbat (development branch)"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
$ cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c
     80  Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz

我没有将 go-judge 安装到系统目录下；我启动 go-judge 的方法是：

$ nohup sudo ./go-judge/usr/bin/go-judge &
[1] 1104542
$ nohup: ignoring input and appending output to 'nohup.out'

最后，我使用 python 脚本向 go-judge 发送 POST 请求，脚本的内容为：

# compile.py
import requests
import json

url = 'http://localhost:5050/run'
data = {
    'cmd': [
        {
            'args': ['/usr/bin/mpic++', 'test.cpp', '-o', 'test'],
            'env': ['PATH=/usr/bin:/bin'],
            'files': [
                {'content': ''},
                {'name': 'stdout', 'max': 10240},
                {'name': 'stderr', 'max': 10240}
            ],
            'cpuLimit': 10000000000,
            'memoryLimit': 104857600,
            'procLimit': 50,
            'copyIn': {
                'test.cpp': {
                    'content': 
'''
#include "mpi.h"

int main(int argc, char ** argv) {
    MPI_Init(&argc, &argv);
    MPI_Barrier(MPI_COMM_WORLD);
    MPI_Finalize();
}
'''
                }
            },
            'copyOut': ['stdout', 'stderr'],
            'copyOutCached': ['test']
        }
    ]
}

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36 MicroMessenger/7.0.9.501 NetType/WIFI MiniProgramEnv/Windows WindowsWechat",
    "Accept-Encoding": "gzip, deflate",
    "Accept": "application/json, text/javascript, */*; q=0.01",
    "Connection": "keep-alive",
    "Content-Type": "application/json",
    "Host": "lazytools.feidee.cn"
}

response = requests.post(url, data=json.dumps(data), headers=headers)

j = json.loads(response.text)

print(json.dumps(j, indent=4))

import requests
import json
import argparse

parser = argparse.ArgumentParser(description='Delete a file')
parser.add_argument('--id', type=str, help='The file ID to delete')
args = parser.parse_args()

url = 'http://localhost:5050/run'
data = {
    'cmd': [
        {
            'args': ['mpirun', '-np', '8', '--allow-run-as-root', 'test'],
            'env': ['PATH=/usr/bin:/bin'],
            'files': [
                {'content': '1 1'},
                {'name': 'stdout', 'max': 10240},
                {'name': 'stderr', 'max': 10240}
            ],
            'cpuLimit': 10000000000,
            'memoryLimit': 104857600,
            'procLimit': 50,
            'copyIn': {
                'test': {'fileId': f'{args.id}'}
            }
        }
    ]
}

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36 MicroMessenger/7.0.9.501 NetType/WIFI MiniProgramEnv/Windows WindowsWechat",
    "Accept-Encoding": "gzip, deflate",
    "Accept": "application/json, text/javascript, */*; q=0.01",
    "Connection": "keep-alive",
    "Content-Type": "application/json",
    "Host": "lazytools.feidee.cn"
}

response = requests.post(url, data=json.dumps(data), headers=headers)

j = json.loads(response.text)
print(json.dumps(j, indent=4))

执行的指令及输出为：

$ python3 compile.py 
[
    {
        "status": "Accepted",
        "exitStatus": 0,
        "time": 822298000,
        "memory": 62218240,
        "runTime": 824558182,
        "files": {
            "stderr": "[executor_server:00012] opal_ifinit: unable to find network interfaces.\n",
            "stdout": ""
        },
        "fileIds": {
            "test": "7IJSAUUO"
        }
    }
]
$ python3 run.py --id 7IJSAUUO
[
    {
        "status": "Signalled",
        "exitStatus": 11,
        "time": 58446000,
        "memory": 7520256,
        "runTime": 253555407,
        "files": {
            "stderr": "[executor_server:00018] opal_ifinit: unable to find network interfaces.\nhwloc/linux: failed to find sysfs cpu topology directory, aborting linux discovery.\n[executor_server:00018] *** Process received signal ***\n[executor_server:00018] Signal: Segmentation fault (11)\n[executor_server:00018] Signal code: Address not mapped (1)\n[executor_server:00018] Failing at address: 0x20\n[executor_server:00018] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x45320)[0x7400e8845320]\n[executor_server:00018] [ 1] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_home_directory+0x28)[0x7400e56e4ac8]\n[executor_server:00018] [ 2] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_mca_base_var_cache_files+0x32)[0x7400e56e0d52]\n[executor_server:00018] [ 3] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_mca_base_var_init+0x247)[0x7400e56e16a7]\n[executor_server:00018] [ 4] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_init_util+0x89)[0x7400e56af609]\n[executor_server:00018] [ 5] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_rte_init+0x77)[0x7400e56afc27]\n[executor_server:00018] [ 6] /lib/x86_64-linux-gnu/libpmix.so.2(PMIx_server_init+0x30e)[0x7400e5674d3e]\n[executor_server:00018] [ 7] /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_pmix_ext3x.so(ext3x_server_init+0x251)[0x7400e848e731]\n[executor_server:00018] [ 8] /lib/x86_64-linux-gnu/libopen-rte.so.40(pmix_server_init+0x31f)[0x7400e8d0944f]\n[executor_server:00018] [ 9] /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_ess_hnp.so(+0x3f18)[0x7400e87aaf18]\n[executor_server:00018] [10] /lib/x86_64-linux-gnu/libopen-rte.so.40(orte_init+0x2aa)[0x7400e8d5915a]\n[executor_server:00018] [11] /lib/x86_64-linux-gnu/libopen-rte.so.40(orte_submit_init+0x911)[0x7400e8d030e1]\n[executor_server:00018] [12] /usr/bin/mpirun(+0x11e8)[0x63b8bae9b1e8]\n[executor_server:00018] [13] /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7400e882a1ca]\n[executor_server:00018] [14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7400e882a28b]\n[executor_server:00018] [15] /usr/bin/mpirun(+0x1415)[0x63b8bae9b415]\n[executor_server:00018] *** End of error message ***\n",
            "stdout": ""
        }
    }
]

[ConnectionFailedd]

对了，我使用的 MPI 版本是：

$ apt list openmpi-bin openmpi-doc libopenmpi-dev
Listing... Done
libopenmpi-dev/noble,now 4.1.6-7ubuntu2 amd64 [installed]
openmpi-bin/noble,now 4.1.6-7ubuntu2 amd64 [installed]
openmpi-doc/noble,now 4.1.6-7ubuntu2 all [installed]

go-judge 版本是 v1.8.2 release 中的 go-judge_1.8.2_linux_amd64.deb 版本，并使用 dpkg -X 命令直接解压到了家目录中。

[criyle]

目前在沙箱环境中是完全没有网络访问的（包括本地环回地址 127.0.0.1 或者 ::1）您可以 在测试环境中 尝试添加 -net-share 命令行参数使用主机的网络来测试以便分别是否造成故障的原因是沙箱没有网络。

此外，如果想调节沙箱内运行的用户，请将仓库中的 mount.yaml 拷贝到沙箱的工作目录或者使用 -mount-conf 参数指定该配置文件的位置。

其中 uid: 1536 可以手动更改为您期望的用户 id （使用非0值，0为root）

此外，报错内容尝试读取了 /sys/devices/system 内容，可以尝试在 mount.yaml 中挂载对应的主机目录，比如添加下面的控制项在列表的末尾：

  - type: bind
    source: /sys/devices/system
    target: /sys/devices/system
    readonly: true

[ConnectionFailedd]

我尝试将 /sys/devices/system 挂载到容器中，但是运行的时候失败了，报错信息如下：

2024-04-16T16:01:10.408+0800	INFO	go-judge/main.go:58	config loaded: &{ContainerInitPath: PreFork:1 TmpFsParam:size=128m,nr_inodes=4k NetShare:false MountConf:mount.yaml SeccompConf:seccomp.yaml Parallelism:80 CgroupPrefix:gojudge ContainerCredStart:0 SrcPrefix:[] Dir: TimeLimitCheckerInterval:100ms ExtraMemoryLimit:16.0 KiB OutputLimit:256.0 MiB CopyOutLimit:256.0 MiB OpenFileLimit:256 Cpuset: EnableCPURate:false CPUCfsPeriod:100ms FileTimeout:0s HTTPAddr:localhost:5050 EnableGRPC:false GRPCAddr:localhost:5051 MonitorAddr:localhost:5052 AuthToken: GRPCMsgSize:64.0 MiB EnableDebug:false EnableMetrics:false Release:false Silent:false ForceGCTarget:20.0 MiB ForceGCInterval:5s Version:false}
2024-04-16T16:01:10.408+0800	INFO	env/env_linux.go:62	Created container mount at:Mounts: bind[/bin:bin:ro], bind[/lib:lib:ro], bind[/lib64:lib64:ro], bind[/usr:usr:ro], bind[/etc/ld.so.cache:etc/ld.so.cache:ro], bind[/etc/alternatives:etc/alternatives:ro], bind[/dev/null:dev/null:rw], bind[/dev/urandom:dev/urandom:rw], bind[/dev/random:dev/random:rw], bind[/dev/zero:dev/zero:rw], bind[/dev/full:dev/full:rw], bind[/sys/devices/system:sys/devices/system:ro], tmpfs[w], tmpfs[tmp], proc[ro]
2024-04-16T16:01:10.408+0800	INFO	env/env_linux.go:100	Creating container builder: hostName=go_judge, domainName=go_judge, workDir=/w
2024-04-16T16:01:10.409+0800	INFO	env/env_linux.go:160	Running with cgroup v2, connecting systemd dbus to create cgroup
2024-04-16T16:01:10.416+0800	INFO	env/env_linux.go:175	Connected to systemd bus, attempting to create transient unit: gojudge.scope
2024-04-16T16:01:10.441+0800	INFO	env/env_linux.go:195	Current cgroup is system.slice/gojudge.scope
2024-04-16T16:01:10.441+0800	INFO	env/env_linux.go:213	Creating nesting api cgroup v2(/sys/fs/cgroup/system.slice/gojudge.scope)[cpu, cpuset, memory, pids]
2024-04-16T16:01:10.443+0800	INFO	env/env_linux.go:224	Creating containers cgroup
2024-04-16T16:01:10.444+0800	INFO	go-judge/main.go:275	create 1 prefork containers
container_exit: serve: failed to execute cmd init_fs: mount bind[/sys/devices/system:sys/devices/system:ro] operation not permitted
2024/04/16 16:01:10 prefork environment failed  conf: recvAck RecvMsg: read unixpacket @->@: EOF

貌似正是在挂载 /sys/devices/system 目录的时候失败了。我尝试将 uid 和 gid 都改成 0 再运行，但还是出现了同样报错。

[criyle]

可以尝试去掉 readonly: true 再尝试，类似

  - type: bind
    source: /sys/devices/system
    target: /sys/devices/system
    readonly: false

同时，你可能需要添加一个 /dev/shm ，类似

  - type: tmpfs
    target: /dev/shm
    data: size=64m,nr_inodes=4k

[ConnectionFailedd]

删除 readonly: true 行后 go-judge 成功运行了，然而 MPI 程序无法正常运行。在添加 /dev/shm 和运行时添加 -net-share 命令后依然无法运行，输出的 stderr 如下：

[go_judge:00018] *** Process received signal ***
[go_judge:00018] Signal: Segmentation fault (11)
[go_judge:00018] Signal code: Address not mapped (1)
[go_judge:00018] Failing at address: 0x20
[go_judge:00018] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x45320)[0x7993dfe45320]
[go_judge:00018] [ 1] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_home_directory+0x28)[0x7993dcce4ac8]
[go_judge:00018] [ 2] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_mca_base_var_cache_files+0x32)[0x7993dcce0d52]
[go_judge:00018] [ 3] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_mca_base_var_init+0x247)[0x7993dcce16a7]
[go_judge:00018] [ 4] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_init_util+0x89)[0x7993dccaf609]
[go_judge:00018] [ 5] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_rte_init+0x77)[0x7993dccafc27]
[go_judge:00018] [ 6] /lib/x86_64-linux-gnu/libpmix.so.2(PMIx_server_init+0x30e)[0x7993dcc74d3e]
[go_judge:00018] [ 7] /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_pmix_ext3x.so(ext3x_server_init+0x251)[0x7993dd5cb731]
[go_judge:00018] [ 8] /lib/x86_64-linux-gnu/libopen-rte.so.40(pmix_server_init+0x31f)[0x7993e016b44f]
[go_judge:00018] [ 9] /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_ess_hnp.so(+0x3f18)[0x7993dfc10f18]
[go_judge:00018] [10] /lib/x86_64-linux-gnu/libopen-rte.so.40(orte_init+0x2aa)[0x7993e01bb15a]
[go_judge:00018] [11] /lib/x86_64-linux-gnu/libopen-rte.so.40(orte_submit_init+0x911)[0x7993e01650e1]
[go_judge:00018] [12] /usr/bin/mpirun(+0x11e8)[0x634189e8e1e8]
[go_judge:00018] [13] /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7993dfe2a1ca]
[go_judge:00018] [14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7993dfe2a28b]
[go_judge:00018] [15] /usr/bin/mpirun(+0x1415)[0x634189e8e415]
[go_judge:00018] *** End of error message ***

和最开始相比，确实少了两行：

[executor_server:00027] opal_ifinit: unable to find network interfaces.
hwloc/linux: failed to find sysfs cpu topology directory, aborting linux discovery.

[criyle]

Address not mapped 通常是内存限制过小，请增加 memoryLimit 再尝试

[ConnectionFailedd]

我的内存限制为 104857600，go-judge 返回中给出的内存使用是 3080192，我觉得应该不是内存限制的问题。

[
    {
        "status": "Signalled",
        "exitStatus": 11,
        "time": 64781000,
        "memory": 3080192,
        "runTime": 269527010,
        "files": {
            "stderr": "[go_judge:00055] *** Process received signal ***\n[go_judge:00055] Signal: Segmentation fault (11)\n[go_judge:00055] Signal code: Address not mapped (1)\n[go_judge:00055] Failing at address: 0x20\n[go_judge:00055] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x45320)[0x7d9d8c245320]\n[go_judge:00055] [ 1] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_home_directory+0x28)[0x7d9d890e4ac8]\n[go_judge:00055] [ 2] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_mca_base_var_cache_files+0x32)[0x7d9d890e0d52]\n[go_judge:00055] [ 3] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_mca_base_var_init+0x247)[0x7d9d890e16a7]\n[go_judge:00055] [ 4] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_init_util+0x89)[0x7d9d890af609]\n[go_judge:00055] [ 5] /lib/x86_64-linux-gnu/libpmix.so.2(pmix_rte_init+0x77)[0x7d9d890afc27]\n[go_judge:00055] [ 6] /lib/x86_64-linux-gnu/libpmix.so.2(PMIx_server_init+0x30e)[0x7d9d89074d3e]\n[go_judge:00055] [ 7] /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_pmix_ext3x.so(ext3x_server_init+0x251)[0x7d9d8bbe5731]\n[go_judge:00055] [ 8] /lib/x86_64-linux-gnu/libopen-rte.so.40(pmix_server_init+0x31f)[0x7d9d8c67144f]\n[go_judge:00055] [ 9] /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_ess_hnp.so(+0x3f18)[0x7d9d8c111f18]\n[go_judge:00055] [10] /lib/x86_64-linux-gnu/libopen-rte.so.40(orte_init+0x2aa)[0x7d9d8c6c115a]\n[go_judge:00055] [11] /lib/x86_64-linux-gnu/libopen-rte.so.40(orte_submit_init+0x911)[0x7d9d8c66b0e1]\n[go_judge:00055] [12] /usr/bin/mpirun(+0x11e8)[0x60b13d31a1e8]\n[go_judge:00055] [13] /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7d9d8c22a1ca]\n[go_judge:00055] [14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7d9d8c22a28b]\n[go_judge:00055] [15] /usr/bin/mpirun(+0x1415)[0x60b13d31a415]\n[go_judge:00055] *** End of error message ***\n",
            "stdout": ""
        }
    }
]

[criyle]

内存限制过小会在内存申请是就失败，并不会变成已使用内存大小。

[ConnectionFailedd]

仍然不行，我已将分配的内存从 100 MB 调整为 10 GB，但是仍然无法正常运行 MPI 程序。
在 compile.py 中可以看到，我尝试运行的是一个很简单的 MPI 程序，理应不会用到大量内存：

#include "mpi.h"

int main(int argc, char ** argv) {
    MPI_Init(&argc, &argv);
    MPI_Barrier(MPI_COMM_WORLD);
    MPI_Finalize();
}

[criyle]

尝试把 bind mount 的路径改成 /sys

[ConnectionFailedd]

报错并没有改变。请问会不会和多进程有关系呢？

[criyle]

那尝试下载 release 里面的 go-judge-shell 运行并粘贴

cat > 1.c << EOF
#include "mpi.h"

int main(int argc, char ** argv) {
    MPI_Init(&argc, &argv);
    MPI_Barrier(MPI_COMM_WORLD);
    MPI_Finalize();
}
EOF
mpic++ 1.c -o 1
./1

看看输出有没有变化

[ConnectionFailedd]

我在 Assets 里面没有找到 go-judge-shell？
我的本地也没有 go 环境，可以麻烦发布一下吗？

[criyle]

在 https://github.com/criyle/go-judge/actions/runs/8565176557 下载或者如果您的 hydrojudge 是最新版，可以使用 hydrojudge terminal 达到相同的效果

[ConnectionFailedd]

使用 ./1 并没有启动多进程，不过启动多进程时给出的输出和之前不同：

$ ./go-judge-shell 
2024/04/16 18:56:24 start [/bin/bash]
bash-5.2# cat > 1.c << EOF
#include "mpi.h"

int main(int argc, char ** argv) {
    MPI_Init(&argc, &argv);
    MPI_Barrier(MPI_COMM_WORLD);
    MPI_Finalize();
}
EOF
mpic++ 1.c -o 1
./1
bash-5.2# mpirun -np 8 --allow-run-as-root ./1
--------------------------------------------------------------------------
While trying to create a regular expression of the node names
used in this application, the regex parser has detected the
presence of an illegal character in the following node name:

  node:  go_judge

Node names must be composed of a combination of ascii letters,
digits, dots, and the hyphen ('-') character. See the following
for an explanation:

  https://en.wikipedia.org/wiki/Hostname

Please correct the error and try again.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
An internal error has occurred in ORTE:

[[11753,0],0] FORCE-TERMINATE AT (null):1 - error ../../../../../orte/mca/plm/base/plm_base_launch_support.c(555)

This is something that should be reported to the developers.
--------------------------------------------------------------------------