mpm fails installing - error message "double free or corruption (out)"

[WDO2]

Issue: mpm fails to install Matlab (a suspicious part of the error message is "double free or corruption (out)")

Overview:
We use mpm to install Matlab in a docker container
(As alternative method we build a .devcontainer which produces basically the same error messages but additionally the hint that "double free or corruption (out)" is an issue.

mpm --version
MATLAB Package Manager 2024.2.1

Reproduction Method:

wget -q https://www.mathworks.com/mpm/glnxa64/mpm
chmod +x mpm
./mpm install     --release=r2023a     --destination=/opt/matlab/r2023a     --products MATLAB     || (echo "MPM Installation Failure. See below for more information:" && cat /tmp/mathworks_root.log && false)

(While retrying the pipeline I find that sometimes it passes, hence that command is unstable)

Expected behavior:
Matlab should be installed

Error Messages from building a container:

#40 12.03 Products will be installed to: /opt/matlab/r2023a
#40 12.22 Starting install
#40 78.02 Finished install
#40 78.02 Completing setup...
#40 78.74 Installation complete
#40 80.89 MPM Installation Failure. See below for more information:
#40 80.89 (Aug 06, 2024 12:59:34) Starting download of product meta data
#40 80.89 (Aug 06, 2024 12:59:34) Finished downloading product meta data
#40 80.89 (Aug 06, 2024 12:59:38) Starting to retrieve support package metadata
#40 80.89 (Aug 06, 2024 12:59:42) Finished retrieving support package metadata
#40 80.89 (Aug 06, 2024 12:59:42) Starting Install
#40 80.89 (Aug 06, 2024 13:00:48) Completed Install
#40 80.89 (Aug 06, 2024 13:00:48) User provided options:	matlabroot : /opt/matlab/r2023a	runmatlabddux : true	
#40 80.89 (Aug 06, 2024 13:00:48) Starting OS Registration Service for MATLAB
#40 80.89 (Aug 06, 2024 13:00:48) Generating Java class path for /opt/matlab/r2023a
#40 80.89 (Aug 06, 2024 13:00:48) Successfully generated Java class path for /opt/matlab/r2023a
#40 80.89 (Aug 06, 2024 13:00:48) Generating pathdef.m for /opt/matlab/r2023a
#40 80.89 (Aug 06, 2024 13:00:48) Successfully generated pathdef.m for /opt/matlab/r2023a
#40 80.89 (Aug 06, 2024 13:00:48) Calling Manifest cache executable
#40 80.89 (Aug 06, 2024 13:00:48) Successfully called Manifest cache executable
#40 80.89 (Aug 06, 2024 13:00:48) Completed OS Registration Service for MATLAB
#40 80.89 (Aug 06, 2024 13:00:48) Starting Support Package Registration Service
#40 80.89 (Aug 06, 2024 13:00:48) Completed Support Package Registration Service

Error Messages from building a .devcontainer:

13.27 Products will be installed to: /opt/matlab/r2023a
13.47 Starting install
114.3 Finished install
114.3 Completing setup...
115.1 Installation complete
116.9 double free or corruption (out)
117.4 MPM Installation Failure. See below for more information:
117.4 (Aug 07, 2024 07:45:02) Starting download of product meta data
117.4 (Aug 07, 2024 07:45:02) Finished downloading product meta data
117.4 (Aug 07, 2024 07:45:05) Starting to retrieve support package metadata
117.4 (Aug 07, 2024 07:45:10) Finished retrieving support package metadata
117.4 (Aug 07, 2024 07:45:10) Starting Install
117.4 (Aug 07, 2024 07:46:51) Completed Install
117.4 (Aug 07, 2024 07:46:51) User provided options:    matlabroot : /opt/matlab/r2023a runmatlabddux : true
117.4 (Aug 07, 2024 07:46:51) Starting OS Registration Service for MATLAB
117.4 (Aug 07, 2024 07:46:51) Generating Java class path for /opt/matlab/r2023a
117.4 (Aug 07, 2024 07:46:51) Successfully generated Java class path for /opt/matlab/r2023a
117.4 (Aug 07, 2024 07:46:51) Generating pathdef.m for /opt/matlab/r2023a
117.4 (Aug 07, 2024 07:46:51) Successfully generated pathdef.m for /opt/matlab/r2023a
117.4 (Aug 07, 2024 07:46:51) Calling Manifest cache executable
117.4 (Aug 07, 2024 07:46:52) Successfully called Manifest cache executable
117.4 (Aug 07, 2024 07:46:52) Completed OS Registration Service for MATLAB
117.4 (Aug 07, 2024 07:46:52) Starting Support Package Registration Service
117.4 (Aug 07, 2024 07:46:52) Completed Support Package Registration Service

[WDO2]

The error might have occured due to low disc space.
However, I would suggest a plausible error message to point the user in the right direction.

[WDO2]

While repeating basically the same commands it sometimes succeeds and sometimes fails and reports one or multiple of the following error messages:

double free or corruption (out)
double free or corruption (fasttop)
munmap_chunk(): invalid pointer
free(): invalid pointer
corrupted size vs. prev_size in fastbins

I regard mpm as unstable.

[pcamach2]

I have the same intermittent issue when using mpm to install MATLAB Toolboxes in an Apptainer build.

Relevant portions of recipe file:

Bootstrap: docker
From: mathworks/matlab:R2022b

#  ...
%post
#  ...
    wget https://www.mathworks.com/mpm/glnxa64/mpm
    chmod +x mpm
    ./mpm install --release=R2022b --products Deep_Learning_Toolbox \
        Parallel_Computing_Toolbox \
        Statistics_and_Machine_Learning_Toolbox \
        Image_Processing_Toolbox
# ...

Error on first try: double free or corruption (fasttop)

Relevant output on second try with the same file:

+ chmod +x mpm
+ ./mpm install --release=R2022b --products Deep_Learning_Toolbox Parallel_Computing_Toolbox Statistics_and_Machine_Learning_Toolbox Image_Processing_Toolbox
Installing with the following parameters:
--doc=false
--release=R2022b
--products=Deep_Learning_Toolbox, Parallel_Computing_Toolbox, Statistics_and_Machine_Learning_Toolbox, Image_Processing_Toolbox
---------------------------------------------
The following required products will also be installed:
MATLAB

The following MathWorks Products are licensed under the The MathWorks, Inc. Software License
Agreement, available in the installation of the MathWorks Product or in the virtual machine image.
Parallel_Computing_Toolbox, Image_Processing_Toolbox, MATLAB, Deep_Learning_Toolbox, Statistics_and_Machine_Learning_Toolbox

Products will be installed to: /usr/local/MATLAB/R2022b
Starting install
Finished install
Completing setup...
Installation complete

[mw-skardile]

Hi @WDO2 and @pcamach2

Thank you for reporting this. We're currently investigating the issue and will keep you updated on our progress.

[borlum]

I have been experiencing the same, but it was caused by lack of disk space. Freeing up space had mpm run successfully again :-)

[WDO2]

@mw-skardile
Any update here?
The pipeline continues to fail frequently due to this bug.

[mw-skardile]

Hi @WDO2 ,

Thank you so much for your patience. We're working on a reliable fix. Could you please share the base image and disk space of your container? Our findings suggest the frequency of the issue might be linked to the Linux distribution and disk space. Thanks.

[WDO2]

sorry, I can not reveal this

[pcamach2]

On my end, the Apptainer build was running on Ubuntu 22.04 LTS. The total disk space used by the .sif file is 11.1GB, including an installation of Freesurfer version 6 performed later in the build recipe.

[WDO2]

@pcamach2
thanks for replying

[tooletime]

So I just ran into this exact problem when building the container for 2022b. I have GPU_Coder, MATLAB_Coder, Parallel Computing Toolbox and Signal_Processing_Toolbox on the addons section. It will fail with all of them until i remove the Signal Processing Toolbox option. So that addon is causing my failure. Waiting on support now.

[michael-c-jablecki]

Same issue with 2024b + Image Processing Toolbox (but 2024a is OK). Issue [ ref:!00Di00Ha1u.!500UU0Of8wj:ref ]