"Could not read archive" error when trying to examine a submission

[parno]

Your environment

We're using AutoLab's Docker (https://github.com/autolab/docker) as of commit b98f28d254036d9ffc6d349ca27191cf37ddd687.

Steps To Reproduce
In the attached zip file submissions.zip,
there are two tar files. If we submit handin-bad.tar to an assignment, the Autograder appears to execute successfully (since the points for the problems populate correctly), but when you click on the more detail link to see the contents of the assignment and the Autograder output, you get a 500 error from the website and an error message saying "Could not read archive".

FWIW, the code that generates this message appears to be this line:

Autolab/app/controllers/submissions_controller.rb

Line 290 in 7677e55

flash[:error] = "Could not read archive."

There are other places that return that message, but I modified the code to indicate which one it is and to print the actual error message, which turns out to be:

Could not read archive /home/app/webapp/courses/[coursename]/buildit/handin/[email_address]_10_handin.tar. Error = "\x80\x00\x00\x00\x00(y\xC3" is not an octal string.

The bad handin.tar file was produced on the linux.ece.cmu.edu servers using:

tar -cf handin.tar --exclude=handin.tar *.c *.h brg_types.h data.c data.h logappend.c logread.c Makefile

where the version of tar is tar (GNU tar) 1.30. We see similar behavior if we run the tar command on the Andrew servers.

However, if I untar the archive on my machine and tar it back up with the same command, we get the handin-good.tar file, which doesn't provoke the error message. My version of tar is bsdtar 3.3.2 - libarchive 3.3.2 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.6.

Current behavior
See above re: error message.

Expected behavior
The submission loads properly.

Any tips or assistance would be much appreciated.

[fanpu]

Hi Bryan, we've faced similar issues previously during testing when tarfiles created from an environment that resembles the Andrew servers results in the error you mentioned. In our case we faced the issue when testing our Git Submission feature on a bare metal installation of Autolab on AFS, and the tar command that Rails forks in order to tar a student's repository creates an archive that cannot be read. @cg2v resolved the issue on the deployment side but we're not sure how.

@cg2v is it possible to elaborate on how you fixed it? Thanks!

[parno]

@fanpu Thanks for your response! I'd be very interested in the workaround that the Andrew folks came up with.

[cg2v]

The internal reader in autolab that's used for submission view uses Gem::Package::TarReader from rubygems to read tar files.

Depending on which exception occurs when reading the archive, either rubygems/rubygems#3146 or rubygems/rubygems#2213 could be the cause. The fix is to upgrade rubygems to at least 3.2 when building the image (gem update --system or gem update --system 3.2.33)

[parno]

@cg2v Thanks very much for the tip! That seems to have done the trick!

FWIW, when editing the Dockerfile (https://github.com/autolab/Autolab/blob/526b3b6767f61c69b922bab8345349f23f1b47cc/Dockerfile), adding gem update --system 3.2.33 at line 28 gave us a system that ran but was unable to send email. There was an error

An error occurred when sending a notification using 'email' notifier.SocketError: getaddrinfo: Name or service not known

deep in the bowels of phusion_passenger.

Moving the gem update line to line 39 of the Dockerfile, however, appears to have succeeded in both addressing the original issue and producing a working system that can send email.

Thanks very much for the help!