fix(codaveri): fix crash on import question with empty data files

[adi-herwana-nus]

Bug

Rollbar flagged FrozenError: can't modify frozen String: "" when importing a
Codaveri programming question, raised from the encoding check in
PythonPackageService#extract_supporting_file:

if content.force_encoding('UTF-8').valid_encoding?

force_encoding re-tags its receiver in place, so it raises as soon as content is frozen.

What exactly triggers it. content originates from
ProgrammingPackage#get_folder_files,
which reads each entry via entry.get_input_stream(&:read). rubyzip short-circuits that read for
exhausted streams and hands back a shared frozen literal:

# zip/ioextras/abstract_input_stream.rb — file carries `# frozen_string_literal: true`
return (maxlen.nil? || maxlen.zero? ? '' : nil) if eof?   # <- frozen literal
...
tbuf.force_encoding(Encoding::ASCII_8BIT)                 # <- freshly allocated, unfrozen

A zero-byte entry is eof? on its very first read, so it never reaches the buffer path and
receives the frozen ''. Every non-empty entry gets a fresh mutable buffer. Measured against the
locked rubyzip:

entry	bytes	frozen	force_encoding
empty.csv	0	true	FrozenError
tiny_binary.dat	5	false	ok
big_binary.dat	1048576	false	ok
utf8.txt	14	false	ok
one_null.dat	1	false	ok

So the trigger is precisely: a package containing a zero-byte file, in the package root or the
tests/, submission/, or solution/ folders, whose name is not on the service's skip list
(Makefile, .meta, append.py, prepend.py, autograde.py, template.py).

Two things this is not sensitive to:

• The file being binary. Binary content of any size arrives unfrozen, and reaches the base64
  branch only after force_encoding has already succeeded. Base64.strict_encode64 does not
  mutate its argument, so that branch never needed a mutable string.
• The language. The freeze happens upstream in the package reader. All eight language services
  carried the identical call, so a zero-byte file broke every one of them (see Scope check).

In the reported package the trigger is tests/empty.csv. tests/append.py is also zero-byte but is
skip-listed, so it never reaches the call.

Regression window. rubyzip 2.4.1's copy of abstract_input_stream.rb has no
frozen_string_literal magic comment, so the identical return '' produced a mutable string and
force_encoding silently worked. The magic comment arrives in rubyzip 3.x, so this has been broken
for every affected package since dc14a58866 ("chore(deps): bump rubyzip from 2.4.1 to 3.2.2").
Verified against the currently locked rubyzip 3.3.0.

Remediation

Tag the encoding on a copy rather than on the string owned by the package reader:

utf8_content = content.dup.force_encoding('UTF-8')
if utf8_encodable?(filename, utf8_content)

While fixing this, extract_supporting_file turned out to be duplicated across all eight
language package services — seven byte-identical, with Java differing by a single clause
(&& filename.to_s.downcase.end_with?('.java')). Every copy carried the same defect, so rather than
patch the same line eight times the method was promoted to
LanguagePackageService
and Java's extra clause extracted into an overridable predicate:

# LanguagePackageService
def utf8_encodable?(_filename, utf8_content)
  utf8_content.valid_encoding?
end

# JavaPackageService
def utf8_encodable?(filename, utf8_content)
  super && filename.to_s.downcase.end_with?('.java')
end

This removes 125 lines of app code. The encoding decision is unchanged for every service: empty
files continue to be sent as utf8 by the seven generic services, and as base64 by Java (whose
predicate has always required a .java extension — for an empty file this is cosmetic either way,
since Base64.strict_encode64('') is ''). The one byte-level difference: the base64 branch now
receives the untouched ASCII-8BIT original instead of a string re-tagged in place, which encodes
identically since Base64 operates on bytes.

Scope check

Audited every force_encoding call across app, lib, and spec. Post-refactor exactly two
remain:

1. LanguagePackageService — this fix.

2. CoursemologyDockerContainer#extract_test_report
   — already guarded, and worth noting because it broke in exactly this way before:

   # this string must be mutable for force_encoding to work
   return (+test_report).force_encoding(Encoding::UTF_8) if test_report

   That guard was added in d6f56afd1a ("chore(ruby): upgrade to ruby 3.3.5", Sept 2024), whose
   body reads "fix error due to tar files being frozen strings". Same bug class — a library reader
   (rubygems' TarReader there, rubyzip here) returning a frozen string into an in-place
   force_encoding. No action needed; it uses +str rather than .dup, which is equally safe.

No other call site mutates strings returned by ProgrammingPackage's file readers.

Tests

Added language_package_service_spec.rb
covering extract_supporting_file and utf8_encodable? at the base class, including the Java
override. The empty-file example asserts the fixture content is genuinely frozen
(expect(content).to be_frozen) so the spec cannot silently stop reproducing the original
condition. This path touches no DB, so no tenant block is needed.

Backed by a new fixture,
spec/fixtures/course/programming_question_template_codaveri_empty_data_file.zip, built on the
existing programming_question_template_codaveri.zip scaffolding plus supporting files covering each
branch: data.csv (utf8), binary.dat (invalid UTF-8, base64), and zero-byte empty.csv,
tests/empty.csv and tests/append.py mirroring the reported package. This is a new fixture
rather than an extension of the existing one, which would have shifted data_files expectations in
current specs.

Verified end-to-end by reconstructing the Codaveri /problem request body for the reported package:
it raises the production FrozenError before this change and serialises the empty file as
{"path": "empty.csv", "content": "", "encoding": "utf8"} after.

[Copilot]

Pull request overview

Fixes a crash when importing Codaveri programming questions that contain zero-byte files by avoiding in-place force_encoding on potentially frozen strings returned by rubyzip, and reduces duplication by centralizing the shared supporting-file extraction logic.

Changes:

• Promote extract_supporting_file into LanguagePackageService, using content.dup.force_encoding('UTF-8') to avoid FrozenError on zero-byte entries.
• Add an overridable utf8_encodable? predicate (Java narrows plaintext handling to .java files as before).
• Add specs + fixture coverage for UTF-8, binary, and empty-file cases (including frozen empty string behavior).

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
spec/services/course/assessment/question/programming_codaveri/language_package_service_spec.rb	Adds regression coverage for frozen empty strings and utf8_encodable? behavior (base + Java override).
app/services/course/assessment/question/programming_codaveri/language_package_service.rb	Centralizes supporting file extraction and avoids mutating frozen zip entry strings.
app/services/course/assessment/question/programming_codaveri/java/java_package_service.rb	Replaces duplicated extraction logic with a utf8_encodable? override to preserve Java’s .java restriction.
app/services/course/assessment/question/programming_codaveri/python/python_package_service.rb	Removes duplicated extract_supporting_file now provided by the base service.
app/services/course/assessment/question/programming_codaveri/java_script/java_script_package_service.rb	Removes duplicated extract_supporting_file now provided by the base service.
app/services/course/assessment/question/programming_codaveri/type_script/type_script_package_service.rb	Removes duplicated extract_supporting_file now provided by the base service.
app/services/course/assessment/question/programming_codaveri/go/go_package_service.rb	Removes duplicated extract_supporting_file now provided by the base service.
app/services/course/assessment/question/programming_codaveri/rust/rust_package_service.rb	Removes duplicated extract_supporting_file now provided by the base service.
app/services/course/assessment/question/programming_codaveri/r/r_package_service.rb	Removes duplicated extract_supporting_file now provided by the base service.
app/services/course/assessment/question/programming_codaveri/c_sharp/c_sharp_package_service.rb	Removes duplicated extract_supporting_file now provided by the base service.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.