truncate csv files in tutor problems api #2568
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mbertrand
approved these changes
Oct 6, 2025
abeglova
force-pushed
the
ab/csv-files-in-tutor-problems
branch
from
c494769 to
45c010b
Compare
abeglova
force-pushed
the
ab/truncate-csv-files-in-tutor-problems
branch
from
fd59275 to
99a538a
Compare
abeglova
force-pushed
the
ab/truncate-csv-files-in-tutor-problems
branch
from
99a538a to
b72f110
Compare
This was referenced Oct 7, 2025
Closed
Closed
Closed
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What are the relevant tickets?
closes https://github.com/mitodl/hq/issues/8770
Description (What does it do?)
#2560 imports csv files as problem set parts . Unfortunately, even with two 1000 line files, I hit the llm token limit while testing the tutor with a problem set that involves csv files. This truncates the output to only show the first few rows of the csv and the column names. Combined with the solution, this seems to be enough for the tutor to be able to answer questions about problems that require data files
How can this be tested?
run docker compose exec web ./manage.py backpopulate_canvas_courses --canvas-ids 34716 (if you don't have it imported already)
Go to http://api.open.odl.local:8065/api/v0/tutor/problems/34716-15.C57_FA25+canvas/Assignment%202/
instead of
contentthe csv files should havetruncated_contentandnumber_of_recordsThe same problem set is also in the test course (14566-kaleba:20211202+canvas) as "15.C57 FA25 - Assignment 2"