This repository contains files I used to compile and share results on local LLM performance on the 2025 AMC 10A. The results were compiled and written about on a blog post here: https://mrjohnsolves.com/can-my-graphics-card-qualify-for-the-american-invitational-mathematics-examination/.

There are three Python scripts (used Gemini's help to generate):

1. solve_problem_set.py

2. score_models.py

3. score_problems.py

4. is used as a prompt for the local model to solve the given set of problems. This uses the llama-cpp-python package. This file creates a results_ and summary_ file with solutions and answers and other statistics. 2)+3) are used to score and create charts (using pandas to organize data and plotly to display it).

I also included the 2025 AMC 10A files in .json format. Note, images were skipped except for Problem 15 where a description as given. (All other questions described the problem fully, but in Problem 15 additional information was included in the image. This additional image is summarized in the description).

The Sample_Results Folder contains the results used to create the blog post listed above. It includes a model_info.txt file that has a list of models used and config settings (as used in the solve_problem_set.py script). It also includes sample results_ and summary_ files.

In particular, the results_ files includes the full solutions generated from the models, which might be interesting to some.

For reference, models were given the following system prompt: SYSTEM INSTRUCTION: You are an expert mathematician competing in a tournament. Solve the user's problem step-by-step using strict Chain-of-Thought logic. Show all your mathematical work clearly. At the very end of your response, write a concluding line formatted exactly like this: 'Final Answer: (X)' where X is the correct multiple-choice option letter.