HistGramBench: Can AI Pass Mr. Dollhopf’s Geometry Class?

You need to be able to sketch a triangle to pass Geometry Accelerated at Deerfield. Consider a hypothetical world where AI policies are non-existent. Should students trust an AI agent to read a geometry sketch and complete their geometry homework? Consider this problem from The Teaching of Geometry (1911): reproduce the semicircle and triangle shown in Figure 3a. Before an agent can solve a geometry problem, it needs to “see” the figure correctly and reconstruct it. One way to rebuild these figures is to redraw them as code. TikZ is a drawing language used with LaTeX, the standard typesetting in mathematics. A TikZ drawing is a short program that compiles the details of a figure so that a computer can read it. Today’s frontier AI models that read images, known as vision-language models, can write such programs from a picture1.

The problem is: how can the accuracy of a redrawn figure be objectively measured? The standard method compares pixels, but differences in line thickness can produce misleading results. Code-by-code comparison is impossible because historical diagrams do not have original code.

We introduce HistGramBench, a benchmark for evaluating how accurately AI models reconstruct historical geometry diagrams. It uses an alternative judging method that checks the facts a reader relies on: which points exist, which points lie on which shape, which lines are parallel to each other, etc. The benchmark seeks to answer a question: How much of a figure’s structure do AI models get right, and how often do they fail to produce a drawing at all?

Existing benchmarks that turn pictures into code use modern material that has original TikZ code: scientific figures in TikZ1, charts in Python8, and web pages in HTML7. These benchmarks grade an AI’s output on how similar the output looks compared to the answer key. HistGramBench’s foundation is built on Inter-GPS, which describes geometry diagrams as lists of objects and relationships6.

The figures come from public-domain historical articles (Figure 1). Each sample is a single figure cropped from its page. An automated check procedure and screening model are introduced to remove non-geometry crops and duplicates. Our final test set contains 2,507 crops from an original dataset of 411 books (Figure 2).

Each figure is given to each model in two ways (Table 1). In Track A, the model sees only the figure and is asked to redraw “the mathematical figure the original intended,” leaving out stains and printing flaws. In Track B, models receive additional captions and nearby text from the same page. In both tracks, the answer must be a single TikZ drawing. Comparison between Track A and Track B shows us to what degree context influences a model’s performance.

Each figure has an answer key: a list of facts of three kinds (Table 2). Entity facts say that an object exists (“there is a point”), relation facts say how objects connect (“Point A lies on the circle”), and label facts show how things are marked (“This point is printed as A”). The answer key avoids recording exact position or size.

Two self-contained AI models, GPT-6.1 Sol and Kimi K2.5, independently write down facts. Their lists are then lined up, and every fact in the combined list U is voted on. Writing a fact counts as a vote for it unless the writer later rejects it. With vw(f) standing for writer w’s vote on fact f, the key is:

So facts enter the key only when both writers agree. The rule also prefers a shorter, reliable key to an overly complete one. Figures with keys that are too thin or too uncertain are omitted from the test set.

The benchmark takes the TikZ code from the model’s answer and runs it on one attempt in a contained environment (Docker) and reads the output picture as a set of points, lines, curves and text (Figure 2, right). If the code fails to compile or contains forbidden commands, the output is considered a complete failure.

The objects in the key are first paired with objects in the drawing. Each fact is then checked by measuring the drawing. For instance, let d(p, x) be the distance from a point p to an object x, and let ∠(ℓ₁, ℓ₂) be the angle between two lines.

A point counts as lying on a circle when it sits within one percent of the drawing’s size from it. Two lines count as parallel when their directions differ by at most 1.5 degrees. The drawing’s size is independent of these criteria because distances are compared with the size of the whole drawing. The checking process is only dependent on written facts, so line thickness, font and colour do not make significant differences. Other relations such as right angles are checked with the same methodology. More complex facts such as shaded regions or marks showing equal lengths go to a third independent judge for further corroboration.

A model’s performance is measured mathematically. Let K_i be the answer key, P_i the facts reproduced, and X_i the facts excluded because a required object is missing. The set difference K_i ∖ X_i contains the checkable facts. Let g_i equal 1 for a valid drawing and 0 for a compilation failure:

The corpus also contains books from adjacent authors so some figures tend to resemble each other. The benchmark tackles this by averaging scores within each group of related books and then averaging the groups:

where S̄_k is the average score in group k and G = 87 is the number of groups. The 95 percent range around each result uses the 87 group averages as the sample3.

Six vision-language models were tested: Gemma 4 26B, Llama 4 Maverick, Ministral 3 14B, Gemma 4 E2B, Claude Haiku 4.5 and Nemotron Nano 12B v2. These models’ outputs shared the same configurations and the same limit on answer length.

Gemma 4 26B does best, with a Track A score of 0.59 (Table 3). Performance of Gemma 4 E2B and Claude Haiku 4.5 is in the same bound through sorting the six into five distinct levels2.

Many answers failed to compile: Of 2,507 Track A answers, Nemotron Nano 12B v2 ran out of answer length on 651, Llama 4 Maverick introduced forbidden commands on 292, and Claude Haiku 4.5 wrote code that failed with syntax errors on 1,218; in 1,007 cases it left out the line opening a LaTeX document. Models struggled the most with relations.

Adding page text does not have a significant influence on the overall performance of models. Individual models move both ways, with gains from Ministral 3 14B to a loss of 0.08 for Gemma 4 26B (Figure 5), and the five levels of Track A merge into two (Figure 4).

To see if ranking is dependent on the family of the AI Judges, each judge’s model family was removed in turn and the six models were reranked. There was no significant correlation between grading and model family.

So, can AI pass Mr. Dollhopf’s geometry class? Six frontier AI models surely did not, at least not yet. Even the best model gets a large part of each figure wrong, and many answers fail to produce drawings, which is like giving a blank page as a submission. Proprietary frontier models like Claude Haiku 4.5 still fail on fundamentals like leaving out the first line of code, equivalent to not signing the Honor Pledge on the test. If you use AI, whether your school’s policy allows it or not, you are setting yourself up to fail geometry class.

For engineers, the quickest step-improvement is configuring the model to produce a valid output. Most failures are mechanical, such as running out of answer length or breaking syntax. In model training, engineers should focus on training models to understand relations instead of just learning how to look at a figure and guess. Fact-level checks in this benchmark can serve as a framework for data that can be used in reinforcement learning. Additionally, researchers should be mindful of page text. It is dependent on individual models, so keeping image-only examples in training can help models learn to read the figure instead of over-relying on context and captions. Finally, bigger does not mean better. Gemma 4 26B, an open-sourced model by Google with only 26 billion parameters, performed better than a proprietary Claude model.

HistGramBench shows that current vision-language models have trouble redrawing historical geometry figures, especially in producing a drawing that adheres to syntax and relations. We hope HistGramBench will encourage work on models that read a figure’s geometry with higher precision and grounding from textbooks that are centuries old to a printed homework worksheet due on a Thursday.