Why AI Miscounts Takeoffs, and How We Cut It by 98.8%
Short answer
An AI can look at a mechanical plan, correctly identify every piece of equipment, and still hand you the wrong count. We know, because we measured it: in one case a model correctly listed 173 objects, then stated numbers in the same answer that added up to 188. The fix is not a bigger model. It is treating the reporting step, turning what the model saw into records, counts, and a reviewable scope, as part of the system to be engineered. Our research did exactly that, and across 50 dense test scenes it cut per-type counting error by up to 98.8% using the same model weights and a single model call per image. That is the difference between a takeoff count you can bid on and one you have to recount by hand. We published the paper, code, and data so anyone can check it.
The problem no one talks about: seeing is not reporting
Most of the conversation about AI on construction drawings is about perception. Can the model find the symbol? Can it read the small stuff? That matters, and we have written about it elsewhere. But there is a second step that quietly decides whether any of it is useful, and almost no one talks about it: turning what the model saw into a number you can act on.
Here is the problem in one example from our study. We gave GPT-6.1 a dense scene. It correctly listed 173 objects, one by one. Then, asked for the total, it stated counts in that same response that summed to 188. The model had the right information. It reported the wrong answer. If you just read the total, you are off by fifteen. If you count the records the model already produced, you get the right number: 173.
For an estimator, this is not an abstract AI problem. A takeoff is a report. A count of receptacles, a length of duct, a number of doors, is only worth something if the number you act on matches what is actually on the sheet. A model that sees everything and then miscounts the total is worse than useless, because it looks right. The gap between seeing and reporting is where a clean-looking takeoff quietly becomes a wrong bid.
What we proved: make reporting part of the system
The paper, From Seeing to Reporting (FSR), treats that gap as the thing to engineer. Instead of asking the model for a total and trusting it, we built a defined chain: take the model’s response, decode it into individual records with a class and a location, assemble those records into the specific collection the task needs, and then compute the answer from the records themselves. The count comes from the list, not from the model’s own arithmetic.
That reframing is the whole idea. A detected item needs to become a usable record. A set of records needs a consistent summary. An estimate needs enough structure that a person can review it and act. Each of those is a step you can define, test, and improve, separate from how good the underlying model is. We also worked out where extra effort actually helps, whether the win comes from acquiring a better candidate answer, holding onto it while the collection is built, or choosing it correctly at the end. Those are three different engineering problems, and treating them as one is why a lot of AI tooling leaves accuracy on the table.
In plainer terms: the right answer is often already inside the model’s response. The job is to get it out intact.
The numbers: up to 98.8% less counting error, with one model call
We tested this hard: 67,364 model calls across six vision-language models, including GPT-6.1, Claude Sonnet 5.5, and Gemini 3.8 Flash, spanning counting, object detection, document layout, screen parsing, and GUI grounding.
The counting result is the one that matters most for takeoff. On 50 dense synthetic scenes, with the same model weights and one model call per image, the full reporting-and-decoding protocol cut mean per-type counting error sharply:
- GPT-6.1: error fell from 10.84 to 1.60, an 85.2% reduction.
- Claude Sonnet 5.5: error fell from 35.02 to 0.42, a 98.8% reduction. Read that again, because the surprising part is what did not change. We did not swap in a better model. We did not add a second pass. Same weights, same single call, a better reporting process, and counting error dropped by up to 98.8%. The accuracy was recoverable all along; it was being lost in the handoff from what the model saw to what it reported.
We also tested the idea beyond counting. In a GUI-grounding task, picking the right spot to click on 600 screenshots, FSR combined candidates from GPT-6.1, Claude Sonnet 5.5, and Gemini 3.8 Flash and selected the best one. It hit 544 of 600 targets, a 90.67% success rate, and cut misses from 76 to 56 versus three GPT-6.1 passes combined by voting, a 26.3% reduction at comparable cost. Different task, same lesson: finding a correct answer and delivering it are separate problems, and connecting them is where the gains are.
The solution: how Quotr turns this into a review-ready estimate
Research is only useful if it changes the product. Here is what the record-first approach does inside Quotr when you upload a plan.
It builds records, not just a total. Quotr does not just tell you “fourteen units.” It keeps each detected item as a record with what it is, where it is on the sheet, and its measurement basis, then builds the count from those records. That is the FSR idea in production: the number you see is computed from a list you can open, not from a guess the model stated.
It preserves the distinctions that drive the estimate. On an HVAC plan, an energy recovery ventilator, an exhaust fan, and a grille are different scope items, and duct fittings and duct runs use different units. Quotr keeps equipment counted in each, duct measured in linear feet, and material and labor separated, with every line keeping its scope description. A drywall area keeps its wall system. A duct length keeps its system and size. A flooring quantity keeps its room and finish. The meaning survives all the way to the line item, which is what lets you price it.
It reconciles multiple looks into one clean scope. When the workflow inspects the same object through several views or model responses, turning those into a single item-level record is its own engineering step, and it is exactly what FSR studies. That is how Quotr avoids the double-counts and dropped items that happen when you naively trust a model’s running total.
It ends with your decision, not the AI’s. The HVAC workspace puts the proposed scope beside the drawing, with quantities, unit prices, labor hours, and line totals visible, and an explicit Approve or Reject. The AI prepares the work. Your team owns the number that goes out.
And because the approach is about structuring records, not a single trade’s symbols, it carries across the board: electrical device counts that keep standard, GFI, and weatherproof distinctions; plumbing organized by system, material, and diameter; concrete reporting volume, formwork area, and rebar through their own rules; painting that retains opening deductions and coat counts. One organized foundation, many deliverables: a quantity review for the estimator, a scope breakdown for the bidder, a material list for the buyer.
Why this makes us confident in Quotr
We did not run 67,364 model calls to publish a paper. We ran them because our software turns your drawings into numbers you bid real money on, and we wanted to know, provably, that the number you see is the number the model actually found.
Most AI takeoff tools stop at perception: they detect, they state a total, they display it. We treat the step after perception as its own problem, because that is where the error we measured actually lives. When a Quotr count matches the sheet, it is not because we happened to use a good model that day. It is because the reporting chain is built to preserve the answer the model already had.
Confidence, for us, is not a line on a sales page. It is a published result with the code and data attached, so you can check it yourself.
The honest caveats
A real result deserves honest edges. The 98.8% figure is per-type counting error on 50 dense synthetic scenes, a controlled benchmark chosen to isolate the reporting step, not a promise that every count on every messy field drawing improves by that much. The GUI-grounding gains are on a specific 600-task benchmark at a stated cost budget. And the method reduces error; it does not eliminate it, which is exactly why Quotr keeps the estimator’s review step instead of hiding it. The paper, the code, and the data are public precisely so the claims can be examined rather than taken on faith.
Read the research
The full paper is on arXiv, and the code and data are on GitHub.
- Paper: From Seeing to Reporting: A Theory of Reliable Visual Set Generation in Vision-Language Models (arXiv:2610.25503)
- Code and data: github.com/shijunzhe/vlm-set-reporting
- The companion paper: Why VLMs Miss Small Objects
- More of our work: Quotr Research
Citation
Junzhe Shi and Yuan Gan. “From Seeing to Reporting: A Theory of Reliable Visual Set Generation in Vision-Language Models.” arXiv preprint arXiv:2610.25503 [cs.CV], 2026. Available at https://arxiv.org/abs/2610.25503.
@article{shi2026fromseeingtoreporting,
title = {From Seeing to Reporting: A Theory of Reliable Visual Set Generation in Vision-Language Models},
author = {Shi, Junzhe and Gan, Yuan},
journal = {arXiv preprint arXiv:2610.25503},
year = {2026},
eprint = {2610.25503},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.25503}
}
FAQ
Why does AI miscount items on construction drawings even when it sees them? Because seeing and reporting are different steps. A model can correctly identify every item and still state a wrong total, the way one model in our study listed 173 objects but reported numbers summing to 188. The fix is to compute the count from the list of detected records rather than trusting the model’s stated total. Doing that cut per-type counting error by up to 98.8% in our tests, with the same model and a single call.
Does using a bigger or newer AI model fix the counting problem? Not by itself. Our result used the same model weights and one model call per image; the improvement came from the reporting process, not a stronger model. Better perception creates information, but it still has to be reported correctly to be useful.
How does this affect my takeoff accuracy in Quotr? Quotr keeps each detected item as a record with its type, location, and measurement basis, and builds counts and quantities from those records. That record-first approach is what the research supports, and it is why a Quotr count can be opened and traced rather than taken on trust.
Does the AI decide the final numbers? No. Quotr prepares the scope, quantities, unit prices, and labor, and presents them next to the drawing with an explicit Approve or Reject. The estimator reviews and owns the final estimate.
Is the research public? Yes. The paper is on arXiv (2610.25503) and the code and data are on GitHub, so anyone can reproduce and check the results.
See it on your own plans
The fastest way to see what record-first reporting does for a takeoff is to run it on a plan you know. Start a free trial or book a 30-minute demo and bring a dense mechanical sheet.
About Quotr.ai
Quotr.ai is an AI-powered construction estimation, takeoff, and procurement platform built for subcontractors, general contractors, and developers. Backed by Llama Ventures and based in San Francisco. Learn more at quotr.ai. The platform has three parts: Quotr Software (AI takeoff, estimating, and bidding, including the AI agent that reads your plans), Quotr Service (done-for-you cost estimates, takeoffs, and pro formas), and Quotr Procurement (factory-direct materials delivered door-to-door).



