Ask an AI assistant where the nearest hospital is, and it answers immediately, confidently, and is often wrong, with no sign that the two might differ. 

A lesson from the Cairngorms

Nearly thirty years ago I finished a winter route in the Northeastern Corries of Lochnagar in a whiteout, the wind stripping the heat out of every exposed surface and building a carapace of ice on everything it touched. My small team knew the ground well and set off confident in our direction-finding. Twenty minutes in, we crossed a fresh set of footprints and took them as reassurance: someone else was out there navigating the same storm. Ten minutes later, we recognised our own tracks. We had walked a full circle back onto ourselves, in a place we thought we knew, confident the whole way round.

We lost thirty minutes finding the summit again, confirming a point we could trust, setting a bearing, and committing to the long walk home. The lesson has only become truer with time: in geography, ninety percent right is one hundred percent wrong. It does not matter how close you get. If you do not reach the destination, the direction was wrong.

Where AI breaks down on “nearest”

That lesson is exactly the failure mode we’re now seeing in AI systems handling spatial queries. A frontier model has a per-question answer budget inside a probabilistic framework, where not answering is unacceptable. So, like the most enthusiastic intern in the world, it blurts out the best answer it has within that budget, confident whether or not it’s right.

I built an agent on top of Claude that’s geospatially capable enough to pass the GISP exam. I asked it to mark two points for local wildfires. It placed both in the wrong spot, cheerfully. When I pointed that out, the reply was: “good catch.” If I hadn’t already known where those fires were, we’d have carried on in confident, complete ignorance.

The same agent can find a genuinely cloud-free satellite image of London without much trouble. Ask it to mark the Tower of London on that image, though, and the point it drops carries a confident label, a precise latitude and longitude, and sits squarely on Tower Bridge instead, a few hundred metres from the actual castle.

Close. Confident. Wrong.

Last-mile delivery learned this lesson the hard way, and so did Apple. Being ninety percent of the way to a location is the same as being nowhere: the parcel still doesn’t arrive, and the person still can’t find the door.

Generative AI, by contrast, tends to treat geography the way it treats everything else: as probabilistic, assembled from patterns in text and images rather than fixed to a coordinate and a timestamp. That treatment is fine for opinions and summaries. It breaks down the moment someone needs to act on the answer.

Geography doesn’t bend

Here’s the asymmetry worth naming. A place, at a particular time, generally is a place. Organisations can and do disagree about the attributes of a location, Taiwan being an obvious example of disputed sovereignty, but almost nobody disputes that the island exists. The geoid is fixed. My house has stood on the same footprint since the seventies, even as the house itself changed around that fixed point. These are deterministic facts: things that existed at a specific place and time, whatever a model happens to believe about them.

Anyone who has orthorectified aerial imagery knows this instinct already. You add control points to the polynomial, and a correct set of points lets you calculate a modelled representation of the ground with the error driven close to zero. Bad points blow the error up. Machines solve that optimisation problem well now. What they are not yet good at is knowing which points are allowed to be control points in the first place, because most frontier models acquire geography implicitly, through text and images, rather than explicitly, through an understanding of the geoid and human distribution.

I’ve seen a model tell someone, flatly, “those locations don’t exist,” about a place that does. That should trouble anyone who treats space and time as fixed features of shared human experience, because it turns one of the few genuinely deterministic categories of knowledge into just another probability to round off.

Building benchmarks that take geography seriously

At Sparkgeo we’ve been building a benchmark aimed squarely at this gap: not how well a model can buffer a polygon or reproject a shapefile, but how well it answers the kind of question a person actually asks a map. We split the problem into three parts.

We also varied who’s asking. An analyst states a query with full operational detail. A professional is conversational but still specific. A consumer is terse and expects the model to infer the rest: “where’s the nearest coffee on the way to the park” rather than a street address and a mode of travel.

Nine question categories, three personas, run blind against six North American cities, with no persona or benchmark knowledge in the model’s context.

A few efforts already point the right way. GeoAnalystBench looks at the quality of geographic code a model produces. GeoBench assesses how well models perform at GeoGuessr-style location inference. Both are useful. Neither is sufficient alone.

The benchmark I keep coming back to is simpler, and harder: ask a model to count the football fields in Georgia. A model with real geographic competence asks which Georgia, which code of football, and whether “field” means the pitch or the whole facility, before it commits to a number. A model without that competence picks one interpretation silently and answers with the same confidence either way. Time and space should be rules that are harder to bend than most, the way they tend to be for us. Right now those rules are largely missing.

This is also where ideas like SpaceTimeIDs matter: a way to track how a boundary or a place name changes over time, fixing a place at a time rather than pretending geography holds still. At Sparkgeo we’ve been doing related work with the Overture Maps Foundation on living point-of-interest data, using social signals to flag when a place has actually changed, rather than assuming a POI dataset built once stays accurate forever. Building that kind of dataset once is the easy part. Keeping it current is the real work, and it’s the part most AI systems skip.

What the numbers show

Ask five frontier models to list every restaurant within a kilometre of a point, and the correct answer is eight. Grok and the reasoning version of ChatGPT came closest, recalling just over half. Gemini returned more than thirty places and got four of them right, largely ignoring the one criterion the question was actually about: distance.

Points fare better than areas. Ask which of two parks is within a five-minute walk, and every model we tested missed at least one, because a park is a shape, not a pin, and spatial reasoning in these models still mostly stops at point-to-point distance.

Then there’s “nearest” itself. In one test city, the closer option by straight-line distance was the wrong answer, because the only bridge to it had burned down two years earlier. One model caught that every time; two caught it most of the time; one sent the directions across a crossing that no longer exists.

Across roughly a hundred questions in four cities, two models came out ahead on overall accuracy and two trailed, with a wide enough gap that which model answers a location question still matters more than most people assume.

This is early, preliminary work, about 500 questions so far, and we’ll publish the full method soon. But the direction is already clear: there’s a lot of room left in both data quality and spatial reasoning.

Geography as the control layer

The practical pattern is the one I’d ask anyone building AI products for the real world to adopt: treat authoritative, cloud-native spatial data as a constraint on what a model is allowed to assert, not as one more document sitting in its context window. Deterministic geospatial operations (geocoding, routing, boundary lookups, distance calculations) belong between the model and the user, checking a claim against ground truth before it reaches anyone who might act on it.

I think about this the same way I think about ground control points pinning down a polynomial for geographic correction. You don’t need to make the model deterministic. You need to give it fixed points it isn’t allowed to argue with, and let it reason non-deterministically on top of those. That’s the shift from a confident guess to a verifiable answer, and it’s the design principle behind how we built Prescient: geospatial data an AI system can query and be corrected by, not just read.

“Nearest” should mean nearest. Getting there means giving AI a set of control points it isn’t allowed to argue with.

I spoke on this at CNG Forum 2026 in Snowbird, Utah, earlier this month. For the longer version of the arguments above, I wrote them up first on Strategic Geospatial: Deterministic Control Points and 90% right is 100% wrong.