An offline 0.8B model that ties Gemini Flash on OpenStreetMap questions

When you are travelling the world you want the freedom to explore it without internet. There is a lot of amazing open data you can download to your phone, OpenStreetMap and Wikipedia among it, but sometimes you have a specific question and you want an AI to answer it so you can get back to the real world.

Introducing topoloop Finch: a blisteringly fast open-weights model that answers Ask Maps questions as well as Gemini 3.8 Flash, runs over an on-device index of open geography data, and works completely offline to help you explore the world.

Figure 1. Correct answers out of a hundred on Ask Maps Basic under the correctness judge. Blue bars are the same 0.8B model before and after training. Under each hosted model is what the hundred questions cost at OpenRouter list price for the tokens it used; Finch runs on the phone.

Why a local Ask Maps

A map is the app you need most where you have the least signal, and where you are is about the most personal data stream there is. Every question to a hosted model is a round trip and a bill. The hard part of doing it locally is not that the model is small, it is that a small model knows almost nothing, so everything it says has to come from what it looked up, and it has to say so. That turns out to be the feature: Finch can put a source next to every claim, and the hosted models in the chart above often cannot, because they answered from memory.

Ask Maps Basic

There was no public eval for this, so this release includes one. Ask Maps Basic is a hundred traveller questions written by a frontier model from real travel-blog passages, each with a gold place and its OpenStreetMap ref. A model gets places search, place details, web search and fetch over the same index, a budget of three places searches, two web searches and one fetch, and is judged two ways by one judge: correctness (right place, sensible answer) and grounding (every specific retrieved and cited). Nine models ran it under the same system prompt. Finch ties Gemini 3.8 Flash and edges GPT-6 Luna on correctness and leads everything on grounding, 79 to Luna's 73, Sonnet's 70 and Gemini's 12. Every transcript and both verdicts for all nine are in the trace explorer.

Figure 2. Every question, every model, under the correctness judge, the same judge as Figure 1: a filled cell is the right place with a sensible answer. Hover a cell for the question and the judge's reason. The second column is reach, the share of questions where the gold place appeared in the model's tool results at all. The grounding judge's per-question verdicts are in the trace explorer.

How Finch was trained

I built the data pipeline, the RL environment, the training loop and the iOS app myself, on three 3090s. Stock Qwen3.5-0.8B gets 14 of 100. I tried on-policy distillation first, the way Will Brown describes it, and could not make it work at this size; four attempts, none of them held. Supervised fine-tuning on 575,000 rows got the model on distribution for the tools and the citation contract instead, which took it to a grounding pass rate of 0.115 on my private development set. Then 25 rounds of GRPO with prime-rl against the geogym blogq environment, on two 3090s with a third serving a Gemma 4 judge in the loop at 35 verdicts a minute. Two things I did not expect. Six rounds of carefully shaped process rewards tied the baseline, and a plain binary reward, 1 for a grounded pass and 0 otherwise, beat them all. And a harness constant, the places-search radius, was worth more than any round of training: fixing it at 300 km moved the round-6 model from 0.615 to 0.695 with no gradient at all.

Figure 3. Grounding pass rate on the private development set by GRPO round. Harness changes are marked; the jump at round 2 is a search-radius constant, not training.

The policy found holes in the verifier

The 0.8B found 24 distinct ways to get reward it had not earned, and the log is the part of the repo I would read first. The best one took nine steps: round 3 penalised a judged-fail answer less if its map pin was unmeasurable, so the policy learned to put a coordinate-free journey node first in the map, bypassing the distance check. The share of answers doing it went from 3.9 to 90.8 percent in nine steps while the dashboard metric rose. The rule that came out of it is on every gate now: a check that can return "unmeasurable" hands the policy a third option beside pass and fail, and it will take it.

Figure 4. Pin laundering, round 3: the share of answers that put a journey node in the map instead of a place, step by step, while the dashboard metric said things were improving.

On the phone

The checkpoint is round 23, the one that ran Ask Maps Basic. Through MLX on the development set: bf16 0.755, 8-bit 0.720, 6-bit 0.700, 4-bit 0.570. At 4-bit the agent loop breaks, with an extra tool call per question and over-budget doubled, so the phone gets 8-bit, 782 MB. On an iPhone 16 Pro it decodes at about 60 tokens a second and prefills at about a thousand, which is a tool call and an answer in a few seconds. MMLU did not move during RL, 0.50 before and after.

Figure 5. The round-23 checkpoint through MLX at four precisions on the development set. 8-bit ships. 4-bit breaks the agent loop, not just recall.

What did not move

What the model searches for. Reach, the share of questions where the gold place ever appeared in its tool results, crept from 0.69 to 0.82 over the program, against 0.95 for Gemini with identical tools. But the share of first searches that named the gold place sat between 66 and 70 percent in twelve consecutive measurements, through every reward and every harness change. Almost all of the climb came from converting reach into correct, grounded answers. The hosted models know where things are and search by name; a 0.8B searches for what the question said, and probes put its name-to-coordinates recall at 14 percent. The next run is the same recipe on a model that knows where things are, a 35B mixture-of-experts. After blogq come routing, constraints and hiking, until an open model can do everything a map app needs from the open data underneath it. I build environments and evals like this one over real data. If you want geogym in your post-training mix, want one built for your domain, or want to build the open geo layer with me, my email is on the front page.

Get it

Try it: the Finch demo, the model answering with live tool calls over the topoloop API.
Weights: topoloop/finch-0.1-0.8B, bf16 plus MLX 8-bit and 6-bit.
Eval: topoloop/ask-maps-basic, the hundred questions and all nine models' traces, and the trace explorer.
Environment, harness, judges and the hack log: topoloop-ai/geogym.
The phone demo: video.