AccessRoute HK
ACCESSROUTE HK / METHOD 02

Retrieval-Augmented Visual Localization

A photo supplies clues. Map records supply coordinates. Follow how the system connects the two.

Try it yourselfMethod overviewRAG data flowFormulation AlgorithmImplementation
EXPLORE THE IDEA

Follow the clues from a sign to a map point.

Read the visible clues, look up nearby map records, then choose a supported location.

See the detailed RAG figure ↓

Prepared visual observations and model evidence; map retrieval and scoring run in your browser.

More uncertainty means a wider search area.

Synthetic landmarks and map nodes explain the mechanism. This walkthrough makes no live model calls.

Step 1 of 5 · Read the visible clues

RAG, in plain language

Retrieval-augmented generation gives the model relevant records before it responds. Here, the records come from a map. The model compares visible clues with those records; the program checks the evidence and looks up the selected node’s stored coordinates. A nearby point alone provides too little evidence to identify a landmark.

Scroll the figure horizontally to read all panels ↔

Three panels show a structured map index, image-to-candidate retrieval and evidence matching, then validation, score-based decisions and graph-node coordinate lookup.
Figure 2. Retrieved map records ground landmark matching and coordinate lookup. (a) Map index construction. (b) Visual observation, bounded retrieval and evidence matching. (c) Evidence validation, candidate ranking and map-node resolution.
Open vector figure ↗ Download PDF ↓ TikZ sources ↓
RETRIEVAL / EVIDENCE / MAP ANCHORS

From retrieved records to a map-grounded location

The expanded data flow exposes the records, model context and evidence that connect an image observation to a stored graph node.

Scroll the figure horizontally to read all panels ↔

Six panels show a symbolic photo query, node-linked map records with a mobility graph, a retrieved-candidate table, a structured evidence-matching prompt, observed-to-mapped text pairs with ranked candidates, and coordinate lookup on the stored graph.
Figure 3. Retrieved records supply the matching context; validated evidence ranks the candidate anchors. (a) Structured photo query. (b) Map records and their graph associations. (c) Bounded Top-20 retrieval. (d) Model input and evidence matching. (e) Span validation, scoring and candidate decisions. (f) Stored coordinate lookup for a ready or confirmed anchor. Solid graph lines denote mobility edges; dashed record links denote stored associations. Retrieval uses floor, spatial and lexical filters. L, M, D and R denote lexical, validated-evidence, spatial and route scores; γ is observation reliability and κ is the association cap.
Open vector figure ↗ Download PDF ↓ TikZ sources ↓
01 / FORMULATION

Map records bind visual identity to graph nodes

The index K combines named map units, amenities, facility anchors and landmark-directory records. A record k stores names and aliases Nk, descriptions Tk, a category, source, graph node vk, optional floor ℓk and association type. The graph supplies the output coordinate x(vk).

For named unit and amenity geometry, indexing selects a junction, entrance or point-of-interest node on the same floor within 20 metres of the feature centre. Landmark-directory entries supply their own node associations. Facility records attach to graph endpoints associated with that facility.

Spatial filtering bounds the candidate set

The vision model maps image I to an observation O: visible names, text, objects, floor label, direction hint and reliability γ. The workflow resolves a visible floor label against the scene manifest. An unknown label or a conflict with the supplied floor produces an empty match set.

r = max(60, 2a) m with photo position;   r = 120 m otherwise
(1)
C = {k ∈ K : landmark(k) ∧ floor-compatible(k) ∧ dist(k, x0) ≤ r}
(2)

The centre x0 is the supplied photo position, or the current graph node. The uncertainty radius a comes from the supplied position accuracy. With no floor constraint, retrieval includes nearby records on all floors. Localization excludes path-edge records.

Text retrieval supplies evidence to the matching model

Normalization applies Unicode NFKC, lowercasing and removal of punctuation and spaces. The lexical matcher compares character-bigram sets with Dice similarity, retaining exact-name matches. Generic facility names receive a 0.35 multiplier; description matches receive a 0.85 multiplier. The shortlist keeps the 20 highest lexical scores, with distance breaking ties.

Dice(A, B) = 2|A ∩ B| / (|A| + |B|)
(3)

The matching model receives the observation and retrieved records, then returns candidate IDs with paired observed and mapped text spans. Programmatic validation keeps a pair only when both normalized spans contain at least two characters and occur in their declared sources. Optional user clues remain a separate evidence source. In the live workflow that accepts these clues, the lexical score takes the maximum of photo-based and user-description similarity.

Retrieval representation. This implementation uses spatial filters, lexical retrieval and model-assisted evidence matching. Its map index stores structured records and text. The evidence indicator below records valid text-span pairs.

Ranking combines identity evidence with spatial context

Sk = round3(min{γ, κk, 0.70Lk + 0.15Mk + 0.10Dk + 0.05Rk})
(4)
Candidate score terms
SymbolDefinition
LkMaximum supported lexical similarity for the candidate.
Mk1 if at least one evidence pair survives validation; 0 otherwise.
Dkmax(0, 1 − dist(k, x0) / 120 m).
Rkmax(0, 1 − distance to the nearest same-floor route node / 50 m); zero if none exists.
γReliability of the visual observation.
κk0.79 for an approximate association; 1 for other associations.

The fixed spatial-score scale is 120 m even when the retrieval radius differs. Scores express retrieval support; the implementation has no probability calibration.

02 / ALGORITHM

Evidence gates the selection of a map anchor

Algorithm 2 Retrieval-Augmented Landmark Localization

Input: Image I, map index K, graph G, position / floor / route context, optional user clues.

Output: ready with a node reference, needs_confirmation with candidates, or no_match.

  1. O ← visual observation of I; resolve its floor against the scene manifest.
  2. if a visible floor label is present but unrecognized, or its resolved floor conflicts with the supplied floor, then return no_match.
  3. C ← nearby, floor-compatible landmark records from K.
  4. C20 ← top 20 records by lexical score, breaking ties by distance.
  5. if C20 = ∅ then return no_match.
  6. H ← model evidence matches between O and C20.
  7. Validate observed and mapped spans in H against their declared sources.
  8. Compute Sk for each candidate; sort by score, then distance.
  9. Let k1 be the top record; let k2 be the highest-ranked record on a different node.
  10. if S1 < 0.20 then return no_match.
  11. if S1 ≥ 0.80 and S1 − S2 ≥ 0.12 then
  12. return ready with node v1 and its floor reference.
  13. return needs_confirmation with the ranked candidates.

Use S2 = 0 when no candidate refers to a different node. Records that share one node represent the same approximate location. Approximate associations remain below the 0.80 ready threshold.

03 / IMPLEMENTATION

Coordinate lookup connects localization to routing

k* → vk* → (lon(vk*), lat(vk*), ℓk*)
(5)

The workflow validates node and floor references against the scene. Pending workflow confirmations expire after five minutes and bind to their originating journey context. In the phone session flow, the user confirms the selected anchor before the session updates its current position and replans the remaining journey. The returned coordinate denotes the associated map anchor near the landmark.

Execution boundary. This pipeline describes live backend inference. The static showcase uses prepared observations and anchor associations. Its preset scores belong to those demonstrations; live matching scores come from the evidence-ranking workflow.
METHOD 01 Policy-constrained route planning