Understand the traveller.
Voice or text becomes mobility settings: stair access, lift requirements, shelter preference and instruction format. The user reviews the settings before applying them.
Language → preferences → route planningMaps, models and routing.
A shared routing engine connects floor graphs, mobility preferences and confirmed observations.
System architecture
Routing architecture.
Scene data
Official floor geometry, pedestrian paths and field anchors.
Routing context
Mobility constraints, preferences, weather and active events.
Journey steps
Dijkstra routes become floor, lift, bridge and outdoor instructions.
React and TypeScript on the phone. Express on the server. Shared domain contracts connect them.
Four AI workflows
Four AI workflows.
Qwen text and vision models work with the loaded map. Each workflow returns structured results.
Locate the affected facility.
Vision extracts the visible barrier and signage. Map retrieval selects nearby facilities or paths. A confirmed match creates a timed event for rerouting.
Photo → facility match → event overlayMatch what the camera sees.
The system reads names, floor signs and visible objects, then compares them with nearby map records. A confirmed candidate becomes the new route origin.
Photo → map candidates → current positionExplain the current step.
Confirmed route facts provide the distance, floor, facility and action. The guidance workflow turns those facts into a short Cantonese instruction for browser speech.
Route facts → Cantonese text → playbackPhoto matchingPhoto-to-map
Photo-to-map
matching.
Separating image observation from map matching keeps every candidate tied to an existing node or facility.
Read the image
The vision role extracts text, floor labels, landmarks and obstacle evidence.
Retrieve map candidates
Floor and position narrow the search. Name and OCR similarity rank nearby records.
Compare the evidence
The matching role checks the observation against the retrieved records. Scores combine identity, evidence, proximity and route context.
Confirm and apply
The user confirms the candidate. The server validates its map ID and current journey context before updating the remaining route.
Routing and state
Session coordination.
The phone and control console use the same server session.
Constraints before preferences
Mobility policies remove unavailable edges. The planner then evaluates distance, exposure, effort or time. Sheltered and indoor objectives compare exposure before secondary cost.
Events over a stable graph
Facility closures and path reports form an event overlay. The base graph stays unchanged, and event validity determines which restrictions apply.
Shared phone and console
HTTP commands update the journey. Server-sent events deliver versioned state snapshots to both screens, including after reconnection.
Current-position rerouting
Completed steps remain in the journey. Confirmed location changes and active events replace the remaining route from the current anchor.
Model and service interfaces
| Component | Implementation | Role |
|---|---|---|
| Language and vision | Qwen-compatible API | Structured preference, observation, matching and guidance output. |
| Speech | Browser speech services | Voice input and device Cantonese playback. |
| Weather | Hong Kong Observatory API | District observations, warnings and cached weather context. |
| Maps | Lands Department and MTR data | Floor geometry, pedestrian networks and station information. |
Text and vision model names are selected in the backend configuration. Data and service references