Researchers unveiled the Trajectory‑Grounded Instruction Translator (TGIT), a front‑end that keeps a frozen aerial vision‑and‑language navigator unchanged while translating terse, intent‑driven commands into executable instructions, slashing the instruction gap and boosting success rates from 11% to 33% on real human inputs.
Core Facts
- TGIT learns from trajectory outcomes to map weak, intent‑centered commands to agent‑friendly actions.
- Success rate climbs from 11.33% to 32.51% on real human instructions, and from 15.27% to 37.93% on weak‑trained inputs.
- Zero‑shot transfer improves held‑out OpenFly performance from 4.95% to 20.79% and yields recovery on CityNav and AirVLN benchmarks.
Background
Vision‑and‑language navigation (VLN) agents traditionally rely on detailed, trajectory‑aligned directives, a mismatch with the concise, intent‑driven language users naturally employ. Prior work addressed this gap by prompting language models with human‑written style examples, but the resulting weak commands still underperformed. TGIT builds on this foundation by anchoring translation to the agent’s own trajectory feedback, effectively bridging the semantic divide without retraining the core navigator.
Future Impact
By decoupling translation from the navigator, TGIT paves the way for rapid deployment of aerial VLN systems across varied domains—delivery drones, search‑and‑rescue, and autonomous inspection—without costly re‑training. Its zero‑shot adaptability suggests a scalable path toward multilingual, multimodal instruction handling, potentially redefining how humans interact with autonomous aerial platforms.