EliteTable introduces a self‑improving program‑search algorithm that leverages large language models to iteratively refine control policies. In a series of Gymnasium benchmarks, the method consistently raised candidate scores and transferred gains to unseen episodes.
Core Mechanics of EliteTable
- Employs a fixed‑capacity table of top solutions to steer new proposals, edits, and remixing.
- LLMs generate candidate programs and evaluate them against a reward function.
- Assessed on six Gymnasium control tasks with ten generations and fifty candidates per generation.
- Median candidate scores rose from first to final generation; winners hit the score ceiling early in CartPole‑v1.
- Transfer to held‑out episodes succeeded in four tasks, though Acrobot‑v1 showed a negative median change.
From LLMs to Control Tasks: The Evolution of Self‑Improving Search
Building on prior work that used language models for symbolic program synthesis, EliteTable extends the approach to continuous control domains. By maintaining a curated table of high‑scoring solutions, the algorithm balances exploration and exploitation, allowing rapid convergence on effective policies across diverse environments such as BipedalWalker and CarRacing.
What Comes Next: Scaling, Benchmarking, and Real‑World Deployment
Future work will benchmark EliteTable against established reinforcement learning baselines and explore larger, more complex tasks. Integrating the method into real‑time robotics and autonomous systems could unlock faster policy iteration and more robust transfer learning, positioning LLM‑guided search as a cornerstone of next‑generation AI control.