Closed loop at the code level - a new philosophy of robotics
Many modern robotic systems are based on language-vision models (VLMs) that generate control decisions. Traditionally, in order for such models to work in practice, they need to be adapted to a specific task - a process known as fine-tuning. This requires large datasets and time. VLCP takes a different approach: instead of modifying the model, it leaves it unchanged (frozen) and uses it to generate a control function in Python. Importantly, it does not need examples or training interference.
The key innovation is closing the control loop at the code level itself, rather than at the decision or subtask level. When a robot makes a mistake - for example, fails to grasp an object - the system does not select a new task or repeat the same function. Instead, every few steps, the model re-analyzes the scene from multiple perspectives and rewrites the control code that just failed. This allows for error correction during a single episode, before it becomes irreversible.
Effectiveness: ten times more successes than the traditional approach
Research conducted on the MuJoCo/RoboVerse task set, encompassing 57 different manipulation tasks, demonstrated a dramatic increase in effectiveness. The VLCP system achieved significantly better results than an identical system operating in open-loop mode (with one query per episode). This difference is not coincidental - confidence intervals do not even overlap for individual scene families.
The main improvement stems from intra-episode error correction. The study showed that the system can recover some of the failed grasp cases - i.e., those situations that traditional open-loop systems treat as the end of the episode. When a robot fails to grasp an object, VLCP does not react at the decision level but rewrites the control code, which may already work correctly in the new situation.
Computational efficiency: low cost and high performance.
Despite the advanced replanning function, VLCP is not computationally expensive. The study shows that a large portion of the input tokens are cached, which significantly reduces response time. For one episode, about 10 short queries are sufficient - no more than needed for a single full processing.
In addition, the control functions generated during replanning are stored in a skill library used in subsequent episodes. This means that the system not only improves during operation but also learns from its own experience - which can lead to long-term competence development without the need for new training.
Limitations and significance for the future of robotics.
The VLCP results are promising, but it should be remembered that they were achieved in simulation - on the MuJoCo/RoboVerse platform. This means that there is no confirmation yet of its operation in the real world with physical robots and sensor noise. It is also worth emphasizing that the system only works for tasks that can be described as short Python functions - which limits its application to specific types of manipulation.
However, the significance of this work is enormous. It shows that robots can not only perform tasks but also analyze and improve their own code in real time. This is a step towards autonomous systems that are not limited to pre-programmed actions, but can adapt at the control level.
