Closed loop at the code level - a new robotics philosophy
Many modern robotic systems are based on language-vision models (VLMs) that generate control decisions. Traditionally, in order for such models to work in practice, they need to be adapted to a specific task - a process known as fine-tuning. This requires large datasets and time. VLCP takes a different approach: instead of modifying the model, it leaves it unchanged (frozen) and uses it to generate a control function in Python. Importantly, it does not need examples or training interference.
The key innovation is closing the control loop at the code level itself, rather than at the decision or subtask level. When a robot makes a mistake - for example, fails to grasp an object - the system does not choose a new task or repeat the same function. Instead, every few steps, the model re-analyzes the scene from multiple perspectives and rewrites the control code that just failed. This allows for error correction during a single episode, before it becomes irreversible.
Effectiveness: ten times more successes than the traditional approach
Research conducted on the MuJoCo/RoboVerse task set, including 57 different manipulation tasks, showed a dramatic increase in effectiveness. The VLCP system achieved significantly better results than an identical system operating in open-loop mode (with one query per episode). This difference is not accidental - confidence intervals do not even overlap for individual scene families.
The main improvement comes from intra-episode error correction. The study showed that the system can recover some of the failed grasp cases - i.e., those situations that traditional open-loop systems treat as the end of the episode. When a robot fails to grasp an object, VLCP does not react at the decision level, but rewrites the control code, which may now work correctly in the new situation.
Computational efficiency: low cost and high performance
Despite the advanced replanning function, VLCP is not computationally expensive. The study shows that a large part of the input tokens goes to the cache, which significantly reduces the response time. For one episode, about 10 short queries are sufficient - no more than needed for one full processing.
Additionally, the control functions generated during replanning are stored in a skill library used in subsequent episodes. This means that the system not only improves during operation but also learns from its own experience - which can lead to long-term competence development without the need for new training.
Limitations and significance for the future of robotics
The results of VLCP are promising, but it should be remembered that they were achieved in a simulation - on the MuJoCo/RoboVerse platform. This means that there is no confirmation yet of its operation in the real world with physical robots and sensor noise. It is also worth emphasizing that the system only works for tasks that can be described as short Python functions - which limits its application to specific types of manipulation.
However, the significance of this work is enormous. It shows that robots can not only perform tasks but also analyze and improve their own code in real time. This is a step towards autonomous systems that are not limited to pre-programmed actions but can adapt at the control level itself.
