The FLARE framework allows manipulation robots to independently correct errors and recover from failures during challenging tasks requiring physical contact, significantly improving performance and robustness even in unforeseen situations.
Problem: Lack of ability for a robot to self-correct errors.
Vision-language action models (VLAs) are currently one of the most promising approaches to manipulation robotics. They enable general understanding and execution of complex, multi-step tasks. However, their main weakness is low robustness to execution errors - they are only trained on ideal demonstrations without errors. In practice, this means that if a robot fails to grasp an object or collides with an obstacle, it cannot independently recover and continue the task.
This limitation is particularly critical in complex scenarios where physical contact with the environment is important - for example, when assembling objects, arranging tools, or manipulating soft objects. The lack of recovery mechanisms makes systems sensitive to even minor deviations and often leads to failure.
Solution: FLARE - a framework with retry and reset mechanisms for autonomous recovery.
FLARE is a new framework designed specifically to handle failures in real time. It is based on a two-stage mechanism: "Retry" and "Reset". The "Retry" mechanism works by introducing disturbances and bridge segments into the training data, which separates the robot's state from the environment's state. This allows the model to learn not only how to perform the task but also how to react to deviations - for example, if a grasp fails on the first attempt, it can try again with a slight change in position.
For more serious out-of-distribution (OOD) failures, such as an object falling or being moved in an unpredictable way, FLARE activates the "Reset" mechanism. In this case, a language-mathematical model is used to analyze the recorded video of the task execution and identify critical states. Based on this, a small set of specialized recovery skills is generated, which restore the environment to a state acceptable for further task execution.
How does FLARE work in practice?
Enlarged imageClose zoomPrevious imageIn real time, FLARE uses an online MLLM (multimodal large language model) to monitor the progress of the task. This model decides whether to continue executing the task or activate the "Reset" procedure. The decision is based on analyzing the current state of the environment and comparing it with the model's expectations. If it detects a deviation exceeding acceptable limits, it automatically launches the appropriate recovery skill.
The integration of these mechanisms allows for a smooth transition between task execution and its correction. Studies conducted on difficult tasks with a high degree of physical contact have shown that FLARE significantly improves the effectiveness and robustness of operation - which is crucial for deployments in real-world industrial or clinical settings.
Significance and limitations of the method
FLARE represents a significant step forward in the development of autonomous robotics. It demonstrates that systems can be not only intelligent but also resilient - capable of recovering from errors, which is essential for their use in dynamic and unpredictable environments. This may accelerate the deployment of robots in manufacturing, logistics or rehabilitation.
At the same time, it is worth noting that FLARE has only been tested on simulated models and under controlled conditions. There is no data on its performance in real industrial installations or in home environments. In addition, efficiency depends on the quality of failure analysis by MLLM - if the model does not recognize an error, it may not activate the appropriate procedure.
It is worth emphasizing that FLARE not only improves operational efficiency but also increases trust in robots in environments where unpredictable situations are the norm. Thanks to the "Retry" mechanism, the system learns to tolerate small deviations, which is crucial when manipulating precise or delicate objects, such as glass or electronic components. The "Reset" mechanism, in turn, ensures resilience to failures of a larger scale - for example, when an object falls off a surface or is moved by an external factor.
In such cases, it not only identifies the problem but also generates a specific strategy for restoring the initial state, which allows the task to be continued without human intervention. Although the framework has been tested mainly in simulations, its architecture is flexible and can be adapted to real systems with sensors and time constraints. A key advantage of FLARE is also its operation in real time - thanks to online MLLM, which monitors the progress of the task, decisions are made without delay, which is essential in dynamic industrial scenarios. However, its effectiveness depends on the quality of failure analysis and the accuracy of the language model.
