The PDDL-ART method allows robots to automatically create detailed manipulation plans based on a single demonstration and natural language description. The study showed high effectiveness in challenging tasks requiring abstract reasoning and memory, thanks to the ability to independently use internal VLM tools.
A task that cannot be solved without abstraction
Many manipulation tasks in real-world environments - from engine servicing to food preparation - require long-term planning and memory. Robots that operate only on images often lose context: they do not understand what 'moving a button' means in a situation where the button looks the same after the action as it did before. This is precisely the key challenge for symbolic robotics - the need for an abstract description of states and actions that are not directly visible in images.
PDDL-ART solves this by integrating language-vision models with a formal approach to planning. Instead of learning only from visual data, the system uses the VLM's ability to 'think' - e.g., calculating distances between objects or analyzing the sequence of actions - which allows it to understand relationships that are not visible in the image.
How PDDL-ART learns from a single demonstration
PDDL-ART works autonomously: it receives a single demonstration of movements, a description of the task in natural language, and a list of available high-level actions. Based on this, it generates a complete description of the PDDL domain - that is, a set of rules, states, and actions - which can be used by a symbolic planner. The key is that it does not require any templates or model fitting - the system independently discovers the structure of the task.
The process does not end with code generation. PDDL-ART runs a multi-stage correction process: first, it checks for syntactic correctness, then semantic correctness (whether the plan makes sense), and finally, execution - whether the robot can execute it in reality. In this last stage, the system uses internal VLM tools to solve geometric and temporal problems that are not visible in the images.
Effectiveness: 93.3% success rate in difficult tasks
Enlarged imageClose zoomPrevious imageResearch conducted on complex tasks - both in a home environment and in an engine service center - shows that PDDL-ART achieves an average success rate of 93.3%. This means that in more than nine out of ten cases, the robot performs the task correctly. Compared to a basic VLM-based planner, which achieved only 78.3%, the difference is significant and demonstrates real progress.
In particular, the system copes with tasks requiring memory - e.g., remembering where an object was placed - and with abstract reasoning, when the final state does not visually differ from the initial one. This shows that PDDL-ART not only "sees" but also "understands" the context and purpose of the task.
What does this mean for the future of robotics?
PDDL-ART is not a product ready for deployment, but it demonstrates the potential of a new approach: integrating language-visual models with formal symbolic planning. If such systems develop, robots may be able to learn complex tasks without the need for manual programming of each action.
However, there are limitations: the method works in controlled environments and requires access to a high-performance VLM. In practice, this may mean the need for powerful computing or cloud connectivity. Furthermore, the system does not confirm whether its plans are safe in the real world - e.g., whether they will cause damage to equipment.
PDDL-ART opens up new possibilities for robotics in industry, medicine and home. Thanks to the ability to automatically generate plans from a single demonstration, the system can significantly reduce the time needed to implement new processes. In factories, where changes in the production line are frequent, robots can quickly adapt to new tasks without manual programming. At home, such solutions can assist elderly or disabled people in everyday activities, such as preparing meals or tidying up. Although the system currently works in controlled environments, its concept can be developed into more flexible and infrastructure-independent solutions.
It is crucial that future versions also take into account safety and ethical aspects - e.g., preventing damage to equipment or avoiding actions potentially dangerous to humans. PDDL-ART not only solves the problem of abstract planning but also translates it into practical applications that can change the way robots interact with people.
