The PDDL-ART method enables robots to automatically create detailed manipulation plans based on a single demonstration and natural language description. The study showed high effectiveness in challenging tasks requiring abstract reasoning and memory, thanks to the ability to independently use internal VLM tools.
A task that cannot be solved without abstraction
Many manipulation tasks in real-world environments - from engine servicing to food preparation - require long-term planning and memory. Robots that operate only on images often lose context: they do not understand what "pressing a button" means when the button looks the same after the action as it did before. This is precisely the kind of problem that becomes a key challenge for symbolic robotics - the need for an abstract description of states and actions that are not directly visible in images.
PDDL-ART solves this by integrating language-visual models with a formal approach to planning. Instead of only learning from visual data, the system uses the VLM's ability to "think" - e.g., calculating distances between objects or analyzing the sequence of actions - which allows it to understand relationships that are not visible in the image.
How does PDDL-ART learn from a single demonstration?
PDDL-ART operates autonomously: it receives a single demonstration of movements, a description of the task in natural language, and a list of available high-level actions. Based on this, it generates a complete PDDL domain description - that is, a set of rules, states, and actions - which can be used by a symbolic planner. Crucially, it does not require any templates or model fitting - the system independently discovers the structure of the task.
The process does not end with code generation. PDDL-ART runs a multi-stage correction process: first, it checks for syntactic correctness, then semantic correctness (whether the plan makes sense), and finally, execution - whether the robot can execute it in reality. In this last stage, the system uses internal VLM tools to solve geometric and temporal problems that are not visible in the images.
Effectiveness: 93.3% success rate in challenging tasks
Research conducted on complex tasks - both in a home environment and in an engine service facility - shows that PDDL-ART achieves an average success rate of 93.3%. This means that in more than nine out of ten cases, the robot performs the task correctly. Compared to a basic planner based on VLM, which achieved only 78.3%, the difference is significant and demonstrates real progress.
In particular, the system handles tasks that require memory - for example, remembering where an object was placed - and abstract reasoning when the final state does not look visually different from the initial one. This shows that PDDL-ART not only "sees", but also "understands" the context and purpose of the task.
What does this mean for the future of robotics?
PDDL-ART is not a ready-to-use product, but it demonstrates the potential of a new approach: integrating language-visual models with formal symbolic planning. If such systems continue to develop, robots may be able to learn complex tasks without the need for manual programming of each action.
However, there are limitations: the method works in controlled environments and requires access to a high-performance VLM. In practice, this may mean the need for significant computing power or cloud connectivity. Furthermore, the system does not verify whether its plans are safe in the real world - for example, whether they will cause damage to equipment.
PDDL-ART opens up new possibilities for robotics in industry, medicine and the home. Thanks to its ability to automatically generate plans from a single demonstration, the system can significantly reduce the time required to implement new processes. In factories, where changes in the production line are frequent, robots can quickly adapt to new tasks without manual programming. At home, such solutions can assist elderly or disabled people with daily activities, such as preparing meals or tidying up. Although the system currently works in controlled environments, its concept can be developed into more flexible and infrastructure-independent solutions.
It is crucial that future versions also take into account safety and ethical aspects - for example, preventing damage to equipment or avoiding actions that may be dangerous to people. PDDL-ART not only solves the problem of abstract planning, but also translates it into practical applications that can change the way robots interact with humans.
