A new standard in data for bimanual manipulation
FineART is the most extensive dataset to date for robots performing bimanual tasks. It contains 40,543 episodes, which corresponds to 1,718 hours of operation and over 533 thousand subtasks distributed across 151 different tasks. Unlike previous datasets, which often contained only one top-level command per episode, FineART offers detailed annotation of each step - meaning that every movement and decision is recorded in detail. This allows models to learn not only "what to do", but also "how to do it" in the context of a specific task.
Such deep annotation is crucial for the development of robots that must operate in real-world environments - e.g. Robotics in additive manufacturing, rehabilitation or home - where a single command is not enough, but complex, multi-step actions are needed. The dataset has been made available to the scientific community and will be available on Hugging Face, allowing researchers and engineers to test new approaches without having to collect their own data.
FineART-VLA model: learning by predicting subsequent steps
Along with the data set, FineART-VLA was also introduced - a vision-language-action policy that can predict the next subtask during operation. Instead of waiting for a complete command from the user, the model decides on its own what to do next, based on the image and context. This approach, known as mid-training, allows for much faster learning and better adaptation.
Research has shown that in spatial reasoning - where the robot must determine which of two similar objects should be moved - accuracy increased from 32.0% to 100.0%. In addition, after minimal adjustment on a new robot, the model requires ten times less data than traditional models without mid-training. This means that the development of systems for new devices can be much faster and cheaper.
Zero-shot generalization - key to robot flexibility
Enlarged imageClose zoomPrevious imageOne of the most important achievements of FineART-VLA is its ability to perform zero-shot generalization. The model can operate on completely new tasks and on new hardware without requiring additional training on data from that device. This means that after a single adaptation, it can be used in various contexts - for example, in a factory, in a clinic, or at home - without the need to re-collect data.
Such flexibility is key to the development of robots that are designed to operate in diverse and dynamic environments. Instead of creating separate models for each device or task, it is possible to use a single solution that learns and adapts. This reduces costs, shortens implementation time, and opens the way to more independent systems.
Significance for the future of robotics
FineART and FineART-VLA are not only new research tools but also a potential step forward in the development of real-world robots. Thanks to detailed data and learning techniques that allow for rapid adaptation to new tasks, the systems can be more independent and flexible. This may be important for the industrial sector, where robots need to handle various products, or in rehabilitation, where individual solutions are needed.
However, it is worth remembering that the model has been tested in a simulated environment and on specific tasks. Its effectiveness in the real world, especially under unpredictable conditions, requires further research. It is not yet a ready-made product for mass deployment, but it represents an important step towards more intelligent and autonomous robots.
The significance of FineART for the future of robotics goes beyond technical achievements - it is a new paradigm in the development of autonomous systems. Thanks to the possibility of zero-shot generalization and low data requirements, models such as FineART-VLA can be quickly deployed in various sectors, from industrial production, where robots need to handle diverse products without the need for reprogramming, to rehabilitation, where an individual approach to the patient requires flexible solutions. At home, such systems can assist elderly or disabled people with daily tasks, adapting to their needs without the need for detailed input of new instructions.
Although the model has been tested in a simulated environment and on a limited set of tasks, its potential for working in the real world is significant - especially when considering scalability and reducing deployment costs. In the future, this may lead to the creation of universal robots that not only perform tasks but also learn to perform them in a way that is understandable to humans, which opens the way to more natural cooperation between people and machines.
