GAM: a new foundational model for robotic manipulation in industry

What is GAM and how does it work?

GAM - Generalized Action Model - is a new type of foundational model for robotic manipulation, developed by a team of Amazon Science researchers. Instead of creating separate models for different types of robot end-effectors, GAM acts as a unified solution that generates manipulation actions based on language and image. The model uses a vision-language-action (VLA) approach, which allows it to understand commands in natural language and translate them into specific movements, regardless of the specific hardware.

One of the key innovations of GAM is its ability to generate different types of actions - from pinching and suction gripping to caging techniques and object placement. All these actions are performed by the same model, which eliminates the need to adapt algorithms for each new end-effector. The model is also equipped with an end-effector encoding mechanism that enables zero-shot transfer - i.e., operation on new types of hardware without additional training.

How was GAM validated and where does it operate?

GAM was tested in real-world industrial conditions, on a fleet of robots working in production systems. During the tests, the model performed over 10 million pick-and-place cycles - i.e., picking up and placing objects - which is one of the most intensive tests in the history of manipulation robotics. The results were very high: grip success exceeded 95%, and placement success exceeded 90%.

An important aspect was also the model's ability to work with hybrid end-effectors, which have different gripping modes. GAM was able to effectively handle such systems without the need for separate training for each mode - which demonstrates its enormous potential for rapid integration in various production environments.

Scalable method for generating data in simulation

One of the biggest challenges in robotics development is the lack of access to large amounts of high-quality interactive data. GAM solves this by using a scalable, offline pipeline for generating data in simulation. The model translates observations from the real world - e.g., images and sensor data - into action candidates and quality labels in a simulated environment.

This method allows for the generation of large datasets without the need for robots to operate continuously in the factory, which significantly reduces the cost and time required to develop new functions. As a result, the model can be quickly adapted to new tasks or environments, even if there is no data from the real world yet.

Significance and limitations of GAM

GAM represents a significant step forward in the development of industrial robotics. Thanks to a unified approach to different end-effectors and the ability for rapid transfer, the model can significantly reduce the time required to deploy new automation systems. This may contribute to greater production flexibility, especially in sectors with high product variability.

At the same time, it is worth emphasizing that the model has only been validated on a specific type of task - pick-and-place - and in industrial conditions. There is no information about its performance in other contexts, such as domestic or clinical settings. Furthermore, although the model works in simulation, its effectiveness in the real world depends on the quality of the input data and the accuracy of the simulation model.

Prospects for industry and robotics integrators

GAM can become a key tool for robotic integrators who want to quickly deploy solutions in various facilities. Thanks to its ability to perform zero-shot transfer and scalable data generation, the company no longer needs to create a separate model for each type of end effector or new workstation.

For robot manufacturers, this may mean the possibility of offering systems with greater flexibility and faster adaptation to customer needs. At the same time, the model is not a commercial product - it is not available as a ready-made product - but rather the result of scientific research that can be used in further development projects.