In the RePAIR project, the Bonn team is responsible for the robotic system. The team has developed planning algorithms for the robot's bimanual arm movements, which enable it to place fresco fragments with great precision. Maren Bennewitz is Professor of Humanoid Robotics and Vice Rector for Digitalization and Information Management. She reports: “We are contributing our many years of experience in motion planning and manipulation by robots to RePAIR. The aim of the project was to investigate whether robots can take over the tedious sorting and puzzle work so that experts can devote their time to areas where human expertise is indispensable.” The University of Bonn was also responsible for integrating and evaluating the entire robotic platform and ensuring that all components fit together seamlessly. To this end, doctoral student Nils Dengler spent a total of five weeks in Pompeii this year.
The project, funded by the European Union, was coordinated by Ca' Foscari University of Venice. It combines state-of-the-art methods of artificial intelligence and robotics with the needs of archaeology. Other project partners include the Italian Institute of Technology (IIT), the Archaeological Park of Pompeii, Ben-Gurion University in the Negev, and the Instituto Superior Técnico in Lisbon.
Bonn expertise for the “Robo-Puzzler”
At the heart of RePAIR is a robotic platform with two robotic arms and soft gripping hands that pick up fragments of wall paintings and place them back in the position calculated by the AI puzzle solver. This is precisely where the Bonn-based research comes in: “Our algorithms calculate the movements for the two robotic arms that will reliably pick up the fragments and carefully place them in the calculated position,” says Nils Dengler.
Fresco puzzle from Pompeii
The innovative system was tested in the Archaeological Park of Pompeii. Among other things, ceiling paintings from the “House of the Painters at Work” and frescoes from the Schola Armaturarum are stored there, which were severely fragmented by the eruption of Vesuvius in 79 AD, as well as by bombing during World War II and a collapse in 2010. The fragments are first digitized using a specially developed 3D scanning system. On this basis, the AI attempts to solve the “puzzle”: it suggests suitable combinations between fragments and calculates how larger image segments can be recreated from hundreds or thousands of pieces.
“The challenge is that, unlike a normal jigsaw puzzle, we don't have a picture on the box,” says Maren Bennewitz. “Many pieces are badly damaged or missing altogether, and fragments from different works are often mixed up. This makes it all the more important for robotics and AI to work closely with archaeological expertise.”
Relief for archaeologists
The reconstruction of broken finds is a task that requires a great deal of patience and often remains undone for years in everyday museum work. By largely automating the time-consuming steps of digitization, proposal calculation, and physical reconstruction, RePAIR allows archaeologists to focus more on scientific evaluation. The knowledge developed in the project is not limited to Pompeii. All over the world, large quantities of fragments of ceramics, wall paintings, and architectural elements are stored in depots and warehouses, their reconstruction never having been started due to lack of time.
Prof. Bennewitz sums up: "It is fascinating to see how robotics not only solves tasks in factories or logistics, but can also help preserve our cultural heritage. RePAIR also exemplifies how interdisciplinary collaboration can work in Europe. For us in Bonn, the project was an important building block in bringing our robotics research even more strongly into socially relevant applications. "
The RePAIR project was funded under the European Union's Horizon 2020 funding program under Grant Agreement No. 964854.
Further information on the project can be found at:
» Website
In this video, you can see the robot at work:
» YouTube
