Professional Engineering

Making a cup of tea or coffee is a relatively straightforward task. It requires coordination of movement, a set of the relevant equipment and an understanding of the correct order of doing things so that you end up with a satisfying brew. It’s not exactly difficult – well, for a human being, at least.

Yet this seemingly unexceptional challenge has been exercising the brains of robotics experts for the last 20 years. It’s a labour of love for Dr Michael Suppa, head of the department of perception and cognition at the Institute of Robotics and Mechatronics at the German Aerospace Centre (DLR). For the past two decades he has led an interdisciplinary team of mechanical and electrical engineers and software developers to get their robot – known as Rollin’ Justin – to make a cup of tea. It’s not as easy as it sounds.

“What began as a mechatronics development project has progressed to one almost entirely focused on the software that will deliver genuine perception and manipulation skills to robots,” says Suppa.

“Looking back, the lightweight robotic arms, the robotic hands, even the stereo vision system and the mobile base, were all relatively straightforward to put together. Now, almost all our research is about software, so that Justin can bring its perception into play in a dynamic environment and deal with change.

“It must be able to assess risk and decide what to do next by using its sensory system to validate data. This is why we’ve set it the tea and coffee-making challenge. Currently, it is being programmed to follow a set series of tasks to make those drinks, yet we are challenging it to make the drinks in much the same way you or I would.”

The first thing Justin must do is interpret the objects it can see with its stereo camera system and understand their significance to the task. Here Suppa is working with researchers across Europe to further develop Justin’s abilities. Dr Andrew Davison at Imperial College London is cooperating with DLR’s researchers in the sphere of navigation. 

“Once the robot starts moving and the scene is updated, the map he builds just gets better and better, but it is very complicated and very graphics-card processing power-hungry,” says Davison.

Once Suppa and his team made Justin mobile, it got much more complicated and the researchers had to go back to the drawing board to rework some fundamental algorithms. “We are constantly challenging Justin and making his life a little bit harder. First he had to find the objects required to make iced tea and understand which tasks to do in what order. Just to be really mean, we introduced transparent mugs, which flummoxed him for a while. Then we introduced a filter coffee machine and he had to use his sensors to realise how much to push. We just keep introducing more and more variants and making our scenario more complicated as a result.”

Even when Justin knows what objects he is working with and what he should be doing with them, he still has to decide where to grip the object and how tightly. The sensors in his fingers allow him to make little adjustments as his initial touch turns into a grip. As research partner Dr Jeremy Wyatt at the University of Birmingham explains: “We aren’t as visually guided as you might imagine.”

Wyatt says: “Trivial tasks are hard. Humans are very good at estimating the weight of an object before they interact with it, so they can answer the question, ‘how will this object move if I apply these forces?’ Bit by bit, Justin is using learning as a predictor for both mass and the forces he needs to apply over distance.

“The ultimate would be a physics engine that mimics the real world, something all the scientists working on his development aspire to. Although the complexity is still mind-numbing for this goal to be achieved, computer processing power has progressed so rapidly it is now a goal that is seen to be achievable.”