robotics r&d

Robotnik at DECENTER: Mobile robotics + artificial intelligence

Interview PhD- Decenter Project Coordinator.

Artificial Intelligence is a technology in constant evolution and its possibilities and influence in different areas is really wide. There are numerous AI applications related to mobile robotics that have a direct relationship with different industrial sectors. 

All these applications, as well as the habits of today's society, make it necessary to focus our attention on one key point: What do we do with the huge amount of data we generate? 

Robotnik is part of DECENTER, a European project within the H2020 program whose aim is precisely to manage, protect and process this data efficiently and securely. 

Ángel Soriano, responsible for Robotnik in this project, tells us a little more about DECENTER, telling us how it has evolved since its inception, at what stage are the objectives, the conclusions they draw and more.

 

1. Starting the conversation, could you give us a summary of what exactly DECENTER consists of?

DECENTER is a project that aims to provide an infrastructure that facilitates the integration and deployment of their services or applications to Artificial Intelligence and Cloud&Edge computing oriented software developers.

The project contemplates 3 different profiles:

  1. Programmers and architects oriented to distributed computing or edge/cloud computing management.
  2. Developers focused on AI algorithms, which normally require large computational capacity and memory.
  3. Operators or technical staff responsible for the proper functioning and integration of the software deployed on the available hardware.

 

 

DECENTER is intended to make it easier for the technician to integrate AI processes and/or algorithms into the system already deployed in their factory, and also to process them on the edge or in the cloud.

An example: we have a professional who is dedicated to developing an algorithm that is able to identify robots in a photograph. On the other hand, we have a worker-operator who knows his system, who is able to take that photograph, but is not a specialist in Artificial Intelligence. How can he integrate the advantages offered by AI without needing to specialize in it?

DECENTER aims to create these connections and distribute the computation (these algorithms require a lot of computing power) at different levels. 

A local level would be to run it inside the same computer that takes the picture. 

A level above that would be to run it on a computer that is not the same as the one taking the picture, but is within the same region or relatively close (Edge).

And a level above that would be to run it in the cloud.

By playing with these 3 levels, the project offers a platform that facilitates this integration and communication.

 

What role does Robotnik play in the project?

Everything explained above applies to 4 use cases that are defined in the project. Robotnik is a supplier of one of the use cases of the project: robotics logistics.

When a robot is performing its autonomous tasks navigating through the warehouse, it has to make certain decisions based on unforeseen events, such as encountering obstacles blocking its path. On certain occasions, if the robot is not able to identify what the obstacle is, the safest option is first for the robot to stop immediately and then to stop: 

  • Either that the robot remains static waiting until the obstacle disappears to ensure that there is no risk of collision. 
  • Or for the robot to re-plan its trajectory to reach its target from another location and try to avoid the obstacle. 

This is a very common situation in fleets of collaborative mobile robots as they often share workspace with each other and with workers in the same warehouse. 

DECENTER opens up the possibility of applying obstacle identification through AI. 

 

 

Robotnik provides the robots and all the infrastructure, as well as developing the integration of the DECENTER platform within the robot and the structure that we already had and thanks to this service offered by the partner and the project, we have been able to branch out the options for managing fleets of robots in warehouses.

 

2. At this point, which of the objectives set have been met and in which aspects will it be necessary to continue working on?

An example of an objective achieved in the specific use case of Robotnik is: we set out to be able to identify another robot with a confidence level of more than 80% and to reduce the CPU usage of robots by 10% by deriving computation to the edge/cloud. Both objectives have been successfully achieved.

 

3. What does this project mean for Industry 4.0?

There are several things that DECENTER brings to Industry 4.0 directly. One of them is the management of computing: being able to route and manage computing between devices. Distributed computing is something that is now the order of the day and it's purely cutting edge. We even talk about doing it in an intelligent way based on objectives or priorities. 

In Robotnik we design and manufacture mobile robots. If we want a robot to run a very heavy algorithm and with a lot of computational capacity (this happens in many occasions: high resolution cameras, LiDAR that captures 320.000 different points every second...), we must enable it to do so.

The straightforward solution is for the robot's CPU to be very powerful, which usually means it is more expensive. However, with the ability to delegate computation, you have the possibility to send all that data using communication with other devices or the cloud, compute the data there and apply the returned result to the robot.

This reduces costs from the point of view of robot production, makes the system more intelligent as it reduces the electrical consumption of the robot, optimises energy and in general optimises resources.

 

4. Taking into account both large companies and SMEs, what is the biggest challenge for the real application of Artificial Intelligence in industry?

The biggest challenge is undoubtedly that AI is constantly evolving and expanding and its integration in industry requires specialized personnel. Currently there is a gap between the specialized AI community and the personnel involved in the industry who have to deal with the problems of their own industry.

DECENTER tries to bring the two profiles together so that industry can benefit from the application of AI without the need to be an AI specialist.

 

 

 

There is another technological challenge involved in applying the DECENTER methodology in a real industry application, communications. When computing is delegated between devices, and these devices require a certain real-time processing speed, communication between devices needs to be fast and secure. 

At DECENTER we have worked at several levels: 

EDGE. Here you do not need internet to communicate with another device and it is considered a much faster level in terms of communications than the cloud.

The cloud. Here you do need an internet connection to access the servers and it is normally assumed to be slower than EDGE. 

Therefore, in general, communications is the great technological challenge of this methodology. Here we could also talk about 5G, which is another field in which Robotnik is actively participating in other projects.

 

5. Which use cases have had more impact or which sectors have shown interest in DECENTER's advances?

DECENTER is mainly related to data processing, its speed in data transmission and computational capacity. Computational resources. 

All projects have a development process in which use cases are always defined to demonstrate that the development is applicable to real applications. In the end, it is necessary to demonstrate in which areas of interest what is developed during the project will work.

In our DECENTER use case, the stakeholders are anyone who uses logistics robotics.

It can be interesting for any warehouse management, both for the savings it will bring to the company and for the technological and productive leap. 

During the DECENTER pilot, the RB-1 BASE was tested with very attractive results. Now I don't have individual robots, but I have a fleet of computers that 'help' each other to optimize processes, not only from a physical point of view, but above all from the data processing and execution of each one of them: if one of your robots is stopped loading, you can use the computer or PC of that robot to process data from the robot that is moving.


industry 4.0

Mobile robots and safety: the experience of Robotnik in HR-RECYCLER project

Collaborative robots have come front and center on the international stage as they’ve become widespread in Industry 4.0. Today we have more powerful, more advanced and more productive robots, so safety has become a key element.

Safety is the key

For Robotnik, as an experimented robot manufacturer and within the collaborative environment of the HR-Recycler project, this aspect is especially important since humans and robots will be working side by side. The solution proposed to routing materials inside a factory has to be done in a safe manner, in this case, the robots designed are the RB-KAIROS+ (mobile robotic manipulator) and the RB-ARES (pallet truck). It’s really important how the mobile robots  will show the intention of motion, elevate or manipulation.  

industry 4.0

 

To ensure the correct operation within the complex framework of this project, Robotnik has equipped its robots with sensors and signalers that allow the robot to proceed safely and show its intentions in advance. 

There are a number of ways manufacturers can introduce safety measures in their automated operations. The type and complexity of these safety measures will vary by the robotic application, with the aim to make the mobile robot safer, there are certain safety rules and standards that these collaborative robots must comply with, in Europe are found in EN ISO 3691-4:2020 and ISO 12100:2010 6.4.3

rb-ares mobile robot

Clarifying the ISO standard

This post aims to give to the reader a brief description about what, why and how all the premises of the ISO will be reached.

First of all, what does the normative include? The standards on warning systems say:

  1. When any movement begins after a stop condition of more than 10 seconds, a visible or acoustic warning signal will be activated for at least 2 seconds before the start of the movement.
  2. A visible or acoustic warning signal will be activated during any movement.
  3. If the human detection means are active, the signal will be different.
  4. When robots change their direction from a straight path, a visible indication will be given of the direction to take before the direction changes in case that the robot is driving autonomously.
  5. When the lift is active, there must be special signage.

The solution proposed is a two-steps software that will manage the signals of the robot, explained after the diagram and on red cells:

The robot_local_control is a manager node, it has information about the status of the whole robot, that is, status of the elevator, goal active, mission ended, etc. On the right side, a group of nodes that manages the movement of the robot with a level of priority:

  • Robotnik_pad_node:  The worker uses a PS4 pad to control the robot and this node will transmit the orders, non autonomous mode. 
  • Path planning nodes: like Move_base, it controls the robot and we speak of it as autonomous mode.

Robotnik has installed on its mobile robots  two ways to alert facility users, acoustic devices or light indicators through the acoustic_safety_driver and leds_driver.

industry 4.0

As you can see, there are two steps to link the top and bottom parts, a node to transform the movement into signals to show the intention of the robot and another one to orchestrate the both signal types and manage the requirements of the normative. 

The turn_signaling_controller aims to solve the first and the fourth requirements of the normative depending on the robot mode (autonomous or not autonomous). 

In non autonomous mode, and as the norm says, the motion depends on an appropriately authorised and trained personnel so it is enough to show that the robot is moving by reading the movement command and checking the velocity applied. 

In autonomous mode the robot navigates to a goal point through a path calculated by the planner, furthermore it manages the AGV to avoid obstacles dynamically and for this reason it is important to alert workers every moment.

What is the process?

This is a very brief description of the function, it bears the plan in mind and recalculates at the same time that the planner does just to be able to show the most up-to-date prediction of motion.

Last but not least, the robot_signal_manager aims to solve the rest of the problems since it has access to the robot status, it shows a light signaling or an acoustic signal 2 seconds before the motion, it gives priority to the emergency signals (consistent with the behaviour of the robot, red signals means that the robot will be stopped) and the signals that are not exclusive are showed using beacons or acoustic signals.

The occupied zone is one of the non exclusive signals, robots have some extra beacons that blinks on red when there is something on the protective zone (close to the intention of motion of the robot, inside the critic zone) and on yellow when there is something on the warning zone (near the protective zone).

 

Summarizing

Safety is not only stopping the robot or avoiding a crash when human-robot collaboration takes place. With the development of these nodes Robotnik aims not only to decrease the probability of accident or comply with the safety ISO premises, but also to help workers feel more comfortable with the mobile robot’s decisions and bring human-robot collaboration closer, showing clear signals about how the robot will perform.


dissasembly area

How to use MoveIT to develop a robotic manipulation application

European Commission funded HR-Recycler project aims at developing a hybrid human-robot collaborative environment for the disassembly of electronic waste. Humans and robots will be working collaboratively sharing different manipulation tasks. One of these tasks takes place in the disassembly area where electronic components are sorted by type into their corresponding boxes. 

An easy-to-use robotic manipulation platform

To speed up the component sorting task Robotnik is developing a mobile robotic manipulator that needs to pick boxes filled with disassembled components from the workbenches and transport them either to their final destination or to further processing areas. MoveIt is an open-source robotic manipulation platform that allows you to develop complex manipulation applications using ROS. 

Here, a brief summary showing how we used MoveIt functionalities to develop a pick and place application will be presented.

Features and benefits of MoveIT

We found MoveIt to be very useful in the early stages of developing a robotic manipulation application. It allowed us to decide on the environment setup, whether our robot is capable of performing the manipulation actions we need it to perform in that setup, how to arrange the setup for the best performance, how to design the components of the workspace the robot has to interact with so that they allow for the correct completion of the manipulation actions needed in the application. 

Workspace layout

MoveIt allows you to build the planning scene environment using mesh objects previously designed in any cad program and allows your robot to interact with them. 

With MoveIt you can plan towards any goal position not only taking into account the environment scene by avoiding objects but also interacting with it by grabbing objects and including them in the planning process. Any MoveIt scene Collision Object can be attached to the desired robot link, MoveIt will then allow collisions between that link and the object, once attached the object will move together with the robot’s link. 

Figure 2: MoveIt Planning Scene with collision objects (green) and attached objects (purple).

This functionality helped us determine from the very beginning whether our robot arm was able to reach objects in a table with a certain height, how far away from the table should the robot position to reach the objects properly, is there enough space to perform the arm movements it needs to perform to manipulate objects around the workspace area. It also helped us design the boxes needed for the task, allowing us to decide on the correct box size that will allow the robot arm to perform the necessary manipulation movements given the restricted working area.  

Motion Planning

MoveIt includes various tools that allow you to perform motion planning to the desired pose with high flexibility, you can adjust the motion planning algorithm to your application to obtain the best performance. This is very useful because it allows you to restrict your robot’s allowed motion to fit very specific criteria, which is an application like ours, with a restricted working space where the robot needs to manipulate objects precisely in an environment shared with humans is very important.

Figure 3: Planning to the desired goal taking into account collisions with the scene.

One of the biggest motion requirements we have is the need for the robot arm to maintain the boxes parallel to the ground when manipulating them as they will be filled with objects that need to be carried between work stations. To plan using constraints can be easily done with MoveIt.

There are various constraints that can be applied, the ones we found more useful for our application are joint constraints and orientation constraints. 

  • With Orientation constraints you can restrict the desired orientation of a robot link, they are very useful to maintain the robot’s end-effector parallel to the ground, needed to manipulate the boxes properly. 
  • Joint constraints limit the position of a joint to be within a certain bound, they are very useful to shape the way you want your robot to move, in our application it allowed us to move the arm maintaining a relative position between the elbow and shoulder, performing more natural movements and avoiding potentially dangerous motions.
Figure 4: Motion Planning with joint and orientation constraints vs without.

Another useful MoveIt motion planning tool is to plan movements to a goal position both in Cartesian and in Joint Space, allowing you to switch between these two options for different desired trajectory outcomes.

  • Cartesian Space planning is used whenever you want to follow a very precise motion with the end effector link. In our application, we made use of these functions when moving down from the box approach position to the grab position and back again. Our robot has to carry the boxes with it, and due to limited space on its base area, all of the boxes are quite close together, using Cartesian planning we could assure we are maintaining verticality while raising the box from its holder avoiding latching between boxes and unnecessary stops. 
  • Joint Space planning is however useful to obtain more natural trajectories when the arm is moving between different grabbing positions making movement smoother. 
Figure 5: Motion Planning in Cartesian Space vs Joint Space.

This is just a brief summary of how we used MoveIt to develop a preliminary robotic pick and place manipulation application, there are still lots of different tools that MoveIt has to offer. Some of MoveIt’s most advanced applications include integrating 3D sensors to build a perception layer used for object recognition in pick and place tasks or using deep learning algorithms for grasp pose generation, areas that will be explored in the next steps. 

Stay tuned for future updates in the development of a robotic manipulation application using MoveIt’s latest implementations. 

Down below you will find a short demonstration of the currently developed application running on a Robotnik’s RB-KAIROS mobile manipulator.

https://www.youtube.com/watch?v=JgyDB57xjDw