Smart camera helps with training and professional development

"We don't learn for school, we learn for life" - this is a piece of wisdom that everyone has probably heard at some point during their school years and possibly acknowledged with a roll of the eyes. What you might not have been aware of at the time due to your age: cramming doesn't stop at the end of school, training or studies. The challenges of the modern working world and Industry 4.0 mean that both employers and employees need to focus on ensuring that knowledge is trained and retained - the keyword here is lifelong learning.

The quality of training and further education measures depends not only on the individual learning type, but also on the type of training and further education. While webinars, where you only watch and listen, have a maximum learning success of 50%, good seminars can already reach 70%. The best learning success, on the other hand, is achieved when as many sensory channels as possible are activated. This also includes haptics or when the learner uses something himself. The latter can lead to a learning success of up to 90%.

This is also the idea behind the KoBeLU project, which is funded by the Federal Ministry of Education and Research and is scheduled to run for three years. The project itself is backed by renowned representatives of industry and science such as Audi AG, Mahle International GmbH, the University of Munich and the University of Tübingen. KoBeLU stands for context-aware learning environment, i.e. it should be possible to work through a learning unit interactively at a station, which projects individually adapted information directly into the work area depending on the person working on it. For example, if a component is touched, the learner sees its properties and where it needs to be soldered. This playful approach, also called gamification, is intended to provide additional motivation. The combination of interactive projection, gesture control, emotion or stress recognition - taking data protection into account - as well as action orientation and a playful approach is also intended to make learning content actually "graspable".

It was only natural that the project partners looked to the world of computer games for the technical implementation and found a suitable candidate in Microsoft's Kinect. The Kinect is a 3D camera that is used in the Xbox game console for contact-free control. The Kinect captures the player so that they interact with the game completely themselves, rather than just through a gamepad.

Initial trials with the Kinect were promising and worked for many scenarios, but it quickly turns out that the Kinect's 2D resolution of 320 × 240 pixels was simply too low for assembly tasks, for example. Even the 512 × 424 pixel resolution of the second generation Kinect was deemed too low.

So the developers set out to find possible solutions. 3D cameras from the industrial image processing sector with higher resolutions were tested, but were far too complicated compared to the API of the Kinect. Through a visit to a trade fair, representatives of the project became aware of the smart camera mvBlueGEMINI with the inspection software mvIMPACT Configuration Studio (mvIMPACT-CS for short) from MATRIX VISION. Although it was a 2D industrial camera, the wizard-based tools and the resolution of 1280 x 1024 pixels had the potential to complement the existing 3D camera with an additional 2D camera with higher resolution. Because to check if a participant of the education and training has soldered a component to the right place, to stay with the example, is nothing else than an inspection and therefore the terrain of the mvBlueGEMINI and mvIMPACT-CS.

The integration of the mvBlueGEMINI offered several advantages. It was designed to allow anyone to create inspections; entirely without programming or image processing knowledge as a prerequisite. For this purpose, a clear number of "tools" were defined, which describe the respective application purpose and use the language of a user. Wizards within the tools then help to automatically evaluate the scene currently being captured by the smart camera and to select the appropriate algorithms and, if required, filters. This ensures that a larger number of people can create or adapt learning units for the learning environment. Another advantage of the mvBlueGEMINI is its web-based approach. This means that the Smart Camera can be accessed from anywhere in the world and thus learning environments can be recorded worldwide. This is ideal for large companies that, for example, want to centrally control their further training measures.

The integration of the Smart Camera is now in full swing and is progressing rapidly. The intuitive and easy-to-learn software has also strengthened the developers in their decision to rely on the solution from MATRIX VISION. Now the task is to equip the learning environment with the required functionality and fill it with the necessary content by the end of the project in 2019. KoBeLU will primarily be used for Audi's and Mahle's own trainees. However, the learning environment will also be offered to external companies. A possible further field of application outside the industry are care facilities. The idea is that the learning environment should serve as an interactive companion while cooking.