Challenge

The University of Sheffield Advanced Manufacturing Research Centre (AMRC), part of the High Value Manufacturing (HVM) Catapult, was tasked with devising a solution by Bristol-based Q5D Technologies, which aims to design the hardware and software tools needed to enable electrical function integration.

To devise a proof-of-concept solution to calculate the position and rotation of a workpiece to enable a robot to fully automate the process of embedding wiring into them for use in the aerospace sector, as well as creating integrated validation processes for quality inspection. 

Background

This work is part of a bigger research and development project that Q5D is leading, named LiveWire – working in collaboration with the AMRC, as well as other companies, including OnePLM, M-Solv and Safran. The Innovate UK-funded project is looking to automate the production of wiring harnesses for the aerospace industry. 

As part of this project, Q5D has created a robotic tool that can take a complex shaped component, such as a moulded part or formed panel and use their 5-axis gantry robot with interchangeable end-effectors to add electrical function using bare or insulated wiring, printed electronics, terminations, surface mounted components and printed FFF structures.

To obtain an accurate position and rotation of the part to automate Q5D’s robot path planning, the AMRC has supported the company in implementing a proof-of-concept solution using a stereo vision camera. 

Adding embedded wiring to the chosen workpiece is a novel solution that Q5D has also been working on. This process requires the ability to automate the wiring inspection, for which the AMRC has also implemented proof-of-concept computer vision solutions. These solutions include algorithms that detect defects such as wire breakages, crushed wires, stripped wire insulation, wires escaping from the track and incomplete track walls.

Q5D tasked the AMRC with making sure the position and rotation of the workpiece produced the best possible accuracy, while maintaining high repeatability and a fast computational process. 

Innovation

AMRC engineers performed a number of trials to identify the right hardware and software that would be best suited for calculating a parts position and rotation angle, as well as the in-process validation of the embedded wiring.

To produce an accurate calculation of the position and rotation angle, the AMRC used an affordable depth camera based on stereo vision technology. The main advantage of stereo vision is that it is less affected by reflections or different types of surfaces. The use of structured light by the stereo vision system also provides good performance with dark and uneven surfaces/materials. 

A computer vision development tool called Halcon was also used – and this tool provides many in-built functionalities for 2D and 3D image processing and one of the in-built functionalities is surface-based matching. It works by producing a point cloud surface model on the tool, which responds to this computer-aided design (CAD) model and helps to work out where the workpiece is positioned and at what angle. These are then matched using the surface-based matching method to the points obtained from the stereo vision camera. 

For the in-process wiring validation, 2D high-resolution grayscale images were used as inputs to bespoke algorithms that exploit image features such as shape contours. A combination of techniques, such as intelligent wire segmentation and shape matching with variance models, were implemented to detect the various defects.

Result 

Engineers at the AMRC successfully achieved the required workpiece position calculations needed and were under 0.5mm and got the orientation requirements accurately worked out under 0.6 degrees when they performed repeatability trials. 

Of the trialled defects for in-process validation, exposed wire detection was the most successful, with a 100 per cent detection rating on flat samples at various angles. This is because the lighting conditions and camera parameters are exploited in real-time and in a repeatable manner to overexpose the visible copper wiring. 

Damaged track walls had a high detection rate, again down to good lighting, exposing the dullness of the damaged walls. The AMRC found that external and repeatable changes to the environment to highlight features in the samples was a more reliable method for defect detection, than accounting for all conditions through software changes.

Impact 

The benefits of automating the robot path using a vision system will provide Q5D’s robotic technology with greater flexibility to adapt to dynamic environments. Making automatic adjustments to the robotic path on the fly will increase productivity as it will decrease the manual programming needed to re-adjust the robot tool path. 

The techniques used in the proof-of-concept defect detection algorithms could lead to the development of a real-time, embedded wiring defect detection system integrated with the machine itself.

Even though this project was looking at a pathway for use in the aerospace sector, this concept can be transferable to any robotics automation system.