AI-based inspection tool for construction

Challenge

Poor productivity, waste and rework is a huge challenge for the construction sector. If a build isn’t accurate, it doesn’t perform as it should and can lead to rework, which adds to the cost. As part of the AROPCQA project - which stands for Augmented Reality for Operative Productivity and Continuous Quality Analysis - the University of Sheffield Advanced Manufacturing Research Centre (AMRC) explored the use of artificial intelligence to automate quality assurance on construction programmes, to digitalise a process that is currently multi-step, labour-intensive and can be prone to errors.

Background

AROPCQA is a collaborative project led by international engineering company Laing O’Rourke and supported by the University of Sheffield Advanced Manufacturing Research Centre (AMRC), Trimble, Offset Services, DesignTech and Workmobile. It set out to increase accuracy and minimise errors and associated rework on construction projects by identifying issues at the design stage through the use of vision and digital technologies. A smart headset was developed bringing together a suite of digital innovations, from augmented reality (AR) and virtual reality (VR), to artificial intelligence (AI), 3D modelling and live video feeds to provide construction operatives with accurate information that supports efficient installation, verifying and recording that it has been done correctly.

Quality assurance in construction is currently too reliant on manual processes. The AROPCQA ambition was to overhaul the manual processes currently impacting on the accuracy and speed of construction programmes, and digitalise them to increase efficiency, productivity and quality within the sector. The work package the AMRC focussed on was the use of AI to automate the quality assurance process for the positioning of precast concrete panels.

Precast panels have embeds of different sizes and shapes in them, into which concrete is poured and the panels are cast. These embeds need to be in the correct position and count within the panel, because failure to do so might result in structurally weak panels.

In order to perform this quality assurance process, inspectors must verify the count and position of embeds and use a multi-stage paper-based system, moving around the panel with a tape measure and a printed sheet of the panel’s computer-aided design (CAD) file. They then click images with a mobile phone and save it to a PC in case any quality issues arise in the future. This process is time consuming, manual and prone to errors. The associated rework can have a significant impact on productivity in the short-term, and quality in the longer term. Also, the time taken to inspect each panel scales up as the panel size increases.

Innovation

The AMRC developed an AI-based tool to help make the panel inspection process quicker, less labour intensive and more robust; cutting the process down from hours to minutes using an overhead camera to capture the actual panel before comparing the image to the design file before reporting back on any missing embeds or embeds outside of tolerance.

To do this, AI-based object detection algorithms were developed in-house at the AMRC to help detect the precast embeds. Using the CAD file of the panel, information such as embed type, count and position were extracted. A web application was built to serve as the front end for the tool to help inspectors carry out the quality assurance process.

The system was tested at Laing O’Rourke’s Centre of Excellence in Worksop, where the precast panels are manufactured, and at the AMRC’s Factory 2050. During testing, an industrial camera was attached to the roof of the Laing O’Rourke factory which captured live images of the panel. The object detection algorithm, along with image processing algorithms would then localise and detect the type of embeds present on the panel.

The entire application ran on a tablet via a web browser. This meant the operator could inspect the panel in front of them, check if the correct embeds are in the correct position and count, and if required make changes based on the correlation between the data from the image and the extracted CAD data. A bill of material (BoM) gives them the count and an overview of the different embeds. Finally, a digital PDF report is automatically generated and saved onto a cloud database, which can later be used for reference in case any quality issue arises.

Results

During demonstration, the following accuracy was achieved:

Anchor pins:

At AMRC test bed: 97%
At LOR factory: 70%

Other embeds:

At AMRC test bed: 95%
At LOR factory: 73%

In the later stages of the project, a final demonstration was carried out at Laing O’Rourke’s factory. Prior to the period leading up to this, the AMRC team had to retrain the models on a new kind of anchor pin (compared with what had been used for the majority of the project).

On the final demonstration day, the detection rate for the new type of anchor pins was 92% on a single image the system was tested on. This new panel didn’t have any other precast embeds other than anchor pins.

The method developed by the AMRC also allowed for a type of inspection which takes the same amount of time regardless of how big the panels are.

Impact

This type of AI-based technology can be incorporated into shop floors to help workers perform inspection of safety critical components much more robustly and quickly, and also reduce the number of errors. It can also be scaled across the factory with multiple vision systems.

The factory team at Laing O’Rourke said the built prototype was innovative and directly solved a real problem faced by the company, and would make things easier and save time. The team also said the AI inspection tool provides certainty that critical-to-safety and quality items are there which will have huge benefit in making the inspection process safer, more efficient and more assured.

For further information, please contact Cameron Christie, sector engagement manager for construction: