Image analysis for the sorting of brick and masonry waste using machine learning methods

This paper describes different machine learning methods for recognizing and distinguishing brick types in masonry debris. Certain types of bricks, such as roof tiles, facing bricks and vertically perforated bricks can be reused and recycled in different ways if it is possible to separate them by optical sorting. The aim of the research was to test different classification methods from machine learning for this task based on high-resolution images. For this purpose, image captures of different bricks were made with an image acquisition system, the data was pre-processed, segmented, significant features selected and different AI methods were applied. A support vector machine (SVM), multilayer perceptron (MLP)


INTRODUCTION AND STATE OF THE ART
Currently, 20-23 million tons of masonry building materials (mortar and plaster, lightweight concrete, aerated concrete, sandlime bricks, and bricks) are produced annually in Germany. The quantity of bricks produced, including roof tiles, is approx. 10-15 million tons [1]. Current guiding strategies and ambitious environmental policy goals increasingly call on manufacturers of mineral building products to introduce material cycles [2], [3].
Pure brick aggregates are currently used in sports field construction, in vegetation applications, in road construction as a proportionate component of frost protection and gravel base courses, and in building construction as recycled aggregate for concrete production [1]. Pure brick recycled aggregates (Figure 1), which are made from low-dense waste and can be obtained from masonry waste, can be returned to brick production as recycled material after being ground again. A prerequisite for this, however, is that no mortar adhesions or other impurities may be present [4]- [6].
Based on the investigations in the projects [5], [7], a leaflet on the use of recycled bricks in the brick industry was drafted. It recommends pure hard-fired material, pure/hard material, low-fired material, and masonry waste as suitable feed material for the production of roof tiles, facing bricks, and vertically perforated bricks. Depending on the type of brick, up to 25 wt.-% can be reused [6], [7].
After grinding, masonry waste can also be used as a cement composite material in the cement industry [8]. Here it is necessary to be able to distinguish and separate low-fired and high-fired brick types, mortar, concrete, and other components from each other. ABSTRACT This paper describes different machine learning methods for recognizing and distinguishing brick types in masonry debris. Certain types of bricks, such as roof tiles, facing bricks and vertically perforated bricks can be reused and recycled in different ways if it is possible to separate them by optical sorting. The aim of the research was to test different classification methods from machine learning for this task based on high-resolution images. For this purpose, image captures of different bricks were made with an image acquisition system, the data was pre-processed, segmented, significant features selected and different AI methods were applied. A support vector machine (SVM), multilayer perceptron (MLP), and k-nearest neighbour (k-NN) classifier were used to classify the images. As a result, a recognition rate of 98 % and higher was achieved for the classification into the three investigated brick classes.
Optical single-grain sorting methods offer a chance for much differentiation even of very similar materials [10, 11, and 12]. Optical analysis and sorting methods are not yet used on a large scale in the sorting of construction and demolition waste. The optical sorting methods are essentially based on innovative detection routines that can be integrated into corresponding software ( Figure 2).
The aim of the investigations is the development of a recognition routine for the differentiation of different brick types for single-variety brick and masonry rubble. Features from the high-resolution image and spectrum and evaluation algorithms from machine learning will be used for the recognition task. The main focus is in a first step on particles with a size of 8/16 mm. Later it will be extended on particle sizes 2/4 and 4/8 mm. Very important is the sorting out impurities, like mortar, gypsum, concrete and so on.
The fundamentals are being created in order to develop more effective sorting processes in the recycling of residual brick masses in the future.
This paper investigates the recognition of types of brick by using RGB images and innovative methods of machine learning are described. This provides the basis for the development of optical sorting procedures.
In the investigations, recognition routines and algorithms are to be tested for the correct differentiation and separation of various brick types based on image features. Figure 3 gives an overview of the test procedure.

Materials and Categories
At the MFPA, initial preliminary investigations were carried out to distinguish between brick types on the basis of visual image information.
In the investigations, different brick products were distinguished, which can be assigned to the following brick categories: • Category II: Roof tiles and facing bricks, • Category III: Vertically perforated bricks and • Category V: Other. At first, a sample collection of a representative nationwide cross-section of brick varieties and adhering mineral building materials were done.
The brick samples in the different categories are composed of different new (unused) and recycled (used) bricks without adherents. Furthermore, other relevant building materials such as aerated concrete, concrete, mortar, sand-lime bricks, etc. (new and recycled) have also been included as impurities.
The measured material parameters water absorption after 24 h and bulk density according to DIN 4226-101 for all samples included in the investigations are shown in the diagram ( Figure 4).
A total of 7,765 image recordings of samples were made. Category I (high-fired material) is still missing from these investigations, as too little material was available for examination. Category IV (brick waste from recycling plant) was also excluded at this point, as the samples were not homogeneous and can therefore belong to different categories. In future works, both categories will be added to the data set. The particle size is 8-16 mm.
The brick samples listed in Table 1 were divided into three classes, which differ in bulk density.

Image acquisition and used software
A data set of high-resolution images of the brick particles was created. The brick samples were examined under different conditions. On the one hand, three different types of lighting and, on the other hand, different combinations of features were used. This allows the influence of the illumination on the analysis results to be investigated. Figure 5 shows the used image acquisition system "QUALILEO". It consists of an RGB matrix camera with a 12 MP sensor and a step-less adjustable lighting system ( Figure 6).

Investigations and Results
The provided algorithms are analysed using the presented data set with individual parameters and the average recognition rate (RR) captured. The achieved RR and standard deviation (Stdev) are used to compare all of the results. All of the researches were carried out in the HALCON programming environment.
The investigation's classifier have different setting parameters. The distance-based nearest k-neighbours value has been set to 5 in the k-NN classifier. The rbf-kernel was used with SVM and the γ parameter was set to 0.02. Also the one-versusall classification mode was used. With this setting a multi-class problem is reduced to a binary decision -each class is compared to all the other classes. For MLP is used a softmax activation function. It performs best at classification tasks with multiple independent classification outputs. In the hidden layer, the number of hidden units is set to 15.
In this study, the data set is split into two parts: 80 % and 20 %. The classifier are trained with a major part of the data set, while the classifier are tested with a smaller fraction of the data.

Results before selection and combination of features
At the beginning, the typical feature groups were analysed. Table 2 shows a summary of these results. The first result was that lighting 2 (half of a ring light that simulates light from the side) always produced significantly worse results, which is why the following only presented the results for lighting L1 (fully ring light on) and L3 (incident light).
This data set showed that colour features alone are the most important feature for this recognition task. The recognition rates achieved are always above 95 % for all data sets and classifiers, as shown in Table 2. The region features have only little impact on recognition. Additionally, the texture features prove more effective with over 80 % recognition rates for SVM and MLP. The k-NN classifier had the lowest recognition results. Aside from colour features, all other accuracy is insufficient. This is because the k-NN algorithm is more vulnerable to redundant features than the robust classifiers such as SVM and MLP. This research found as well that using all of the possible number of features does not always result in a better accuracy, or that the improvement is minor because many of the features are redundant. Besides that, with a large number of features, the classifier's training time increase. A further improvement is to be achieved with feature selection. Only the most important features for the study's problem are calculated by this algorithm.

Results after selection and combination of features
The best results by feature selection are shown in Figure 7 and Figure 8. The very high recognition rates for all classifiers are remarkable.  Step-less adjustable lighting system on the QUALILEO. A look at the individual categories also shows that category III is recognised with an over 99 % recognition rate with both SVM and MLP. Not far from this result, k-NN is just below. Category II is always recognised equally well by SVM and MLP (close to 98 %). Here k-NN is clearly worse in the recognition (about 96 %). Category V shows a lower recognition rate between 96 % and 97 % for all classifiers, but again k-NN shows the lowest results of all.
Compared to the results without feature selection and combination, no large increase in the recognition rates was observed. With SVM and k-NN, the detection rate increased by only about 1 %, with MLP it remained at about the same rate.
Finally, Figure 9 shows the importance of the feature selection procedure in classical classification classifiers. Because it is not useful to use as many features as possible for classification, but only the most important ones. This can be seen as the score progresses. The RR increases with the number of features until it reaches its maximum value. After that, the RR decreases as more characteristics are added. Because a lot of the features are redundant and have a lot of noise in them. As a feature selection method Halcon presents a greedy method. In this algorithm the currently most promising feature is added to the feature vector. After that, it is evaluated whether one of the newly added features is unnecessary.
Summarised, it is important to note that the high recognition rates after feature selection were achieved with a relatively small number of features (Table 3). The average RR is over 95 % for all classifier and both data sets L1 and L3. Hence the different settings on light had only low effect on the result of the optimised classifier. Additionally, the standard deviation is low, indicating a stable classification process. This simplifies the future implementation of the application and reduces the required computing effort and time.

CONCLUSION AND FUTURE WORK
In this work, a method for recognizing different kinds of the brick using image processing and machine learning was investigated. A very good differentiation of the selected brick categories roof tiles and facing bricks or vertically perforated bricks could be demonstrated.
With an overall recognition rate of about 98 %, the three categories are separated. In the future, the data set will be extended by additional categories, and further classifier methods (also Deep Learning) will be tested. The algorithms obtained are to be further used for the development of optical sorting methods.