Introduction
Computer Vision is an important field of artificial intelligence. Its application in various industries such as food, pharmaceutical, automotive, aerospace, and rail dates back to the early 1970s. Until the 1990s, only a few methods of computer vision were used in manufacturing due to limited computational capabilities. In recent years, the development of this discipline has accelerated thanks to the rapid progress in deep learning. Data from 2021 compiled by IBM shows that as much as 51% of the global computer vision market is located in the industry. It is therefore not surprising that this technology is among the 10 most important technologies in the manufacturing sector.
Figure 1. 10 key technologies helping manufacturers achieve their business goals (aitransition.com)
Tasks of Computer Vision
The goal of computer vision is to generate a symbolic description of what is in the image. The extracted information can then be used to control various operations in the process. There are many tasks and algorithms in the field of computer vision such as detection, recognition, and segmentation. Below are some selected issues that have wide applications in many fields.
Image Classification
Image classification is a task where a computer program can distinguish the “class” to which an image belongs (e.g., classifying a product image as correct or damaged). Early image classifications were based on raw pixel data, meaning that the program divided images into individual pixels. The problem is that two images of the same object can look completely different. They can have different backgrounds, angles, poses, etc. This posed a unique challenge for accurate image analysis and classification (Kaur et al. 2021). Nowadays, convolutional neural networks are used for image classification.
Object Segmentation
Image segmentation is a computer vision technique that divides a digital image into distinct groups of pixels. By breaking down complex visual information into separate segments, this technique allows for faster and more advanced image processing. An example of object segmentation using computer vision is the application of the Mask R-CNN algorithm (Regional Convolutional Neural Network) to identify and segment multiple instances of various objects within an image. The neural network is trained on a dataset with images containing appropriate object masks and can then be used to predict masks for objects in new images.
Object Detection and Recognition
The aim of object detection is to detect multiple instances of objects from one or several known classes such as people or cars in an image (Amit et al. 2021). This is one of the fundamental issues in computer vision applications. It can support the quality control process in manufacturing by detecting a given object in an image and deciding whether the inspection was performed correctly or not. It can also provide valuable information about human behavior, for example by detecting the activities of workers in industrial environments. Due to lighting conditions, significant differences in viewpoints, and postures, traditional methods have some limitations. Therefore, much attention is paid to the use of deep learning methods to develop a robust approach to object detection (Wang et al. 2021).
Object Tracking
Tracking a specific object or multiple objects is used in video recordings and real-life interactions, for example in human behavior analysis. This is a challenging task due to occlusions, fast-moving objects, or interactions between objects (Schulter et al. 2017). So far, a number of algorithms have been developed, differing in terms of advancement, performance, and effectiveness. One popular algorithm is the Kalman filter, which is used to predict the position of objects based on their previous movements, considering speed and acceleration, making it effective in simple scenarios. Another noteworthy algorithm is DeepSORT, which combines the Kalman filter with a deep neural network for more accurate real-time object tracking, especially in complex scenarios with occlusions. Different algorithms are applied in various tracking scenarios from simple to more advanced applications requiring accuracy and efficiency.
Additional Issues
Performing the above tasks involves a number of intermediate processes through which information is extracted, allowing for a more detailed interpretation of the image. These processes include, among others, edge detection and feature extraction.
Edge Detection
Edge detection involves extracting the contours of objects against the background of the image. It significantly reduces the amount of data and filters out unwanted or irrelevant information. The selected data is then used in image processing, e.g., for object detection (Bhardwaj and Mittal 2012). Edge detection technology began to be researched in the 20th century and developed rapidly until the 1960s. Since then, it has found wide application in medicine, industry, and agriculture (Shou-Ming Hou et al. 2021). Classic edge detection technology is divided into pixel-level edge detection technology and subpixel-level edge detection technology. In pixel-level edge detection, the main methods include fuzzy theory, genetic theory, morphology, etc. In subpixel-level edge detection, matrix calculus and interpolation are mainly used. Subpixel-level edge detection technology allows for precise determination of edge point coordinates, which has attracted the attention of many researchers and engineers in recent years (Shou-Ming Hou et al. 2021).
Feature Extraction
Feature extraction is the process of extracting important features or patterns from an image. Computer vision algorithms can use the selected information to determine the presence of objects in the image. For example, when identifying animals in an image, feature extraction will allow the algorithm to look for specific features such as color, shape, size, etc., helping it distinguish between different species of animals. Another application is controlling autonomous vehicles. Feature extraction is used to identify important elements in the image such as pedestrians, traffic signs, or vehicles.
Applications of Computer Vision in Manufacturing
Key issues in the industrial application of computer vision technology include quality control, process monitoring, and robot guidance and control. This is confirmed by statistics on the occurrence of words describing different stages of production in computer vision research publications (Fig. 2). They indicate that these methods are most often used in production inspection and assembly. It is also worth noting other applications such as safety control or warehouse management.
Figure 2. Keyword statistics of different production stages in computer vision research publications (Zhou et al., 2023)
Safety
Employees in manufacturing companies are at risk of injury while performing their work. Monitoring compliance with safety rules by employees is often insufficient because the responsible person may not be able to effectively observe multiple screens at once. Using computer vision technology can effectively detect security issues, create reports on dashboards, and send notifications. There is also the possibility of automatically generating alerts in the event of an accident, allowing immediate action to be taken. An example of using computer vision for detecting personal protective equipment is shown in Fig. 3.
Fig. 3. Detection of personal protective equipment (labelyourdata.com)
Computer Vision in Quality Control
Quality control is an important element of production because high-quality products minimize costs and improve brand reputation. Defects in the product can result in the need to stop production, which costs very high, reaching about $22,000 per minute (encord.com). Production defects can include features such as unwanted holes, dents and scratches, corrosion, or damage resulting from material fatigue. Current vision systems can detect defects even in very small components in real-time (Yang et al. 2019). Thanks to this technology, it is possible to achieve quality assessment effectiveness above 95%. During industrial production, defects such as cracks also occur. Edge detection algorithms are useful in identifying the type of these defects and precisely determining their location, which is important for operators in assessing the quality of parts (Shou-Ming Hou et al. 2021).
Figure 4. Manufacturing fault detection using real-world deep learning (Yang et al., 2019)
Warehouse Management
Traditional warehouse management relies on manual work, handheld scanners, and even paper-based systems, which are associated with low efficiency, susceptibility to errors, and high operational costs. Vision systems overcome these difficulties. By equipping the warehouse with cameras, sensors, and advanced algorithms, companies gain greater control over their status. Computer vision allows factories to reduce the time spent searching for items, manual scanning, minimize inventory shortages, and optimize warehouse space (vimaan.ai).
Fig. 5. The use of computer vision in warehouse management (vimaan.ai)
Benefits of Using Computer Vision
Computer vision brings a number of benefits to the industry. It is used in the automation of production and logistics processes through the recognition and detection of objects, which shortens task execution time and reduces costs. It contributes to improving product quality through more accurate control and faster defect detection. Another benefit is the optimization of inventory management by precisely tracking warehouse status and minimizing the risk of shortages and excessive inventory. It increases safety by monitoring the work environment and quickly recognizing hazards. Finally, the use of computer vision increases competitiveness by improving efficiency, reducing costs, and enhancing quality, enabling companies to gain a market advantage.
Summary
Computer vision is one of the key technologies supporting industrial development worldwide. This technology not only increases production efficiency but also improves quality, safety, and management of production and logistics. Thanks to advances in deep learning and modern algorithms, vision systems are becoming an indispensable tool in automation and optimization processes in the industry. Its growing importance is confirmed by numerous studies and practical applications, making it a key element of the future of industrial production.
References
Amit, Y., Felzenszwalb, P., Girshick, R., 2021. Object Detection BT – Computer Vision: A Reference Guide, in: Ikeuchi, K. (Ed.), . Springer International Publishing, Cham, pp. 875–883. https://doi.org/10.1007/978-3-030-63416-2_660
Bhardwaj, S., Mittal, A., 2012. A Survey on Various Edge Detector Techniques. Procedia Technol. 4, 220–226. https://doi.org/https://doi.org/10.1016/j.protcy.2012.05.033
Kaur, P., Singh, S.K., Singh, I., Kumar, S., 2021. Exploring convolutional neural network in computer vision-based image classification, in: International Conference on Smart Systems and Advanced Computing (Syscom-2021).
Schulter, S., Vernaza, P., Choi, W., Chandraker, M., 2017. Deep network flow for multi-object tracking, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 6951–6960.
Shou-Ming Hou, Chao-Lan Jia, Ya-Bing Wanga, Mackenzie Brown, 2021. A Review of the Edge Detection Technology. Sparklinglight Trans. Artif. Intell. Quantum Comput. 1, 26–37. https://doi.org/10.55011/staiqc.2021.1203
Wang, D., Wang, J.-G., Xu, K., 2021. Deep Learning for Object Detection, Classification and Tracking in Industry Applications. Sensors. https://doi.org/10.3390/s21217349
Yang, J., Li, S., Wang, Z., Yang, G., 2019. Real-Time Tiny Part Defect Detection System in Manufacturing Using Deep Learning. IEEE Access 7, 89278–89291. https://doi.org/10.1109/ACCESS.2019.2925561
Zhou, L., Zhang, L., Konz, N., 2023. Computer Vision Techniques in Manufacturing. IEEE Trans. Syst. Man, Cybern. Syst. 53, 105–117. https://doi.org/10.1109/TSMC.2022.3166397
Websites
https://encord.com/blog/computer-vision-manufacturing
https://itransition.com/computer-vision/manufacturing
https://labelyourdata.com/articles/computer-vision-in-manufacturing