Review Article
Open Access

Application of traditional methods and deep learning in breast ultrasound image segmentation

Fangfang Chen
Fangfang Chen
School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China.
,
Miao Zhou
Miao Zhou
Jiangsu Cancer Hospital, Nanjing 213164, Jiangsu Province, China.
,
Jintao Duan
Jintao Duan
School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China.
,
Yuxiang Wang
Yuxiang Wang
School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China.
,
Liangqing Lin
Liangqing Lin
The First Hospital of Putian, Putian 351100, Fujian Province, China.
,
Wenhui Guo
Wenhui Guo
wendyguo17@outlook.com
School of Anesthesiology, Naval Medical University, Shanghai 200433, China.
,
Yongchu Hu
Yongchu Hu
Adsfoxcn@sina.com.cn
The Department of Anesthesiology, Long March Hospital, Shanghai 200003, China.
Address correspondence to
Article notes
Highlights
Wenhui Guo, School of Anesthesiology, Naval Medical University, No.25 Zhongyuan Road, Shanghai 200093, China. E-mail: wendyguo17@outlook.com. Yongchu Hu, The Department of Anesthesiology, Long March Hospital, No.415 Fengyang Road, Shanghai 200003, China. E-mail address: Adsfoxcn@sina.com.cn.

Received September 27, 2024; Accepted January 22, 2025; Published March 31, 2025

  • Comparison of Methods: This article compares traditional techniques, such as thresholding and edge detection, with deep learning-based methods for breast ultrasound segmentation.

  • U-Net's Effectiveness: U-Net is identified as the benchmark for medical image segmentation due to its efficiency and ability to preserve details.

  • Advantages of Deep Learning: Deep learning models, like CNNs and FCNs, improve segmentation accuracy by learning directly from raw data, while also mitigating noise.

Review Article
Open Access
Application of traditional methods and deep learning in breast ultrasound image segmentation
Fangfang Chen
Fangfang Chen
School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China.
,
Miao Zhou
Miao Zhou
Jiangsu Cancer Hospital, Nanjing 213164, Jiangsu Province, China.
,
Jintao Duan
Jintao Duan
School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China.
,
Yuxiang Wang
Yuxiang Wang
School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China.
,
Liangqing Lin
Liangqing Lin
The First Hospital of Putian, Putian 351100, Fujian Province, China.
,
Wenhui Guo
Wenhui Guo
wendyguo17@outlook.com
School of Anesthesiology, Naval Medical University, Shanghai 200433, China.
,
Yongchu Hu
Yongchu Hu
Adsfoxcn@sina.com.cn
The Department of Anesthesiology, Long March Hospital, Shanghai 200003, China.
Address correspondence to
Wenhui Guo, School of Anesthesiology, Naval Medical University, No.25 Zhongyuan Road, Shanghai 200093, China. E-mail: wendyguo17@outlook.com. Yongchu Hu, The Department of Anesthesiology, Long March Hospital, No.415 Fengyang Road, Shanghai 200003, China. E-mail address: Adsfoxcn@sina.com.cn.
Article notes

Received September 27, 2024; Accepted January 22, 2025; Published March 31, 2025

Highlights
  • Comparison of Methods: This article compares traditional techniques, such as thresholding and edge detection, with deep learning-based methods for breast ultrasound segmentation.

  • U-Net's Effectiveness: U-Net is identified as the benchmark for medical image segmentation due to its efficiency and ability to preserve details.

  • Advantages of Deep Learning: Deep learning models, like CNNs and FCNs, improve segmentation accuracy by learning directly from raw data, while also mitigating noise.

2025 Apr;1(1):14-26
PDF
On This Page
CITE
Accesses: 44

Abstract

Breast ultrasound image segmentation is vital in medical imaging, enabling precise delineation of tissues and lesions, which contributes to the diagnosis and treatment of breast diseases. This article reviews both traditional methods and recent advancements in deep learning techniques for breast ultrasound image segmentation. The discussion begins by highlighting the significance of image segmentation in breast disease diagnosis and its background within medical imaging. Traditional segmentation methods, such as thresholding, edge detection, and region growing, are examined, with an analysis of their applications and limitations in breast ultrasound segmentation. Subsequently, the focus shifts to deep learning approaches, including classic models like Convolutional Neural Networks, Fully Convolutional Networks, and U-Net, along with their improved algorithms. These methods learn hierarchical features directly from raw data, reducing reliance on manual preprocessing. U-Net, in particular, is highlighted as the benchmark for medical image segmentation due to its efficient data usage and ability to preserve fine-grained details. Comparative analysis demonstrates the advantages of deep learning in enhancing segmentation accuracy, reducing noise, and handling complex texture structures. The article concludes by summarizing current achievements and challenges in the field, while offering an outlook on the future developments aimed at advancing breast ultrasound image segmentation for improved diagnosis and treatment of breast diseases.

Keywords: Breast ultrasound image segmentation, medical imaging, deep learning

Introduction

Breast diseases encompass a range of conditions affecting breast tissue, including mastitis, breast hyperplasia, fibroadenoma, and breast cancer, with female being the primary demographic affected [1]. In 2023, the National Health Commission of the People’s Republic of China underscored the significance of breast cancer as a leading malignancy threatening women’s health, emphasizing the need for prioritizing the prevention and management of breast diseases.


Early diagnosis is crucial for improving treatment outcomes and survival rates for breast cancer patients. Breast ultrasound plays a key role in early screening and diagnosis by accurately segmenting breast tissue and other regions in ultrasound images, thereby providing high-resolution, detailed tissue information. This segmentation process is vital in the diagnosis and treatment of breast diseases. However, challenges arise due to the complex nature of breast tissue, including noise, low contrast, and image blur, which complicate the differentiation between the breast area and surrounding tissues. Moreover, the interpretation of ultrasound images heavily relies on the experience and skills of clinicians, leading to variability in diagnostic accuracy.


Accurate breast ultrasound image segmentation enables rapid and accurate identification of disease types and locations. Recent advancements in image processing and computer vision have significantly enhanced the segmentation efficiency. While traditional image segmentation techniques, such as thresholding, edge detection, and region growing, have been extensively applied in breast ultrasound image segmentation, they often struggle to handle the complex textures and indistinct boundaries in ultrasound images, resulting in inaccurate or inconsistent segmentation. To overcome these limitations, there has been a growing focus on deep learning-based methods for breast ultrasound image segmentation. Deep learning techniques, particularly convolutional neural networks (CNNs), can automatically learn deep features from large datasets, demonstrating remarkable performance improvements in image segmentation tasks [2]. These approaches not only enhance the differentiation of breast tissues and lesions but also reduce the impact of individual and device-related image variations, providing doctors with more reliable diagnostic tools.

Traditional breast ultrasound segmentation methods

Threshold based segmentation method


Threshold-based segmentation methods are commonly categorized into two types: binary segmentation and multi-level segmentation, depending on the number of resulting segments [3]. Binary segmentation is the most used threshold segmentation method, dividing the image into two regions: foreground (e.g., white) and background (e.g., black), as illustrated in Figure 1 [4]. This technique is generally applied in simpler image segmentation tasks, such as object detection and image binarization.

Figure 1. Breast ultrasound image segmentation based on threshold value. This figure is cited from [4].

In contrast, multi-level segmentation is employed for more complex images, where the image is divided into multiple regions, each representing a different category. This method uses several thresholds to allocate pixels to different categories based on their grayscale values, creating a multi-valued image suitable for complex image segmentation tasks. However, in practical applications, using multi-level thresholds for image segmentation may encounter difficulties, such as parameter selection, over-segmentation, high computational complexity, and sensitivity to noise, necessitating a careful consideration of the trade-offs based on specific circumstances when selecting an appropriate segmentation method [3]. Breast ultrasound delineation involves accurately and automatically segmenting breast tissue from surrounding tissues in ultrasound images. This process integrates principles from multiple fields, including image processing and machine learning. By comparing the pixel grayscale values of breast ultrasound images with predefined thresholds, the image can be segmented into breast tissue and background regions. The selection of thresholds significantly influences the results and is most effective for relatively simple image scenarios.


In the 1990s, with advancements in digital image processing and computer vision, the utilization of computer algorithms for medical images analysis started to be explored. Automatic segmentation of breast ultrasound images has emerged as a significant research direction, attracting increasing attention. In 2001, Horsch et al. proposed a computationally efficient algorithm for segmenting breast masses in ultrasound images [5]. This algorithm maximizes utility functions defined on segmentation boundaries using grayscale value thresholds in preprocessed images. The performance of the segmentation algorithm was evaluated in a database containing 400 cases using two methods. The first method compared the computer-delineated boundaries with the manually outlined ones, and the second method compared the performance of computer-aided diagnostic techniques on edges delineated by the computer versus those delineated manually [5]. The results demonstrated that the automatic classifier developed by Horsch et al. achieved Area under the ROC Curve (AUC) values of 0.91 and 0.87 for distinguishing between benign and malignant breast lesions in fully automated segmentation and manual segmentation, respectively.


Furthermore, when integrated with their automatic classifier, the performance of the algorithm in distinguishing breast lesions was comparable to manual segmentation [5]. In another study, Cui et al. designed a method for automatically segmenting breast masses in ultrasound images. This method automatically estimated an initial contour based on a manually identified point near the mass center [6]. It then employed a two-stage Active Contour (AC) approach to iteratively refine the initial contour. The segmentation results were subject to self-inspection and correction, with performance evaluation comparing computer-based segmentation to those performed by experienced radiologists. The results demonstrated that their Computer-Aided Diagnosis (CAD) system performed similarly to manual segmentation by experienced professionals [6].


Recently, researchers have made new advancements. Zebari et al. proposed an improved segmentation model based on thresholding and trainable techniques for extracting regions of interest [7]. Furthermore, they developed a hybrid segmentation method for breast region and chest muscle boundary segmentation in breast X-ray images, integrating thresholding and machine learning techniques. In this approach, wavelet transform bands were removed to highlight the breast region and optimize the estimated breast boundary. This novel thresholding technique contributes to determining the initial breast boundary and partially overcomes the limitations of manual image segmentation.

Method based on edge detection


Edge detection is a commonly used method for image segmentation that involves detecting edge information within an image [8]. Edges typically correspond to boundaries between objects or transitions between regions, making them crucial for segmenting an image into distinct areas or objects [9]. This method segments the image by detecting contours and boundaries, effectively outlining the contours of the breast region in ultrasound images. The edge detection-based segmentation approach generally follows these steps: first, edge detection is performed; next, edge tracking, edge linking, or region merging is conducted based on the detected edge information to achieve image segmentation. This approach is suitable for breast tissues with clearly defined edges but may be less effective for complex or poorly defined breast tissue structures.


To address these challenges, researchers have been developing enhanced techniques. In 2019, Rampun et al. proposed a modified version of the Holistically-Nested Edge Detection network method to accurately segment the pectoralis major muscle from breast X-ray images, showing that the modified Holistically-Nested Edge Detection network model could segment the muscle more effectively with higher accuracy and robustness compared to traditional methods [10]. In 2022, Daoud et al. proposed an edge-based approach to improve the localization of Region of Interest (RoI) in breast ultrasound images, overcoming the limitations of single-object detection models in diverse scenarios [11]. This method integrates results from multiple deep learning-based object detection models to process ultrasound images and obtain their respective RoI localizations. Subsequently, edge detection algorithms, such as the Canny edge detector, are utilized to extract edge information from the images. Based on the edge information, the RoIs generated by the different models are filtered and merged to achieve more precise RoI localization. Further details on the application of edge detection in breast-related research can be found in Table 1.

Table 1. Literature on the application of edge detection in breast diagnosis
Note: HOG, Histograms of oriented gradient; SVM, Support Vector Machine.

Region growing algorithm


The region growing algorithm is a widely used image segmentation technique based on seed points, which partitions images into regions with similar characteristics. Starting with one or several seed points, it progressively clusters adjacent pixels into a unified region via incremental growth or pixel amalgamation, particularly suitable for segmenting homogeneous backgrounds and contiguous breast regions. Its fundamental principle involves expanding the region from the seed point by adding neighboring pixels that share similar characteristics, typically based on grayscale or color value differences, until a predefined stopping criterion is met [17]. Pixels whose differences fall below a specified threshold are grouped into the same region. Despite demonstrating resilience to noise and the ability to manage local image features without a prerequisite on the number of regions to be segmented, this algorithm still has limitations. Its performance is sensitive to the initial seed point selection, and the process of setting appropriate parameters can be complex. To address these challenges, researchers introduced enhancements in seed point selection and region propagation in the early 21st century. For example, Xia et al. introduced a best-first approach to enhance seed accuracy, which simplifies the bipartite matching process to unipartite matching by utilizing epipolar and continuity constraints [18]. Additionally, they implemented a dynamically adaptive window, replacing the traditionally used larger one, thus improving both the temporal and spatial efficiency of the search process.


However, in practical applications, the region growing algorithm is infrequently employed in isolation. Instead, it is commonly combined with other image processing techniques to support breast cancer diagnosis. Nonetheless, it offers effective segmentation, feature extraction, and quantitative analysis in breast ultrasound image processing, aiding in the diagnosis and treatment of breast diseases. Consequently, research into enhancing the region growing algorithm remains an ongoing endeavor. Recently, Wang et al. introduced a segmentation method for breast ultrasound images, leveraging adaptive region growing with variable level set techniques [19]. This approach seeks to address challenges associated with breast ultrasound images, including high noise levels, blurred boundaries, and substandard imaging quality. Notably, the results demonstrated diagnostic values exceeding 0.99 for both Jaccard and Dice indices in benign tumor segmentation, highlighting the method’s potential to improve segmentation accuracy in breast ultrasound images.

Application of deep learning in breast ultrasound segmentation

Deep learning, a subset of artificial intelligence, is a pivotal machine learning technique that primarily employs neural network models to capture and represent intricate data patterns [19, 20]. Its defining feature is the ability to perform feature learning and representation learning via multi-layer neural networks, which facilitates the automatic extraction of relevant features from data, enabling the modeling and resolution of complex problems. Common neural network structures utilized in deep learning include Multi-Layer Perceptrons (MLP), CNNs, and Recurrent Neural Networks [21]. These structures refine their weight parameters via the backpropagation algorithm, consistently enhancing the learning capabilities and overall performance.


Deep learning has led to remarkable achievements in various fields, including computer vision, natural language processing, and speech recognition. In image processing, it has proven effective in tasks such as image classification, object detection, and image segmentation. In medical imaging, deep learning is widely employed in tasks like breast ultrasound image segmentation, tumor detection, and disease diagnosis, significantly enhancing both accuracy and efficiency. Traditional image segmentation methods often require manual design of feature extractors and segmentation rules, limiting their performance and making them vulnerable to noise and other interferences. In contrast, deep learning techniques, through multi-layer neural networks, can automatically learn features from breast ultrasound images and offer robust capabilities for nonlinear fitting, making it more capable of handling complex image structures and variations [2]. In fact, deep learning has superseded conventional detection methods. For instance, Freitas et al. demonstrated the use of a novel neural network approach to replace the Sobel filter, utilizing it to an automatic method for calculating depth edges in breast contours [22].

Deep learning common neural networks


MLP


MLP is one of the most fundamental and widely utilized structures within artificial neural networks [23]. It consists of an input layer, one or several hidden layers, and an output layer, forming a fundamental architecture, as depicted in Figure 2.

Figure 2. Multi-layer perceptron.

The input layer functions as the gateway for the MLP, receiving feature vectors as input data. The hidden layers, or the intermediate layer, which may contain one or multiple layers of neurons, process this input. Each neuron receives the output from the preceding layer as its input and applies a nonlinear transformation via an activation function. The output layer, located at the final stage, produces the MLP’s results. The number of neurons within the output layer depends on the specific task: a single neuron is sufficient for binary classification, whereas multiple neurons are required for multi-class classification, with sigmoid or softmax activation functions typically used for classification.


Despite its simplicity and effectiveness, the MLP has limitations when handling complex tasks such as image processing. Its expressive power is constrained, making more advanced architectures like CNNs better suited for such tasks.

CNNs


CNNs emerge as one of the most prominent architectures in artificial neural networks, widely used for tasks such as image recognition and computer vision. As a type of feedforward neural network, CNNs are particularly effective at automatically extracting features from images and processing data with a grid-like structure. Key components of CNNs include convolutional layers, pooling layers, and fully connected layers [24]. A simplified structure of a CNNs is depicted in Figure 3 [25]. The convolutional layers perform feature extraction by convolving the input image with specific convolutional kernels, resulting in a series of feature maps. Pooling layers reduce the dimensionality of these feature maps, thus decreasing computational complexity and improving the network’s robustness. Fully connected layers then map the feature maps from the pooling layers to their corresponding labels, completing the classification process.

Figure 3. A simple CNN architecture. This figure is cited from [25]. CNNs, convolutional neural networks.

In addition to the previously mentioned components, CNNs also incorporate activation functions, loss functions, and the backpropagation algorithm. Figure 4 [25] depicts a representative CNNs structure, where activation functions (e.g., ReLUs) are applied between the convolutional and pooling layers, followed by one or more fully connected layers also utilizing ReLUs activations to enhance learning and feature mapping.

Figure 4. Common CNN structures. This figure is cited from [25]. CNNs, convolutional neural networks.

Fully Convolutional Network (FCNs)


FCNs, are specialized type of neural network introduced by Jonathan Long, Evan Shelhamer, and Trevor Darrell in 2015 [26]. Their innovation lies in applying FCNS to the complex task of image semantic segmentation. The key distinction of FCNs is the replacement of traditional CNN’s fully connected layers with convolutional layers. This modification enables FCNs to process input images of arbitrary sizes. In standard CNNs, convolutional layers are responsible for extracting pertinent image features, whereas fully connected layers subsequently map these features into classification or regression outputs. Conversely, FCNs accomplish the intricate transformation from feature maps to output maps solely through convolutional layers, allowing them to handle images of diverse sizes and generate output images with corresponding dimensions. The development of FCNS was motivated by the need to address a critical limitation of traditional CNNs, namely their inability to process input images of varying sizes [26, 27].

Deep learning algorithm applied to image segmentation


U-Net


U-Net, a specialized form of CNNs, is specifically tailored for image segmentation tasks. Its architecture, illustrated in Figure 5, operates at a low resolution of 32×32 pixels. Initially proposed by Ronneberger et al. in 2015, U-Net addresses the challenge of spatial information loss during the inter-layer transmission within CNNs, a limitation that can potentially compromise image segmentation accuracy [28]. Unlike traditional CNNs, U-Net incorporates skip connections to preserve information, enabling decoder layers to directly access both shallow and deep features from the encoder. Consequently, it mitigates issues such as information loss and vanishing gradients, thereby enhancing the spatial accuracy and semantic coherence of segmentation results. Owing to these advantages, U-Net is well-suited for tasks like breast ultrasound image segmentation.

Figure 5. U-net architecture. This figure is cited from [28].

The introduction of U-Net established a solid foundation for its subsequent utilization in breast ultrasound image segmentation. In 2018, Almajalid et al. implemented the U-Net architecture to segment breast ultrasound images [29]. Their approach began with preprocessing techniques, including histogram equalization to enhance contrast and anisotropic diffusion filtering to reduce speckle noise. Subsequently, to expand the training dataset, they employed data augmentation techniques, such as rotation and elastic deformation. Following this, noise regions were excised from the segmentation outcomes. Employing double cross-validation, the U-Net model was trained and tested on a dataset encompassing 221 breast ultrasound images, achieving an average DICE score of 82.52%, demonstrating its robustness and accuracy for tumor image segmentation. Similarly, Yap et al. proposed a patch-based LeNet and U-Net model, along with a transfer learning approach utilizing a pre-trained FCNS-AlexNet [30]. Their study compared these methods to four advanced lesion detection algorithms: radial gradient index, multifractal filtering, rule-based region ranking, and deformable part models. Additionally, they conducted a comparative analysis on datasets obtained from two distinct ultrasound systems, improving the overall performance of deep learning-based techniques for lesion detection.


Multiple automatic methods have been proposed for detecting and segmenting breast lesions. Nevertheless, challenges persist due to ultrasound artifacts and the complex shapes and locations of lesions, which hinder accurate segmentation. To address these issues, Amiri et al. introduced a two-stage approach, where U-Net was initially employed for lesion detection, followed by a second U-Net for segmenting the identified lesion areas [31]. They also developed a test-time augmentation technique to evaluate the effectiveness of the detection phase, which resulted in a 1.8% improvement in the average Dice score. In 2021, Guo et al. proposed an enhanced U-Net with an extended training method, which preserved the textural nuances and edge characteristics of breast tumors in output maps [32].


Notably, training U-Net with grayscale probability labels accelerated the process compared to conventional labeling methods, leading to clearer tumor boundaries and improving the safety and precision of robot-assisted breast surgeries. In 2022, Zhao et al. proposed a breast tumor ultrasound image segmentation method based on the U-Net framework, combining residual blocks and attention mechanisms [33]. Meanwhile, Yan and Liu et al. developed an attention-enhanced U-Net model with mixed dilated convolutions (AE U-net with HDC) for tumor segmentation in breast ultrasound images [34]. Recently, Pramanik et al. presented a novel network structure, dual branch U-Net (DBU-Net), which improves upon the classical U-Net architecture [35]. DBU-Net incorporates a dual-branch structure to enhance segmentation accuracy and robustness by handling feature information at different scales. One branch captures global and contextual information, while the other branch focuses on local and detailed information. This dual-branch approach allows DBU-Net to better capture tumor structures in breast ultrasound images, demonstrating significant improvements in segmentation performance over traditional U-Net. These advancements highlight the potential clinical applicability of DBU-Net in breast tumor image segmentation.

UNet++


UNet++, an extended version of the U-Net algorithm, was proposed in 2018 by a research team led by Zhou from Indonesia. Its network structure, shown in Figure 6, aims to enhance the performance of image segmentation tasks [36]. The primary improvement of UNet++ lies in its feature propagation and reuse capabilities, which are achieved by introducing multiple down-sampling and up-sampling pathways [36, 37]. Unlike the original U-Net architecture, which has a single down-sampling and up-sampling path, UNet++ introduces deeper feature fusion pathways by incorporating multiple nested, dense skip connections. These skip connections enable the network to propagate and reuse feature information across different levels, significantly improving segmentation accuracy.

Figure 6. U-net++ architecture. This figure is cited from [36].

UNet++ has found wide applications in medical image segmentation due to its numerous advantages. Wu et al. utilized UNet++ as the base structure and substituted its standard convolutions with dilated convolutions [38]. This modification increased the receptive field, allowing the model to incorporate more global contextual information, thereby enhancing organ segmentation accuracy.


Currently, the utilization of UNet++ in breast-related tasks primarily focuses on segmenting breast cancer cell nuclei. In 2020, Wang et al. proposed an improved breast cancer cell nucleus segmentation method using UNet++, with Inception-ResNet-V2 as the backbone [39]. The performance of this enhanced UNet++ was compared with the original UNet++ architecture using a breast cancer cell nucleus segmentation database, demonstrating the superiority of the improved version. The following year, Dinh et al. introduced a novel approach combining UNet++ with pre-trained EfficientNet as the backbone architecture for breast tumor cell nucleus segmentation, along with multi-organ transfer learning methods [40]. This new approach outperformed other techniques in segmenting breast tumor cell nuclei, significantly increasing accuracy. More recently, You et al. proposed a novel EfficientUNet++ model for breast tumor segmentation [41]. This model integrates ResNet18 as the encoder, a channel attention mechanism, and deep supervision to address the issue of gradient vanishing and enhance feature extraction. The channel attention module further improves the model’s accuracy, particularly in handling tumor boundaries.

SegNet


SegNet is a CNN architecture designed for semantic image segmentation tasks. Initially proposed in 2017 by a team from the Computer Laboratory at the University of Cambridge, including Badrinarayanan, its network structure is shown in Figure 7 [42]. SegNet’s design focuses on pixel-level image segmentation, incorporating an encoder for feature extraction and a decoder for feature restoration and segmentation.

Figure 7. SegNet architecture. This figure is cited from [42].
Compared to U-Net, it offers satisfactory segmentation performance while requiring lower computational and storage resources, making it suitable for resource-constrained environments. In contrast, U-Net is specifically optimized for medical image segmentation, considering factors such as high resolution and complex structures, which contribute to its superior performance in medical imaging tasks [43].


U-Net and SegNet were both applied to lesion segmentation in breast ultrasound images by Vianna et al. in 2021, with comparative analysis conducted using paired sample t-tests [44]. The results demonstrated U-Net’s superior performance in their task settings. Among the configurations tested, U-Net with Dice as the loss function and ReLU as the activation unit achieved the best results, with higher Dice similarity coefficients and shorter training times. This study concluded that, despite SegNet’s efficiency, U-Net outperformed it in dataset segmentation tasks. In the same year, Vianna et al. presented a performance analysis of SegNet for automatic breast ultrasound image segmentation [45]. The results showed that SegNet achieved accuracy similar to other models, although performance varied depending on the specific configurations used.


While the U-Net algorithm remains the primary choice for breast ultrasound image segmentation, SegNet offers advantages such as high computational efficiency and low memory usage, making it a promising candidate for other medical image processing tasks. SegNet has also been successfully applied to retinal vessel segmentation in fundus images, where accurate segmentation of vessel networks is crucial for the early detection and treatment of conditions like diabetic retinopathy and hypertensive retinopathy [46]. Moreover, SegNet has demonstrated utility in lesion segmentation in liver and lung CT images, as well as in brain MRI images, particularly in applications related to brain imaging [47-49]. However, the applicability and performance of SegNet should be validated experimentally for each specific domain, depending on the task and dataset.

DeepLab


DeepLab is a series of image segmentation algorithms proposed by the Google team, utilizing deep convolutional neural networks and probabilistic graphical models, such as conditional random fields, for pixel-level image classification. Since its first introduction in 2014 by Chen et al., the DeepLab series has evolved through multiple versions, including DeepLab v1, v2, v3, and v3+ [50, 51].


The DeepLab algorithms have been continuously optimized by researchers, improving the structure and parameters of deep convolutional neural networks and probabilistic models to enhance the accuracy of semantic image segmentation [52]. This progression has contributed significantly to the field of computer vision, offering valuable insights for understanding and analyzing breast ultrasound images. Shia et al. proposed a method for semantic segmentation of breast ultrasound images using the DeepLab v3+ model [53]. They analyzed a total of 684 breast tumor images and evaluated and found that DeepLab v3+ with a ResNet-50 decoder outperformed previous architectures in semantic segmentation tasks, particularly in identifying malignant features associated with BI-RADS malignant classifications in breast ultrasound (BUS) images. Additionally, Zhou et al. proposed a new deep learning-based breast region extraction method that integrates preprocessing techniques, including median filtering for noise suppression and contrast limited adaptive histogram equalization (CLAHE) for contrast enhancement, followed by semantic segmentation using the DeepLab v3+ model [54]. Moreover, this approach demonstrated state-of-the-art performance in extracting breast regions from mammography images. While DeepLab is less frequently applied in breast ultrasound segmentation, it has been widely used for segmenting human organs such as the brain [55]. Other literature on the application of deep learning in breast ultrasound image segmentation is presented in Table 2.

Table 2. New techniques for breast diagnosis

Note: CNN, Convolutional Neural Network; SVM, Support Vector Machine; DSM, Dual scan mammoscope; MS, Multiscale; MI, Multi-Instance; SMU-Net, Saliency-guided morphology-aware U-Net; W-Net, Waveform-Net; C-Net, Cascaded Convolutional Neural Network; AAU-net, adaptive attention U-net; GAN, generative adversarial networks; U-Net3+, A Full-Scale Connected U-Net; HCTNet, hybrid CNN-transformer network; AUC, Area Under the Curve; ACC, Accuracy; Dice, Dice Similarity Coefficient; IoU, Intersection over Union; MAD, Mean Absolute Deviation; F1, F1-score; MIoU, Mean Intersection over Union.

Discussion

Early research in breast disease diagnosis primarily focused on traditional image processing techniques for breast ultrasound image segmentation, such as thresholding, edge detection, and region growing. These methods typically rely on features such as grayscale intensity, texture, and shape for segmentation. For instance, tumor regions can be extracted using thresholding segmentation methods, which classify image pixels into distinct categories based on one or more thresholds; Edge detection methods, utilizing operators like Canny, Sobel, Roberts, and Laplacian, detect edges in the image to delineate the boundaries of the tumor region; Region growing methods, on the other hand, progressively merge adjacent pixels or regions into a unified area, starting from seed points and continuing until predefined stopping criteria are met.


However, as the incidence of breast diseases has risen globally, traditional ultrasound image segmentation methods have shown limitations like strong subjectivity and high sensitivity to noise, making them less effective for clinical applications. Consequently, breast ultrasound image segmentation has become a widely studied topic, garnering significant attention from numerous researchers.


In recent years, with the rapid advancement of deep learning technology, breast ultrasound image segmentation methods based on deep learning have garnered extensive attention and research. These methods typically use deep learning models such as CNNs to automatically learn and extract features from large datasets, achieving more accurate and robust segmentation results. Notable deep learning models, such as U-Net, UNet++, SegNet, and DeepLab, have shown remarkable success in medical image segmentation. Among them, U-Net algorithm has demonstrated strong performance in breast ultrasound image segmentation, while UNet++, SegNet, and DeepLab algorithms, although less commonly used in breast imaging, have proven effective in other medical domains, showcasing substantial potential for broader use.


This study primarily focuses on algorithm improvements, evaluation of segmentation performance, and their clinical applications in breast disease diagnosis. Despite progress, challenges persist, such as poor image quality and the complex morphology of tumors, which continue to hinder segmentation accuracy. Furthermore, issues related to computational efficiency and real-time performance remain significant obstacles. As a result, no single algorithm has yet proven optimal across all scenarios. In practical applications, it is crucial to consider the specific characteristics of the images and the requirements of the task, selecting or combining algorithms accordingly for the best results.

Conclusion

This article reviews the various breast ultrasound image segmentation techniques, with a focus on traditional methods and the application of deep learning. Segmentation plays a crucial role in early breast cancer screening, tumor measurement, and surgical planning. While traditional techniques like thresholding and edge detection provide some value, they often struggle with noise and complex shapes. In contrast, deep learning methods offer improved accuracy, though challenges like data annotation and high computational demands remain. Future advancements in this field are expected to focus on multimodal image fusion, improvements in deep learning approaches, and the integration of ultrasound with MRI and X-ray for enhanced diagnostic accuracy and clinical utility.

Declarations

Author contributions: Fangfang Chen: Responsible for the primary research work of the paper, including literature review, data analysis, and article writing. Miao Zhou: Responsible for part of the literature review and writing of specific chapters, particularly those related to relevant work and background. Jintao Duan: Participated in data collection and organization, contributing to the discussion of technical details and methods. Yuxiang Wang: Participated in literature search, provided experimental data support, and created some of the figures and tables. Liangqing Lin: Participated in the revision and organization of the paper, mainly helping with formatting and language. Wenhui Guo, Yongchu Hu: Responsible for the overall conception and review of the paper, guiding the research direction, and finalizing and revising the paper.

References

[1] Ilesanmi AE, Chaumrattanakul U, Makhanov SS. Methods for the segmentation and classification of breast ultrasound images: a review. J Ultrasound 2021;24(4):367-382.

[2] Liu S, Wang Y, Yang X, et al. Deep learning in medical ultrasound analysis: a review. Engineering 2019;5(2):261-275.

[3] Jia H, Ma J, Song W. Multilevel thresholding segmentation for color image using modified moth-flame optimization. IEEE Access 2019;7:44097-44134.

[4] Rodrigues PS, Giraldi GA. Improving the non-extensive medical image segmentation based on Tsallis entropy. Pattern Anal Applic 2011;14(4):369-379.

[5] Horsch K, Giger ML, Venta LA, et al. Automatic segmentation of breast lesions on ultrasound. Med Phys 2001;28(8):1652-1659.

[6] Cui J, Sahiner B, Chan HP, et al. A new automated method for the segmentation and characterization of breast masses on ultrasound images. Med Phys 2009;36(5):1553-1565.

[7] Zebari DA, Zeebaree DQ, Abdulazeez AM, et al. Improved threshold based and trainable fully automated segmentation for breast cancer boundary and pectoral muscle in mammogram images. IEEE Access 2020;8:203097-203116.

[8] Ghassan MGA, Ahmed MA. Abushaaala. Edge detection methods. 2015 2nd World Symposium on Web Applications and Networking (WSWAN) 2015;1-7.

[9] Shao D, Xu C, Xiang Y, et al. Ultrasound image segmentation with multilevel threshold based on differential search algorithm. IET Image Process 2019;13(6):998-1005.

[10] Rampun A, Lopez-Linares K, Morrow PJ, et al. Breast pectoral muscle segmentation in mammograms using a modified holistically-nested edge detection network. Med Image Anal 2019;57:1-17.

[11] Daoud MI, Al-Ali A, Alazrai R, et al. An Edge-Based Selection Method for Improving Regions-of-Interest Localizations Obtained Using Multiple Deep Learning Object-Detection Models in Breast Ultrasound Images. Sensors (Basel) 2022;22(18):6721.

[12] Chakraborty S, Bhowmik MK, Ghosh AK, et al. Automated edge detection of breast masses on mammograms. 2016 IEEE Region 10 Conference (TENCON) 2016;1241-1245.

[13] Guo G, Razmjoo N. A new interval differential equation for edge detection and determining breast cancer regions in mammography images. Syst Sci Control Eng 2019;7(1):346-356.

[14] Anjum R, Dipti RR, Rashid HO, et al. An Efficient Breast Cancer Analysis Technique by Using a Combination of HOG and Canny Edge Detection Techniques. 2021 5th International Conference on Trends in Electronics and Informatics (ICOEI) 2021;1290-1295.

[15] Gupta KK, Vijay R, Pahadiya P. Detection of abnormality in breast thermograms using Canny edge detection algorithm for thermography images. Int J Medical Eng Informatics 2022;14:31-42.

[16] Triwibowo DN, Dewa BP, Sumantri RBB, et al. Identification of Breast Tumors With Image Processing Using Canny Edge Detection. J Adv Health Inf Res 2023;1(1):28-34.

[17] Angelina S, Suresh LP, Veni SHK. Image segmentation based on genetic algorithm for region growth and region merging. 2012 International Conference on Computing, Electronics and Electrical Technologies (ICCEET) 2012;970-974.

[18] Xia X, Gai S. An Improved Dense Matching Algorithm for Face Based on Region Growing. Computer Engineering and Networking 2014;625-633.

[19] Wang Y, Ye J, Wang T, et al. Breast Ultrasound Image Segmentation Algorithm Using Adaptive Region Growing and Variation Level Sets. 2022;2022:1752390.

[20] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553):436-444.

[21] Deng L, Yu D. Deep Learning: Methods and Applications. Found Trends Signal Process 2014;7:197-387.

[22] Freitas N, Silva D, Mavioso C, et al. Deep Edge Detection Methods for the Automatic Calculation of the Breast Contour. Bioengineering (Basel) 2023;10(4):401.

[23] Taud H, Mas JF. Multilayer Perceptron (MLP). In: Camacho Olmedo MT, Paegelow M, Mas JF, Escobar F, editors. Geomatic Approaches for Modeling Land Change Scenarios. Cham. Springer International Publishing, 2018. p.451-455.

[24] Gu J, Wang Z, Kuen J, et al. Recent advances in convolutional neural networks. Pattern Recognit 2018;77:354-377.

[25] O'Shea K, Nash R, "An Introduction to Convolutional Neural Networks." Available: http://arxiv.org/abs/1511.08458

[26] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015;3431-3440.

[27] Wang J, Song L, Li Z, et al. End-to-End Object Detection with Fully Convolutional Network. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021;15844-15853.

[28] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 2015;234-241.

[29] Almajalid R, Shan J, Du Y, et al. Development of a Deep-Learning-Based Method for Breast Ultrasound Image Segmentation. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA) 2018;1103-1108.

[30] Yap MH, Pons G, Marti J, et al. Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J Biomed Health Inform 2018;22(4):1218-1226.

[31] Amiri M, Brooks R, Behboodi B, et al. Two-stage ultrasound image segmentation using U-Net and test time augmentation. Int J Comput Assist Radiol Surg 2020;15(6):981-988.

[32] Guo Y, Duan X, Wang C, et al. Segmentation and recognition of breast ultrasound images based on an expanded U-Net. PLoS One 2021;16(6):e0253202.

[33] Zhao T, Dai HJCI, Neuroscience. Breast Tumor Ultrasound Image Segmentation Method Based on Improved Residual U-Net Network. Comput Intell Neurosci 2022;2022:3905998.

[34] Yan Y, Liu Y, Wu Y, et al. Accurate segmentation of breast tumors using AE U-net with HDC model in ultrasound images. Biomed Signal Process Control 2022;72:103299.

[35] Pramanik P, Pramanik R, Schwenker F, et al. DBU-Net: Dual branch U-Net for tumor segmentation in breast ultrasound images. DBU-Net: Dual branch U-Net for tumor segmentation in breast ultrasound images. PLoS One 2023;18(11):e0293615.

[36] Zhou Z, Siddiquee MMR, Tajbakhsh N, et al. Unet++: A nested u-net architecture for medical image segmentation. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support (2018) 2018;11045:3-11.

[37] Zhou Z, Siddiquee MMR, Tajbakhsh N, et al. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 2020;39(6):1856-1867.

[38] Wu S, Wang Z, Liu C, et al. Automatic Segmentation of Pelvic Organs After Hysterectomy by using Dilated Convolution U-Net++ 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C) 2019;362-367.

[39] Wu Z, Wen Z, Chen Y, et al. An Improved Cell nuclei Segmentation Method Based on U-Net++ 2023 International Conference on Machine Vision, Image Processing and Imaging Technology (MVIPIT) 2023;78-84.

[40] Dinh Tuan L, Kwon S-G, Lee S-H, et al. Breast Tumor Cell Nuclei Segmentation in Histopathology Images using EfficientUnet++ and Multi-organ Transfer Learning. J Korea Multimedia Soc 2021;24(8):1000-1011.

[41] You G, Yang X, Lee X, et al. EfficientUNet: An efficient solution for breast tumour segmentation in ultrasound images. IET Image Process 2024;18(2):523-534.

[42] Badrinarayanan V, Kendall A, Cipolla RJltopa, et al. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 2017;39(12):2481-2495.

[43] Siddique N, Paheding S, Elkin C P, et al. U-Net and its variants for medical image segmentation: A review of theory and applications. Siddique N, Paheding S, Elkin CP, et al. U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications. IEEE Access 2021;9:82031-82057.

[44] Vianna P, Farias R, de Albuquerque Pereira WC. U-Net and SegNet performances on lesion segmentation of breast ultrasonography images. Res Biomed Eng 2021;37(2):171-179.

[45] Vianna PO, Farias R, Pereira WCA. Performance of the SegNet in the Segmentation of Breast Ultrasound Lesions. 2021 Global Medical Engineering Physics Exchanges/Pan American Health Care Exchanges (GMEPE/PAHCE) 2021;1-4.

[46] Ryu J, Rehman MU, Nizami IF, et al. SegRNet: A deep learning framework with multiscale feature fusion for robust retinal vessel segmentation. Comput Biol Med 2023;163:107132.

[47] Saood A, Hatem IJBMl. COVID-19 lung CT image segmentation using deep learning methods: U-Net versus SegNet. BMC Med Imaging 2021;21(1):19.

[48] Almotairi S, Kareem G, Aouf M, et al. Liver tumor segmentation in CT scans using modified SegNet. Sensors (Basel) 2020;20(5):1516.

[49] Alqazzaz S, Sun X, Yang X, et al. Automated brain tumor segmentation on multi-modal MR image using SegNet. 2019;5(2):209-219.

[50] Chen LC, Papandreou G, Kokkinos I, et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Trans Pattern Anal Mach Intell 2018;40(4):834-848.

[51] Chen LC, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. 2017;arXiv:1706.05587.

[52] Chen LC, Zhu Y, Papandreou G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Computer Vision - ECCV 2018 2018;833-851.

[53] Shia WC, Hsu FR, Dai ST, et al. Semantic segmentation of the malignant breast imaging reporting and data system lexicon on breast ultrasound images by using DeepLab v3+. Sensors (Basel) 2022;22(14):5352.

[54] Zhou K, Li W, Zhao D, et al. Deep learning-based breast region extraction of mammographic images combining pre-processing methods and semantic segmentation supported by DeepLab v3+. Technol Health Care 2022;30(S1):173-190.

[55] Roy Choudhury A, Vanguri R, Jambawalikar SR, et al. Segmentation of Brain Tumors Using DeepLabv3+. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries 2019;154-167.

[56] Shu X, Zhang L, Wang Z, et al. Deep neural networks with region-based pooling structures for mammographic image classification. IEEE Trans Med Imaging 2020;39(6):2246-2255.

[57] Ekici S, Jawzal HJmh. Breast cancer diagnosis using thermography and convolutional neural networks. Med Hypotheses 2020;137:109542.

[58] Heidari M, Mirniaharikandehei S, Liu W, et al. Development and assessment of a new global mammographic image feature analysis scheme to predict likelihood of malignant cases. IEEE Trans Med Imaging 2020;39(4):1235-1244.

[59] Nyayapathi N, Lim R, Zhang H, et al. Dual scan mammoscope (DSM)- a new portable photoacoustic breast imaging system with scanning in craniocaudal plane. IEEE Trans Biomed Eng 2020;67(5):1321-1327.

[60] Wang Z, Zhang L, Shu X, et al. An End-to-End Mammogram Diagnosis: A New Multi-Instance and Multiscale Method Based on Single-Image Feature. IEEE Trans Cogn Dev Syst 2021;13(3):535-545.

[61] Ning Z, Zhong S, Feng Q, et al. SMU-Net: Saliency-guided morphology-aware U-Net for breast lesion segmentation in ultrasound image. IEEE Trans Med Imaging 2022;41(2):476-490.

[62] Gare GR, Li J, Joshi R, et al. W-Net: Dense and diagnostic semantic segmentation of subcutaneous and breast tissue in ultrasound images by incorporating ultrasound RF waveform data. Med Image Anal 2022;76:102326.

[63] Chen G, Dai Y, Zhang JJCM, et al. C-Net: Cascaded convolutional neural network with global guidance and refinement residuals for breast ultrasound images segmentation. Comput Methods Programs Biomed 2022;225:107086.

[64] Chen G, Li L, Dai Y, et al. AAU-net: an adaptive attention U-net for breast lesions segmentation in ultrasound images. IEEE Trans Med Imaging 2023;42(5):1289-1300.

[65] Alruily M, Said W, Mostafa AM, et al. Breast ultrasound images augmentation and segmentation using GAN with identity block and modified U-Net 3+. Sensors (Basel) 2023;23(20):8599.

[66] Tekin E, Yazici C, Kusetogullari H, et al. Tubule-U-Net: a novel dataset and deep learning-based tubule segmentation framework in whole slide images of breast cancer. Sci Rep 2023;13(1):128.

[67] He Q, Yang Q, Xie M, et al. HCTNet: A hybrid CNN-transformer network for breast ultrasound image segmentation. Comput Biol Med 2023;155:106629.

Medical Artificial Intelligence

ISSN: 2957-5524

Volume 1, Issue 1

April 2025

Pages: 1-48

PDF CITE Accesses: 44
On This Page
Latest Issue
Abstract
Introduction
Traditional breast ultrasound segmentation methods
Application of deep learning in breast ultrasound segmentation
Discussion
Conclusion
Declarations
References
Medical Artificial Intelligence
ISSN: 2957-5524
ZENTIME PUBLISHING CORPORATION LIMITED
On This Page
CITE
On This Page
Abstract
Introduction
Traditional breast ultrasound segmentation methods
Application of deep learning in breast ultrasound segmentation
Discussion
Conclusion
Declarations
References