Background: The segmentation of ultrasound images is a key component of automated medical image analysis and enables detection of organs, tissues and other structures. The challenges associated with ultrasound image segmentation arise from varying image quality, the need to distinguish between different tissue types, and strong dependence on image geometry and artifacts generated by ultrasound imaging process (1). In recent years, advanced artificial intelligence methods, particularly convolutional neural networks, have achieved promising results in improving segmentation accuracy (2). In our study, we performed unsupervised segmentation of joint ultrasound images by Vision Transformer using spectral clustering (3), and evaluated the quality of the segmentation. Vision Transformer is a novel Deep learning method that applies sequences of image pathes to analyze the image entirely (5).

Objectives: To test the performance of unsupervised segmentation of joint ultrasound images from clinical practice using Vision Transformer and spectral clustering method.

Methods: Joint ultrasound examinations were performed using the General Electric LOGIQ E10 ultrasound machine. The images were exported in DICOM format. Subsequently, the DICOM images were converted to JPEG format so that patient-related and technical data were not included. Only B-mode images without the application of Doppler signals were analyzed. Images of the MCP, PIP, and MTP joints were then automatically separated and classified according to the OMERACT criteria (4) based on the activity score. Ultrasound images with insufficient quality were excluded. To achieve an equal number of images per group, the number of images in each group was adjusted to match that of the smallest group.

A deep learning approach using a Vision Transformer and spectral image segmentation was then applied. After image segmentation, the quality of the method was assessed by two doctors by analyzing the accuracy of the segmentation with respect to bone structure, the separation of the boundaries between the two bones of a joint, the boundaries between bone and soft tissue, recognition of the joint capsule, and synovitis. The quality of the segmentation was evaluated semi-quantitatively for each feature in each image as either 0 (low agreement), 0,5 (moderate agreement), or 1,0 (high agreement), resulting in a maximum quality score of 5 points per image

Results: 9.897 ultrasound images were extracted from the ultrasound device. 4.660 B-mode images could be automatically separated for further analysis. Only images with MCP, PIP, and MTP joints were included, comprising 912 images without evidence of synovitis, 419 images with grade 1 synovitis, 112 images with grade 2 synovitis, and 61 images with grade 3 synovitis. Through randomization and removing of extra images, the number of images per group was reduced to 61.

The segmentation model was trained using 244 images (61 images per group). Segmentation was performed using six clusters per image (Figure 1). Images in each group were analyzed for segmentation quality, with a maximum score of 61 points per feature (Table 1). The delineation between the bones themselves as well as between bones and soft tissues performed relatively well across all grades of synovitis. Homogeneity of the bone structure could only be identified with moderate success using the current model and showed no significant differences between the groups. Segmentation of synovitis differed significantly in favor of grade 3 synovitis. Recognition of the joint capsule was only possible in some cases.

Conclusions: Unsupervised segmentation of ultrasound images using unsupervised deep learning-based spectral clustering analysis using Vision Transformer can achieve a certain degree of success in identifying the boundaries between the bones of a joint, as well as between bone and soft tissue, and in delineating synovitis.

Figure 1.

Table 1.

REFERENCES: NIL.

Acknowledgments: NIL.

Disclosure of Interests: Viktor Korendovych Abbvie, Lilly, Novartis, Jan-Gerd Rademacher AbbVie, GSK, Lilly., Abbvie, AstraZeneca, Johnson & Johnson, Roche, UCB.