ИСТИНА

Aims: Artificial intelligence is rapidly gaining ground in online detection, endoscopic and morphological characterization of colon epithelial neoplasms. Even for pathologists identification of metaplasia and dysplasia in the epithelium of the mucous glands could be an extremely difficult task. The same task in vivo, directly during the endoscopic examination is no less difficult, therefore the development of auxiliary mathematical models for image recognition is requested. Methods: We propose a new design of a convolutional neural network (CNN) based on U-Net model and use it for mucous glands segmentation. The main distinctive ideas of the proposed CNN lay in the multiscale architecture, using non-local blocks to capture long-range dependencies in the image and using a contour-aware loss function. The network was first trained on the public Warwick-QU dataset with non-linear augmentation process and was afterthat fine-tuned on the manually labeled histological images obtained from paraffin sections of endoscopic biopsy material of the colon. Results: The multiscale architecture of the proposed segmentational CNN makes it less sensitive to the scale of the input image. Due to the specific loss function it is able to detect and separate “stuck” glands. The used non-linear blocks have a positive effect on the time needed for model to converge. Altogether this leads to the accurate segmentation of glands on histology images (Dice coefficient = 0.87 for Warwick-QU dataset, Dice coefficient = 0.83 for the obtained dataset). Conclusions: The generalization ability of the proposed algorithm enables it to effectively segment individual glands as well as to perform inner-gland segmentation (detect nuclei, lumen and cytoplasm) in histological images. The subsequent development of this gland segmentation technology can allow to detect changes in the lumen shape (serration) of glands, in the nuclear-cytoplasmic ratio inside mucus-forming cells, and in the character of the expression of immunohistochemical markers.