Automatic segmentation of histological images for quality evaluation in tissue engineering

M. Jiřík, I. Gruber, V. Moulisová, V. Liška et al.

Oncological and surgical treatment of malignant focal processes of the liver reaches its limits in cases where the amount of affected parenchyma is too large or where the hilar structures are involved – stages at which a complete removal of the tumor tissue equals a total loss of the organ. In such patients, restoration of the liver function by means of conventional liver transplantation is generally out of the question, as it requires the administration of drugs to suppress the immune system’s response. Since the immune system fights not only pathogens but also cancerous cells, immunosuppression can lead to a rapid recurrence of the disease originating from the remnants of the malignant cells left in the body. The hope for a change lies in tissue engineering research, which promises to create an artificial replacement organ using the patient’s own cells, thereby reducing the need for immunosuppression.

Liver scaffold in the final stage of preparation

An important step in our approach to the production of an artificial organ is the preparation of an inanimate skeleton of the liver (scaffold), which is obtained from the liver of a domestic pig and is intended for subsequent colonization by living cells. To evaluate the quality of this scaffold, it is necessary to distinguish the areas of the liver lobules and their surroundings (septa). To follow the previously described quality assessment methodology, it has hitherto been necessary to manually label parts of the scaffold samples in histological slides stained with hematoxylin and eosin. In cooperation with colleagues from the Biomedical Center of the Faculty of Medicine in Pilsen of Charles University and Universitätsklinikum Jena, we designed an algorithm that produces the label automatically. The algorithm, which is based on convolutional neural networks, was published in the Sensors journal (Q1, IF=3.275).

Hematoxylin-and-eosin-stained liver scaffold

Compared to classical methods of computer vision, convolutional neural networks usually offer better results, but require a considerable amount of training data. Therefore, we designed a two-phase training procedure. In the first phase, we use the methods of classical computer vision for semi-automatic segmentation using texture features. In this phase, we use training datasets where less than 5% of the image area is annotated. The output data are then used in the second phase for training a neural network. The architecture of the neural network is inspired by the U-Net network. After the training, full segmentation is produced automatically.


Two-phase training procedure

Automatic histological image processing allows extensive skeletal quality measurements to be made. An important benefit of this method is the repeatability of measurements and their independence from the human factor, once the network is trained. In the follow-up research, our team is going to integrate the proposed method into a freely available application, which will make it possible to compare the results of laboratories from all over the world.

References


[Liska2019]
Liška V, Moulisová V, Pálek R, Rosendorf J, Červenková L, Bolek L, Třeška V. Repopulation of decellularized pig scaffolds: Promising approach for liver tissue engineering. Rozhl Chir. 2019 Fall;98(10):388-393. English. PMID: 31842567.

[Moulisova2020]
Moulisová V, Jiřík M, Schindler C, et al. Novel morphological multi-scale evaluation system for quality assessment of decellularized liver scaffolds. Journal of Tissue Engineering. January 2020. doi:10.1177/2041731420921121

[Jirik2020]
Jirik, M.; Gruber, I.; Moulisova, V.; Schindler, C.; Cervenkova, L.; Palek, R.; Rosendorf, J.; Arlt, J.; Bolek, L.; Dejmek, J.; Dahmen, U.; Zelezny, M.; Liska, V. Semantic Segmentation of Intralobular and Extralobular Tissue from Liver Scaffold H&E Images. Sensors 2020, 20, 7063.

Environment Classification System for Mobile Robots: How to distinguish Between Indoor from Outdoor Environment

The research of environment classification for mobile robots focuses on using 1D non-visual sensors such as thermometer, humidity sensor, air pressure sensor, or ceiling height sensor to detect the transition between defined environments. The robot’s behavior is then adapted based on this information.

The crucial part of this research is an image classifier trained to recognize the environment’s type. In particular, in our paper [1], we focused on the classification of an indoor and outdoor environment. It is the most extensive comparison of description methods and classifiers (including neural networks) performed on a single dataset to our best knowledge.

In our papers [2] and [3], the system for environment change detection and environment classification is proposed. The system uses non-visual sensors to generate triggers when the transition between environments is detected. Triggers activate the classification of the environment. Thus, its main purpose is to reduce the time requirements of the environment classification compared to the system, which depends only on images.

References:

[1] Neduchal, P., Gruber, I., & Železný, M. (2020, October). Indoor vs. Outdoor Scene Classification for Mobile Robots. In International Conference on Interactive Collaborative Robotics (pp. 243-252). Springer, Cham.

[2] Neduchal, P., & Železný, M. (2020, September). Environment Classification Approach for Mobile Robots. In Proceedings of 15th International Conference on Electromechanics and Robotics” Zavalishin’s Readings” (pp. 421-432). Springer, Singapore.

[3] Neduchal, P., Bureš, L., & Železný, M. (2019). Environment detection system for localization and mapping purposes. IFAC-PapersOnLine, 52(27), 323-328.


HANDS19 Challenge

Our colleagues Marek Hrúz, Jakub Kanis, and Zdeněk Krňoul won the HANDS19 Challenge Task 2. HANDS19 Challenge is a public competition designed for the evaluation of the task of 3D hand pose estimation in both depth and color modalities in the presence and absence of objects. The goal of Task 2 is depth-based 3D Hand Pose Estimation while Interacting with Objects: This task builds on F-PHAB dataset. Objects appear to be manipulated by a subject in an egocentric viewpoint. Some hand shapes and objects are strategically excluded from the training set in order to measure the interpolation and extrapolation capabilities of submissions. The first prize was announced and presented at a workshop held at the International Conference on Computer Vision (ICCV) 2019.

Neural Sign Language Synthesis: Words Are Our Glosses

Jan Zelinka, Jakub Kanis

Our work deals with a text-to-video sign language synthesis. Instead of direct video production, we focused on skeletal model production. Our main goal was to design a fully end-to-end automatic sign language synthesis system trained only on available free data (daily TV broadcasting). Thus, we excluded any manual video annotation. Furthermore, our designed approach even does not rely on any video segmentation. A proposed feed-forward transformer and recurrent transformer were investigated. To improve the performance of our sequence-to-sequence transformer, soft non-monotonic attention was employed in our training process. A benefit of character-level features was compared with word-level features. We focused our experiments on a weather forecasting dataset in the Czech Sign Language.


J. Zelinka and J. Kanis, “Neural Sign Language Synthesis: Words Are Our Glosses,” 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA, 2020, pp. 3384-3392, doi: 10.1109/WACV45572.2020.9093516.

Towards Regression Task-Oriented Annotation Tool for Microscopic Images

Annotating a dataset for training a Supervised Machine Learning algorithm is time and annotator’s attention intensive. Our colleagues from the Department of Cybernetics (Miroslav Jirik, Ivan Gruber, and Milos Zelezny) developed a procedure for producing the datasets based on microscopic whole slide images for regression tasks. Together with colleagues from the Biomedical Center of Charles University, they prepared an open-source application for creating annotations of the dataset with minimal demands on the expert’s time. The work was published in Lecture Notes in Computer Science (SJR=0.283).

Jirik M. et al. (2020) MicrAnt: Towards Regression Task Oriented Annotation Tool for Microscopic Images. In: Lukić T., Barneva R., Brimkov V., Čomić L., Sladoje N. (eds) Combinatorial Image Analysis. IWCIA 2020. Lecture Notes in Computer Science, vol 12148. Springer, Cham. https://doi.org/10.1007/978-3-030-51002-2_15

Miroslav Jirik awarded for the Lisa – Application to Support Liver Cancer Research

Our colleague from the Department of Cybernetics and NTIS Research Center, Ing. Miroslav Jiřík, Ph.D. and his application  Lisa (Liver Surgery Analyzer) have been awarded the Czech DIGI @ MED Award 2019 by the Roche Biotechnology Company. In 2013, it ranked 3rd among the largest pharmaceutical and biotechnology companies in the world (in terms of sales). The Evaluation Jury, composed of renowned long-term health care professionals and IT technology consultants, aims to stimulate young innovators and promote innovative ideas using digital technologies in healthcare and health promotion. The award ceremony took place on 26.11.2019 in Carolina.

Ing. Miroslav Jirik, Ph.D. is an employee of the Biomedical Center of the Faculty of Medicine in Pilsen and the NTIS Research Center of the University of West Bohemia. His long-term research topic is in computer vision and machine learning methods in the field of medical applications. For our departments, it mediates contact with medical research conducted at the Biomedical Center of the Faculty of Medicine.

The user of Lisa is a doctor, a researcher who has specific liver-motivated requirements that cannot be satisfied with common commercial tools. The application serves as a platform to support the research of the blood perfusion in the liver. The basic idea is to extend the possibilities of current clinical care of patients with liver cancer and to allow surgeries at the margin of operability by modeling the outcome of surgery and estimating the restoration potential of the rest of the organ. A significant shift in the knowledge line in favor of patient rescue is made possible by combining the latest technology and medical research, which is being developed jointly by the Faculty of Medicine in Pilsen, Charles University and the Faculty of Applied Sciences, University of West Bohemia. These include the currently implemented AMTMI project – Application of modern technologies in medicine and industry (reg. No .: CZ.02.1.01 / 0.0 / 0.0 / 17_048 / 0007280) supported within the framework of the Pre-Application Research call for the ITI Pilsen Metropolitan Areas.