Computational Medicine with Deep Learning (Danny Z. Chen)

Abstract

Computer technology plays a vital role in modern medicine, health care, and life sciences, especially in medical imaging, human genome study, clinical diagnosis and prognosis, treatment planning and optimization, and medical data management and analysis. As computing technology continues to evolve, computer science research and development will inevitably become an integral part of modern medicine and health care. Computational research and applications on modeling, formulating, and solving core problems in medicine and health care are not only crucially needed, but are actually indispensable.

Deep learning (DL) techniques have achieved remarkable performance for many computer vision tasks (e.g., image classification, object detection, and semantic segmentation). In this talk, we present new approaches based on DL techniques for solving a number of medical problems: identifying and classify immune cells in H&E staining histology images for diagnosis and treatment response monitoring of inflammation diseases (e.g., rheumatoid arthritis and inflammatory bowel disease) based on CNN and FCN, detecting and analyzing glial cells interacting with tumors in the brain microenvironment of metastatic breast cancer in 3D microscopy images based on U-Net, identifying and classifying glands and villi in histology colon images for diagnosis of inflammatory bowel disease based on CNN, tracking bacteria motion in time-lapse images based on RNN, and segmenting and analyzing fungus cells that collectively control their host’s behaviors based on U-Net. The image sizes in our medical applications tend to be very large. For example, even one 2D H&E histology image is of size 30000 * 30000 (for diagnosing inflammation diseases); one 3D whole brain image has over 1011 voxels (for studying brain metastasis of breast cancer).

We show that simply applying DL alone is often insufficient to solve our medical problems. Thus, we construct new procedures or components to complement and work with DL techniques. For example, we devise a new geometric “context” model based on the Voronoi diagram of clusters to capture cell context information, which complements the object identification capability of CNN, for identifying immune cells. We combine CNN with a process of computing maximal independent set to identify glands and villi in colon images. We incorporate U-Net with the Earth Mover’s Distance based matching model to segment fungus cells in 3D images for building fungi interaction networks. A key point is that DL is used as one main step in our approaches, which is complemented by other main steps. Further, we carefully select and combine parts (e.g., layers) of the DL networks for specific applications. For example, we develop a multi-scale FCN based approach for identifying medical regions with vastly different sizes and shapes, by reorganizing a set of FCNs of different scales and combine layers of FCNs to form new DL networks. We show experimental data and results to illustrate the clinical applications of our approaches.

Time

Speaker

Room