Supervisé.e par : Jean-François Carrier
Université de Montréal
Enhancement of quantitative estimation of metabolism and vascularization with positron emission tomography (PET) and Ultrafast Ultrasound Localization Microscopy (UULM) Using Deep Learning
« 1 Problem and context
Structural and functional imaging of tissue vasculature has been studied using various imaging modalities such as positron emission tomography (PET), magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound. Among all the molecular imaging modalities, no single modality is perfect and sufficient to obtain all the necessary information for any questions of interest. A recent novel technique inspired by super-resolved fluorescence microscopy called ultrasound localization microscopy (ULM), improved spatial resolution of vascular images, from hundreds to a few microns in vivo via the detection at thousands of frames per second of millions of individual microbubbles injected in the bloodstream. Our group (led by my co-supervisor J. Provost, IVADO member) has recently shown the feasibility of extending ULM to dynamic acquisitions in three dimensions using novel imaging sequences and reconstruction algorithms.
With the introduction of deep learning algorithms, research focusing on multimodality medical imaging has increased exponentially, such as image segmentation, denoising, and image reconstruction. In this work, we propose to combine ULM with ultra-resolution images and dynamic PET imaging to estimate parametric images of dynamic PET based on deep learning, using compartment pharmacokinetic modeling. Moreover, I aim to enhance the quality of dynamic PET images and extraction of perfusion and vasculature parameters to have a precise model of tissue behavior with denoising and precise segmentation.
In this project, we aim to combine the microvascular information from 3D ULM to the molecular information of dynamic PET imaging in order to enhance the resolution and quantification of PET dynamic acquisitions. The first step is an iterative parametric image reconstruction using a deep neural network. I will not use prior training pairs, but only the same ULM image. I will utilize the ULM images from the same image as anatomical prior (blood content in every voxel) to guiding the parametric image reconstruction through the neural network. The neural network will be inserted into the iterative parametric image reconstruction framework and pharmacokinetic modeling to achieve more precise kinetic parameters, rather than using it as a post-processing tool.
The second step of the project will be on denoising of dynamic PET images because of the high-level noises of these images. Generally, deep learning with convolutional neural networks (CNN), requires the preparation of large training image datasets. This presents a challenge in a clinical setting because it would be very difficult to prepare large, high-quality datasets. Recently, the deep image prior (DIP) approach suggests that CNN structures have an intrinsic ability to solve inverse problems such as denoising without any pre-training. The DIP approach iterates learning using a pair of random noise and corrupted images and a denoised image is obtained by the network output with moderate iterations. The third step will be the segmentation of PET images, which will be organ detection based on unsupervised learning. The main idea for this is that better image representation gets better clustering, and better clustering results helps to get better image representation.
Modified network structures are developed based on the 3D U-net for each step which consists of an “encode” part and a “decode” part. The encode part of architecture consisting of the repetitive applications of 3D convolution layers, each followed by a batch normalization (BN) and a leaky rectified linear unit (LReLU), and convolutional layers for downsampling. The decoding part consists of a deconvolution layer, followed by the BN and the LReLU, transposed convolutional layers for up-sampling, skip connection with the corresponding linked feature map from the encoding part. In each step, the network structure and the loss function are modified to have the best performance in that step.
We will extract perfusion and vasculature parameters to have a precise model of tissue behavior. The extracted parametrical maps non-invasively depict beneficiary information about tissue microvasculature and are used as input in the PET compartment pharmacokinetic model. The technique will be used on pre-clinical dynamic data for small animal microPET and on numerical phantoms for dynamic quantitative analysis, validation and sensitivity study. Applications for human PET in nuclear medicine – including tumor microenvironment parametrization – will be developed. «