Using deep neural networks to automate cardiac measurements in 2D ultrasound in real-time
This video shows an application we are currently working on at the Centre for Innovative Ultrasound Solutions (CIUS). The application is developed using FAST which is integrated with TensorFlow for fast neural network inference.
A scientific publication on this topic can be downloaded from here or from IEEE at https://doi.org/10.1109/ULTSYM.2018.8579886
Ejection fraction and mitral annular plane systolic excursion (MAPSE) are two common ultrasound measurements of the heart. In order to calculate the ejection fraction, we need to calculate the volume of the heart chamber. To do this, clinicians have to acquire images from two perpendicular angles, also known as views, of the heart, and then manually draw the heart chamber wall on the images. This can be time-consuming, and is also subject to large interobserver variability, meaning that two different doctors may draw the heart chamber wall quite differently, thereby resulting in different volume measurements.
Two ultrasound images acquired at different angles of the heart with the left ventricle wall drawn in green.
Illustrations by Helene Mørk (www.helemork.com).
The application we have made recognizes the different views automatically, performs segmentation of the heart chamber wall and calculates the volume, ejection fraction and MAPSE in real-time while the user is scanning. This is all done without any interaction between the user and the computer, the user simply scans the patient as shown in the video above. The view classification and segmentation was performed using deep convolutional neural networks. These networks were trained using datasets acquired and annotated at St. Olavs Hospital in Norway and CREATIS in France.
Together with clinician Ivar Mjåland Salte at the Sørlandet Hospital in Kristiansand, we performed a study to see how well our methods can do these measurements automatically compared to manually drawing the heart wall contour. Ultrasound recordings of 72 patients was collected and manually annotated. The neural network was able to automatically and quite accurately calculate ejection fraction for all patients, even for patients with low image quality ultrasound images.
The final segmentation of apical four chamber and two chamber views in both end-diastole and end-systole. The horizontal show the lines which are used to measure the volume by Simpson’s methods.
