BMDLab – Biomedical data analysis laboratory

AI in digital pathology

Digitally scanned Whole slide images (WSI) are provided from the Department of Pathology, Stavanger University Hospital (SUS). Several projects are currently ongoing looking at segmentation, diagnostic classification and prognostic prediction based on image processing and artificial intelligence.

Urinary bladder cancer

Urothelial TaT1 carcinoma, i.e. urinary bladder cancer, where a large population based dataset of patients with follow-up data on different prognoses and further development is available. Globally there has been an enormous increase in bladder cancer incidents the past decades, with the number of deaths increased by 49 per cent from 1990 to 2010. Correct prognosis of recurrence and progression is essential to avoid under- or over-treatment of the patient, as well as unnecessary suffering and cost. To diagnose the cancer grade and stage, pathologists study the histological images. However, this is a time-consuming process and reproducibility among pathologists are low.

In this project, a method for automatic classification of H&E stained whole slide images (WSI) into six different classes is proposed, where the classes are defined as; urothelium, stroma, muscle tissue, damaged tissue, blood and background. The method is based on convolutional neural networks (CNN). A second stager are using segmented areas for providing diagnosis classification.

Prostate cancer

Using a large population based database we will validate and implement an AI-algorithm for the automatic recognition of cancer vs. non-cancerous tissue in prostate biopsies in Helse-Vest. Also we will develop and test an AI-algorithm for the analyses of prostate cancer biopsies in order to establish the Gleason score and ultimately establish an individual risk of recurrence score for each patient. Furthermore an AI-algorithm for the automated identification of high grade prostatic intraepithelial neoplasia, a lesion considered the most likely precursor of prostatic carcinoma, will be developed and tested.

Melanoma of the skin

The cancer type with the largest increase in incidence during the last decade, the incidence of skin melanoma in Rogaland is currently amongst the highest in the world. Goal for this project is to develop a Computer-aided Diagnostic (CAD) system for whole slide images that will help pathology departments to diagnose malignant melanoma more efficiently, by (1) reducing examination time, (2) reducing diagnostic variations and (3) increase diagnostic accuracy. At SUS have 65.000 biopsies available, one of the largest databases of its kind in the world.

Funding: HelseVest

Triple Negative Breast cancer

This is the type of breast cancer with the worse survival due its heterogeneity which causes difficulty to treat the disease. In this project (part of CLARIFY) we will study, by image analysis, histopathological patterns and features of Triple negative breast cancer, together with immunohistochemical, genetic and clinical information, in order to improve the reproducibility and prognostic value of the pathology diagnosis and classification for this subgroup of breast cancers with the worst prognostic outcome.

Researchers taking part from BMDLab: Trygve Eftestøl, Emiel Janssen, Kjersti Engan, Rune Wetteland, Anders Blilie, Vebjørn Kvikstad and Erlend Tøssebro.

Cloud artificial intelligence for pathology (CLARIFY)

CLARIFY’s main goal is to develop a digital diagnostic environment that facilitates whole-slide-image (WSI) interpretation and diagnosis everywhere. We aim to maximize the benefits of digital pathology using artificial intelligence and cloud-oriented data algorithms.

BMDLab is the beneficiary of two ECR Early stage researchers of this project, which started in the fall of 2020:

Saul Fuster Navarro, with the project “Extracting diagnostic and prognostic information from histological images of non-muscle invasive bladder cancer”.

Neel Kanwal, with the project “Preprocessing, segmentation, and anonymization of whole slide images”

Local partners are UiS, CIPSI, bitYoga and Stavanger University Hospital.

From BMDLab: Kjersti Engan, Trygve Eftestøl, Emiel Janssen, Mahdieh Khanmohammadi, Saul Fuster Navarro, Neel Kanwal.

More information on the Clarify project website.

Safer Births

The Safer Births collaboration started in 2013 and involves many partners. Full information on the Safer Births website (external link).

Stillbirths are a worldwide challenge, with an estimated 2.6 million stillbirths in 2015, of these 1.3 million are estimated to have died during labour and birth, i.e. fresh stillbirth. In addition, one million newborns die within their first and only day of life. Birth asphyxia is the primary cause of these very early deaths. Birth asphyxia can also cause cerebral palsy and long term damage.

Within this project, observational and signal data describing labours and the newborn have been collected from the first FHR assessment on admission until 24 hours after the time of birth at partner hospitals in Tanzania.

Taking part from BMDLab: Hege Ersdal, Helge Myklebust, Trygve Eftestøl, Kjersti Engan, Jörn Schulz, Jan Terje Kvaløy. PhD students from BMDLab (all finished): Huyen Vu, Øyvind Meinich-Bache, Jarle Urdal.
Collaborators: See information at http://www.saferbirths.com.

AI for newborn survival

Subprojects at BMDLab:

Fetal heart rate analysis

Fetal heart rate (FHR) monitoring is a widely used method to assess the status of the fetus during pregnancy, labour, and birth. In high resource countries, continuous monitoring of the FHR is done using cardiotocography (CTG) for labours categorized as high risk. As CTG is not an alternative in low resource setting due to the cost, we use the Moyo fetal heart rate monitor within the Safer Birth to monitor the fetus during labour.

To extend the possibilities when using a continuous FHR monitors, such as Moyo, our group has proposed methods to detect noise, estimate missing data, and showed how contractions can be indicated using an accelerometer mounted in close proximity of the Doppler Ultrasound sensor. We have also explored how the FHR develops during labour based on the neonatal outcome.

Continued research on FHR signals acquired using Moyo has received funding from Idella and Lyse, starting fall of 2020.

From BMDLab: Jarle Urdal, Trygve Eftestøl, Kjersti Engan, Hege Ersdal, Helge Myklebust.

Funding: Idella and Lyse

Newborn resuscitation

In cases where the newborn is unable to start breathing, immediate intervention from the health care providers are required. The general guideline is to start the resuscitation within the first minute after birth, but a gap between the medical guidelines and what is performed has been observed.

To help increase the understanding of how the various resuscitation activities affect the resuscitation outcome, our group has proposed methods for automatically annotate resuscitation activities such as ventilation and stimulation.

Using ECG and movement of the newborn in combination with pressure, flow, and CO2 measurements from the bag-mask-resuscitator, we have proposed methods to automatically annotate when stimulation and ventilation is being performed. The data in this work has been acquired using the Laerdal newborn resuscitation monitor, a prototype of the NeoBeat.

In cases where a video of the resuscitation is available the possibilities for extracting information related to the resuscitation episode increases. Video allow us to capture events and activities performed by the health care providers (HCP) and by studying them together with Moyo and NeoBeat data we can get a more reliable and complete picture of the events before, during and after birth.

Manual video annotations are both time consuming and entails privacy issues, thus we aim to create activity timelines automatically. Our current work on activity recognition is based on supervised learning and utilization of convolutional Neural Networks. Step 1 is to detect and track objects relevant for the resuscitation activity and to propose regions to further analyze. Step 2 is to perform temporal analysis of the proposed regions, recognize the activities of interest, and create activity timelines.

At BMDLab: Øyvind Meinich-Bache, Jarle Urdal, Hege Ersdal, Trygve Eftestøl, Kjersti Engan, Helge Myklebust.

Partly funded by Laerdal Medical

NewbornTime

Deprivation of oxygen to an infant during and after birth might lead to birth asphyxia, one of the leading causes of newborn deaths, cerebral palsy and other long-term damage. According to guidelines, a newborn in need of help to start breathing should be resuscitated immediately after birth. Resuscitation activities include stimulating, clearing airways, and perform bag-mask-ventilation. In Norway, approximately 10% of term infants need stimulation and around 3% need bag-mask ventilation.

NewbornTime will produce a timeline describing events and activities performed on a newborn. Accurate time of birth will be detected using AI models from thermal videos collected in the delivery room. Activity recognition will be performed using AI in the form of deep convolutional neural networks (CNN) on thermal and RGB video from the resuscitation. The system will be designed to recognize multiple time-overlapping activities. Care will be given to make the AI models robust, reliable, general, and adaptive to be able to use it at different hospitals and settings. The timelines will be used to evaluate compliance to guidelines and identify successful resuscitation activity patterns. It can further be useful in a de-briefing and quality improvement tool.

The project is a collaboration between University of Stavanger (UiS), Stavanger University Hospital (SUS), Laerdal Medical and BitYoga. UiS, SUS and Laerdal has long experience in collaborative research on newborn care. They have documented promising results on detecting activities using resuscitation videos from a hospital in Tanzania. In NewbornTime the data collection will be performed at SUS. BitYoga and Laerdal will ensure smart GDPR compliant data-contracts and data-platforms. UiS will develop site-adaptive AI methods for activity recognition in video.

More information on the NewbornTime project website.

Smartphone use in CPR

Out-of-hospital cardiac arrest (OHCA) is a major cause of mortality throughout many regions of the world. Calling an emergency number should be the first thing the bystander does.

Today almost everybody has a smart phone, permitting the incorporation of other functions in addition to speech by using an emergency app. In case of emergency, the app is activated and takes control over the phone.  A dispatcher receives the phone call, and in addition to talking to the bystander, the app provides the dispatcher with GPS coordinates.   

At UiS researchers are working on including chest compression measurement using the video camera of the smart phone placed beside the patient during bystander cardiopulmonary resuscitation (CPR). Image processing is performed on the smart phone, and the dispatcher will be provided with the detected compression rate, if any. 

The app is now available at Google Play and Appstore under the name TCPR link See also TCPR Link Demo (YouTube). 

North sea race endurance exercise study (NEEDED)  

The NEEDED (North Sea Race Endurance Exercise Study) research program currently consists of three studies (NEEDED 2013, 2014 and 2018), including data from more than 1040 presumably healthy subjects exposed to a prolonged episode of high intensity exercise.

We add specific data on heart injury by using the cardiac specific biomarker: cardiac Troponin I. In the present work, we relate data derived from ECG and sport watches to the biomarker response in healthy and sick individuals. We will perform new studies providing additional data on the impact of exercise intensity and duration on the sensor data output and biomarker response in specific populations ranging from healthy Olympic athletes to patients with severe heart disease.    

The following figure shows the effect of the heart rate on different groups of participants.  The color codes refers to different levels of troponin, where the one in green indicates a known cardiac health issue. One can see how these participants’ reaction to the hill climb and following descent is  different from the other groups.  

From BMDLab: Stein Ørn, Kjell Le, Tomasz Wittorski, Kjersti Engan and Trygve Eftestøl. Collaborators: Magnus Friestad Bjørkavoll-Bergseth.

Analysis of magnetic resonance imaging (MR) and intracardial ECG from patients with myocardial infarction

Some patients who have had myocardial infarction will be at increased risk of arrhythmias and acute cardiac arrest. In this regard, it is interesting to study changes in the heart's ability to direct electrical impulses, as such changes can lead to the development of dangerous arrhythmias.

We study MRI images of the heart from patients with myocardial infarction. We have subprojects that engage in automatic segmentation of cardiac muscle from surrounding tissues, and automatic segmentation of scar tissue from healthy tissues from MR images of the heart.

Since there are hypotheses that there are boundaries between healthy and dead tissues that induce arrhythmias, we are working to quantify the degree of injury as a probability mapping of the myocardium where the likelihood of scar is visualized.

Using image analysis techniques characterise the texture of the myocardium, machine learning is applied to develop a probability function (Pmap) to indicate the presence of scarred tissue. With the probability mapping we can define myocardial regions, zooming in on regions in the scar border zone and on the scar itself.

These regions can be characterized and analysed to identify which carries information of diagnostic importance, for example for risk of dangerous arrhythmia.  

From BMDLab: Stein Ørn, Kjersti Engan, Trygve Eftestøl. Collaborators: Vidar Frøysa, Gøran Janson Berg.

Resuscitation data analysis

In Norway there are approximately 3000 annual cases of patients experiencing cardiac arrest outside hospitals in addition to a significant number in hospitals.  In out of hospital cardiac arrest, rescuers provide chest compressions, ventilations and electrical shock in an attempt to resuscitate the patient.  In our attempt to understand the relationship between therapy and the patient’s response.  We have developed methods to detect chest compression and recognize the cardiac rhythm to do this. 

Mortality from cardiac arrest outside hospitals is unfortunately high. Significant focus and research on systematic treatment of cardiac arrest has nevertheless improved the prognosis considerably, especially for those who survive until they arrive at the hospital.  The key elements of therapy are cardiac compressions, ventilations and electric shock to reestablish circulation. until the ambulance arrives.

Ongoing research is about developing new signal processing algorithms and statistical models to be able to determine which effect the various aspects of therapy have on survival.  Currently we are working on (1) developing indicator for monitoring the state of the patient, and (2) an indicator for monitoring the quality of the therapy. 

To establish these indicators, we have worked with signal processing and machine learning methods on the electrocardiograms, chest compression and ventilations signals measured throughout the resuscitation episodes.  We have worked with adaptive filters to remove the noise from chest compressions, we have developed chest compression and ventilation detectors so that the therapy can be characterised.  Furthermore, we have used machine learning to develop methods to determine the cardiac rhythms automatically.

With these methods in place, we want to study the relationship between therapy and the patient response to identify which factors are important for successful outcomes during therapy and for survival.

Various parts of this research are conducted in collaboration with several research groups, both nationally and internationally.

From BMDLab: Trygve Eftestøl, Jan T. Kvaløy

Collaborators:  Ali Bahrami Rad, Unai Irusta, Elisabete Aramendi, Erik Alonso, Trond Nordseth, Eirik Skogvoll, Lars Wik, Jo Kramer-Johansen Funding: NRC, Laerdal Foundation, Stiftelsen Norsk Luftambulanse

Estimation of coronary blood flow velocities from sparse and uncertain tomographic projections

This project is on coronary flow reserve study conducted as a collaboration between the University of Stavanger, Stavanger university hospital, Norway and University of Copenhagen, Denmark.

Coronary flow reserve is the ratio between the maximal flow and the resting flow down a coronary vessel. Impaired coronary flow reserve is associated with increased morbidity and mortality in patients with angina pectoris. The final goal of the project is to assess coronary flow reserve by estimating the ratio between blood flow during full hyperemia using adenosine infusion and blood flow velocity at rest.

Clinical standard procedures to assess the coronary flow reserve is angiography and ultrasounds. For angiography, a contrast agent is injected into the vessels, and the flow of the agent is visualized using X-ray. In ultrasound procedures a wire is inserted into the vessels and by the doppler effect the flow of blood is measured. Both techniques are invasive and performed in an operation room.

In this project, we seek to estimate the velocity of the blood inside the major vessels of the human heart based on image sequences taken using a C-arm during angiography operation. The project consists of two problems: (I) the creation of 3D, personalised models of beating coronary arteries on the surface of the heart and (II) the estimation of the time-varying velocity field of the blood inside the arteries.

Finally, we note that the quality of the reconstruction depends on which viewing angles, the C-arm is positioned. The C-arm has more degrees of freedom than a typical computed tomography system, which implies that fewer views may be needed to obtain necessary measurements for estimating the coronary flow reserve. However, in the long run, it is essential that the system runs in real-time, and that it can give suggestions to the operator on additional angles to optimize the estimations.

From BMDLab: Mahdieh Khanmohammadi (K.Engan, T. Eftestøl)  

Collaborators: Yuan Wang, Jon Sporring (Univeristy of Copenhagen), Charlotte Sæland, Alf Inge Larsen (SUS)

Analysis of brain magnetic resonance (MR) images from patients with dementia

Today, a combination of markers is used to determine diagnosis. In the project we look at hyperintense areas in the white substance part of the brain. These hyperintense areas are known to increase in size also in normal, as a function of age. However, the size and location are also linked to different forms of dementia.

We aim to develop an individualized disease severity index for prodromal dementia using a multi-biomarker and deep learning approach

From BMDLab:  Ketil Oppedal, Solveig Kristina Hammonds, Kathinka D. Kurz, Trygve Eftestøl

Analysis of perfusion CT (CTP) images in acute stroke patients

We are exploring different methods, including deep neural network approaches, to segment and study the infarcted regions after a stroke in order to contribute in accelerating the medical decision making.

A cerebral stroke is a severe neurological condition that can cause lasting brain damage, long-term disability, and death. Ischemic stroke is the most common type, accounting for 87% of all cerebral strokes.  An ischemic stroke occurs when an artery that supplies blood to the brain is obstructed. Neurological disorders are the leading cause of disability in the world with cerebral stroke accounting for the highest disability and mortality.  

Cerebral stroke is the third cause of death among adults in Norway and the second most common cause worldwide. Thus, cerebral stroke has a huge socio-economic impact for society and a tremendous impact on the quality of life of every single patient.

Rapid recognition of stroke symptoms, patient transfer and acute treatment are of vital importance in the acute setting and significantly improve outcomes in acute stroke patients.

One common initial procedure for patients with suspected acute ichemic stroke is the acquisition of Computed Tomography Perfusion (CTP) scanned over the injection period of a contrast agent. A series of parametric color-coded maps are generated from the CTP based on calculations on blood perfusion. These maps are interpreted by a radiologist to help assess the severity of the stroke and to identify the different infarcted regions inside the brain.

Whether treatment is applied is dependent on time from symptoms onset to hospital admission but also largely on imaging results with CTP being the key-modality for patient selection. However, calculations on blood perfusion in the ischemic area based on CTP are far from perfect in diagnostic accuracy, and further improvement of the methods in use is needed.

In our project we are exploring different methods, including deep neural network approaches, to segment the infarcted regions in a fully automated way in order to contribute in accelerating the medical decision making.

We will also use a parametric tool called capillary transit time heteogeniety (CTH) which assesses the environment on the capillary level during an ischemic event in the brain. These analyzations will be done with the use of an external software (Cercare Medical ApS). CTH has shown to be one of the most promising calculation tools having the capasity to select more patients to treatment. 

Currently working on the project:

• Technical PhD student: Luca Tomasetti, start August 2019. Project: Image analysis and artificial intelligence on computed tomography (CT) perfusion and angiography images of acute stroke patients. Supervisors: Mahdieh Khanmohammadi, Kjersti Engan, Kathinka Kurz.
• Medical PhD student: Liv Jorun Høllesli, start nov 2019. Project: In depth analysis of perfusion computed tomography (CTP) in patients with acute stroke. Supervisors: Kathinka Kurz, Kjersti Engan.

Statistical shape analysis with application to Parkinson’s disease

Today between 6.000 and 8.000 people with Parkinson live in Norway. This number will increase further in an increasingly older society. Understanding of risk factors and pathological changes in Parkinson patients is important to improve health care for patients.

The aim of this project is to study advanced three-dimensional models for human shapes like brain structures that contain a rich set of geometrical information.

Further, based on the shape models, we like to develop methods that allow for sensitive statistical inferences in order to detect global and local shape changes in Parkinson and to find associations with other medical covariates.

Taking part from BMDLab: Ass.prof. Jörn Schulz, professor Jan Terje Kvaløy, PhD student Mohsen Taheri Shalmani