Abstract

     The increased accuracy and exhaustivity of modern Artificial Intelligence techniques in supporting the analysis of complex data, such as medical images, have exponentially increased real-world data collection for research purposes. This fact has led to the development of international repositories and high-performance computing solutions to deal with the computational demand for training models. However, other stages in the development of medical imaging biomarkers do not require such intensive computing resources, which has led to the convenience of integrating different computing backends tailored for the processing demands of the various stages of processing workflows. We present in this article a distributed and federated repository architecture for the development and application of medical image biomarkers that combines multiple cloud storages with cloud and HPC processing backends. The architecture has been deployed to serve the PRIMAGE (H2020 826494) project, aiming to collect and manage data from paediatric cancer. The repository seamlessly integrates distributed storage backends, an elastic Kubernetes cluster on a cloud on-premises and a supercomputer. Processing jobs are handled through a single control platform, synchronising data on demand. The article shows the specification of the different types of applications and a validation through a use case that make use of most of the features of the platform.

Introduction

     Increasingly, radiology is based on objective and quantifiable data extracted from Quantitative Imaging Biomarkers (QIBs). QIBs are quantitative indicators generated from structural, functional, physiological or biological characteristics of pathological lesions [1]. In the workflow development of QIBs, complex computational functions and models automatically extract attributes, namely radiomics features, from different types of radiological images to correlate them to the phenotype or genetic signatures of the lesions. These analyses aim to early detect and classify anomalies to predict prognostics, define follow-up results, or non-invasively assess the treatment response.

     In the last years, developers have analysed the images by learning from retrospective data, enriching radiomics features with demographic, clinical, liquid biopsies and genomic data because they improve the clinical value of the biomarkers [2]. Thus, gathering data processes are crucial to developing useful Clinical Decision support Systems (CDSS) based on QIBs in clinical practice, requiring a massive storage and high-performance computing capacity [3] for managing data on image biobanks. Furthermore, the huge amount of data makes traditional statistical analyses impractical, leading to a transition to novel textitArtificial Intelligence (AI) solutions such as Deep Learning [4]. Running AI algorithms efficiently requires high-computing performance resources [5] connected to the data storage backends.

Conclusions

     This work describes the design, implementation and validation of a software architecture to support the development and application of Quantitative Image Biomarkers. It implement a federated model to synchronise data among the different storage backends linked to different processing environments, including both Cloud and HPC resources. The architecture provides a federated Authentication and Authorisation Infrastructure based on Virtual Organisations that provide coherent and scalable authorisation management across the different providers. The processing backend is supported by a Kubernetes container management platform that runs the platform services and customised applications. The architecture is the outcome of a requirement elicitation process and uses mainstream, widely available components.

      The architecture uses the abstraction of the batch, High-Throughput-Compute, High-Performance computing and Interactive jobs to provide a simplified framework to develop applications with POSIX access to the distributed storage backends. HPC Jobs are managed through a mirror Kubernetes job that interacts with HPC batch queues to provide a seamless and coherent environment.