CRAN Task View: Medical Image Analysis

Medical images are produced by systems such as magnetic resonance imaging (MRI), computed tomography (CT) and positron emission tomography (PET) scanners. They are often three-dimensional, and sometimes also have a dimension that varies with time or orientation. Moreover, they typically include important metadata relating to the details of the scan and the image’s spatial relationship with the scan subject. This information is stored with the images in one of several file formats designed for the domain.

The packages in this task view are designed to read and write these files, visualize medical images and process them in various ways. Some of them are applicable to conventional images as well, and some general-purpose image-processing package can also be used with medical image data. The image intensities, stored per pixel or voxel (3D pixel), generally map naturally into an R array, which is a standard data structure and therefore suitable for interoperable working with base R and other code.

Data Input/Output

DICOM

The industry standard format, for data coming off a clinical imaging device, is DICOM (Digital Imaging and Communications in Medicine). The DICOM “standard” is very broad and very complicated. Roughly speaking each DICOM-compliant file is a collection of fields organized into two four-byte sequences (group,element) that are represented as hexadecimal numbers and form a tag . The (group,element) combination announces what type of information is coming next. There is no fixed number of bytes for a DICOM header. The final (group,element) tag should be the “data” tag (7FE0,0010), such that all subsequent information is related to the image(s). In practice, there are many vendor-specific quirks associated with real DICOM files, which makes consistent handling a major challenge.

• The packages oro.dicom, divest and tractor.base provide R functions for general-purpose reading of DICOM files and converting them to ANALYZE or NIfTI format.
• Packages fmri and dti offer more specialized functions focussed on reading particular types of scans (see also below).
• Package DICOMread contains a simple wrapper around some DICOM-handling facilities from MATLAB.

ANALYZE and NIfTI

Although the industry standard for medical imaging data is DICOM, another format has come to be heavily used in the image analysis community. The ANALYZE format was originally developed in conjunction with an image processing system (of the same name) at the Mayo Foundation. An ANALYZE (7.5) format image is comprised of two files, the “hdr” and “img” files, that contain information about the acquisition and the image data itself, respectively. A more recent adaption of this format is known as NIfTI-1 and is a product of the Data Format Working Group (DFWG) from the Neuroimaging Informatics Technology Initiative (NIfTI). The NIfTI-1 data format is almost identical to the ANALYZE format, but offers a few improvements: merging of the header and image information into one file (.nii), re-organization of the 348-byte fixed header into more relevant categories and the possibility of extending the header information.

• The packages RNifti, fmri, tractor.base, oro.nifti, neuroim and nifti.io all provide functions that read/write ANALYZE and NIfTI files. They use a variety of internal data structures, but in each case the pixel or voxel data can be converted to a standard R array quite straightforwardly.
• Several other packages outlined below use one of these to perform their file I/O.

Other formats

There are a number of other formats that are specific to certain other software packages or applications.

• The gifti package reads the GIFTI geometry format, and the cifti package reads the CIFTI connectivity format. Both are related to the NIfTI image format mentioned above.
• Package tractor.base can read FreeSurfer’s MGH/MGZ image format, and freesurferformats can read this plus several other file formats that FreeSurfer uses for morphometry and surface meshes.

Visualization

• The package brainR includes functions for creating three-dimensional (3D) and four-dimensional (4D) images using WebGL, RGL, and JavaScript commands. This package relies on the X ToolKit (XTK).
• Morpho is a collection of tools for statistical shape analysis and visualization of point based shape representations (landmarks, meshes). Apart from the core functions such as General Procrustes Analysis and sliding of semi-landmarks, Morpho is sporting a variety of statistical procedures to assess group differences and asymmetry, most of them based on permutation/bootstrapping methods. For registration purposes there are functions to calculate landmark transforms (rigid, similarity, affine and thin-plate spline) as well as iterative closest point registration and automated alignment exploiting the shapes’ principal axes. To deal with missing/erroneous data there are imputation methods available for missing landmarks and interactive outlier detection. For visualization there are functions to create interactive 3D plots of distance maps as well as visualizing differences between point clouds by deforming rectangular grids, both in 2D and 3D. Additionally, it includes an algorithm to retrodeform surface meshes representing structures that have suffered a series of locally affine deformations (e.g. fossils).
• Rvcg interfaces VCGLIB to provide functions for manipulating triangular surface meshes; e.g., surfaces generated from medical image segmentations. Among those manipulations are quadric-edge collapse decimation, smoothing, subsampling, closest point search or uniform remeshing. Additionally it allows the generation of isosurfaces from 3D arrays. It has capabilities for import/export of STL, PLY and OBJ files, both in binary and ASCII format.

Magnetic Resonance Imaging (MRI)

Diffusion MRI

• The tractor.base package supports diffusion MRI specific metadata such as diffusion sensitization gradient directions and b-values. It is part of the wider TractoR project, which offers R-based tools for diffusion tensor estimation, fiber tracking and structural connectome estimation, all of which are based on diffusion MRI.
• The dti package provides functionality for diffusion tensor imaging (DTI), diffusion kurtosis imaging (DKI), modeling for high angular resolution diffusion weighted imaging (HARDI) using Q-ball reconstruction and tensor mixture models, several methods for structural adaptive smoothing, and fiber tracking.
• The dmri.tracking package also implements a fiber tracking algorithm.

Functional MRI

• The R package fmri provides tools for the analysis of functional MRI data. The core is the implementation of a new class of adaptive smoothing methods. These methods allow for a significant signal enhancement and reduction of false positive detections without, in contrast to traditional non-adaptive smoothing methods, reducing the effective spatial resolution. This property is especially of interest in the analysis of high-resolution functional MRI. The package includes functions for input/output of some standard imaging formats (ANALYZE, NIfTI, AFNI, DICOM) as well as for linear modelling the data and signal detection using Random Field Theory. It also includes ICA and NGCA (non-Gaussian Components Analysis) based methods.
• Neuroimage is an R package (currently only available within the neuroim project on R-Forge) that provides data structures and input/output routines for functional brain imaging data. It reads and writes NIfTI-1 data and provides S4 classes for handling multi-dimensional images.

Structural MRI

• The package mritc provides tools for MRI tissue classification using normal mixture models and (partial volume, higher resolution) hidden Markov normal mixture models fitted by various methods. Functions to obtain initial values and spatial parameters are available. Facilities for visualization and evaluation of classification results are provided. To improve the speed, table lookup methods are used in various places, vectorization is used to take advantage of conditional independence, and some computations are performed by embedded C code.
• Package qMRI supports the estimation of quantitative relaxometry maps from multi-parameter mapping (MPM) MRI acquisitions, including adaptive smoothing.

Simulation

• The package neuRosim (archived) allows users to generate fMRI time series or 4D data. Some high-level functions are created for fast data generation with only a few arguments and a diversity of functions to define activation and noise. For more advanced users it is possible to use the low-level functions and manipulate the arguments.

Magnetic Resonance Spectroscopy (MRS)

MRS uses the same basic scanner technology as MRI, but focuses on using it to obtain chemical spectra. This is used to measure concentrations of various chemical compounds including, in the medical context, metabolites with important biochemical roles.

• Package spant includes tools for reading, visualizing and processing MRS data, including methods for spectral fitting and spectral alignment.

General Image Processing

• adimpro is a package for 2D digital (color and B/W) images, actually not specific to medical imaging, but for general image processing.
• The package bayesImageS implements several algorithms for segmentation of 2D and 3D images (such as CT and MRI). It provides full Bayesian inference for hidden Markov normal mixture models, including the posterior distribution for the smoothing parameter. The pixel labels can be sampled using checkerboard Gibbs or Swendsen-Wang. MCMC algorithms for the smoothing parameter include the approximate exchange algorithm (AEA), pseudolikelihood (PL), thermodynamic integration (TI), and approximate Bayesian computation (ABC-MCMC and ABC-SMC). An external field prior can be used when an anatomical atlas or other spatial information is available.
• EBImage is an R package which provides general purpose functionality for the reading, writing, processing and analysis of images. Furthermore, in the context of microscopy-based cellular assays, this package offers tools to transform the images, segment cells and extract quantitative cellular descriptors.
• The imbibe package provides a set of fast, chainable image-processing operations which are applicable to images of two, three or four dimensions, particularly medical images.
• The package mmand (Mathematical Morphology in Any Number of Dimensions) provides morphological operations like erode and dilate, opening and closing, as well as smoothing and kernel-based image processing. It operates on arrays or array-like data of arbitrary dimension.
• The RNiftyReg provides an interface to the NiftyReg image registration tools. Rigid-body, affine and non-linear registrations are available and may be applied in 2D-to-2D, 3D-to-2D and 4D-to-3D procedures.
• The package fslr contains wrapper functions that interface with the FMRIB Sofware Library (FSL), a powerful and widely-used neuroimaging software library, using system commands. The goal with this package is to interface with FSL completely in R, where you pass R-based NIfTI objects and the function executes an FSL command and returns an R-based NIfTI object.

Positron Emission Tomography (PET)

• The occ package provides a generic function for estimating PET neuro-receptor occupancies by a drug, from the total volumes of distribution of a set of regions of interest (ROI). Fittings methods include the reference region, the ordinary least squares (OLS, sometimes known as “occupancy plot”) and the restricted maximum likelihood estimation (REML).
• The oro.pet package contains several parameter estimation routines for PET experiments including: the standard uptake value (SUV), occupancy, the simplified reference tissue model (SRTM), the multilinear reference tissue model (MRTM) and the half maximal inhibitory concentration (IC50).

Electroencephalography (EEG)

• edfReader reads some of the most popular file formats in EEG recordings.
• The EEG package (currently only available within the eeg project on R-Forge) reads in single trial EEG (currently only ascii-exported pre-processed and trial segmented in Brain Vision Analyzer), computes averages (i.e., event-related potentials or ERP’s) and stores ERP’s from multiple data sets in a data.frame like object, such that statistical analysis (linear model, (M)ANOVA) can be done using the familiar R modeling framework.
• eegkit includes many useful functions for analysing EEG signals (among others, visualizing positions of electrodes).
• PTAk is an R package that uses a multiway method to decompose a tensor (array) of any order, as a generalisation of a singular value decomposition (SVD) also supporting non-identity metrics and penalisations. A 2-way SVD with these extensions is also available. The package also includes additional multiway methods: PCAn (Tucker-n) and PARAFAC/CANDECOMP with these extensions. Applications include the analysis of EEG and functional MRI data.
• The raveio package supports the “R analysis and visualization of human intracranial electroencephalography data” (RAVE) project for analysis of EEG data from depth or surface recordings.

CRAN packages

Related links

• Journal of Statistical Software special volume on Magnetic Resonance Imaging in R.
• Neuroconductor is a Bioconductor-like platform for rapid testing and dissemination of reproducible computational imaging software in R.
• ANTsR is a framework that incorporates ITK and ANTs-based image processing methods into the R programming language.
• SimpleITK is a simplified layer built on top of ITK, intended to facilitate its use in rapid prototyping, education, interpreted languages. SimpleITK provides support for 2D and 3D images, and a selected set of pixel types for them. Different image filters may support a different collection of pixel types, in many cases due to computational requirements. The library is wrapped for interpreted languages by using SWIG. In particular, the following wrappings are available: Python, Java, Tcl, Lua, R and Ruby.

Other resources

• Bioconductor Package: EBImage
• R-Forge Project: eeg
• R-Forge Project: neuroim