Approximating Ideal-Observer Performance Using Fisher Information and the Extreme Value Distribution in Detection Tasks

Abstract

When building an imaging system for signal detection tasks, one needs to evaluate the system performance before optimizing it. One evaluation method is to compute the performance of the Bayesian ideal observer. An observer's performance can be illustrated by its receiver operating characteristic (ROC) curve. The area under the ROC curve (AUC) can be used as the figure of merit. The ideal-observer AUC is often computationally expensive, if possible, therefore it is very desirable to have approximations to it.In detection tasks, one usually has two probability densities, signal-present and signal-absent, for the data vector. We use a single probability density with a variable scalar or vector parameter to represent the corresponding densities under the two hypotheses. We have developed approximations to the ideal-observer detectability, which is a monotonic function of the ideal-observer AUC. Our approximations are functions of signal parameters, and uses the Fisher information matrix, which is normally used in estimation tasks. The accuracy of the approximations is examined in analytical examples and lumpy-background simulations. If we plot the ideal-observer detectability as a function of the signal parameter, our approximation is able to predict the slope at the null parameter value. Even without an analytical expression for ideal-observer detectability we are able to compute analytical forms for its derivatives in terms of the Fisher information matrix and similarly defined statistical moments.In the clinic, one often needs to perform detection and localization tasks. One way to evaluate system performance in such tasks is to study the localization-ROC (LROC) curve and the area under the LROC curve (ALORC). We use the ideal ALROC as the figure of merit. We attempt to capture the distribution of the ideal-LROC test statistic with the extreme value distribution. We have also derived an expression for the ideal ALROC using the distribution of the ideal-LROC test statistic of signal-absent data only. In a different approach, by defining a parameterized probability density function of the data distribution, we have derived another approximation to the ideal ALROC for weak signals. This approximation results in an expression similar to the Fisher information approximation in detection tasks.