Installation and usage of this package.

Only three R-codes are included for user. After installing R language, execute the followings on R console.

# Installation
install.packages("BayesianFROC", dependencies = TRUE)

# Load the package
library(BayesianFROC);

# Run the function
BayesianFROC::fit_GUI_Shiny() # Variables are not required.

#The above three are R codes, 
#so please execute them on R console or R-studio console.

So, this package gives only one function BayesianFROC::fit_GUI_Shiny() to users.

Conventional Notation

In the following, we use the conventional likelihood notation;

\[ f(y|\theta), \]

where \(\LARGE{\color{red}{y}}\) denotes data, \(\LARGE{\color{blue}{\theta}}\) a model parameter. E.g., \(f(y|\theta) = \theta^y(1-\theta)^{1-y}\) for \(\{0,1\}\) valued random variable \(y\) defined by Bernoulli trial of success rate \(\theta, 0 \leq \theta \leq 1\). In the next section, we show that what trial arise and what is \(\theta\) and what is \(y\) in context of FROC analysis.

Data \(\large{\color{red}{y}}\) and trial

First of all, we introduce a trial from which we obtain a data-set to be fitted a model defined later.

Suppose that there are a physician, radiographs and a data-analyst. In the trial, the physician localizes shadows in radiographs if and only if he thinks shadows are lesions. In addition, the physician assigns to each localized shadow a confidence level which is a number as the following table.

E.g., if the physician localized a shadow with his highest confidence, then he assigns the number 3 to the localized shadow, which means that, in his point of view, the shadow is definitely a lesion. For example, if physician marks 10 positions in radiographs, then he also assigns 10 confidence levels for 10 marked positions.

After the physician’s lesion finding task, the data-analyst evaluates physician’s localization, i.e. the data-analyst counts physician’s truly localized positions (hits) and incorrectly marked positions (false alarms) for each confidence level. Here, we assume that data-analyst knows the truth (Gold-Standard of Diagnosis). Consequently, we will obtain the following table reflecting the physician’s recognition performance ability.

In this table, each component \(H_c\) and \(F_c\) are non negative integer valued random variables. For example, \(H_{3}\) denotes the number of hit with reader’s confidence level \(3^\text{rd}\).

So, in conventional notation we may write

\[\large{\color{red}{y}} = (H_0,H_1,H_2,H_3;F_1,F_2,F_3),\]

where we set \(H_\color{red}{0}:=N_L-(H_1+H_2+H_3)\) and \(N_L\) denotes the number of lesions among all radiographs. We note that \(\Sigma_{c=1,2,3} H_c \leq N_L\).

In the next section, we provide the probability law of these random variables \(H_c\) and \(F_c\) as the notion of image measure or the so-called push-forward measure, namely, likelihood function.

Likelihood \(f(\large{\color{red}{y}}|\large{\color{blue}{\theta}})\)

For model of 1 reader, 1 modality and 3 confidence levels. Let \(N_I\) denote the number of radiographs. Define the model by \[ \{H_{c};c=\color{red}{0},1,2,\cdots C\} \sim \color{green}{\text{Multinomial}}(\{p_{c,}(\theta)\}_{c=\color{red}{0},1,2,\cdots C}),\\ F_{c} \sim \text{Poisson}(q_c(\theta)N_I),\\ \]

where

\[ p_{c}(\theta) := \int_{\theta_c}^{\theta_{c+1}}\text{Gaussian}_{}(x|\mu,\sigma)dx,\\ q_c(\theta) := \int_{\theta_c}^{\theta_{c+1}} \biggl( \frac{d}{dz} \biggr) \log \Phi(z)dz. \]

from which, we can calculate the most important characteristic indicating the observer recognition performance ability as

\[ AUC := \Phi (\frac{\mu/\sigma}{\sqrt{(1/\sigma)^2+1}}), \\ \]

which is a real number between 0 and 1. If AUC is higher, then it means that radiologist recognition ability is also higher.

Note that model parameter is \(\LARGE{\color{blue}{\theta}}\) \(= (\theta_1,\theta_2,\theta_3,...\theta_C;\mu,\sigma)\) which should be estimated and \(\Phi\) denotes the cumulative distribution functions of the canonical Gaussian. Note that \(\theta_{C+1} = \infty\) and \(\theta_{0} = -\infty\).

library("BayesianFROC")
fit_GUI_Shiny()

MCS

I am a patient suffering from Chemical Sensitivity. Don’t wear volatile organic compounds (VOC), or huge burden to me. I hate prurigo nodularis in my body, syndet is the cause, (VOCs is also).

Explanatory variable for FROC models (Future development plan)

Here, the author announces how to include explanatory variable \(X \in \mathbb{R}^d\) in FROC models.

You know, to include fucking explanatory variable in FROCs, what should we do is very simple.

To do so, it is sufficient to consider the following model:

Define the model by \[ \{H_{c};c=\color{red}{0},1,2,\cdots C\} \sim \color{green}{\text{Multinomial}}(\{p_{c,}(\theta)\}_{c=\color{red}{0},1,2,\cdots C}),\\ F_{c} \sim \text{Poisson}(q_c(\theta)N_I),\\ \]

where

\[ p_{c}(\theta) := \int_{\theta_c}^{\theta_{c+1}}\text{Gaussian}_{}(x|\mu(\alpha,X),\sigma(\beta,X))dx,\\ q_c(\theta) := \int_{\theta_c}^{\theta_{c+1}} \biggl( \frac{d}{dz} \biggr) \log \Phi(z)dz. \]

and we modify the model parameters by using the following fucking link functions as follows.

\[\mu(\alpha,X):= \alpha X = \mu_0 + \sum \alpha_i X_i,\]

\[\sigma(\beta,X) := \sigma_0 \exp(\beta X)= \sigma_0\exp(\sum \beta_i X_i).\]

Let \(X=(x_!,x_2)\) be a explanatory variable of dummies indicating some categories. Then the data to be fitted a model is the following form!

For ref of this section,

A Bayesian approach to a general regression models for ROC curves; …et al; 1998;18;436-443

Sayonara! Good Bye \(\star\)

\(( \dot{} \omega \dot{})\) Love love me do! \(('; \omega ;)\) You know I love you \(\star\)

A short course on ROC

What is the difference between FROC and ROC? In ROC case, both of hits \(H_c\) and false alarms \(F_c\) have upper bounds, namely, there are positive integers \(N,M\) such that

\(\Sigma H_c \leq N\) and \(\Sigma F_c \leq M\), On the other hand, in FROC case, \(F_c\) dose not have any upper bounds.

If we assume such an upper bound for \(F_c\), say \(M\), then, by putting \(F_0 := M - \Sigma F_c\), the following ROC model is available.

\[ \{H_{c};c=\color{red}{0},1,2,\cdots C\} \sim \color{green}{\text{Multinomial}}(\{p_{c,}(\theta)\}_{c=\color{red}{0},1,2,\cdots C}),\\ \{F_{c};c=\color{red}{0},1,2,\cdots C\} \sim \color{green}{\text{Multinomial}}(\{q_{c,}(\theta)\}_{c=\color{red}{0},1,2,\cdots C}),\\ \]

where

\[ p_{c}(\theta) := \int_{\theta_c}^{\theta_{c+1}}\text{Gaussian}_{}(x|\mu,\sigma)dx,\\ q_{c}(\theta) := \int_{\theta_c}^{\theta_{c+1}}\text{Gaussian}_{}(x| 0,1)dx.\\ \]

Unfortunately, it is not so good, according to the following paper, which suggests that reductions of parameter makes MCMC to be more stable. E.g., by regarding \(\mu\) and \(\sigma\) as functions of parameter \(\theta\), i.e., \(\mu=\mu(\theta), \sigma=\sigma(\theta)\), we can reduce the parameter.

I have tried the above VERY VERY BAD model ROC model as follows.

library("BayesianFROC")
# Make an ROC dataset 
d<-BayesianFROC::ROC_data_creator2()

# Fit the above bad ROC model
f<-BayesianFROC::fit_srsc_ROC(d,ite  = 111, summary = FALSE,  cha = 1,)

# Check the model. The result indicate MCMC does not converge.
rstan::check_hmc_diagnostics(f)

# Plot looks bad so BAD MODEL...
rstan::traceplot(f)

# Plot looks good but BAD MODEL...
BayesianFROC::draw_ROC_Curve_from_fitted_model(f)
BayesianFROC::fit_GUI_ROC() #FROC user interface is used, so under construction...

Why the above ROC model is bad? I am not sure, but I guess level surface of likelihood function relates, or several critical points of likelihood function relates… Reference Bayesian analysis of a ROC curve for categorical data using a skew-binormal model; Balgobin Nandram and Thelge Buddika Peiris (This paper is nice!); 2018 volume 11 369-384; Statistics and its interface

Reference As another reference of ROC or FROC, see page 109 equation (6.3.5) of the book: Chakraborty, Dev P - Observer performance methods for diagnostic imaging foundations, modeling, and applications with R-based examples-CRC Press (2017)__

Next section, ….

we consider the comparison of imaging modality, such as MRI, CT, PET, …

This model also can be interpreted as a subject-specific random effect model if we forget the modality and use it as image instead.

I implement this because Bayesian is suitable for including individual differences,…. the author think this is a benefit of Bayesian ….

 


fit <- fit_Bayesian_FROC(
   ite  = 1111, 
   cha = 1,
   summary = TRUE,  
   Null.Hypothesis = F, 
   dataList = dd # example data to be fitted a model
   )


# Or GUI by Shiny

fit_GUI_Shiny_MRMC()

stanModel <- stan_model_of_sbc()

Simulation_Based_Calibration_single_reader_single_modality_via_rstan_sbc(
stanModel = stanModel,
ite     = 233,
M       = 111, # or 1111 which tooks a lot of times
epsilon = 0.04,BBB = 1.1,AAA =0.0091,sbc_from_rstan = TRUE)

# or 

stanModel <- stan_model_of_sbc()

Simulation_Based_Calibration_single_reader_single_modality_via_rstan_sbc(
stanModel = stanModel,
ite     = 233,
M       = 1111, 
epsilon = 0.04,BBB = 1.1,AAA =0.0091,sbc_from_rstan = TRUE)