README

The FarmTest library implements the Factor-Adjusted Robust Multiple Testing method proposed by Fan et al., 2019. Let X be a p-dimensional random vector with mean μ = (μ₁,…,μ_p)^T. This library carries out simultaneous inference on the p hypotheses H_0j : μ_j = μ_0j. To explicitly caputre the strong dependency among features, we assume that the data vectors X_i that are independently drawn from X following a factor model: X_i = μ + Bf_i + ε_i, where f_i are the common factors, B denotes the factor loading matrix, and ε_i are idiosyncratic errors. Specifically, we consider three different scenarios with (i) observable factors, (ii) latent factors and (iii) a mixture of covariates and latent factors. Assume f_i and ε_i are independent and have zero means. The number of hypotheses p may be comparable to or considerably exceed the sample size n.

FarmTest implements a series of adaptive Huber methods combined with fast data-driven tuning schemes to estimate model parameters and construct test statistics that are robust against heavy-tailed and/or asymetric error distributions. Extensions to two-sample simultaneous mean comparison are also included. As by-products, this library also contains functions that compute adaptive Huber mean and covariance matrix estimators that are of independent interest.

Main updates for version 2.0.0

The FarmTest method involves multiple tuning parameters for fitting the factor models. In the case of latent factors, the algorithm first computes a robust covariance matrix estimator, and then use the eigenvalue ratio method (Ahn and Horenstein, 2013) along with SVD to estimate the number of factors and loading vectors. It is therefore computationally expenstive to select all the tuning parameters via cross-validation. Instead, the current version makes use of the fast data-driven tuning scheme proposed by Ke et al., 2019, which significantly reduces the computational cost.

Installation

FarmTest is available on CRAN, and it can be installed into R environment using the command:

install.packages("FarmTest")

Functions

Getting help

Help on the functions can be accessed by typing ?, followed by function name at the R command prompt.

For example, ?farm.test will present a detailed documentation with inputs, outputs and examples of the function farm.test.

Examples

First generate data from a three-factor model X = μ + Bf + ε. The sample size and dimension (the number of hypotheses) are taken to be 50 and 100, respectively. The number of nonnulls is 5.

library(FarmTest)
n = 50
p = 100
K = 3
muX = rep(0, p)
muX[1:5] = 2
set.seed(2019)
epsilonX = matrix(rnorm(p * n, 0, 1), nrow = n)
BX = matrix(runif(p * K, -2, 2), nrow = p)
fX = matrix(rnorm(K * n, 0, 1), nrow = n)
X = rep(1, n) %*% t(muX) + fX %*% t(BX) + epsilonX

In this case, the factors are unobservable and thus need to be recovered from data. Assume one is interested in simultaneous inference on the means with two-sided alternatives. For a desired FDR level α=0.05, run FarmTest as follows:

output = farm.test(X)

The library includes summary.farm.test, print.farm.test and plot.farm.test functions, which summarize, print and visualize the results of farm.test:

summary(output)
print(output)
plot(output)

Based on 100 simulations, we report below the average values of the true positive rate (TPR), false positive rate (FPR) and false discover rate (FDR).

TPR	FPR	FDR
1.000	0.002	0.026

In addition, we illustrate the use of FarmTest under different circumstances. For one-sided alternatives, modify the alternative argument to be less or greater:

output = farm.test(X, alternative = "less")

The number of factors can be user-specified. It should be a non-negative integer that is less than the minumum between sample size and number of hypotheses. However, without any subjective ground of the data, this is not recommended.

output = farm.test(X, KX = 10)

As a special case, when we set number of factors to be zero, a robust test without factor adjustment will be conducted.

output = farm.test(X, KX = 0)

In the situation with observable factors, put the n by K factor matrix into argument fX:

output = farm.test(X, fX = fX)

Finally, as an extension to two-sample problems, we generate another sample Y with the same dimension 100, and conduct a two-sided test with latent factors.

muY = rep(0, p)
muY[1:5] = 4
epsilonY = matrix(rnorm(p * n, 0, 1), nrow = n)
BY = matrix(runif(p * K, -2, 2), nrow = p)
fY = matrix(rnorm(K * n, 0, 1), nrow = n)
Y = rep(1, n) %*% t(muY) + fY %*% t(BY) + epsilonY
output = farm.test(X, Y = Y)

As by-products, robust mean and covariance matrix estimation is not only an important step in the FarmTest, but also of independent interest in many other problems. We write separate functions huber.mean and huber.cov for this purpose.

library(FarmTest)
set.seed(1)
n = 1000
X = rlnorm(n, 0, 1.5)
huberMean = huber.mean(X)

n = 100
d = 50
X = matrix(rt(n * d, df = 3), n, d)
huberCov = huber.cov(X)

Remark

This library is built upon an earlier version written by Bose, K., Ke, Y. and Zhou, W.-X. (GitHub). Another library named tfHuber that implements data-driven robust mean and covariance matrix estimation as well as standard and l₁-regularized Huber regression can be found here.

License

System requirements

Author(s)

Maintainer

References

Ahn, S. C. and Horenstein, A. R. (2013). Eigenvalue ratio test for the number of factors. Econometrica 81 1203–1227. Paper

Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B. Stat. Methodol. 57 289–300. Paper

Bose, K., Fan, J., Ke, Y., Pan, X. and Zhou, W.-X. (2019). FarmTest: An R package for factor-adjusted robust multiple testing. Preprint

Eddelbuettel, D. and Francois, R. (2011). Rcpp: Seamless R and C++ integration. J. Stat. Softw. 40 1-18. Paper

Eddelbuettel, D. and Sanderson, C. (2014). RcppArmadillo: Accelerating R with high-performance C++ linear algebra. Comput. Statist. Data Anal. 71 1054-1063. Paper

Fan, J., Ke, Y., Sun, Q. and Zhou, W.-X. (2019). FarmTest: Factor-adjusted robust multiple testing with approximate false discovery control. J. Amer. Statist. Assoc. 114 1880-1893. Paper

Huber, P. J. (1964). Robust estimation of a location parameter. Ann. Math. Statist. 35 73-101. Paper

Ke, Y., Minsker, S., Ren, Z., Sun, Q. and Zhou, W.-X. (2019). User-friendly covariance estimation for heavy-tailed distributions. Statis. Sci. 34 454-471. Paper

Sanderson, C. and Curtin, R. (2016). Armadillo: A template-based C++ library for linear algebra. J. Open Source Softw. 1 26. Paper

Storey, J. D. (2002). A direct approach to false discovery rates. J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 479–498. Paper

Sun, Q., Zhou, W.-X. and Fan, J. (2020). Adaptive Huber regression. J. Amer. Statist. Assoc. 115 254-265. Paper

Zhou, W.-X., Bose, K., Fan, J. and Liu, H. (2018). A new perspective on robust M-estimation: Finite sample theory and applications to dependence-adjusted multiple testing. Ann. Statist. 46 1904-1931. Paper

FarmTest

Description