13.12 Lab: Multiple Testing

Review of Hypothesis Tests

We begin by performing some one-sample $t$-tests using the t.test() function.

First we create 100 variables, each consisting of 10 observations. The first 50 variables have mean $0.5$ and variance $1$, while the others have mean $0$ and variance $1$.

set.seed(6)
x <- matrix(rnorm(10 * 100), 10, 100)
x[, 1:50] <- x[, 1:50] + 0.5

Calculate the t-test.

t.test(x[, 1], mu = 0)

## 
##  One Sample t-test
## 
## data:  x[, 1]
## t = 2.0841, df = 9, p-value = 0.06682
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.05171076  1.26242719
## sample estimates:
## mean of x 
## 0.6053582

And with a for we calculate the t-tests and the pvalues:

p.values <- rep(0, 100)

for (i in 1:100)
  p.values[i] <- t.test(x[, i], mu = 0)$p.value

decision <- rep("Do not reject H0", 100)
decision[p.values <= .05] <- "Reject H0"

table(decision,
    c(rep("H0 is False", 50), rep("H0 is True", 50))
  )

##                   
## decision           H0 is False H0 is True
##   Do not reject H0          40         47
##   Reject H0                 10          3

Repeate the t-test:

x <- matrix(rnorm(10 * 100), 10, 100)
x[, 1:50] <- x[, 1:50] + 1

for (i in 1:100)
  p.values[i] <- t.test(x[, i], mu = 0)$p.value


decision <- rep("Do not reject H0", 100)
decision[p.values <= .05] <- "Reject H0"


table(decision,
    c(rep("H0 is False", 50), rep("H0 is True", 50))
  )

##                   
## decision           H0 is False H0 is True
##   Do not reject H0           9         49
##   Reject H0                 41          1