13.12 Lab: Multiple Testing

Review of Hypothesis Tests

We begin by performing some one-sample \(t\)-tests using the t.test() function.

First we create 100 variables, each consisting of 10 observations. The first 50 variables have mean \(0.5\) and variance \(1\), while the others have mean \(0\) and variance \(1\).

set.seed(6)
x <- matrix(rnorm(10 * 100), 10, 100)
x[, 1:50] <- x[, 1:50] + 0.5

Calculate the t-test.

t.test(x[, 1], mu = 0)

## 
##  One Sample t-test
## 
## data:  x[, 1]
## t = 2.0841, df = 9, p-value = 0.06682
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.05171076  1.26242719
## sample estimates:
## mean of x 
## 0.6053582

And with a for we calculate the t-tests and the pvalues:

p.values <- rep(0, 100)

for (i in 1:100)
  p.values[i] <- t.test(x[, i], mu = 0)$p.value

decision <- rep("Do not reject H0", 100)
decision[p.values <= .05] <- "Reject H0"

table(decision,
    c(rep("H0 is False", 50), rep("H0 is True", 50))
  )

##                   
## decision           H0 is False H0 is True
##   Do not reject H0          40         47
##   Reject H0                 10          3

Repeate the t-test:

x <- matrix(rnorm(10 * 100), 10, 100)
x[, 1:50] <- x[, 1:50] + 1

for (i in 1:100)
  p.values[i] <- t.test(x[, i], mu = 0)$p.value


decision <- rep("Do not reject H0", 100)
decision[p.values <= .05] <- "Reject H0"


table(decision,
    c(rep("H0 is False", 50), rep("H0 is True", 50))
  )

##                   
## decision           H0 is False H0 is True
##   Do not reject H0           9         49
##   Reject H0                 41          1