Many definitions

• Discrete case, joint PMF is straightforward:

\[ P_{X,Y}(x,y) = \mathbb{P}[X = x \text{ and } Y = y] \]

• Continuous case,the joint PDF is a function \(f_{X,Y}(x,y)\) that can be integrated to find the probability for an event \(A\):

\[ \mathbb{P}[A] = \int_{A} f_{X,Y}(x,y)dxdy \]

• Marginal PDF / PMF is defined as the integral / sum over the other variable, e.g.

\[ f_X(x) = \int_{\Omega_Y}f_{X,Y}(x,y)dy \]

I.e. the probabilty density for x when we dont care about y.

• Independent random variables: you can factor the pmf/pdf

\[ f_{X,Y}(x,y) = f_X(x)f_Y(y) \text{ for continious }\\ p_{X,Y}(x,y) = p_X(x)p_Y(y) \text{ for discrete } \]

• Independent and Identically Distributed (i.i.d.)

A collection of random variables is i.i.d. if all the variables or independent (the pdf/pmf can be factored) and they all have the same exact distribution, so that the joint distribution is:

\[ f_{X_1,...,X_N}(x_1,...,x_N)= \prod_{n=1}^{N} f_{X_1}(x_n) \]