Many definitions

• Discrete case, joint PMF is straightforward:

$$P_{X,Y}(x,y) = \mathbb{P}[X = x \text{ and } Y = y]$$

• Continuous case,the joint PDF is a function $f_{X,Y}(x,y)$ that can be integrated to find the probability for an event $A$:

$$\mathbb{P}[A] = \int_{A} f_{X,Y}(x,y)dxdy$$

• Marginal PDF / PMF is defined as the integral / sum over the other variable, e.g.

$$f_X(x) = \int_{\Omega_Y}f_{X,Y}(x,y)dy$$

I.e. the probabilty density for x when we dont care about y.

• Independent random variables: you can factor the pmf/pdf

$$f_{X,Y}(x,y) = f_X(x)f_Y(y) \text{ for continious }\\ p_{X,Y}(x,y) = p_X(x)p_Y(y) \text{ for discrete }$$

• Independent and Identically Distributed (i.i.d.)

A collection of random variables is i.i.d. if all the variables or independent (the pdf/pmf can be factored) and they all have the same exact distribution, so that the joint distribution is:

$$f_{X_1,...,X_N}(x_1,...,x_N)= \prod_{n=1}^{N} f_{X_1}(x_n)$$