9.10 Radial Kernels

image credit: Manin Bocss

• There are other options besides polynomial kernel functions, and a popular one is a radial kernel.

$$K(x, x_{i}) = \text{exp}\left(-\gamma\sum_{j=1}^p(x_{ij} - x_{i'j})^2\right), \quad \gamma > 0$$

• For a given test observations $x^*$, if it is far from $x_{i}$, then $K(x^*, x_{i})$ will be small given the negative $\gamma$ and large $\sum_{j=1}^p(x^*_{j} - x_{ij})^2)$.
• Thus, $x_{i}$ will play little role in $f(x^*)$.
• The predicted class for $x^*$ is based on the sign of $f(x^*)$, so training observations far from a given test point play little part in determining the label for a test observation.
• The radial kernel therefore exhibits local behavior with respect to other observations.