9.10 Radial Kernels

image credit: Manin Bocss

• There are other options besides polynomial kernel functions, and a popular one is a radial kernel.

\[K(x, x_{i}) = \text{exp}\left(-\gamma\sum_{j=1}^p(x_{ij} - x_{i'j})^2\right), \quad \gamma > 0\]

• For a given test observations \(x^*\), if it is far from \(x_{i}\), then \(K(x^*, x_{i})\) will be small given the negative \(\gamma\) and large \(\sum_{j=1}^p(x^*_{j} - x_{ij})^2)\).
• Thus, \(x_{i}\) will play little role in \(f(x^*)\).
• The predicted class for \(x^*\) is based on the sign of \(f(x^*)\), so training observations far from a given test point play little part in determining the label for a test observation.
• The radial kernel therefore exhibits local behavior with respect to other observations.