14.1 SVM model as motivating example

We’re interested in developing a classification model to classify sex for the palmers penguins dataset using a radial basis function support vector machine (svm).

library(tidyverse)
library(tidymodels)
library(palmerpenguins)
library(patchwork)
library(finetune)

penguins_df <- penguins %>%
  filter(!is.na(sex)) %>% # discarding NA obs
  select(-year, -island) # not useful

splits <- initial_split(penguins_df, strata = sex)

penguins_folds <- vfold_cv(training(splits), v = 5, strata = sex)

set.seed(420)

roc_res <- metric_set(roc_auc) # accuracy, a classification metric

Let’s fit a radial basis function support vector machine to the palmers penguins and tune the SVM cost parameter (cost()) and the σ parameter in the kernel function (rbf_sigma):

svm_rec <- 
  recipe(sex ~ ., data = penguins_df) 

svm_spec <-
  svm_rbf(cost = tune(), rbf_sigma = tune()) %>%
  set_mode("classification") %>%
  set_engine("kernlab")

Now, let’s set up our workflow() and feeding it our svm model

svm_wflow <- 
  workflow() %>% 
  add_model(svm_spec) %>% 
  add_recipe(svm_rec)

Let’s zoom in on the default parameter values for our two tuning parameters:

cost()

## Cost (quantitative)
## Transformer: log-2 [1e-100, Inf]
## Range (transformed scale): [-10, 5]

rbf_sigma()

## Radial Basis Function sigma (quantitative)
## Transformer: log-10 [1e-100, Inf]
## Range (transformed scale): [-10, 0]

We can change them:

svm_param <- 
  svm_wflow %>% 
  parameters() %>% 
  update(rbf_sigma = rbf_sigma(c(-7, -1)))

Because the methods that we will go over later need some resampled performance statistics before proceeding, we can use tune_grid() function to resample these values:

start_grid <- 
  svm_param %>% 
  update(
    cost = cost(c(-6, 1)),
    rbf_sigma = rbf_sigma(c(-6, -4))) %>% 
  grid_regular(levels = 2)

set.seed(2)

svm_initial <- 
  svm_wflow %>% 
  tune_grid(resamples = penguins_folds, grid = start_grid, metrics = roc_res)

collect_metrics(svm_initial)

## # A tibble: 4 × 8
##     cost rbf_sigma .metric .estimator  mean     n std_err .config             
##    <dbl>     <dbl> <chr>   <chr>      <dbl> <int>   <dbl> <chr>               
## 1 0.0156  0.000001 roc_auc binary     0.588     5  0.171  Preprocessor1_Model1
## 2 2       0.000001 roc_auc binary     0.588     5  0.171  Preprocessor1_Model2
## 3 0.0156  0.0001   roc_auc binary     0.588     5  0.172  Preprocessor1_Model3
## 4 2       0.0001   roc_auc binary     0.836     5  0.0545 Preprocessor1_Model4

We can see that there’s one point in which the performance is better. There results can be fed into iterative tuning functions as initial values, which we’ll see shortly.