Tidy Modeling with R Book Club
Welcome
Book club meetings
1
Software for modeling
1.1
The pit of success
1.2
Types of models
1.3
Terminology
1.4
The data analysis process
1.5
The modeling process
1.6
Meeting Videos
1.6.1
Cohort 1
1.6.2
Cohort 2
1.6.3
Cohort 3
1.6.4
Cohort 4
2
A tidyverse primer
2.1
Tidyverse design Principles
2.2
Design for Humans - Overview
2.3
Design for Humans and the Tidyverse
2.4
Reusing existing data structures
2.5
Designed for the pipe
2.6
Designed for Functional Programming
2.7
Tibbles vs. Data Frames
2.8
How to read and wrangle data
2.9
Further Reading
2.10
Meeting Videos
2.10.1
Cohort 1
2.10.2
Cohort 2
2.10.3
Cohort 3
2.10.4
Cohort 4
3
A review of R modeling fundamentals
3.1
R formula syntax
3.1.1
Recap
3.2
Inspecting and developing models
3.3
More of
{base}
and
{stats}
3.4
Why Tidy Principles and
{tidymodels}
?
3.5
Meeting Videos
3.5.1
Cohort 1
3.5.2
Cohort 2
3.5.3
Cohort 3
3.5.4
Cohort 4
Basics
4
The Ames housing data
4.1
Pittsburgh: a parallel real world example
4.2
Meeting Videos
4.2.1
Cohort 1
4.2.2
Cohort 2
4.2.3
Cohort 3
4.2.4
Cohort 4
5
Spending our data
5.1
Spending our data
5.2
Common methods for splitting data
5.3
Class imbalance
5.3.1
Stratified sampling simulation
5.4
Continuous outcome data
5.5
Time series data
5.6
Multi-level data
5.7
What proportion should be used?
5.8
Summary
5.8.1
References
5.9
Meeting Videos
5.9.1
Cohort 1
5.9.2
Cohort 2
5.9.3
Cohort 3
5.9.4
Cohort 4
6
Fitting models with parsnip
6.1
Create a Model
6.1.1
Different Model Interfaces
6.1.2
Model Specification
6.1.3
Model Fitting
6.1.4
Generalized Model Arguments
6.2
Use Model results
6.3
Make Predictions
6.4
{tidymodels}-Adjacent Packages
6.5
Summary
6.6
Meeting Videos
6.6.1
Cohort 1
6.6.2
Cohort 2
6.6.3
Cohort 3
6.6.4
Cohort 4
AI Ethics
6.7
Meeting Videos
6.7.1
Cohort 2
7
A model workflow
7.1
Workflows
7.2
Demonstration
7.2.1
Some data exploration and cleaning
7.3
Modeling with workflows
7.3.1
Different model, same recipe
7.3.2
Same model, different preprocessing
7.4
Managing many workflows
7.5
Notes
7.6
Meeting Videos
7.6.1
Cohort 1
7.6.2
Cohort 2
7.6.3
Cohort 3
7.6.4
Cohort 4
8
Feature engineering with recipes
8.1
In summary
8.1.1
Intoduction
8.2
A SIMPLE RECIPE FOR THE AMES HOUSING DATA
8.3
USING RECIPES
8.4
HOW DATA ARE USED BY THE RECIPE
8.4.1
EXAMPLES OF RECIPE STEPS:
8.5
SKIPPING STEPS FOR NEW DATA
8.6
TIDY A RECIPE
8.7
COLUMN ROLES
8.8
Resources
8.9
Meeting Videos
8.9.1
Cohort 1
8.9.2
Cohort 2
8.9.3
Cohort 3
8.9.4
Cohort 4
9
Judging model effectiveness
9.1
Performance Metrics and Inference
9.2
A little Recap of previous chapters
9.2.1
Case Study 1
9.2.2
Tidymodels: modeling as a step by step mode
9.2.3
Workflow: to combine models and recipes
9.2.4
Case Study 2
9.3
Functions used to measure predictive strengths of a model
9.3.1
Case Study 3
9.3.2
Conclusion
9.4
Measures of Model Fit - Case Study (Cohort 1)
9.5
Disclaimers
9.6
Regression Metrics
9.7
Binary Classification Metrics
9.8
References
9.9
Meeting Videos
9.9.1
Cohort 1
9.9.2
Cohort 2
9.9.3
Cohort 3
9.9.4
Cohort 4
Review of chapters 4-9
9.10
Meeting Videos
9.10.1
Cohort 1
Tools for Creating Effective Models
10
Resampling for evaluating performance
10.1
Why?
10.2
Resubstitution approach
10.3
Resampling methods
10.3.1
Cross-validation
10.3.2
Validation sets
10.3.3
Boostrapping
10.3.4
Rolling forecasting origin resampling
10.4
Estimating performance
10.5
Parallel processing
10.6
Saving the resampled objects
10.7
Meeting Videos
10.7.1
Cohort 1
10.7.2
Cohort 2
10.7.3
Cohort 3
10.7.4
Cohort 4
11
Comparing models with resampling
11.1
Calculate performance statistics
11.2
Calculate performance statistics: {workflowsets}
11.3
Within-resample correlation
11.4
Practical effect size
11.5
Simple Comparison
11.6
Bayesian methods
11.7
Meeting Videos
11.7.1
Cohort 1
11.7.2
Cohort 2
11.7.3
Cohort 3
11.7.4
Cohort 4
12
Model tuning and the dangers of overfitting
12.1
What is a Tuning Parameter?
12.1.1
Examples
12.2
When not to tune
12.3
Decisions, Decisions…
12.4
What Metric Should We Use?
12.5
Can we make our model
too
good?
12.6
Tuning Parameter Optimization Strategies
12.7
Tuning Parameters in tidymodels
{dials}
12.8
Let’s try an example:
12.9
Build our random forest model:
12.10
Add tuning parameters:
12.11
Updating tuning parameters:
12.12
Finalizing tuning parameters:
12.13
What is next?
12.14
Meeting Videos
12.14.1
Cohort 1
12.14.2
Cohort 2
12.14.3
Cohort 3
12.14.4
Cohort 4
13
Grid search
13.1
Regular and non-regular grids
13.1.1
Regular Grids
13.1.2
Irregular Grids
13.2
Evaluating the grid
13.3
Finalizing the model
13.4
Tools for efficient grid search
13.4.1
Submodel optimization
13.4.2
Parallel processing
13.4.3
Benchmarking Parallel with boosted trees
13.4.4
Racing Methods
13.5
Chapter Summary
13.6
Meeting Videos
13.6.1
Cohort 1
13.6.2
Cohort 2
13.6.3
Cohort 3
13.6.4
Cohort 4
14
Iterative search
14.1
SVM model as motivating example
14.2
Bayesian Optimization
14.2.1
Gaussian process model, at a high level
14.3
Simulated annealing
14.3.1
How it works
14.3.2
The tune_sim_anneal() function
14.4
References
14.5
Meeting Videos
14.5.1
Cohort 1
14.5.2
Cohort 2
14.5.3
Cohort 3
14.5.4
Cohort 4
15
Screening Many Models
15.1
Obligatory Setup
15.2
Creating
workflow_set
s
15.3
Ranking models
15.4
Finalizing the model
15.5
Meeting Videos
15.5.1
Cohort 1
15.5.2
Cohort 2
15.5.3
Cohort 3
15.5.4
Cohort 4
Review of chapters 10-15
15.6
Meeting Videos
15.6.1
Cohort 1
16
Dimensionality reduction
16.1
{recipes} without {workflows}
16.2
Why do dimensionality reduction?
16.3
Introducing the beans dataset
16.4
Prepare the beans data using recipes
16.5
Principal Component Analysis (PCA)
16.6
Partial Least Squares (PLS)
16.7
Independent Component Anysis (ICA)
16.8
Uniform Manifold Approximation and Projection (UMAP)
16.9
Modeling
16.10
Meeting Videos
16.10.1
Cohort 1
16.10.2
Cohort 3
16.10.3
Cohort 4
Other Topics
17
Encoding categorical data
17.1
Effect of Encoding
17.2
Encoding methods:
17.2.1
Cohort 4
18
Explaining models and predictions
18.1
Chapter 18 Setup
18.2
Overview
18.3
Local Explanations
18.4
Local Explanations for Interactions
18.5
Global Explanations
18.6
Global Explanations from Local Explanations
18.7
References
18.8
Meeting Videos
18.8.1
Cohort 1
18.8.2
Cohort 3
18.8.3
Cohort 4
19
When should you trust predictions?
19.1
Equivocal Results
19.2
Model Applicability
19.3
Meeting Videos
19.3.1
Cohort 1
19.3.2
Cohort 3
19.3.3
Cohort 4
20
Ensembles of models
20.1
Ensembling
20.2
Ensembling with
stacks
!
20.3
Define some models
20.4
Initialize and add members to stack.
20.5
Blend, fit, predict
20.6
Case Study - Patient Risk Profiles
20.6.1
Loading necessary libraries
20.6.2
Loading data
20.6.3
Have a look at the data
20.6.4
Data wrangling
20.6.5
Data to use in the Model
20.6.6
Spending Data
20.6.7
Preprocessing and Recipes
20.6.8
Make many models
20.6.9
Build the Workflow
20.6.10
Tuning: Grid and Race
20.6.11
Stacking (Ensembles of models)
20.7
Meeting Videos
20.7.1
Cohort 1
20.7.2
Cohort 3
20.7.3
Cohort 4
21
Inferential analysis
21.1
Dataset used for demonstrating inference
21.2
Tidy method from the {broom} package
21.3
{infer} for simple, high level hypothesis testing
21.3.1
p value for idependence based on simulation with permutation
21.3.2
Confidence interval for correlation based on simulation with bootstrapping
21.3.3
Use theory instead of simulation
21.3.4
Linear models with multiple explanatory variables
21.4
reg_intervals from {rsample}
21.5
Inference with lower level helpers
21.6
Meeting Videos
21.6.1
Cohort 3
21.6.2
Cohort 4
Published with bookdown
Tidy Modeling with R Book Club
11.4
Practical effect size
It’s a good idea to think about how big of a difference matters to you.
Maybe a change will be statistically significant, but is it worth the trouble of deploying a new model?