51. Tokenization Beyond Text

Learning objectives

• Talk about tokenization beyond text

• You can add notes on each slide with blocks like this!
• Load a deck in the browser and type “s” to see these notes.

Tokenization Beyond Text

Text

tokenization of text

• image source: Murilo Gustineli

Images

Vectorization

tokenization of images

• images are partitioned into patches

• each patch is flattened into a vector

• image source: Shusen Wang

Positional Encoding

positional encoding of patches

• image source: Shusen Wang

Audio

Abstraction

audio abstraction

• image source: Valerio Velardo

Music

• beat

• timbre

• pitch

• harmony

• …

• source: Valerio Velardo

Fourier

signal domain

• image source: Valerio Velardo

Trends

• digital signal processing \(\rightarrow\) rule-based systems

• traditional ML \(\rightarrow\) feature engineering

• deep learning \(\rightarrow\) automatic feature engineering

• source: Valerio Velardo