3.5 Broadcasting

Example: Want to add two tensors of shape 3x7x1 and 1x5

t1 shape: 3 7 1
t2 shape:   1 5

Broadcast (from right to left):

t1 shape: 3 7 5
t2 shape:   7 5

Next, virtual expansion:

t1 shape: 3 7 5
t2 shape: 1 7 5

And, broadcast again:

t1 shape: 3 7 5
t2 shape: 3 7 5

Let’s see it in action:

t1 <- torch_ones(3, 7, 1)
t2 <- torch_zeros(1, 5)
torch_add(t1, t2)

## torch_tensor
## (1,.,.) = 
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
## 
## (2,.,.) = 
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
## 
## (3,.,.) = 
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
##   1  1  1  1  1
## [ CPUFloatType{3,7,5} ]

The following will NOT work:

t1 <- torch_ones(3, 7, 1)
t2 <- torch_zeros(6, 5)
torch_add(t1, t2)