In convolution networks, there are “SAME” and “VALID” paddings. Here is the tutorial:
Understand the Difference Between ‘SAME’ and ‘VALID’ Padding in Convolution Networks
However, there also exists causal padding. In this tutorial, we will introduce what it is.
What is causal padding?
Causal padding looks like:
We can find: the convolutional output at time \(t\) only depends on the previous time steps (less than \(t\)).
For example:
As to the yellow output 5. It depends on its previous blue time steps 1, 2, 3, 4
This is causal padding.
We also can find: as to the first yellow output 2, we have to padding some empty circles.
SAME or VALID padding vs causal padding
Comparing the traditional convolutional computation, we can find the difference between them.
The output with SAME or VALID padding does not only depend on its previous time steps, but also its next time steps.