Understand PyTorch inplace Parameter with Examples – PyTorch Tutorial

Parameter inplace appears in many pytorch functions. In this tutorial, we will use some example to help you understand it.

Parameter inplace

We can find it in these functions:

torch.nn.ReLU6(inplace=False)
torch.nn.ReLU(inplace=False)
torch.nn.CELU(alpha=1.0, inplace=False)

It will change the value of inputs.

For example:

import torch
import torch.nn as nn

m = nn.ReLU6(inplace=False)
input = torch.randn(6)
output = m(input)

print("input=",input)
print("output=",output)

Here inplace=False

We can find input is not changed. input and output are different.

input= tensor([-0.6441,  0.2386,  0.0593, -0.8481,  0.3325,  0.9892])
output= tensor([0.0000, 0.2386, 0.0593, 0.0000, 0.3325, 0.9892])

If inplace = True?

n = nn.ReLU6(inplace=True)
output = n(input)

print("input=",input)
print("output=",output)

Here inplace = True, which means the input is also be changed.

The input and output are the same.

input= tensor([0.0000, 0.2386, 0.0593, 0.0000, 0.3325, 0.9892])
output= tensor([0.0000, 0.2386, 0.0593, 0.0000, 0.3325, 0.9892])