Understand PyTorch model.named_parameters() with Examples

PyTorch model.named_parameters() is often used when trainning a model. In this tutorial, we will use an example to show you what it is.

Look at example below:

import torch.nn as nn
from torch.autograd import Variable
import torch.optim as optim
class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(2, 4)
        self.fc2 = nn.Linear(4, 3)
        self.out = nn.Linear(3, 1)
        self.out_act = nn.Sigmoid()

    def forward(self, inputs):
        a1 = self.fc1(inputs)
        a2 = self.fc2(a1)
        a3 = self.out(a2)
        y = self.out_act(a3)
        return y

model = Net()

Then, we can use model.named_parameters() to print all parameters and values in this model.

params = model.named_parameters()
print(params)

Run this code, we will see:

<generator object Module.named_parameters at 0x000001ED9F6C7D58>

It means model.named_parameters() will return a generateor.

We can convert it to a python list.

params = list(model.named_parameters())
print(params)

Then, we will see:

[('fc1.weight', Parameter containing:
tensor([[-0.2501, -0.1425],
        [-0.3438,  0.7018],
        [-0.2928, -0.5517],
        [ 0.3835, -0.5121]], requires_grad=True)), ('fc1.bias', Parameter containing:
tensor([-0.3924,  0.3195,  0.0454,  0.1041], requires_grad=True)), ('fc2.weight', Parameter containing:
tensor([[-0.4710, -0.2547,  0.0455,  0.4160],
        [-0.4388, -0.1971,  0.3316, -0.1419],
        [ 0.4574,  0.4238,  0.4872, -0.4997]], requires_grad=True)), ('fc2.bias', Parameter containing:
tensor([0.2411, 0.4258, 0.4584], requires_grad=True)), ('out.weight', Parameter containing:
tensor([[-0.1198, -0.4459, -0.5505]], requires_grad=True)), ('out.bias', Parameter containing:
tensor([0.5574], requires_grad=True))]

We will get a list which contains [(name1, value1), (name2, value2), ….]

model.named_parameters() is similar to model.state_dict()

For example:

params = model.state_dict()
print(params)

We will see:

OrderedDict([('fc1.weight', tensor([[-0.2501, -0.1425],
        [-0.3438,  0.7018],
        [-0.2928, -0.5517],
        [ 0.3835, -0.5121]])), ('fc1.bias', tensor([-0.3924,  0.3195,  0.0454,  0.1041])), ('fc2.weight', tensor([[-0.4710, -0.2547,  0.0455,  0.4160],
        [-0.4388, -0.1971,  0.3316, -0.1419],
        [ 0.4574,  0.4238,  0.4872, -0.4997]])), ('fc2.bias', tensor([0.2411, 0.4258, 0.4584])), ('out.weight', tensor([[-0.1198, -0.4459, -0.5505]])), ('out.bias', tensor([0.5574]))])

Understand PyTorch model.state_dict() – PyTorch Tutorial