Understand PyTorch F.linear() with Examples – PyTorch Tutorial

In this tutorial, we will use some pytorch examples to show you how to use F.linear() function. This function is widely used in many pytorch scripts.

F.linear()

This function is:

torch.nn.functional.linear(input, weight, bias=None)

In order to use it, we should import:

import torch.nn.functional as F

This function will compute:

\(y = xA^T+b\)

Here input = x, weight = A and bias = b

input: (∗,in_features)

weight: $(out\_features,in\_features)$ or $(in\_features)$

bias: $(out\_features)$ or ()

output: (∗,out_features) or (∗)

How to use F.linear()?

Here is an example:

Create weight

import torch
import torch.nn as nn
import torch.nn.functional as F

out_features = 50
in_features = 10

weight = nn.Parameter(torch.FloatTensor(out_features, in_features))
nn.init.xavier_uniform_(weight)

Create input

batch_size = 30

input = nn.Parameter(torch.FloatTensor(batch_size, in_features))
nn.init.xavier_uniform_(input)

In order to know how to initialize a pytorch parameter, you can view:

Understand torch.nn.init.xavier_uniform_() and torch.nn.init.xavier_normal_() with Examples – PyTorch Tutorial

Finally, we can can compute \(y = xA^T+b\)

y = F.linear(input, weight)
print(y.shape)

Here bias = 0, run this code, we will get:

torch.Size([30, 50])

By using F.linear(), we can create a linear layer in pytorch. However, we have create weight and bias parameter. If you do not want to create them, we can use nn.Linear() function. Here is a tutorial:

Create a MLP with Dropout in PyTorch – PyTorch Tutorial