Understand tf.name_scope(): Manage TensorFlow Variables

TensorFlow tf.name_scope() is a good way to manage variables, especially when you are creating variables with same name. In this tutorial, we will introduce some basic knowlege on it for tensorflow beginners.

Syntax

tf.name_scope() is a tensorflow class, which can create a context to manage tensorflow variables. The initialized method is:

__init__(
    name,
    default_name=None,
    values=None
)

where name is the name of context manager, tensorflow will use this name to manage variables.

How to use tf.name_scope()

The basic way to use tf.name_scope() is:

with tf.name_scope(name):
    #create tensorflow variables

We will use some examples to show you how to use it.

Create variables with tf.name_scope()

import tensorflow as tf
import numpy as np

with tf.name_scope("weights"):
    w1 = tf.Variable(np.array([[1, 2, 3, 4],[5, 6, 7, 8]]), dtype = tf.float32, name = 'w1')
    w2 = tf.Variable(np.array([[8, 7, 6, 5],[4, 3, 2, 1]]), dtype = tf.float32, name = 'w2')

In code above, we have created two tensorflow variables with name ‘w1‘ and ‘w2‘ in the name scope ‘weights‘.

Output variable name

print(w1.name)
print(w2.name)

The names are:

weights/w1:0
weights/w2:0

From result we find: The way of tf.name_scope() to manage tensorflow variables is to add a scope name before the name of variables.

For example, we have created a variable w1 with name ‘w1‘ in name scope ‘weights‘. The final name of variable w1 is weights/w1:0. If we have not created w1 in in name scope ‘weights‘, the name of w1 will be w1:0.

Create two name scopes with the same name

Look at example below:

with tf.name_scope("weights"):
    w1 = tf.Variable(np.array([[1, 2, 3, 4],[5, 6, 7, 8]]), dtype = tf.float32, name = 'w1')
    w2 = tf.Variable(np.array([[8, 7, 6, 5],[4, 3, 2, 1]]), dtype = tf.float32, name = 'w2')

with tf.name_scope("weights"):
    w3 = tf.Variable(np.array([[1, 2, 3, 4],[5, 6, 7, 8]]), dtype = tf.float32, name = 'w3')
    w4 = tf.Variable(np.array([[8, 7, 6, 5],[4, 3, 2, 1]]), dtype = tf.float32, name = 'w4')

In this example, we have created two name scopes with the same name ‘weights‘. Do you think the variable w3 and w4 are in the same name scope with w1 or w2, which means the name w3 and w4 are:

weights/w3:0
weights/w4:0

Run this code and output the name of w3 and w4.

print(w3.name)
print(w4.name)

The name of w3 and w4 are:

weights_1/w3:0
weights_1/w4:0

From the result we can find: If we create two name scopes with the same name, these two name scopes are different.

How about tf.get_variable() in tf.name_scope()

We have known tf.get_variable() can create or get a tensorflow variable, can tf.name_scope() affect the name of variables got or created by tf.get_variable()? The answer is not.

Look at example below:

with tf.name_scope("weights"):
    w1 = tf.Variable(np.array([[1, 2, 3, 4],[5, 6, 7, 8]]), dtype = tf.float32, name = 'w1')
    w2 = tf.Variable(np.array([[8, 7, 6, 5],[4, 3, 2, 1]]), dtype = tf.float32, name = 'w2')
    w5 = tf.get_variable(name = 'w5', initializer = tf.random_uniform([10, 10], -0.01, 0.01))

In this example, we have created 3 tensorflow variables in name scope ‘weights‘. We have known the name of w1 and w2 are:

weights/w1:0
weights/w2:0

Is the name of w5 is: weights/w5:0?

We output the name of w5.

print(w5.name)

The name of it is: w5:0, which does not start with name scope name ‘weights‘. This fact means tf.name_scope can not affect the name of tensorflow variable created by tf.get_shape().

Understand tf.name_scope(): Manage TensorFlow Variables – TensorFlow Tutorial