Understand the Difference Between tf.name_scope() and tf.variable_scope(): A Beginner Guide – TensorFlow Tutorial

By | December 11, 2019
0 Comment

Both of tensortlow tf.name_scope() and tf.variable_scope() can manage tensorflow variables, how about the difference between them? In this tutorial, we will discuss this topic for tensorflow beginners.

The difference between tf.name_scope() and tf.variable_scope()

The difference can be summarized to:

Name Description
tf.name_scope() only affect tf.Variable() variables, can not affect tf.get_variable()
tf.variable_scope() can affect tf.Variable() and tf.get_variable(), can share tensorflow variables

We will use some examples to show their differences.

1.tf.name_scope() and tf.variable_scope() can affect tf.Variable()

We will create some tensorflow variables in tf.name_scope() and tf.variable_scope(). Here is an example:

import tensorflow as tf
import numpy as np

with tf.name_scope("name_scope"):
    w1 = tf.Variable(np.array([[1, 2, 3, 4],[5, 6, 7, 8]]), dtype = tf.float32, name = 'w1')
    w2 = tf.Variable(np.array([[8, 7, 6, 5],[4, 3, 2, 1]]), dtype = tf.float32, name = 'w2')
with tf.variable_scope("variable_scope"):
    w3 = tf.Variable(np.array([[1, 2, 3, 4],[5, 6, 7, 8]]), dtype = tf.float32, name = 'w3')
    w4 = tf.Variable(np.array([[8, 7, 6, 5],[4, 3, 2, 1]]), dtype = tf.float32, name = 'w4')

Print the name of w1, w2, w3 and w4.

print(w1.name)
print(w2.name)
print(w3.name)
print(w4.name)

The result is:

name_scope/w1:0
name_scope/w2:0
variable_scope/w3:0
variable_scope/w4:0

From the result we can find: tf.name_scope() and tf.variable_scope() can affect tensorflow variables created by tf.Variable()

2.tf.name_scope() can not affect tf.get_variable(), however, tf.variable_scope() can affect

We will use tf.get_variable() to create some tensorflow variables in tf.name_scope() and tf.variable_scope(). Here is an example:

with tf.name_scope("name_scope"):
    w1 = tf.Variable(np.array([[1, 2, 3, 4],[5, 6, 7, 8]]), dtype = tf.float32, name = 'w1')
    w2 = tf.Variable(np.array([[8, 7, 6, 5],[4, 3, 2, 1]]), dtype = tf.float32, name = 'w2')
    w5 = tf.get_variable(name = 'w5', initializer = tf.random_uniform([10, 10], -0.01, 0.01))
with tf.variable_scope("variable_scope"):
    w3 = tf.Variable(np.array([[1, 2, 3, 4],[5, 6, 7, 8]]), dtype = tf.float32, name = 'w3')
    w4 = tf.Variable(np.array([[8, 7, 6, 5],[4, 3, 2, 1]]), dtype = tf.float32, name = 'w4')
    w6 = tf.get_variable(name = 'w6', initializer = tf.random_uniform([10, 10], -0.01, 0.01))

How about the name of six variables?

The name of these variables are:

name_scope/w1:0
name_scope/w2:0
w5:0
variable_scope/w3:0
variable_scope/w4:0
variable_scope/w6:0

From result we can find: the name of w5 variable in tf.name_scope() is w5:0, which means tf.name_scope() can not affect variables create by tf.get_variable(). The name of w6 variable tf.variable_scope() is variable_scope/w6:0, which means tf.variable_scope() can affect tf.get_variable().

3.tf.variable_scope() can share variables with tf.get_variable()

We can use tf.variable_scope() to share tensorflow variables, here is an example:

def getWeight():  
    with tf.variable_scope("variable_scope", reuse = tf.AUTO_REUSE):
        w = tf.get_variable(name = 'w', initializer = tf.random_uniform([10, 10], -0.01, 0.01))  
        return w

w1 = getWeight()
w2 = getWeight()

We will find w1 and w2 are the same, the name of them are:

variable_scope/w:0
variable_scope/w:0

To know more about how to share variables with tf.variable_scope(), you can read these tutorials.

Understand tf.variable_scope(): A Beginner Guide

Understand tf.variable_scope() Reuse Mode for Beginners

Leave a Reply