An Explain to sess.run(tf.global_variables_initializer()) for Beginners – TensorFlow Tutorial

sess.run(tf.global_variables_initializer()) is often used when we are using tensorflow. In this tutorial, we will introduce some tips on using it.

Why we use sess.run(tf.global_variables_initializer())?

Look at an example.

import numpy as np
import tensorflow as tf
x = tf.Variable(tf.orthogonal_initializer()([3, 3]), name="x")
y = tf.Variable(tf.orthogonal_initializer()([3, 3]), name="y")
z = x + y

init = tf.global_variables_initializer()
with tf.Session() as sess:
    #sess.run([init])
    np.set_printoptions(precision=4, suppress=True)
    sum = sess.run(z)
    print(sum)

Run this code, you will get an error: Attempting to use uninitialized value x

In tensorflow 1.x, in order to use a variable, there are two steps:

1.Create a variable

We can use tf.Variable() or tf.get_variable() to create a variable, however, there is no value in this variable.

2.Assign initialized value to variable

We can use tf.global_variables_initializer() to initialize global variables, or use tf.local_variables_initializer() to initialize local variables.

tf.global_variables_initializer()

tf.global_variables_initializer() is defined as:

@tf_export(v1=["initializers.global_variables", "global_variables_initializer"])
def global_variables_initializer():
  """Returns an Op that initializes global variables.

  This is just a shortcut for `variables_initializer(global_variables())`

  Returns:
    An Op that initializes global variables in the graph.
  """
  if context.executing_eagerly():
    return control_flow_ops.no_op(name="global_variables_initializer")
  return variables_initializer(global_variables())

It will call variables_initializer() function to initialize global variables.

variables_initializer() is defined as:

@tf_export(v1=["initializers.variables", "variables_initializer"])
def variables_initializer(var_list, name="init"):
  """Returns an Op that initializes a list of variables.

  After you launch the graph in a session, you can run the returned Op to
  initialize all the variables in `var_list`. This Op runs all the
  initializers of the variables in `var_list` in parallel.

  Calling `initialize_variables()` is equivalent to passing the list of
  initializers to `Group()`.

  If `var_list` is empty, however, the function still returns an Op that can
  be run. That Op just has no effect.

  Args:
    var_list: List of `Variable` objects to initialize.
    name: Optional name for the returned operation.

  Returns:
    An Op that run the initializers of all the specified variables.
  """
  if var_list and not context.executing_eagerly():
    return control_flow_ops.group(*[v.initializer for v in var_list], name=name)
  return control_flow_ops.no_op(name=name)

We should notice:

global_variables() will return a variable list, which contains all gloabal variables.

To summarize, we use sess.run(tf.global_variables_initializer()) in our code, it will initialize all global variables. However, it may cause some errors if we load a pre-trained model for fine-tuning.