Understand The Relations of Multiple Tensorflow Sessions: A Beginner Guide

By | April 18, 2020
0 Comment

You may use multiple sessions in a tensorflow application, what is their relationship? In this tutorial, we will discuss this topic for tensorflow beginners.

The Relations of Multiple Tensorflow Sessions

Each session is independent in tensorflow

We can create some sessions using tf.Session(), however, each session object is independent. One session can not affect others, each session manage its memory space.

We have known session can run a tensorflow graph. A graph can be run in different sessions.

Here is an example:

import tensorflow as tf
import numpy as np

graph = tf.Graph()
with graph.as_default():
    w1 = tf.Variable(np.array([1,2], dtype = np.float32))
    w2 = tf.Variable(np.array([2,2], dtype = np.float32))
  
    w = tf.multiply(w1, w2)
  
    initialize = tf.global_variables_initializer()
    
with tf.Session(graph=graph) as sess_1:
    sess_1.run(initialize)
    print(sess_1.run([w]))
    
with tf.Session(graph=graph) as sess_2:
    print(sess_2.run([w]))# erro occur
    sess_2.run(initialize)
    print(sess_2.run([w]))

We have created two sessions sess_1 and sess_2 to run a same graph. sess_1 and sess_2 are independent.

Initialize operation assigns some memory space to tensorflow variables and assigns initialized values for them.

Run this code, you will find this error:

Attempting to use uninitialized value Variable_1

Why does this error occur?

In sess_1, we have initialized variables in graph. However, in sess_2, we also need to initialize variables in graph. sess_1 and sess_2 are independent. Operations in sess_1 can net affect sess_2.

Look at this example:

with tf.Session(graph=graph) as sess_1:
    sess_1.run(initialize)
    print(sess_1.run([w]))
    w = tf.multiply(w, w2)
    print(sess_1.run([w]))
    
with tf.Session(graph=graph) as sess_2:
    sess_2.run(initialize)
    print(sess_2.run([w]))

Run this python code, you will get the result like this:

[array([2., 4.], dtype=float32)]
[array([4., 8.], dtype=float32)]
[array([4., 8.], dtype=float32)]

We have changed the variable w in sess_1, however, w is also changed in sess_2. Why?

sess_1 and sess_2 are independent, it only means their action on calling operations of graph are independent. For example, sess_1 has called initialize operation of graph to initialize varaibles, it allocates memory space in its environment, however, these memory spaces are not access to sess_2, it have to spare memory space to initialize varaibles. However, they share the same structure of graph. If sess_1 change the structure of graph, sess_2 will also can use new structure of graph.

So variable w in sess_1 is:

[array([4., 8.], dtype=float32)]

w in sess_2 is also:

[array([4., 8.], dtype=float32)]

Leave a Reply