Understand Gated End-to-End Memory Networks – Deep Learning Tutorial

Gated End-to-End Memory Networks is proposed by Fei Liu in 2017 (https://www.aclweb.org/anthology/E17-1001), which is an improvement of End-to-End Memory Networks.

Standard End-to-End Memory Networks like this:

The output u^k+1 is:

As this equation, here is an problem: u^k and o^k contribute equally to u^k+1?

If not, how to evaluate this kind of different contribution?

In this paper, author use an sigmoid function to create a gate to control this weight, which likes in LSTM.

The equation is:

We can see, the key idea of this paper is to use u^k to create a sigmoid function to control the different weight between o^k and u^k.

Here is a problem, why not use u^k and o^k to create a sigmoid function like:

I think it is feasible.

On the other hand, this paper processed two kind of W and b.

The experiment results show Hop-specific is better than Global, which is easy understand and obvious. Because Global W and b will be disturbed easily.