Linear-Chain CRF Layer in TensorFlow: A Beginner Guide

Linear-Chain CRF, which is also called linear-chain conditional random field algorithm, is widely used for Named Entity Recognition (NER). In this tutorial, we will introduce some basic knowlege on it in tensorflow.

Linear-chain crf in tensorflow

In order to implement linear-chain crf in tensorflow, there are some methods. They are:

• tf.contrib.crf.crf_sequence_score
• tf.contrib.crf.crf_log_norm
• tf.contrib.crf.crf_log_likelihood
• tf.contrib.crf.crf_unary_score
• tf.contrib.crf.crf_binary_score
• tf.contrib.crf.CrfForwardRnnCell
• tf.contrib.crf.viterbi_decode

Generally, we will use tf.contrib.crf.crf_log_likelihood() to compute a loss function to train. Here is an example code:

log_likelihood, transition_params = tf.contrib.crf.crf_log_likelihood(
    unary_scores, gold_tags, sequence_lengths)
loss = tf.reduce_mean(-log_likelihood)
train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss)

How to compute the loss in tf.contrib.crf.crf_log_likelihood()?

View the source code of tf.contrib.crf.crf_log_likelihood(), we can find:

  if transition_params is None:
    transition_params = vs.get_variable("transitions", [num_tags, num_tags])

  sequence_scores = crf_sequence_score(inputs, tag_indices, sequence_lengths,
                                       transition_params)
  log_norm = crf_log_norm(inputs, sequence_lengths, transition_params)

  # Normalize the scores to get the log-likelihood per example.
  log_likelihood = sequence_scores - log_norm
  return log_likelihood, transition_params

The log_likelihood is computed by sequence_scores and log_norm. Here is the structure among these functions.