Linear-Chain CRF, which is also called linear-chain conditional random field algorithm, is widely used for Named Entity Recognition (NER). In this tutorial, we will introduce some basic knowlege on it in tensorflow.
Linear-chain crf in tensorflow
In order to implement linear-chain crf in tensorflow, there are some methods. They are:
Generally, we will use tf.contrib.crf.crf_log_likelihood() to compute a loss function to train. Here is an example code:
log_likelihood, transition_params = tf.contrib.crf.crf_log_likelihood( unary_scores, gold_tags, sequence_lengths) loss = tf.reduce_mean(-log_likelihood) train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
How to compute the loss in tf.contrib.crf.crf_log_likelihood()?
View the source code of tf.contrib.crf.crf_log_likelihood(), we can find:
if transition_params is None: transition_params = vs.get_variable("transitions", [num_tags, num_tags]) sequence_scores = crf_sequence_score(inputs, tag_indices, sequence_lengths, transition_params) log_norm = crf_log_norm(inputs, sequence_lengths, transition_params) # Normalize the scores to get the log-likelihood per example. log_likelihood = sequence_scores - log_norm return log_likelihood, transition_params
The log_likelihood is computed by sequence_scores and log_norm. Here is the structure among these functions.