To shield some elements in tensor, we can use a mask tesnor. In this tutorial, we will introduce tf.sequence_mask(), which can create a mask tensor.
tf.sequence_mask( lengths, maxlen=None, dtype=tf.bool, name=None )
Create a mask tensor.
lengths: integer tensor, all its values <= maxlen.
maxlen: scalar integer tensor, size of last dimension of returned tensor. Default is the maximum value in lengths
dtype: output type of the resulting tensor.
How to use tf.sequence_mask()?
We will write some examples to illustrate how to use it.
Here is an example.
import tensorflow as tf mask = tf.sequence_mask([2, 3, 1], maxlen = 4) with tf.Session() as sess: sess.run(tf.global_variables_initializer()) print(sess.run(mask))
Run this code, you will get the result:
From the result, we can find the dimension of lengths is (1, ), the dimension of mask is (lengths, maxlen)
Create a float32 mask tensor
We can set dtype for tf.sequence_mask() to create a float32 mask tensor. Here is an example:
mask = tf.sequence_mask([2, 3, 1], maxlen = 4, dtype = tf.float32)
The mask is:
[[1. 1. 0. 0.] [1. 1. 1. 0.] [1. 0. 0. 0.]]
True will be converted to 1.0 and False to 0.
How to use tf.sequence_mask() in NLP?
In nlp, the length of sentences is not fixed, in order to calculate the atttention we need to use tf.sequence_mask().
Here is an example: