Compute Model Trainable Variables Count or Memory Size in TensorFlow – TensorFlow Tutorial

By | March 11, 2022

As to a tensorflow model, it is easy to get all trainable or untrainable variables. Here is the tutorial:

List All Trainable and Untrainable Variables in TensorFlow – TensorFlow Tutorial

However, we can not know the total variable count in a trainable variable. For example, as to 4*50 weight variable, the variable is 1, however, it contains 200 trainable variables.

In this tutorial, we will introduce you how to compute model trainable variable count and estmate memory used in a tensorflow model.

Compute Model Trainable Variables Count or Memory Size in TensorFlow - TensorFlow Tutorial

For example:

import tensorflow as tf
import numpy as np

i = np.array([1,2, 20, 10, 100])
i = tf.convert_to_tensor(i, dtype = tf.float32)

w = tf.get_variable('weight', shape=[5, 200], dtype = tf.float32)

trainable_vars = tf.trainable_variables()
print("trainable vars count:", len(trainable_vars))
# check variables memory
variable_count = np.sum(np.array([np.prod(np.array(v.get_shape().as_list())) for v in trainable_vars]))
print("total variables :", variable_count)

Run this code, we will see:

trainable vars count: 1
total variables : 1000

In this code, we only create a trainable variable w, it contains 5*200 = 1000 variables.

Leave a Reply

Your email address will not be published. Required fields are marked *