Implement TensorFlow CNN Networks for MNIST Handwritten Digits Classification – TensorFlow Tutorial

By | August 19, 2020

CNN networks are widely used in deep learning, in this tutorial, we will build a cnn network for mnist handwritten digits classification. It will help you understand how to use cnn in deep learning.

The structure of CNN network

The basic structure fo a CNN network looks like:

the structure of convolutional networks

We will use this structure to build a CNN network for mnist handwritten digits classification.

Load MNIST data

We should import MNIST data first.

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
import os
import numpy as np
import random

mnist = input_data.read_data_sets(os.getcwd() + "/MNIST-data/", one_hot=True)

Then, we can prepare data that can be used by a cnn network.

Prepare data

x = tf.placeholder(tf.float32, [None, 784])
y_label = tf.placeholder(tf.float32, [None, 10])
X_images = tf.reshape(x, [-1, 28, 28, 1])

This code will make X_images can be passed into a cnn network.

Create some hyperparameters

These hyperparameters can be used in our example.

learning_rate = 1e-3

total_steps = 1000
category_num = 10
steps_per_validate = 15
steps_per_test = 15
batch_size = 64

Then we can start to build a cnn network.

We will use tensorflow tf.nn.conv2d() and tf.nn.max_pool(). You can learn how to use them in these two tutorials.

Understand tf.nn.conv2d(): Compute a 2-D Convolution in TensorFlow

Understand TensorFlow tf.nn.max_pool(): Implement Max Pooling for Convolutional Network

How to implement Convolution+ReLU

Convolution+ReLU is the basic operation of a cnn notwork, we can use tf.nn.conv2d() and tf.nn.relu() to implement it. Here is an example:

conv1_Weights = tf.Variable(tf.truncated_normal([5, 5, 1, 32], stddev=0.1), name='conv1_Weights')
# out_channels = 32
conv1_biases = tf.Variable(tf.constant(0.1, shape=[32]), name='conv1_biases')
#[batch, out_height, out_width, out_channels]
#out_channels = 32
conv1_conv2d = tf.nn.conv2d(X_images, conv1_Weights, strides=[1, 1, 1, 1], padding='SAME') + conv1_biases
conv1_activated = tf.nn.relu(conv1_conv2d)

How to implement pooling

In this tutorial, we will use max_pooling operation, here is an example:

# channels = 32
conv1_pooled = tf.nn.max_pool(conv1_activated, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

Then we can repeat Convolution+ReLU+pooling operations

conv2_Weights = tf.Variable(tf.truncated_normal([5, 5, 32, 64], stddev=0.1), name='conv2_Weights')
conv2_biases = tf.Variable(tf.constant(0.1, shape=[64]), name='conv2_biases')
conv2_conv2d = tf.nn.conv2d(conv1_pooled, conv2_Weights, strides=[1, 1, 1, 1], padding='SAME') + conv2_biases
conv2_activated = tf.nn.relu(conv2_conv2d)
#[batch, out_height, out_width, channels]
# channels = 64
conv2_pooled = tf.nn.max_pool(conv2_activated, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

How to implement Fully Connected operation

A fully connected layer can be created as following:

dim_w1= 7 * 7 * 64
connect1_flat = tf.reshape(conv2_pooled, [-1, dim_w1])
connect1_Weights = tf.Variable(tf.truncated_normal([dim_w1, 1024], stddev=0.1), name='connect1_Weights')
connect1_biases = tf.Variable(tf.constant(0.1, shape=[1024]), name='connect1_biases')
connect1_Wx_plus_b = tf.add(tf.matmul(connect1_flat, connect1_Weights), connect1_biases)
connect1_activated = tf.nn.relu(connect1_Wx_plus_b)

Then we can get final output to classify.

#full connected layer 2
connect2_Weights = tf.Variable(tf.truncated_normal([1024, 10], stddev=0.1), name='connect2_Weights')
connect2_biases = tf.Variable(tf.constant(0.1, shape=[10]), name='connect2_biases')
y = tf.add(tf.matmul(connect1_activated, connect2_Weights), connect2_biases)

Make predication

# Loss
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_label, logits=y)
train = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cross_entropy)

# Prediction
correction_prediction = tf.equal(tf.argmax(y, axis=1), tf.argmax(y_label, axis=1))
accuracy = tf.reduce_mean(tf.cast(correction_prediction, tf.float32))

Finally, we can start to train this model

Start to train cnn network

init = tf.global_variables_initializer()
try:
    
    with tf.Session() as sess:
        sess.run(init)
        
        test_acc = 0.
        dev_acc = 0.
        better_acc = 0.0
        
        #set train times
        for step in range(total_steps + 1):
            batch_x, batch_y = mnist.train.next_batch(batch_size)
            _, acc = sess.run([train, accuracy] , feed_dict={x: batch_x, y_label: batch_y})
            print("train step="+str(step) +" accuracy = " + str(acc))
           
            if step % steps_per_validate == 0:
                
                dev_x, dev_y = mnist.validation.images, mnist.validation.labels
                
                
                dev_acc = sess.run(accuracy,feed_dict = {x: dev_x, y_label: dev_y})
                
                print("dev step="+str(step) +" accuracy = " + str(dev_acc))
                
                if better_acc < dev_acc:
                    test_x, test_y = mnist.test.images, mnist.test.labels
                    test_acc = sess.run(accuracy,feed_dict = {x: test_x, y_label: test_y})
                    print("test step="+str(step) +" accuracy = " + str(test_acc))
                    better_acc = dev_acc
        
        #final
        dev_x, dev_y = mnist.validation.images, mnist.validation.labels
        dev_acc = sess.run(accuracy,feed_dict = {x: dev_x, y_label: dev_y})
except Exception as e:
    print(e)

Run this code, you will get the training result.

Implement TensorFlow CNN Networks for MNIST Handwritten Digits Classification - TensorFlow Tutorial

Leave a Reply

Your email address will not be published. Required fields are marked *