Tesla

Subscribe
SPEED UP TRAINING WITH GPU-ACCELERATED TENSORFLOW
The fastest, easiest way to get started with deep learning on GPUs
Speed Up Training with GPU-Accelerated TensorFlow

Training Models

TensorFlow can be used via Python or C++ APIs, while its core functionality is provided by a C++ backend. The API provides an interface for manipulating tensors (N-dimensional arrays) similar to Numpy, and includes automatic differentiation capabilities for computing gradients for use in optimization routines.

The library comes with a large number of built-in operations, including matrix multiplications, convolutions, pooling and activation functions, loss functions, optimizers and many more. Once a graph of computations has been defined, TensorFlow enables it to be executed efficiently and portably on desktop, server, and mobile platforms.

To run the example codes below, first change to your TensorFlow directory 1:

$ cd (tensorflow directory)
$ git clone -b update-models-1.0 https://github.com/tensorflow/models

Image recognition is one of the tasks that Deep Learning excels in. While human brains make this task of recognising images seem easy, it is a challenging task for the computer. However, there have been significant advancements over the past few years to the extent of surpassing human abilities. What makes this possible is the convolutional neural network (CNN) and ongoing research has demonstrated steady advancements in computer vision, validated against ImageNet - an academic benchmark for computer vision.

Quick Demo using Inception-v3

First, let’s run the following commands and see what computer vision can do:

$ cd (tensorflow directory)/models/tutorials/image/imagenet
$ python classify_image.py

classify_image.py downloads the trained Inception-v3 model from tensorflow.org when the program is run for the first time. You'll need about 200M of free space available on your hard disk. The above command will classify a supplied image of a panda bear (found in /tmp/imagenet/cropped_panda.jpg) and a successful execution of the model will return results that look like:

giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca (score = 0.89107)
indri, indris, Indri indri, Indri brevicaudatus (score = 0.00779)
lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens (score = 0.00296)
custard apple (score = 0.00147)
earthstar (score = 0.00117)

You may also test other JPEG images by using the --image_file file argument:

$ python classify_image.py --image_file <path to the JPEG file>
(e.g. python classify_image.py --image_file /tmp/imagenet/cropped_pand.jpg)

CIFAR-10

CIFAR-10 classification is a common benchmark task in machine learning. The task is to classify RGB 32x32 pixel images across 10 categories (airplane, automobile, bird, cat, deer, dog, frog, horse, ship and truck).

The model used references the architecture described by Alex Krizhevsky, with a few differences in the top few layers. It is a multi-layer architecture consisting of alternating convolutions and nonlinearities, followed by fully connected layers leading into a softmax classifier.

First, let’s train the model:

$ cd (tensorflow directory)/models/tutorials/image/cifar10
$ python cifar10_train.py

If successful, you will see something similar as follow:

Filling queue with 20000 CIFAR images before starting to train. This will take a few minutes.
……
2017-03-06 14:59:09.089282: step 10230, loss = 2.12 (1809.1 examples/sec; 0.071 sec/batch)
2017-03-06 14:59:09.760439: step 10240, loss = 2.12 (1902.4 examples/sec; 0.067 sec/batch)
2017-03-06 14:59:10.417867: step 10250, loss = 2.02 (1931.8 examples/sec; 0.066 sec/batch)
2017-03-06 14:59:11.097919: step 10260, loss = 2.04 (1900.3 examples/sec; 0.067 sec/batch)
2017-03-06 14:59:11.754801: step 10270, loss = 2.05 (1919.6 examples/sec; 0.067 sec/batch)
2017-03-06 14:59:12.416152: step 10280, loss = 2.08 (1942.0 examples/sec; 0.066 sec/batch)
……

Congratulations, you have just started training your first model!

Following the training, you can evaluate how well the trained model performs by using the cifar10_eval.py script. It calculates the precision at 1: how often the top prediction matches the true label of the image.

$ python cifar10_eval.py

If successful, you will see something similar as follow:

2017-03-06 15:34:27.604924: precision @ 1 = 0.499

Using a pre-trained Inception v3 on new dataset

Next, let’s revisit Google’s Inception v3 and get more involved with a deeper use case. Inception v3 is a cutting-edge convolutional network designed for image classification. Training this model from scratch is very intensive and can take from several days up to weeks of training time. An alternative approach is to download the pre-trained model, and re-train it on another dataset. We will walkthrough how this is done using the flowers dataset.

 

Download the flowers dataset:

$ cd (tensorflow directory)/models/inception
$ export FLOWERS_DATA_DIR=$HOME/flowers-data
$ bazel build inception/download_and_preprocess_flowers
$ bazel-bin/inception/download_and_preprocess_flowers "${FLOWERS_DATA_DIR}"

If successful, you will see a message similar to “Finished writing all 3170 images in data set.”.

Download the pre-trained Inception v3 model, which includes the checkpoint file:

$ DATA_DIR=$HOME/inception-v3-model
$ mkdir ${DATA_DIR}
$ cd ${DATA_DIR}
$ curl -O http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz
$ tar xzf inception-v3-2016-03-01.tar.gz

Retrain a pre-trained Inception v3 model on the flowers dataset:

$ cd (tensorflow directory)/models/inception
$ bazel build inception/flowers_train
$ MODEL_PATH=”${DATA_DIR}/inception-v3/model.ckpt-157585”
$ TRAIN_DIR=$HOME/flowers_train
$ bazel-bin/inception/flowers_train \
--train_dir="${TRAIN_DIR}" \
--data_dir="${FLOWERS_DATA_DIR}" \
--pretrained_model_checkpoint_path="${MODEL_PATH}" \
--fine_tune=True \
--initial_learning_rate=0.001 \
--input_queue_memory_factor=1

step 450, loss = 1.24 (50.4 examples/sec; 0.634 sec/batch)
step 460, loss = 1.17 (50.4 examples/sec; 0.635 sec/batch)
step 470, loss = 1.10 (48.5 examples/sec; 0.660 sec/batch)
step 480, loss = 1.14 (48.6 examples/sec; 0.658 sec/batch)
step 490, loss = 1.17 (50.4 examples/sec; 0.635 sec/batch)

The re-trained model can now be evaluated on the validation dataset:

$ bazel build inception/flowers_eval
$ mkdir -p $HOME/flowers_eval
$ EVAL_DIR=$HOME/flowers_eval
$ bazel-bin/inception/flowers_eval \
--eval_dir="${EVAL_DIR}" \
--data_dir="${FLOWERS_DATA_DIR}" \
--subset=validation \
--num_examples=500 \
--checkpoint_dir="${TRAIN_DIR}" \
--input_queue_memory_factor=1 \
--run_once

starting evaluation on (validation).
Successfully loaded model from $HOME/flowers_train/model.ckpt-1999 at step=1999.
precision @ 1 = 0.9551 recall @ 5 = 0.9980 [512 examples]

The evaluation script will return something as above, providing you with the classification accuracy. For more details on using the Inception v3 model, see the README document.

1 Typically /usr/local/python2.7/dist-packages

SEE HOW THE PERFORMANCE STACKS UP