Learning Tensorflow

In this writing, I will present some fundamental problems and lessons that I encountered during learning Tensorflow for computer vision problems.  Most of them are from the internet that will come with the original link.

A. Feed the graph

1. For small data, we can use either tf.place holder or files to feed the graph

2. For large data, for example, 150GB, it is recommended to use queued binary file. For more details: https://indico.io/blog/tensorflow-data-inputs-part1-placeholders-protobufs-queues/

 

B. Compact code

While TensorFlow has been developed rapidly, building a pure-Tensorflow graph is not that short as compared to Keras.  A good news is that Keras code using Tensorflow as backend is highly compatible with Tensorflow. Thus, we can definitely use Keras layers as an interface to Tensorflow.

https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html

However, I feel it is complex and hard to use since it is a high-level neural networks API, capable of running on top of either TensorFlowCNTK or Theano. I still cannot figure out how to divide an input tensor to feed different branches of a network. There is an alternative which I feel easier and more Tensorflow-style which is slim. We can get it by putting slim = tf.contrib.slim at the beginning of the code.

 

c. Deconvolutional Layer

The best tutorial that I found:

http://cv-tricks.com/image-segmentation/transpose-convolution-in-tensorflow/

D. Saving and Restoring Models

1. Save Models

 

E. Use only One GPU memory

There is a chance during the testing, your model eats up all GPUs’ s memory so that you cannot run any other job even though just one GPU is working. To avoid this situation, one simple solution is as follows.

 import os
# Set only one GPU available 
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"   # see issue #152
os.environ["CUDA_VISIBLE_EVICES"]="1"    # 1 mean use GPU 1. Set 0 to use GPU 0
 

Then, 
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True, device_count={'GPU':1})) # 1 means use only one GPU 
It will still log out GPU:0, for example, but still use only GPU 1 in  this case.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s