Tensorflow Convolutional Networks for semantic segmentation
Tensorflow implementation of Fully Convolutional Networks.
This is playable fork of https://github.com/MarvinTeichmann/tensorflow-fcn
Start this project, all the dependencies, libraries, packages are in place.
This is a one file Tensorflow implementation of Fully Convolutional Networks in Tensorflow. The code can easily be integrated in your semantic segmentation pipeline. The network can be applied directly or finetuned to perform semantic segmentation using tensorflow training code.
Deconvolution Layers are initialized as bilinear upsampling. Conv and FCN layer weights using VGG weights. Numpy load is used to read VGG weights. No Caffe or Caffe-Tensorflow is required to run this.
The .npy file for [VGG16] to be downloaded before using this needwork.
curl ftp://mi.eng.cam.ac.uk/pub/mttt2/models/vgg16.npy -o vgg16.npy
No Pascal VOC finetuning was applied to the weights. The model is meant to be finetuned on your own data. The model can be applied to an image directly (see
test_fcn32_vgg.py) but the result will be rather coarse.
In addition to tensorflow the following packages are required:
- tensorflow v1.2.1
All the requirements are already installed on PLON, so this step is not neceserry when you launch this at PLON.IO. However if you try this on your local machine:
Those packages can be installed by running
pip install -r requirements.txt or
pip install numpy scipy pillow matplotlib.
python test_fcn32_vgg.py to test the implementation.
Use this to build the VGG object for finetuning:
vgg = vgg16.Vgg16() vgg.build(images, train=True, num_classes=num_classes, random_init_fc8=True)
images is a tensor with shape
[None, h, w, 3]. Where
w can have arbitrary size.
Trick: the tensor can be a placeholder, a variable or even a constant.
Be aware, that
num_classes influences the way
score_fr (the original
fc8 layer) is initialized. For finetuning I recommend using the option
Example code for training can be found in the KittiSeg project repository.
Finetuning and training
For training build the graph using
vgg.build(images, train=True, num_classes=num_classes) were images is q queue yielding image batches. Use a softmax_cross_entropy loss function on top of the output of vgg.up. An Implementation of the loss function can be found in
To train the graph you need an input producer and a training script. Have a look at TensorVision to see how to build those.
I had success finetuning the network using Adam Optimizer with a learning rate of
Currently the following Models are provided:
The deconv layer of tensorflow allows to provide a shape. The crop layer of the original implementation is therefore not needed.
I have slightly altered the naming of the upscore layer.
Field of View
The receptive field (also known as or
field of view) of the provided model is:
( ( ( ( ( 7 ) * 2 + 6 ) * 2 + 6 ) * 2 + 6 ) * 2 + 4 ) * 2 + 4 = 404
Weights were generated using Caffe to Tensorflow. The VGG implementation is based on tensorflow-vgg16 and numpy loading is based on tensorflow-vgg. You do not need any of the above cited code to run the model, not do you need caffe.
Installing matplotlib from pip requires the following packages to be installed
pkg-config. On Debian, Linux Mint and Ubuntu Systems type:
sudo apt-get install libpng-dev libjpeg8-dev libfreetype6-dev pkg-config
pip install -r requirements.txt
- Provide finetuned FCN weights.
- Provide general training code
- [2017-09-01 10:46:23] v0.1.1: First version
More from this author
- SVM MNIST handwritten digit classification
- Linear regression models
- Introduction to matplotlib
- Tensorflow numbers recognition
- Tensorflow MNIST convolutional neural networks
- Numpy tutorial
- The World Bank eastern Europe GDP Analysis using Python Pandas and Seaborn
- Aymeric Damien TensorFlow Examples in Python 3
- TensorFlow Examples by aymericdamien
- Image convolution in python