Batch size is an important hyper-parameter for Deep Learning model training. When using GPU accelerated frameworks for your models the amount of memory available on the GPU is a limiting factor. In this post I look at the effect of setting the batch size for a few CNN’s running with TensorFlow on 1080Ti and Titan V with 12GB memory, and GV100 with 32GB memory.
Tensor-cores are one of the compelling new features of the NVIDIA Volta architecture. In this post I discuss the some thought on mixed precision and FP16 related to Tensor-cores. I have some performance results for large convolution neural network training that makes a good argument for trying to use them. Performance looks very good.
Building TensorFlow from source is challenging but the end result can be a version tailored to your needs. This post will provide step-by-step instructions for building TensorFlow 1.7 linked with Anaconda3 Python, CUDA 9.1, cuDNN7.1, and Intel MKL-ML. I do the build in a docker container and show how the container is generated from a Dockerfile.
In this post I go through how to use Docker to create a container with all of the libraries and tools needed to compile TensorFlow 1.7. The build will include links to Intel MKL-ML (Intel’s math kernel library plus extensions for Machine Learning) and optimizations for AVX512.
NVIDIA’s Graphics Technology Conference (GTC) is probably my all-time favorite conference. It’s an interesting blend of “Scientific Research meeting” and Trade-Show. It’s put on by a hardware vendor but still feels like a scientific meeting. It’s not just a “Kool-Aid” fest! In this post I go present some of my thoughts about this years conference.
TensorFlow is a very powerful numerical computing framework. However, like any large research level program it can be challenging to install and configure. In this post I’ll try to give some guidance on relatively easy ways to get started with TensorFlow. I’ll only look at relatively simple “CPU only” Installs with “standard” Python and Anaconda Python in this post. (I also have a quick test with Intel Python.)
TensorFlow is on it’s way to becoming the “standard” framework for machine learning. There are many reasons for that, and, it is not just for machine learning! In this post I’ll give a descriptive introduction to TensorFlow. This is the first post in a series on how to work with TensorFlow. Hopefully after reading thsi you will have a better understanding of the What? and Why? of TensorFlow.
This post will look at the molecular dynamics program, NAMD. NAMD has good GPU acceleration but is heavily dependent on CPU performance as well. It achieves best performance when there is a proper balance between CPU and GPU. The system under test has 2 Xeon 8180 28-core CPU’s. That’s the current top of the line Intel processor. We’ll see how many GPU’s we can add to those Xeon 8180 CPU’s to get optimal CPU/GPU compute balance with NAMD.
In this post I present some Multi-GPU scaling tests running TensorFlow on a very nice system with 8 1080Ti GPU’s. I use the Docker Workstation setup that I have recently written about. The job I ran for this testing was the “Billion Words Benchmark” using an LSTM model. Results were very good and better than expected.
This should be the last post in this series dealing with the Docker setup for accessing the NVIDIA NCG Docker registry on your workstation. There are a couple of configuration tuning changes that you may want to make. These will improve performance and ensure that you have proper system “user limit” resources to handle large application and job runs with docker.
Puget Systems builds custom workstations, servers and storage solutions tailored for your work.
We provide:
Extensive performance testing
making you more productive and giving better value for your money
Reliable computers
with fewer crashes means more time working & less time waiting
Support that understands
your complex workflows and can get you back up & running ASAP
A proven track record
as shown by our case studies and customer testimonials
