I attended the Microsoft Build 2018 developers conference this week and really enjoyed it. I wanted to share my “big picture” feelings about it and some of the things that stood out to me. I’m not going to give you a “reporters” view or repeat press-release items. This is just my personal impression of the conference.
I have been qualifying a 4 GPU workstation for Machine Learning and HPC use. The last confirmation testing I wanted to do was running it with TensorFlow benchmarks on 4 NVIDIA Titan V GPU’s. I have that systems up and running and the multi-GPU scaling looks very good.
Batch size is an important hyper-parameter for Deep Learning model training. When using GPU accelerated frameworks for your models the amount of memory available on the GPU is a limiting factor. In this post I look at the effect of setting the batch size for a few CNN’s running with TensorFlow on 1080Ti and Titan V with 12GB memory, and GV100 with 32GB memory.
Tensor-cores are one of the compelling new features of the NVIDIA Volta architecture. In this post I discuss the some thought on mixed precision and FP16 related to Tensor-cores. I have some performance results for large convolution neural network training that makes a good argument for trying to use them. Performance looks very good.
NVIDIA’s Graphics Technology Conference (GTC) is probably my all-time favorite conference. It’s an interesting blend of “Scientific Research meeting” and Trade-Show. It’s put on by a hardware vendor but still feels like a scientific meeting. It’s not just a “Kool-Aid” fest! In this post I go present some of my thoughts about this years conference.
TensorFlow is a very powerful numerical computing framework. However, like any large research level program it can be challenging to install and configure. In this post I’ll try to give some guidance on relatively easy ways to get started with TensorFlow. I’ll only look at relatively simple “CPU only” Installs with “standard” Python and Anaconda Python in this post. (I also have a quick test with Intel Python.)
TensorFlow is on it’s way to becoming the “standard” framework for machine learning. There are many reasons for that, and, it is not just for machine learning! In this post I’ll give a descriptive introduction to TensorFlow. This is the first post in a series on how to work with TensorFlow. Hopefully after reading thsi you will have a better understanding of the What? and Why? of TensorFlow.
This post will look at the molecular dynamics program, NAMD. NAMD has good GPU acceleration but is heavily dependent on CPU performance as well. It achieves best performance when there is a proper balance between CPU and GPU. The system under test has 2 Xeon 8180 28-core CPU’s. That’s the current top of the line Intel processor. We’ll see how many GPU’s we can add to those Xeon 8180 CPU’s to get optimal CPU/GPU compute balance with NAMD.
In this post I present some Multi-GPU scaling tests running TensorFlow on a very nice system with 8 1080Ti GPU’s. I use the Docker Workstation setup that I have recently written about. The job I ran for this testing was the “Billion Words Benchmark” using an LSTM model. Results were very good and better than expected.
The Intel CPU flaw and the Meltdown and Spectre security exploits are causing a lot of concern. There is a possibility of application slowdown from the kernel patches to mitigate the exploits. This slowdown concern is a concern for GPU accelerated application because of the systems calls they require for moving data between CPU and GPU memory space. I did some testing on a couple of large Tensorflow and Caffe machine learning jobs along with the creation of a LMDA database from 1.3 million images.
Puget Systems builds custom workstations, servers and storage solutions tailored for your work.
We provide:
Extensive performance testing
making you more productive and giving better value for your money
Reliable computers
with fewer crashes means more time working & less time waiting
Support that understands
your complex workflows and can get you back up & running ASAP
A proven track record
as shown by our case studies and customer testimonials
