Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/1187
Dr Donald Kinghorn (Scientific Computing Advisor )

The Best Way to Install TensorFlow with GPU Support on Windows 10 (Without Installing CUDA)

Written on June 21, 2018 by Dr Donald Kinghorn
Share:


A couple of weeks ago I wrote a post titled Install TensorFlow with GPU Support on Windows 10 (without a full CUDA install). What you are reading now is a replacement for that post.

In that older post I couldn't find a way around installing at least some of CUDA. I tried to minimize it by installing only the essential DLL's to make things work. It did require making changes to the "User PATH" which I would rather not have done. I have now found a way around this using only Anaconda Python packages.

I was doing a pytorch install on Linux and Windows 10 and I noticed that they were adding the needed CUDA and cuDNN libraries with a separate, single, Anaconda package (pytorch/cuda90 or 91). I tried that package with TensorFlow but it didn't work. However, I searched Anaconda Cloud and found Anaconda supported packages for CUDA 9.0 and cuDNN 7 that did work! All I had to do was install these two packages in the conda virtual environment for TensorFlow. [ The Linux TensorFlow Anaconda package includes CUDA and cuDNN internally in the same package. ]

The focus here is to get a good GPU accelerated work environment for TensorFlow (with Keras and Jupyter notebook) up and running for Windows 10. You will not need to install CUDA for this!

In this post I'll walk you through the best way I have found so far to get a good TensorFlow work environment on Windows 10 including GPU acceleration. I'll also go through setting up Anaconda Python and create an environment for TensorFlow and how to make that available for use with Jupyter notebook. As a "non-trivial" example of using this setup we'll go through training LeNet-5 with Keras using TensorFlow with GPU acceleration. We'll get a setup that is 18 times faster than using the CPU alone.

Python Environment Setup with Anaconda Python

I highly recommend you use Anaconda Python. If you need some arguments for using Python take a look at my post Should You Learn to Program with Python. For arguments on why you should use the Anaconda Python distribution see, How to Install Anaconda Python and First Steps for Linux and Windows. The best reason for using Anaconda Python in the context of installing GPU accelerated TensorFlow is that by doing so you will not have to do a CUDA install on the system.

Anaconda is focused toward data-science and machine learning. It installs cleanly on your system in a single directory so it doesn't make a mess in your systems application and library directories. It is also performance optimized and links important numerical packages like numpy to Intel's MKL.


Install Anaconda Python

1) Download and check the installer

  • Go to the Anaconda downloads page https://www.anaconda.com/downloads and get the Python 3.6 version.

  • It's good practice to check the file hash to be sure you got a good copy. [ I have to admit that I don't always do this myself. ]

    • Open Powershell and cd to the directory where you downloaded the Anaconda installer exe file. In my case that is the Downloads directory.
cd Downloads

Then run

 Get-FileHash .\Anaconda3-5.2.0-Windows-x86_64.exe -Algorithm SHA256

Run the installer

Since you have Powershell open in the directory with the Anaconda installer exe file you can start it by just typing it's name (type A and hit tab to expand the name) and hitting return. [ You could alternatively just double click on the download install exe. from your file browser. ]

.\Anaconda3-5.1.0-Windows-x86_64.exe

The installer GUI should now be running.

  • You will be asked to accept a license agreement ...
  • "Select Install Type" I recommend you chose "Just Me" since this is part of your personal development environment.
  • "Chose Install Location" I recommend you keep the default which is at the top level of you user directory.
  • "Advanced Installation Options"
    Advanced install opts
    My recommendation is to check both boxes. Make Anaconda Python 3 your default Python. And, as a developer you really should be aware of your PATH environment variable. So yes, go ahead and let the installer add the Anaconda bin directory to your PATH. If you haven't looked at your environment variables in awhile you should have a look. Do a search from the Windows menu for "environment variables". You should find a settings panel that will show your account environment and system wide environment. After the Anaconda install you will see that its application and library directories have been prepended to your user PATH. We'll look at it, and modify it, after installing the CUDA libraries.

VSCode?

  • Next you will be asked if you want to install Microsoft VSCode. VSCode is a really good editor and it is available for free on Windows, Linux and MacOS. However, if you are interested in trying it out I would recommend that you go to the VSCode website and check it out first. If you think you want to try it, then go ahead and download it and install it yourself. I like VSCode but I usually use the Atom editor which also runs on Windows, Linux and MacOS. If you are checking out editors I recommend you try both of these as well as Sublime Text. They are all great editors!

Check your install

If you still have Powershell open you will need to close it and restart it so that it will re-read your environment variables and pick up your PATH which now includes the Anaconda Python directories. With Powershell reopened you can check that you now have Anaconda Python 3 as your default Python.

python --version

Python 3.6.5 :: Anaconda custom (64-bit)

Update your base Anaconda packages

conda is a powerful package and environment management tool for Anaconda. We'll use conda from Powershell to update our base Python install. Run the following commands. It may take some time to do this since there are a lot of modules to update.

conda update conda
conda update anaconda
conda update python
conda update --all

That should bring your entire base Anaconda install up to the latest packages. (Anaconda 5.2 had just been released when I wrote this and nearly everything was fully up-to-date.)

Anaconda Navigator

There is a GUI for Anaconda called anaconda-navigator. I personally find it distracting/confusing/annoying and prefer using conda from the command-line. Your taste may differ! ... and my opinion is subject to change if they keep improving it. If you are new to Anaconda then I recommend you read up on conda even (or especially!) if you are thinking about using the "navigator" GUI.


Create a Python "virtual environment" for TensorFlow using conda

You should set up an environment for TensorFlow separate from your base Anaconda Python environment. This keeps your base clean and will give TensorFlow a space for all of it's dependencies. It is in general good practice to keep separate environments for projects especially when they have special package dependencies. Think of it as a separate "name-space" for your project.

There are many possible options when creating an environment with conda including adding packages with specific version numbers and specific Python base versions. This is sometimes useful if you want fine control and it also helps with version dependencies resolution. Here we will keep it simple and just create a named environment and then activate that environment and install the packages we want inside of that.

From a command line do,

conda create --name tf-gpu

I named the environment 'tf-gpu' but you can use any name you want.

"activate" the environment

Now activate the environment, (I'll show my full terminal prompt and output instead of just the commands)

Note: for some reason Powershell will not run the "activate" script! You will need to start "CMD" shell to do this. You can start CMD shell from Powershell (notice how the "PS" that was at the beginning of the Powershell prompt disappears). Having to switch to CMD is an annoyance but you can easily switch back and forth in a Powershell window

PS C:\Users\don> cmd
Microsoft Windows [Version 10.0.16299.461]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\don> activate tf-gpu

(tf-gpu) C:\Users\don>

You can see that my CMD shell prompt is now preceded by the the name of the environment (tf-gpu). Any conda package (or pip) installs will now be local to this environment.

Install TensorFlow-GPU from the Anaconda Cloud Repositories

My preference would be to install the "official" Anaconda maintained TensorFlow-GPU package like I did for Ubuntu 18.04, unfortunately the Anaconda maintained Windows version of TensorFlow is way out-of-date (version 1.1). There is a current CPU-only version 1.8 for Windows but we want GPU acceleration.

A search for "tensorflow" on the Anaconda Cloud will list the available packages from Anaconda and the community. There is a package "aaronzs / tensorflow-gpu 1.8.0" listed near the top that has builds for Linux and Windows. This is the only up-to-date package I know of that is working correctly with Windows 10. This package was built by, and is being nicely maintained by, Aaron Sun. You can check out his GitHub page for the project.

Lets install TensorFlow with GPU acceleration in this conda environment.

(tf-gpu) C:\Users\don> conda install -c aaronzs tensorflow-gpu

Now, we can do the CUDA and cuDNN dependencies,

(tf-gpu) C:\Users\don> conda install -c anaconda cudatoolkit
(tf-gpu) C:\Users\don> conda install -c anaconda cudnn

Note that I explicitly use the -c flag to specify the "anaconda" "channel". That would be default if you leave out the channel name but in this case I wanted to be explicit about where the packages came from. The links are, cudatoolkit current is 9.0 and cudnn current is 7.1.4. You should check version numbers when you install.

That's it! You do not need to do a CUDA install on your system.


Check That TensorFlow is working with your GPU

Close any Powershell or CMD shells you had open and reopen one. You need to do that so that your new PATH settings get read in. You can use a CMD shell to activate your tf-gpu environment start Python and run the following lines,

>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))

My session including the output looked like this, (there was a long delay during this "first run" session startup )

PS C:\Users\don> cmd
Microsoft Windows [Version 10.0.16299.461]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\don> activate tf-gpu

(tf-gpu) C:\Users\don>python
Python 3.6.5 |Anaconda custom (64-bit)| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow')
>>> sess = tf.Session()
2018-06-01 16:37:57.666250: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2018-06-01 16:37:57.967130: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:01:00.0
totalMemory: 8.00GiB freeMemory: 6.62GiB
2018-06-01 16:37:57.975868: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1435] Adding visible gpu devices: 0
2018-06-01 16:40:10.162112: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-06-01 16:40:10.168554: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:929]      0
2018-06-01 16:40:10.171214: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:942] 0:   N
2018-06-01 16:40:10.174162: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6400 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
>>> print(sess.run(hello))
b'Hello, TensorFlow'
>>>

Yea! PATH's are correct and everything is working. You can see that it is has GPU support.

Next we will do something a little more useful and fun with Keras, after we configure Jupyter notebook to use our 'tf-gpu' environment.


Create a Jupyter Notebook Kernel for the TensorFlow Environment

You can work with an editor and the command line and you often want to do that, but, Jupyter notebooks are great for doing machine learning development work. In order to get Jupyter notebook to work the way you want with this new TensorFlow environment you will need to add a "kernel" for it.

With your tf-gpu environment activated do,

(tf-gpu) C:\Users\don>conda install ipykernel

Now create the Jupyter kernel,

(tf-gpu) C:\Users\don>python -m ipykernel install --user --name tf-gpu --display-name "TensorFlow-GPU"

With this "tf-gpu" kernel installed, when you start Jupyter notebook you will now have an option to to open a new notebook using this kernel.

Jupyter kernel for TF


An Example using Keras with TensorFlow Backend

In order to check everything out lets setup LeNet-5 using Keras (with our TensorFlow backend) using a Jupyter notebook with our "TensorFlow-GPU" kernel. We'll train the model on the MNIST digits data-set and then open TensorBoard to look at some plots of the job run.

Install Keras

With the tf-gpu environment activated do,

(tf-gpu) C:\Users\don\projects>conda install keras-gpu

You now have Keras installed utilizing your GPU accelerated TensorFlow.

Launch a Jupyter Notebook

With the tf-gpu environment activated start Jupyter,

(tf-gpu) C:\Users\don>jupyter notebook

From the 'New' drop-down menu select the 'TensorFlow-GPU' kernel that you added (as seen in the image in the last section). You can now start writing code!

MNIST example

Following are Python snippets you can copy into cells in your Jupyter notebook to setup and train LeNet-5 with MNIST digits data.

Import dependencies

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.layers import Flatten,  MaxPooling2D, Conv2D
from keras.callbacks import TensorBoard

Load and process the MNIST data

(X_train,y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(60000,28,28,1).astype('float32')
X_test = X_test.reshape(10000,28,28,1).astype('float32')

X_train /= 255
X_test /= 255

n_classes = 10
y_train = keras.utils.to_categorical(y_train, n_classes)
y_test = keras.utils.to_categorical(y_test, n_classes)

Create the LeNet-5 neural network architecture

model = Sequential()
model.add(Conv2D(32, kernel_size=(3,3), activation='relu', input_shape=(28,28,1)) )
model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))
model.add(Flatten())          
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(n_classes, activation='softmax'))

Compile the model

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

Set log data to feed to TensorBoard for visual analysis

tensor_board = TensorBoard('./logs/LeNet-MNIST-1')

Train the model

model.fit(X_train, y_train, batch_size=128, epochs=15, verbose=1,
          validation_data=(X_test,y_test), callbacks=[tensor_board])

The results

After running that training for 15 epochs the last epoch gave,

Epoch 15/15
60000/60000 [==============================] - 6s 102us/step - loss: 0.0192 - acc: 0.9936 - val_loss: 0.0290 - val_acc: 0.9914

Not bad! Training accuracy 99.36% and Validation accuracy 99.14%


Look at the job run with TensorBoard

You will need "bleach" for TensorBoard so install it first,

(tf-gpu) C:\Users\don>conda install bleach

Start TensorBoard

 (tf-gpu) C:\Users\don\projects>tensorboard --logdir=./logs --port 6006

It will give you an address similar to http://stratw:6006 Open that in your browser and you will be greeted with (the wonderful) TensorBoard. These are the plots it had for that job run,
TensorBoard output

That was a model with 1.2 million training parameters and a dataset with 60,000 images. It took 1 minute and 26 seconds utilizing the NVIDIA GeForce 1070 in my laptop system! For reference it took 26 minutes using all cores at 100% of the Intel 6700HQ CPU in that system. That's an 18 fold speedup on the GPU!

Happy computing! --dbk

Tags: Windows 10, TensorFlow, Anaconda Python, GPU acceleration, CUDA
Nathan Copier

I tried following your tutorial to the "t", but had issues loading the DLLs. However, it appears the official Anaconda TensorFlow-GPU library has been updated. I tried using the official version and it worked for me.

Posted on 2018-07-26 06:01:50
Donald Kinghorn

Yes, that happened after I wrote this. They finally update that! That is definitely the way to go.
Thanks for adding your comment!
Best wishes --Don

Posted on 2018-08-01 21:18:17
sahar

i followed this tutorial and it worked for me. Now i want to install opencv can you give the corresponding command to install opencv in the tf-gpu environment

Posted on 2018-09-23 20:59:23
Donald Kinghorn

There is a build of opencv on Anaconda cloud that is supported for Linux and Windows so you should be able to "activate" your environment and then just do

conda install opencv

However, I will warn you that sometimes opencv can be troublesome. There are lots of capabilities that can be compiled in and occasionally people have trouble with various builds. If the default install listed above give you trouble then you can try uninstalling that and try this other build.

conda install -c conda-forge opencv

Posted on 2018-09-24 23:50:37
Anurag Gupta

conda install -c conda-forge opencv has limitations. this is complete contrib version of opencv and has never disappointed: conda install -c michael_wild opencv-contrib or pip install opencv-contrib-python

Posted on 2018-10-01 14:19:00
Anurag Gupta

conda install -c michael_wild opencv-contrib
or
pip install opencv-contrib-python

Posted on 2018-10-02 07:51:37
Donald Kinghorn

thanks for posting that! It seems like every time I have needed openCV I've had to dig around for a build with what i need linked in or just build it myself.

Posted on 2018-10-03 01:21:16
jokkebk1

Wow, this actually worked. I am amazed, after a couple of hours banging my head to the wall, I now have a GPU device working with tensorflow. Huge thanks!

Posted on 2018-09-30 16:15:16
吳重昇

its really helpful for me when I update cuda and cudnn to latest version

Posted on 2018-10-07 12:11:02
Sasha Moiseev

Well I would highly(!) recommend to intall original python 3.6 from python.org, then its better create separate python virtual enviroment for tensorflow-gpu and packages you need.
After activating virtual environment, install packages over pip: tensorflow-gpu, then keras, install packages you need (like jupyter, ipykernel, scikit-learn, matplotlib, pydot and etc.) . You can use method of installing nvidia's dll as described in this article or packages by pip. Also please pay attention on the version requirements of python and cuda dlls are provided by tensorflow.

The reason of those recommendations is that if you are doing image preprocessing by ImageDataGenerator functions in keras (which is faster instead of writing own generators) - you will face lots of problems (like AttributeError: 'ImageDataGenerator' object has no attribute 'flow_from_dataframe') due the old versions of packagies are in anaconda or conda-forge and their dependencies. Be aware of anaconda...

Posted on 2018-10-07 21:19:41
Donald Kinghorn

You have a good point about some of the builds on Anaconda cloud. It is especially a problem for Windows versions. I've had Linux packages that did what I needed and then pulled the Windows version only to find it's out of date or wasn't linked with the lib I needed.

I've used both python.org python and envs and Anaconda with conda. There are some things that bug me about anaconda but overall it is an incredible service they are doing. I haven't hit many problems lately and I am 99% on Linux. I have had to do pip installs in anaconda envs and that works fine really.

But like you pointed you can run into problems now and then. --Don

Posted on 2018-10-08 19:28:31
Sasha Moiseev

Yes, agree. Francois Sholle says that builds for Windows (i think he meant to say it about Anaconda) are kind of 'big mistake'. And mostly they recommend install APIs by pip. Thanks for the message!

Posted on 2018-10-08 20:19:07
Donald Kinghorn

The problem is that it if you try to install TensorFlow in Windows without using Anaconda you are going to have to install CUDA too. And that is a not fun on Windows. If you can use Anaconda, especially on Windows, it can make the difference between getting some work done or just being frustrated. If you go with pip packages then you have to have whatever external environment is need by it i.e. with multiple packages you could end up needing different version of CUDA installed. The Anaconda packages can eliminate that dependence.

Posted on 2018-10-15 15:11:50
gokul s

sir i tried to install through the above steps but i get some error like https://uploads.disquscdn.c...

Posted on 2018-10-14 12:31:33
Donald Kinghorn

The key is in the line "ImportError: DLL load failed ..." that is telling you that your PATH and or LD_LIBRARY_PATH is not right. I've mostly seen that in Jupyter notebooks when I have forgotten to create (and then use) a Jupyter kernel for the python environment I'm trying to use.

Also, it looks a bit like you have multiple versions of stuff installed (maybe multiple Python distributions) ?? I'm not sure why anything would be looking your AppData\Roaming\Python path. Check your PATH variable ???

Posted on 2018-10-15 15:20:08
akshay 895

Hi,
I installed the tensorflow-gpu using the anaconda navigator method.
Just ticked the tensorflow-gpu in the env tab and it installed everything, this worked for me but it worked for only one day, my gpu was getting detected by tensorflow and was being used by python. But the next day when I tried to run tensorflow again, it gave me the exact same error as gokul s , I did not update anything or change anything.

Posted on 2018-10-25 05:59:58
Donald Kinghorn

I've never really used navigator but it looks like you are on the right track. It may just be setting path in the local shell that it launches the notebook from. In that case you would have to repeat that process each time. exe and DLL path issues are common for developers. It doesn't matter what language or OS you are using :-)

Posted on 2018-10-26 15:18:49
Colin Wearring

Greetings, this was extremely helpful, but i seem to have encountered a problem with the different CUDA libraries that prevents the execution of cudainit. I have a gForce GTX 960 GPU running on Windows 10, machine name = Alien. Here is the terminal response from the tensorflow test:

>>> sess = tf.Session()
2018-10-16 16:10:56.076143: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 (found this error described as a warning)
2018-10-16 16:10:56.124025: E tensorflow/stream_executor/cuda/cuda_driver.cc:300] failed call to cuInit: CUDA_ERROR_UNKNOWN: unknown error
2018-10-16 16:10:56.131741: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] retrieving CUDA diagnostic information for host: Alien
2018-10-16 16:10:56.135542: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:170] hostname: Alien

I tried a few iterations,
first using Aaron Sun's tensorflow-gpu, which created a couple of upgrade/downgrade messages during install of the anaconda channel cudatoolkit and cudnn.
cudatoolkit => The following packages will be DOWNGRADED: certifi: 2018.10.15-py36_0 --> 2018.8.24-py36_1 anaconda

second using the anaconda tensorflow-gpu based on comments from this Q&A => same results as for Aaron's install
third trying the cudatookit/cudnn from the default rather than the anaconda channel.

If you would be kind enough to suggest some additional diagnostics, share the location of the files that are in the cuInit call, or provide any feasible path forward, it would be greatly appreciated.

Posted on 2018-10-16 20:45:48
Donald Kinghorn

Hi Colin, There are a couple of things that could be giving you trouble (all related). It usually comes down to a PATH problem.
I'm not great on Windows since I don't use it very often but the same problems happen on Linux (but I understand Linux better under-the-hood)
Things to check/be careful about:
How you start your Python environment makes a difference. If you start a dos shell from the anaconda GUI it sets your PATH for that shell and you still need to "activate" your the environment. If you try to use Powershell (which I really like) you can't load environments! You have to use the "dos" command in PS to switch how it reads environment vars and such and then you can activate. Also, if you start up a Jupyter notebook you have to remember to do so from a shell that has the proper env activated and you still have to then open a notebook using the proper kernel.

Other problems are usually PATH related too. If the environment you are can't find the cuda libs you will get a cuinit error. That can also happen if a program can't find the NVIDIA driver libs! This is a big problem on laptops because of that power saving junk that switches back to on-board Intel graphics.

There are two uses that often end up in PATH messes: Developers and system administrators. As a developer you have to constantly be aware of how lib/dll's are linked and what your local shell see for PATH and DLL paths.

For a little insight more insight you might benefit from the the similarly titled post "Install TensorFlow with GPU Support on Windows 10 (without a full CUDA install)" https://www.pugetsystems.co...

Posted on 2018-10-17 16:08:25
Colin Wearring

Thanks Donald, I activated the environment from the command shell and looked for the cuda and nvidia dlls in the path.

Found two nvidia dlls in the system32/bin folder (nvcuda.dll and nvcuvid.dll), so these nvidia libraries seem to visible from the tensorflow environment (i called mine cw-gpu).
Then i confirmed that nvrtc64_90.dll, nvrtc-builtins64_90.dll, nvToolsExt64_1.dll, nvvm64_32_0.dll, cudnn64_7.dll and cudart64_90.dll were visible in the directory Anaconda3\envs\cw-gpu\Library\bin.
I appreciate that you are not familiar with windows, but do these look like the Cuda and Nvida driver libraries that I need in my path? (hope the naming is similar on linux) Any guidance is appreciated.

Posted on 2018-10-18 21:47:40
Donald Kinghorn

I rebooted my laptop into Win10 had to wait a loooong time for the 1803 update to download and install :-) I checked what I had done on that system ... I had the install with the DLL's like I had done in the earlier post. But I searched the full system and the only place I had anything to do with cuda was in my user directory. I'm guessing that you have some cuda version installed globally on your system since you are seeing libs in system32. In fact I don't even have \bin in system32

Try this, open the Environment settings panel and see what is set in there for your PATH. That tool has a nice editor. Move anything there to a position below the Anaconda path settings. Then open up a new shell and try again. I think that will work. ??? It may require a reboot??

Posted on 2018-10-18 23:51:40
Colin Wearring

Donald,
You're being very generous with your time. thank you. I updated the precedence of the Anaconda paths to appear first. I removed all the nvidia packages and rebooted. Then I deleted the nvidia dlls from the system32 path. I then tried to import tensorflow within Python and failed with a missing DLL. I reinstalled tensorflow and received these interesting version/update statements:

>conda install tensorflow-gpu ===> The following packages will be UPDATED: certifi: 2018.10.15-py36_0 anaconda --> 2018.10.15-py36_0
>conda install -c anaconda cudatoolkit ===> The follo...be UPDATED: certifi: 2018.10.15-py36_0 --> 2018.10.15-py36_0 anaconda
>conda install -c anaconda cudnn ==> All requested packages already installed.

Start new CMD shell, activate cw-gpu environment and run >where *cud*
C:\Anaconda3\envs\cw-gpu\Library\bin\cudart64_90.dll
C:\Anaconda3\envs\cw-gpu\Library\bin\cudatoolkit_config.yaml
C:\Anaconda3\envs\cw-gpu\Library\bin\cudnn64_7.dll
C:\Anaconda3\Library\bin\icudt.dll C:\Anaconda3\Library\bin\icudt58.dll

When I tried to import tensorflow, it failed with the same missing DLL error.
I then reinstalled the Nvidia graphics drivers (416.34-desktop-win10-64bit-international-whql) which updated the available cuda dlls to:

C:\Anaconda3\envs\cw-gpu\Library\bin\cudart64_90.dll
C:\Anaconda3\envs\cw-gpu\Library\bin\cudatoolkit_config.yaml
C:\Anaconda3\envs\cw-gpu\Library\bin\cudnn64_7.dll
C:\Anaconda3\Library\bin\icudt.dll
C:\Anaconda3\Library\bin\icudt58.dll
C:\Windows\System32\nvcuda.dll
C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common\cudart32_65.dll
C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common\cudart64_65.dll

Started Python:
Python 3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 11:27:44) [MSC v.1900 64 bit (AMD64)] on win32
Reinstalled tensorflow and cuda packages and received the same update messages.
Tried the test script, and it worked!
Many thanks Donald,

I include this detail in case some one else has the same issues.
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
2018-10-22 12:37:40.352102: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2018-10-22 12:37:40.561837: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1411] Found device 0 with properties:
name: GeForce GTX 960 major: 5 minor: 2 memoryClockRate(GHz): 1.2005
pciBusID: 0000:01:00.0
totalMemory: 4.00GiB freeMemory: 3.33GiB
2018-10-22 12:37:40.570778: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1490] Adding visible gpu devices: 0
2018-10-22 12:37:43.938204: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-10-22 12:37:43.943315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] 0
2018-10-22 12:37:43.945757: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0: N
2018-10-22 12:37:43.950135: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1103] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3041 MB memory) -> physical GPU (device: 0, name: GeForce GTX 960, pci bus id: 0000:01:00.0, compute capability: 5.2)
>>> print(sess.run(hello))
b'Hello, TensorFlow!'

Posted on 2018-10-22 16:42:05
Eric Xu

This is amazing. Made my day. I've used Dell XPS 15 9560 and try to install GPU support for Tensorflow and Pytorch on Win10. Spent three days meeting all (all) kinds of trouble, version issue, installed tensorflow but it still use CPU, a lot of issues. Following your instruction, I made it in one hour! Kudos to you.

Posted on 2018-10-28 03:48:03
CPL593H

I created a Disqus account just to say that, you sir, are a heaven sent! It's been weeks, several panic attacks, and an endless amount of tutorials, that have only ended in disappointment and failure... But not your tutorial sir! FINALLY, Tensorflow is reading my GTX 1060! Thanks to you my project is NOT impossible or inconvenient in carrying out. Thank you for sharing this information! I hope this comment helps out with your SEO because you're a saint and I know from reading countless posts, that there are way too many people like myself seeking exactly this information, but the responses they receive are full of ego, convoluted methods, and condescending remarks. The internet needs more folks like you. Thank you again

Posted on 2018-10-31 02:03:49
Donald Kinghorn

and, you are very welcome! :-)

Posted on 2018-10-31 19:53:24
Paul Lipman

I followed your tutorial, but when I entered "sess = tf.Session()", I got the following and then no mention of the GPU (I also have a GeForce GTX 1070). I've struggled for weeks trying to get TF working with my GPU. I would be most grateful for any suggestions!

2018-11-04 14:47:54.274248: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

Posted on 2018-11-04 22:52:27
Donald Kinghorn

That message is OK. The first thing TF does is run that cpu_feature_guard code. It checks the capabilities of your processors and then then tells you about any features that are present that it wasn't compiled against (or that are missing that it was compiled against) By default TF is not linked to AVX (or any CPU vector libs) It gives you that warning when it starts up. It's what comes after all of those messages that's important.

Do this; from a dos shell (assuming you named your environment tf-gpu)
activate tf-gpu
python -c "import tensorflow as tf; tf.test.gpu_device_name()"

That should return your detected GPU ... if it doesn't then there is a problem ...

note: if that python line doesn't work from dos (I'm in Linux right now) then do this;
start python in your tf env

activate tf-gpu

python
...then in that python shell do,

import tensorflow as tf
tf.test.gpu_device_name()

Posted on 2018-11-05 16:08:13
Paul Lipman

Hi Donald. I tried that, and just got the same "CPU supports AVX2" message, and then it dumped me back out to the tg-gpu command prompt with no mention of the GPU. I can see the GPU in windows task manager, so it's definitely physically connected. I've also ensured that the GPU has the current drivers installed. I'm sure that there's something obvious that I'm missing. I'd really appreciate any suggestions for what to try next!

Posted on 2018-11-05 23:46:02
Donald Kinghorn

Hey Paul, try this:

open an Anaconda Shell and then activate tf-gpu, then start python and do that from ... timport command followed byt the device list thing.

(base) C:\Users\don>
(base) C:\Users\don>activate tf-gpu

(tf-gpu) C:\Users\don>python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>>> from tensorflow.python.client import device_lib
>>> device_lib.list_local_devices()

It should give you output that has a bunch of the TF start up messages and then a list with the detected devices. I get the following, (ignoring all the TF messages at the begining)

[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 10925237602644881371
, name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 6710956523
locality {
bus_id: 1
links {
}
}
incarnation: 1490371656099133640
physical_device_desc: "device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1"
]

What do you get?

Posted on 2018-11-06 17:47:14
Paul Lipman

Unfortunately I just get the same CPU message, no mention of the GPU:

(base) C:\Users\Ben Lipman>activate tf-gpu

(tf-gpu) C:\Users\Ben Lipman>python
Python 3.6.7 |Anaconda, Inc.| (default, Oct 28 2018, 19:44:12) [MSC v.1915 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from tensorflow.python.client import device_lib
>>> device_lib.list_local_devices()
2018-11-06 16:59:33.992171: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2

Posted on 2018-11-07 01:01:37
DriesR

Hi Don,

thanks for the tutorial.

All installations went well, except I get the following error when testing the GPU:

Internal: cudaGetDevice() failed. Status: CUDA driver version is insufficient for CUDA runtime version..

Print-out of conda list and sess = tf.Session() below

thanks in advance

Dries

(tf-gpu) c:\Users\cvbrun>conda list
# packages in environment at C:\Users\cvbrun\Anaconda3\envs\tf-gpu:
#
# Name Version Build Channel
absl-py 0.6.1 py36_0
astor 0.7.1 py36_0
blas 1.0 mkl
certifi 2018.10.15 py36_0 anaconda
cudatoolkit 9.0 1 anaconda
cudnn 7.1.4 cuda9.0_0 anaconda
gast 0.2.0 py36_0
grpcio 1.12.1 py36h1a1b453_0
icc_rt 2017.0.4 h97af966_0
intel-openmp 2019.0 118
libprotobuf 3.6.1 h7bd577a_0
markdown 3.0.1 py36_0
mkl 2019.0 118
mkl_fft 1.0.6 py36hdbbee80_0
mkl_random 1.0.1 py36h77b88f5_1
numpy 1.14.5 py36h9fa60d3_4
numpy-base 1.14.5 py36h5c71026_4
pip 18.1 py36_0
protobuf 3.6.1 py36h33f27b4_0
python 3.6.7 h33f27b4_1
setuptools 40.5.0 py36_0
six 1.11.0 py36_1
tensorboard 1.10.0 py36_0 aaronzs
tensorflow-gpu 1.10.0 py36_0 aaronzs
termcolor 1.1.0 py36_1
vc 14.1 h21ff451_3 anaconda
vs2015_runtime 15.5.2 3 anaconda
werkzeug 0.14.1 py36_0
wheel 0.32.2 py36_0
wincertstore 0.2 py36h7fe50ca_0
zlib 1.2.11 h8395fce_2

(tf-gpu) c:\Users\cvbrun>python
Python 3.6.7 |Anaconda, Inc.| (default, Oct 28 2018, 19:44:12) [MSC v.1915 64 bi
t (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
2018-11-06 21:00:23.481624: I T:\src\github\tensorflow\tensorflow\core\platform\
cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow bi
nary was not compiled to use: AVX2
2018-11-06 21:00:23.994731: I T:\src\github\tensorflow\tensorflow\core\common_ru
ntime\gpu\gpu_device.cc:1405] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.835
pciBusID: 0000:03:00.0
totalMemory: 8.00GiB freeMemory: 7.05GiB
2018-11-06 21:00:23.995233: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1484] Adding visible gpu devices: 0
2018-11-06 21:00:23.995633: E T:\src\github\tensorflow\tensorflow\core\common_runtime\direct_session.cc:158] Internal: cudaGetDevice() failed. Status: CUDA driver version is insufficient for CUDA runtime version
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\cvbrun\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow\python\client\session.py", line 1494, in __init__super(Session, self).__init__(target, graph, config=config)
File "C:\Users\cvbrun\Anaconda3\envs\tf-gpu\lib\site-packages\tensorflow\python\client\session.py", line 626, in __init__self._session = tf_session.TF_NewSession(self._graph._c_graph, opts)tensorflow.python.framework.errors_impl.InternalError: Failed to create session.

Posted on 2018-11-06 20:17:10
Yongyao Jiang

Helps a lot. My only question is whether I need to install cuda driver first. I also posted a question on stackoverflow https://stackoverflow.com/q...

Posted on 2018-11-14 00:47:17
Caim Astraea

Ufff can't get it to work :(
Getting
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "Y:\anaconda\envs\tf-gpu\lib\site-packages\tensorflow\python\client\session.py", line 1511, in __init__
super(Session, self).__init__(target, graph, config=config)
File "Y:\anaconda\envs\tf-gpu\lib\site-packages\tensorflow\python\client\session.py", line 634, in __init__
self._session = tf_session.TF_NewSessionRef(self._graph._c_graph, opts)
tensorflow.python.framework.errors_impl.InternalError: failed initializing StreamExecutor for CUDA device ordinal 0: Internal: failed call to cuDevicePrimaryCtxRetain: CUDA_ERROR_UNKNOWN: unknown error

Posted on 2018-11-14 20:00:50
Donald Kinghorn

Hi Caim, look at what I put in the comment above. That should help. The main thing is to get Anaconda Python setup on your machine and then setup an environment for TF like above --Don

Posted on 2018-11-15 17:17:41
Caim Astraea

Ah thanks!! :) Yea got it to work using a config option for tf.session but trying to get openCV working on windows defeatead me :) Had to throw in the towel after 5-6 hours of moving dll and wheel file and whatever else around. Will be building a sweet ubuntu 18 PC soon so looking forward to that.

Posted on 2018-11-16 20:14:27
Donald Kinghorn

I think you'll like Ubuntu 18.04 and it will be more cooperative :-) I just saw that Canonical is going to be supporting it as LTS for 10 years! That's a strong commitment on their part, very welcomed...

Posted on 2018-11-19 17:30:14
Sulaiman Almani

Thanks. I was running into a lot of problems but your tutorial was simple and ran like a charm!

Posted on 2018-11-16 18:29:53
Rania

Thank you! that is really helpful. I did all what you said, and now the TensorFlow works if I used Jupyter Notebook, It doesn't work if I use spyder. I get that error when I run Tenserflow on Spyder "
File "<ipython-input-1-64156d691fe5>", line 1, in <module>
import tensorflow as tf

ModuleNotFoundError: No module named 'tensorflow'"

I'm not sure what's the problem here

Posted on 2018-11-16 23:19:54
Rania

I think I solved my problem. I had to install spyder in my gpu environment

so what I did is:

(base) C:\Users\.... >activate tf-gpu
(tf-gpu) C:\Users\....> conda install spyder

and now I'm using spyder(tf-gpu). Really thank you. I've been trying to install tenserflow- gpu for a week. Thanks for your tutorial!

Posted on 2018-11-18 03:29:41
Donald Kinghorn

I was thinking it was something with the env not getting set for Spyder ... glad you worked it out!

Posted on 2018-11-19 17:33:09
Paul Lipman

Donald - I re-installed Windows on the PC and then followed your instructions with the fresh machine. Worked perfectly! Thank you so much for this tutorial!

Posted on 2018-11-17 01:28:24
Donald Kinghorn

Great! I'm sorry you had to do a fresh install but honestly it's often the easier thing to do! I'm pretty quick to do reinstalls (Windows and Linux both). I keep good backups of important stuff and try to keep my overall systems as "clean and lean" as possible.

Posted on 2018-11-19 17:39:22
Kseniia Palin

Hi! Thanks for the great tutorial! I've ran the installation with only minor change:
(tf-gpu) C:\Users\don> conda install -c anaconda tensorflow-gpu

(it already includes cudatoolkit and cudnn)

instead of :
(tf-gpu) C:\Users\don> conda install -c aaronzs tensorflow-gpu
(tf-gpu) C:\Users\don> conda install -c anaconda cudatoolkit
(tf-gpu) C:\Users\don> conda install -c anaconda cudnn

The installation text code and tensorboard visualization works fine.

My question is: have you tried using CuDNNGRU or CuDNNLTSM?

I am currently using the tf-gpu environment to run a model with CuDNNGRU and is gives me the error on training start:

UnknownError: Fail to find the dnn implementation.
[[{{node bidirectional_1/CudnnRNN_1}} = CudnnRNN[T=DT_FLOAT, _class=["loc:@training/Adam/gradients/bidirectional_1/CudnnRNN_1_grad/CudnnRNNBackprop"], direction="unidirectional", dropout=0, input_mode="linear_input", is_training=true, rnn_mode="gru", seed=87654321, seed2=0, _device="/job:localhost/replica:0/task:0/device:GPU:0"](bidirectional_1/transpose_2, bidirectional_1/ExpandDims_3, bidirectional_1/Const_1, bidirectional_1/concat_1)]]
[[{{node metrics/acc/Mean_1/_143}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1418_metrics/acc/Mean_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Posted on 2018-11-27 11:46:19
Donald Kinghorn

Yes, good, what you did using the new Anaconda TF package is the way to go.

I have not run anything like what you mention. It looks like it's not finding the package?? You will probably have to explicitly import tf.contrib.cudnn_rnn I'm not sure exactly what's needed but I think that is basically what is going on ... OK I just checked to be sure the cudnn_rnn stuff is in there, looks like it is. You probably need to use something like

from tensorflow.contrib.cudnn_rnn import CudnnLSTM

That loaded OK for me ...
If you are doing that and it is still failing then I don't know what is going on

Posted on 2018-11-27 23:26:23
VK

Hi Donald, just to let you know that this made my day. worked liked a charm. fortunately, I landed on this article in couple of hours after building a new pc with GPU(RTX 2070). Back of my mind, I was hating to get ready for days/weeks of reading, trying and errors.

Thanks again, V

Posted on 2018-11-29 04:54:32

I am really impressed with the details that you have provided and replied in the detail on diferent comments. Huge kudos to you. I was banging my head since 3 weeks on my new laptop trying avrious things and now only got something working, thanks to you. Great community work indeed.

Posted on 2018-12-02 19:30:53
Donald Kinghorn

We are all in it together my friend! :-) I am happy to have the opportunity to help

Posted on 2018-12-03 18:45:44
Julian Appel

Just want to add my deep appreciation and thanks for this tutorial. While I could install PyTorch in a moment on Windows 10 with the latest Python (3.7) and CUDA (10),
Tensorflow resisted any reasonable effort. Finally I found this tutorial and all went smoothly with Python 3.6 (from Anaconda) and the suggested CUDA 9 libraries.
Many, many thanks (Chapeau bas from up North)

Posted on 2018-12-13 02:27:59
pranjal saxena

thanks man!

Posted on 2018-12-13 06:23:48