In this post we look at using a testing Lab of Windows systems as a benchmarking platform for Linux scientific application using network boot with nfsroot and home mounts. Linux is boot on the systems "diskless" leaving the Windows installs untouched. LTSP turned out to be a great time saver for setting up the configuration.
This post presents testing data showing that power-limit reduction on NVIDIA GPUs have give significant benefits for both high wattage and lower wattage GPUs. Power-limit vs Performance data is presented for 1-4 A5000 and 1-4 RTX3090 GPUs.
In this post I am referencing a Bash shell script I recently put together for setting up automatic NVIDIA GPU power-limit lowering at system boot. This allows a reliable way to configure and maintain multi-GPU systems for stable operation under heavy load.
In this post I'll show you how to setup isolated conda envs for Python without having a base conda install! I'll cover Linux and Windows including an example to get you started. Read on to learn about the wonderful micromamba project.
This post will guide you through the process of creating an Ubuntu 20.04 (or newer) autoinstall ISO by modifying the default installer ISO. The install configuration will be done using cloud-init cloud-config method that is now used for the Ubuntu server installer.
NVIDIA Enroot has a unique feature that will let you easily create an executable, self-contained, single-file package with a container image AND the runtime to start it up! This allows creation of a container package that will run itself on a system with or without Enroot installed on it! "Enroot Bundles".
Enroot is a simple and modern way to run "docker" or OCI containers. It provides an unprivileged user "sandbox" that integrates easily with a "normal" end user workflow. I like it for running development environments and especially for running NVIDIA NGC containers. In this post I'll go through steps for installing enroot and some simple usage examples including running NVIDIA NGC containers.
When you install Miniconda3 or Anaconda3 on Windows it adds a PowerShell shortcut that has the necessary environment setup and initialization for conda. It's listed in the Windows menu as "Anaconda Powershell Prompt (Anaconda3)". However, this opens a separate/detached PowerShell instance and it would be nice to have this as an optional shell from Windows Terminal! In this post we will add that functionality as a new shell option in Windows Terminal.
The current JupyterHub version 2.5.1 does not allow user installed extension for JupyterLab when it is being served from JupyterHub. This should be remedied in version 3. However, even when this is "fixed" it is still useful to be able to install extensions globally for all users on a multi-user system. This note will show you how.
WSL on Windows 10 does not (currently) provide a direct way to copy a Linux distribution that was installed from the "Microsoft Store". The following guide will show you a way to make a working copy of an installed distribution with a new name.
In this note I'll go through creating self-signed SSL certificates and adding them to a JupyterHub configuration running on a LAN or VPN. This will allow encrypted access to the server using https in a browser.
This is a quick note about setting up a JupyterHub server and JupyterLab using conda with Anaconda Python.
It's time for a "Docker with NVIDIA GPU support" update. This post will guide you through a useful Workstation setup (including User-name-spaces and performance tuning) with the new versions of Docker and the NVIDIA GPU container toolkit.
Docker is a great Workstation tool. It is mostly used for command-line application or servers but, ... What if you want to run an application in a container, AND, use an X Window GUI with it? What if you are doing development work with CUDA and are including OpenGL graphic visualization along with it? You CAN do that!
Install TensorFlow 2 beta1 (GPU) on Windows 10 and Linux with Anaconda Python (no CUDA install needed)Written on June 26, 2019 by Dr Donald Kinghorn
TensorFlow 2.0.0-beta1 is available now and ready for testing. What if you want to try it but don't want to mess with doing an NVIDIA CUDA install on your system. The official TensorFlow install documentations has you do that, but it's really not necessary.
Being able to run Jupyter Notebooks on remote systems adds tremendously to the versatility of your workflow. In this post I will show a simple way to do this by taking advantage of some nifty features of secure shell (ssh). What I'll do is mostly OS independent but I am putting an emphasis on Windows 10 since many people are not familiar with tools like ssh on that OS.
This post is a setup guide and introduction to ssh client and server on Windows 10. Microsoft has a native OpenSSH client AND server on Windows. They are standard (and in stable versions) on Windows 10 since the 1809 "October Update". This guide should helpful to both Windows and Linux users who want better interoperability.
Being able to get Docker and the NVIDIA-Docker runtime working on Ubuntu 19.04 makes this new and (currently) mostly unsupported Linux distribution a lot more useful. In this post I'll go through the steps that I used to get everything working nicely.
This post is the needed update to a post I wrote nearly a year ago (June 2018) with essentially the same title. This time I have presented more details in an effort to prevent many of the "gotchas" that some people had with the old guide. This is a detailed guide for getting the latest TensorFlow working with GPU acceleration without needing to do a CUDA install.
Ubuntu 19.04 will be released soon so I decided to see if CUDA 10.1 could be installed on it. Yes, it can and it seems to work fine. In this post I walk through the install and show that docker and nvidia-docker also work. I ran TensorFlow 2.0- alpha on Ubuntu 19.04 beta.
NVIDIA recently released version 10.0 of CUDA. This is an upgrade from the 9.x series and has support for the new Turing GPU architecture. This CUDA version has full support for Ubuntu 18.4 as well as 16.04 and 14.04. The CUDA 10.0 release is bundled with the new 410.x display driver for Linux which will be needed for the 20xx Turing GPU's. If you are doing development work with CUDA or running packages that require you to have the CUDA toolkit installed then you will probably want to upgrade to this. I'll go though how to do the install of CUDA 10.0 either by itself or along with an existing CUDA 9.2 install.
PyTorch for Scientific Computing - Quantum Mechanics Example Part 4) Full Code Optimizations -- 16000 times faster on a Titan V GPUWritten on September 14, 2018 by Dr Donald Kinghorn
This is the 16000 times speedup code optimizations for the scientific computing with PyTorch Quantum Mechanics example. The following quote says a lot, "The big magic is that on the Titan V GPU, with batched tensor algorithms, those million terms are all computed in the same time it would take to compute 1!!!"