How-To: Make Ubuntu Autoinstall ISO with Cloud-init
Written on September 1, 2021 by Dr Donald KinghornThis post will guide you through the process of creating an Ubuntu 20.04 (or newer) autoinstall ISO by modifying the default installer ISO. The install configuration will be done using cloud-init cloud-config method that is now used for the Ubuntu server installer.
Intel Ice Lake Xeon-W vs AMD TR Pro Compute Performance (HPL, HPCG, NAMD, Numpy)
Written on July 29, 2021 by Dr Donald KinghornThe single socket version of Intel third generation Xeon SP is out, the Ice Lake Xeon W 33xx. This is a much better platform with faster large capacity 8 channel memory and PCIe v4 with plenty of lanes. The new Intel platform is very much like the AMD Threadripper Pro (single socket version of EPYC Rome) so this is the obvious comparison to make. Read on to see how the numerical computing testing went!
Self Contained Executable Containers Using Enroot Bundles
Written on July 14, 2021 by Dr Donald KinghornNVIDIA Enroot has a unique feature that will let you easily create an executable, self-contained, single-file package with a container image AND the runtime to start it up! This allows creation of a container package that will run itself on a system with or without Enroot installed on it! "Enroot Bundles".
NVIDIA 3080Ti Compute Performance ML/AI HPC
Written on June 18, 2021 by Dr Donald KinghornFor computing tasks like Machine Learning and some Scientific computing the RTX3080TI is an alternative to the RTX3090 when the 12GB of GDDR6X is sufficient. (Compared to the 24GB available of the RTX3090). 12GB is in line with former NVIDIA GPUs that were "work horses" for ML/AI like the wonderful 2080Ti.
Outstanding Performance of NVIDIA A100 PCIe on HPL, HPL-AI, HPCG Benchmarks
Written on May 21, 2021 by Dr Donald KinghornThe NVIDIA A100 (Compute) GPU is an extraordinary computing device. It's not just for ML/AI types of workloads. General scientific computing tasks requiring high performance numerical linear algebra run exceptionally well on the A100.
Run "Docker" Containers with NVIDIA Enroot
Written on May 11, 2021 by Dr Donald KinghornEnroot is a simple and modern way to run "docker" or OCI containers. It provides an unprivileged user "sandbox" that integrates easily with a "normal" end user workflow. I like it for running development environments and especially for running NVIDIA NGC containers. In this post I'll go through steps for installing enroot and some simple usage examples including running NVIDIA NGC containers.
Intel Rocket Lake Compute Performance Results HPL HPCG NAMD and Numpy
Written on March 31, 2021 by Dr Donald KinghornThe new Intel Rocket Lake CPUs have been officially released. There were numerous posts and reviews before the official release date of March 30 2021, but I haven't seen anything about the numerical compute performance. I've had access to a Core-i9 11900KF 8-core CPU and have compared it with (my own) AMD 5800X system.
AMD Threadripper Pro 3995x HPL HPCG NAMD Performance Testing (Preliminary)
Written on March 5, 2021 by Dr Donald KinghornThreadripper Pro! AMD has released the long awaited Threadripper Pro CPUs. I was able to spend a (long) day (and night) running compute performance testing on the flagship 64-core TR Pro 3995WX. In this post I've got some HPC workload benchmark results from putting this excellent CPU through its compute paces.
Intel oneAPI AI Analytics Toolkit -- Introduction and Install with conda
Written on February 17, 2021 by Dr Donald KinghornI recently wrote a post introducing Intel oneAPI that included a simple installation guide of the Base Toolkit. In that post I promised a follow-up about the the oneAPI AI Analytics Toolkit. This is it! I'll describe what it is and give recommendations for doing an install setup of the AI toolkits using conda with Anaconda Python.
Intel oneAPI Developer Tools -- Introduction and Install
Written on February 3, 2021 by Dr Donald KinghornIntel oneAPI is a massive collection of very high quality developer tools, and, it's free to use! In this post I'll give you a little background on what oneAPI is and my recommendations for doing an install setup to get started exploring the collection of tool-kits.
How To Install TensorFlow 1.15 for NVIDIA RTX30 GPUs (without docker or CUDA install)
Written on December 9, 2020 by Dr Donald KinghornIn this post I will show you how to install NVIDIA's build of TensorFlow 1.15 into an Anaconda Python conda environment. This is the same TensorFlow 1.15 that you would have in the NGC docker container, but no docker install required and no local system CUDA install needed either.
Quad RTX3090 GPU Power Limiting with Systemd and Nvidia-smi
Written on November 24, 2020 by Dr Donald KinghornThis is a follow up post to "Quad RTX3090 GPU Wattage Limited "MaxQ" TensorFlow Performance". This post will show you a way to have GPU power limits set automatically at boot by using a simple script and a systemd service Unit file.
Quad RTX3090 GPU Wattage Limited "MaxQ" TensorFlow Performance
Written on November 13, 2020 by Dr Donald KinghornCan you run 4 RTX3090's in a system under heavy compute load? Yes, by using nvidia-smi I was able to reduce the power limit on 4 GPUs from 350W to 280W and achieve over 95% of maximum performance. The total power load "at the wall" was reasonable for a single power supply and a modest US residential 110V, 15A power line.
RTX3070 (and RTX3090 refresh) TensorFlow and NAMD Performance on Linux (Preliminary)
Written on October 29, 2020 by Dr Donald KinghornThe GeForce RTX3070 has been released. The RTX3070 is loaded with 8GB of memory making it less suited for compute task than the 3080 and 3090 GPUs. we have some preliminary results for TensorFlow, NAMD and HPCG.
Note: Adding Anaconda PowerShell to Windows Terminal
Written on October 1, 2020 by Dr Donald KinghornWhen you install Miniconda3 or Anaconda3 on Windows it adds a PowerShell shortcut that has the necessary environment setup and initialization for conda. It's listed in the Windows menu as "Anaconda Powershell Prompt (Anaconda3)". However, this opens a separate/detached PowerShell instance and it would be nice to have this as an optional shell from Windows Terminal! In this post we will add that functionality as a new shell option in Windows Terminal.
RTX3090 TensorFlow, NAMD and HPCG Performance on Linux (Preliminary)
Written on September 24, 2020 by Dr Donald KinghornThe second new NVIDIA RTX30 series card, the GeForce RTX3090 has been released. The RTX3090 is loaded with 24GB of memory making it a good replacement for the RTX Titan... at significantly less cost! The performance for Machine Learning and Molecular Dynamics on the RTX3090 is quite good, as expected.
RTX3080 TensorFlow and NAMD Performance on Linux (Preliminary)
Written on September 17, 2020 by Dr Donald KinghornThe much anticipated NVIDIA GeForce RTX3080 has been released. How good is it with TensorFlow for machine learning? How about molecular dynamics with NAMD? I've got some preliminary numbers for you!
Does Enabling WSL2 Affect Performance of Windows 10 Applications
Written on July 17, 2020 by Dr Donald KinghornWSL2 offers improved performance over version 1 by providing more direct access to the host hardware drivers. Recent "Insider Dev Channel" builds of Win10 even allows access to the Windows NVIDIA display driver for GPU computing applications for WSL2 Linux applications! The performance improvements with WSL2 are largely because this version is running as a privileged virtual machine on to of MS Hyper-V. This means that at least low level support for the Hyper-V virtualization layer needs to be enabled to use it. In particular, the Windows feature "VirtualMachinePlatform" must be enabled for WSL2. We tested to see if there was any negative application performance impact.
Note: How To Install JupyterLab Extensions (Globally for a JupyterHub Server)
Written on July 15, 2020 by Dr Donald KinghornThe current JupyterHub version 2.5.1 does not allow user installed extension for JupyterLab when it is being served from JupyterHub. This should be remedied in version 3. However, even when this is "fixed" it is still useful to be able to install extensions globally for all users on a multi-user system. This note will show you how.
Note: How To Copy and Rename a Microsoft WSL Linux Distribution
Written on June 19, 2020 by Dr Donald KinghornWSL on Windows 10 does not (currently) provide a direct way to copy a Linux distribution that was installed from the "Microsoft Store". The following guide will show you a way to make a working copy of an installed distribution with a new name.
Note: Self-Signed SSL Certificate for (local) JupyterHub
Written on May 7, 2020 by Dr Donald KinghornIn this note I'll go through creating self-signed SSL certificates and adding them to a JupyterHub configuration running on a LAN or VPN. This will allow encrypted access to the server using https in a browser.
Note: JupyterHub with JupyterLab Install using Conda
Written on April 17, 2020 by Dr Donald KinghornThis is a quick note about setting up a JupyterHub server and JupyterLab using conda with Anaconda Python.
HPC Parallel Performance for 3rd gen Threadripper, Xeon 3265W and EPYC 7742 (HPL HPCG Numpy NAMD)
Written on April 9, 2020 by Dr Donald KinghornOn March 19, 2020 I did a webinar titled, "AMD Threadripper 3rd Gen HPC Parallel Performance and Scaling ++(Xeon 3265W and EPYC 7742)" The "++(Xeon 3265W and EPYC 7742)" part of that title was added after we had scheduled the webinar. It made the presentation a lot more interesting than the original Threadripper only title! This is a follow up post with the charts and plots of testing results presented in that webinar.
Threadripper 3990x vs 3970x Performance and Scaling (HPL, Numpy, NAMD plus GPUs)
Written on March 6, 2020 by Dr Donald KinghornIs 32-cores enough? I had some testing time again on an AMD Threadripper 32-core 3970x and thought it would be interesting to compare that to the 64-core 3990x. In this post I take a comparative look at parallel performance and scaling for HPL Linpack, Python numpy and the NAMD molecular dynamics program.
Threadripper 3990x 64-core Parallel Scaling
Written on February 25, 2020 by Dr Donald Kinghorn64 cores is a lot of cores! How well will parallel applications scale on that many cores? The answer, of course, is, it depends on the application. In this post I look at Amdhal's Law parallel scaling for HPL Linpack, Python numpy and the NAMD molecular dynamics program.