Agisoft PhotoScan 1.4.1 - Multi GPU Scaling

Always look at the date when you read an article. Some of the content in this article is most likely out of date, as it was written on May 2, 2018. For newer information, see our more recent articles.

Table of Contents

Introduction

PhotoScan is a photogrammetry program: an application that takes a set of images and combines them to create a 3D model or map. This article is part of a series looking at how different aspects of computer hardware affect PhotoScan performance. For more information on this software, or to see the other entries, check out our introductory article.

PhotoScan makes use of the video cards in a computer to assist with the computation of certain steps. As such, both the model of video card used and the number of them present in a system can have an impact on the amount of time those steps take. In this article, we take a look at how multiple GeForce GTX 1080 Ti cards scale in performance across a few CPU platforms – different chipsets supporting up to 2, 3, or 4 GPUs.

Methodology and Test Hardware

The primary GPU accelerated steps within PhotoScan are aligning photos and building a dense point cloud (a 3D representation of physical points that are seen from multiple angles in different photos). We have already looked at how different GPU models compare in these calculations, so now we are going to take the top-performing card from that roundup – the GeForce GTX 1080 Ti – and stack multiple cards in a single system to see how they scale. Different CPUs and chipsets also support different numbers of video cards, so we will look at 1-2 cards on Z370 (Intel Coffee Lake), 1-3 cards on X299 (Intel Core X), and 1-4 cards on X399 (AMD Threadripper). The goal here is not to find the best CPU for PhotoScan, which will be the focus of a future article, but instead to see how much performance gain there is from each additional card. In keeping with the rest of this series, all tests were conducted with "High" quality settings.

If you would like more details about the full hardware configurations we tested on, and the image sets we used within PhotoScan, simply click here to expand the following section.

Testing Hardware
Motherboard:	Gigabyte Z370 AORUS 5	Gigabyte X299 Designare	Gigabyte X399 AORUS 7
CPU:	Intel Core i7 8700K 3.7GHz (4.7GHz Turbo) Six Core	Intel Core i9 7960X 2.8GHz (4.4GHz Turbo) Sixteen Core	AMD Threadripper 1950X 3.4GHz (4GHz Boost) Sixteen Core
RAM:	4x Crucial DDR4-2666 16GB (64GB total)		8x Crucial DDR4-2666 16GB (128GB total)
GPU:	1 – 4 x NVIDIA GeForce GTX 1080 Ti 11GB
Hard Drive:	Samsung 960 Pro 1TB M.2 PCI-E x4 NVMe SSD
OS:	Windows 10 Pro 64-bit
PSU:	Antec HPC Platinum 1000	EVGA SuperNova 1000W P2	EVGA SuperNova 1600W P2
Software:	Agisoft PhotoScan 1.4.1

Image Sets (from PhotoScan website)
Monument (32 photos)	Building (50 photos)

Results

Here are results for 1 – 2 GPU scaling on the Z370 platform, with an Intel Core i7 8700K processor:

And here are results for 1 – 3 GPUs on the X299 platform, with Intel's Core i9 7960X:

And finally, here are the results for 1 – 4 GPUs on the X399 platform, using AMD's Threadripper 1950X processor:

Analysis

We can see in the graphs above that there is a substantial improvement in performance when going from one to two GPUs, on all three of these platforms. There is a smaller, but measurable, increase with a third card as well. However, there is almost no change going from three to four video cards on the Threadripper system (the only one we tested here which supported that many video cards). That seems to be pretty straightforward, though having seen these results I would like to test another quad-GPU system to see if the trend holds. This will likely happen when we get to testing dual CPU systems with Intel's new Xeon Scalable processors, which I hope will be soon.

One thing to keep in mind with the results shown above is that we did our tests at "High" quality. There is an option above that for building the dense cloud, "Ultra High", which will take substantially longer. Many users might also work with larger image sets, which also take more processing time. Because of that, the performance difference between video cards will vary somewhat based on your exact workload – but in general, having more video cards (up to 3 at least) should have a measurable impact on PhotoScan calculation times.

Conclusion

If you run PhotoScan with either High or Ultra High quality in the Build Dense Cloud step, on a single socket CPU workstation, then having 2 or possibly 3 video cards will be worthwhile. The fourth video card in our tests did not substantially impact performance, and even the third card is debatable: I would say it is probably more important if you like to build the dense cloud at Ultra High settings, since that substantially increases the processing time and therefore could benefit more from the time savings of a third GPU.