Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/1472
Article Thumbnail

Metashape 1.5.1: NVIDIA GeForce, Titan, and AMD Radeon Performance Comparison

Written on June 11, 2019 by William George
Share:

Introduction

Metashape, formerly known as PhotoScan, is a photogrammetry program that takes a set of images and combines them to create a 3D model or map. Processing of those images into point clouds and 3D meshes/textures is time-consuming, heavily using a computer's CPU and GPU. We have updated our benchmark tools, following the recent rebranding of this software, and recently published a CPU comparison article. Next up, we are taking a look at how different AMD and NVIDIA video cards perform in this application.

Test Hardware

Here is a list of the hardware we tested Metashape on. We selected the Intel Core i9 9900K for the processor in this system since it topped the results in our recent CPU roundup. The other specs were kept the same across all test runs, to ensure a fair comparison.

Benchmark Details

We have improved our benchmarking for Metashape compared to previous articles. Now we have four image sets that we own the rights to, covering both smaller and larger size Model and Map projects. The smaller image sets are included in our public Metashape benchmark, which you can download and run if you want to compare your system's performance to what we measured in our testing.

  • Rock Model - 36 photos at 20 megapixels each
  • School Map - 51 photos at 18 megapixels each
  • School Model - 439 photos at 18 megapixels each
  • Park Map - 792 photos at 18 megapixels each

We also split our "model" testing into two branches: one using the building of a dense point cloud to creating the model / mesh, and the other skipping that step and using just the depth maps for mesh creation. These methods utilize the hardware in a computer differently, and we wanted to reflect both approaches in our benchmark. This is also why we ran Metashape 1.5.1 instead of the newer 1.5.2: it was reported to us by a reader than 1.5.2 has some odd behavior with depth maps, and we didn't want that to affect these results.

Each image set was processed 2-3 times on each video card, and the fastest overall result was used for the comparisons below.

Results

Here are charts for each of the six test projects, showing the total time (in seconds) they took to process on each GPU. The video cards are listed in the same order on all charts, regardless of performance, to make it easy to see how they stack up. AMD models are shown in red, with NVIDIA in green, and the fastest result on each project is highlighted (bold) to ensure it stands out from the crowd.

For those who want to dig further into the differences in how each GPU performed, here is a table showing the times for each step within Metashape on each of the image sets:

Metashape 1.5.1 NVIDIA GeForce, Titan, and AMD Radeon Performance Table

Multi GPU Performance

What about using two GPUs instead of one? We did a brief test on that by adding a second RTX 2080 Ti to our test system:

Metashape 1.5.1 Multi GPU Scaling with GeForce RTX 2080 Ti

Analysis

Across most of our testing, there is a clear but small increase in performance as you move to higher-end video cards - both on the AMD and NVIDIA sides of the spectrum. However, it isn't a massive difference. With one exception (the School Model when using Depth Maps to build the mesh) the difference between NVIDIA's top-end Titan RTX and mid-range GTX 1660 Ti was only around 10%, and those cards retail for $2500 versus only ~$300! Unless you need every ounce of speed you can get, the highest-end video cards are overkill.

In some situations, a strong alternative to those individual top-end cards is a pair of less expensive models. Specifically, if you are working with models (rather than maps) and plan to calculate meshes using depth maps then you should see a nice boost (~5-20%) from going with two GPUs. The impact on other types of projects appears to be negligible, though, so it isn't the best option for everyone.

Problems with AMD Radeon Cards

While they did turn in reasonable performance in a couple of tests, there is some additional information worth mentioning about AMD's Radeon GPUs in our Metashape testing. Every test run with NVIDIA cards went perfectly, but we ran into at least one crash with each of the three Radeon cards we tested. Specifically, all three crashed in one of the Park Map runs - during the Build Depth Maps step. Thankfully we looped the test three times, so we had two successful runs for each card from which to gather data for this article, but the fact that we encountered such crashes at all - and especially that they happened on all three of the AMD Radeon cards - is a red flag.

The Radeon RX Vega 64 also had one other problem, where during a test run (I cannot remember on which project) it kept processing for over 13 hours. I eventually had to terminate the test manually, and I did notice an OpenCL error when skimming through the Metashape log afterward. That sort of error only happened the one time, but it delayed getting this article published by a full day.

Encountering such delays in a real work environment would be bad, so we cannot recommend AMD cards for Metashape at this time.

Conclusion

For the most part, simply selecting the NVIDIA GeForce card which best matches your budget is the way to go for Metashape.

If you know that your work will heavily involve calculating meshes based on depth maths, then consider dual GPUs as a way to boost performance - but keep in mind that not all projects will see a big benefit from going that way. And for now, avoid AMD Radeon cards.

Relative Performance of NVIDIA Graphics Cards in Metashape 1.5.1

In case you missed it, you can check out our recent CPU performance article. We also make our smaller image sets publicly available to test via our Metashape benchmark, so that you can compare your own system's performance to what we have measured in our articles.

Looking for a
Photogrammetry Workstation?

Puget Systems offers a range workstations tailored for Pix4D, Metashape, and RealityCapture. Even in the most demanding situations, our workstations and designed to minimize downtime and allow you to work as efficiently as possible.

Tags: Metashape, Performance, AMD, Radeon, Radeon Vega 64, Radeon VII, RX 580, NVIDIA, GeForce, Titan RTX, RTX, RTX 2060, RTX 2070, RTX 2080, RTX 2080 Ti, 1660 Ti, GPU
Mort

Great article!

Your older multi-GPU tests showed a larger performance gain going from 1-2 GPU than was seen in this test.
(https://www.pugetsystems.co...

I am curious the comparison between 1x Titan RTX vs dual GTX 1660 Ti? In other words, is the performance gain from dual GPU cards greater for lower-end GPU (for a given CPU)?

Thanks again.

Posted on 2019-11-06 17:56:58

I was surprised by the difference in relative GPU performance too - both between models and one vs two cards. I haven't had a chance to revisit that again, but I suspect that under-the-hood stuff must have changed from version 1.4.x to 1.5.x (which also coincided with the name change from PhotoScan to Metashape). It is also very likely that multi-GPU is more helpful with lower-end cards than with higher-end models, but again I haven't had a chance to go back and check that yet.

Posted on 2019-11-11 21:55:21
Ryan

Excellent article, I too have issues with Metashape Pro while using my Radeon VII. The only stable Adrenaline drivers that work for me are the 19.11.1.

Instead of using the Adrenaline drivers I am using the Radeon Pro SSG drivers. No crashing or hanging for me since I switched. Currently using the 20.Q1 version.

Posted on 2020-02-18 04:45:27
Steva

Hi, I analyzed the benchmark results in this article and it seems that Metashape/PhotoScan is not using the full power of GPUs?.
From your summary report it can be seen that high-end Titan RTX performs only 10% better than mid-range GTX 1660 (while it was 2x faster on external generic GPU benchmarks https://gpu.userbenchmark.c... )?
Is it due to only 10% utilization of GPU power, compared to >95% CPU utilization, that we noticed during our tests? Or we maybe missed some configuration details for GPU full power utilization?

Posted on 2020-02-18 14:05:46

I'm not certain where the bottleneck lies, but it does seem that newer versions of Metashape see less benefit from faster video cards (or more of them) compared to older versions. But that could also be due to newer cards simply being faster, rather than actual changes on the processing side. I am not aware of any configuration options to select, though, other than making sure the video card(s) are checked in the preferences section and that the option to use the CPU during GPU processing is un-checked (de-selected, as it seems to have a negative affect on performance overall... at least in my experience).

Posted on 2020-02-18 17:03:45
Oteph

You dataset is simply to small. The issue is that Metashape splits the datasets into batches. If the batches are to small the gpu will not be utilized properly.
Also multiple gpus will not be utilized at all. If you increase the img count by a factor of 10, this would probably be enough.

This is also the reason why high thread count cpus underperform. If you have a Threadripper with 64 Threads but you only have 20 images, most cores wont have any jobs for very long stretches in the pipeline.

The tests as they are right now are very misleading, as hardly anyone ever has a dataset this small in real life.

Posted on 2020-06-17 16:44:34

The image sets we test range from 36 to 792 photos. These sizes were selected to provide both tests that could finish in a matter of minutes, for quick testing, as well as tests that took a few hours for more thorough analysis. Going for something 10 times larger than our largest data set would result in literal days of processing, which would greatly hamper the number of configurations we could test for a given article. I have done some limited testing with image sets in the 1,000 to 2,000 photo range, in some other photogrammetry applications, and didn't see much difference in their behavior compared to our largest projects. Sometimes the really high core count processors did benefit a tiny bit with such big image sets, but I don't think that would be the case in Metashape given how it appears to behave.

Out of curiosity, what size of image set would you say is typical in your experience? Most of the time when I've had people talk to me they're working with things in the multiple hundreds to a few thousands of images.

Posted on 2020-06-17 17:05:46
Oteph

I actually have so say that i was under the assumption that you datasets are still the same as the last time i read one of your test (2018 i think). Sorry for that.
The tests as they are now are probably fine for most people. Especially the two big ones. Our datastes are much larger, 2000 images at 100MP, but that's certainly not common.

I wonder what they messed up within metashape so that the scaling we saw with your old tests went away completely. Also for the Cpu test it seems like single thread performance can predict nearly all of the variability of the test scores.

The fact that the large core count threadrippers, as well as Multi GPU are this bad (and that running multiple MS Instances is bad as well), is also a pretty clear indication of some fundamental issues within metashape. (Frequent cache flushes for example could be a cause, High interrupt load on Core0 and so on).

So benchmarking different ram speed might be great.
Taking an i9 10900 with different disabled core counts would also be really interesting. It might even be the case that an i9 with some cores disabled is actually faster, then one with all of them enabled.

Posted on 2020-06-17 18:19:10

I am actually doing some testing right now in all of our photogrammetry applications (Metashape, Pix4D, and RealityCapture) to look at performance with HT (for Intel) and SMT (for AMD) enabled vs disabled. I have done some testing like that before, when the 3990X came out, because of concerns that having 128 threads available might actually hurt performance more than it helped... and indeed, SMT off on that CPU was faster than with it on in some of these applications. I am testing this time with a few more processors: the 9900K (the 10900K we have was busy in other testing, sadly), the 10980XE, the 3950X, and the 3970X - basically one sample each from Core, Core X, Ryzen, and Threadripper. I should have the results in a couple of days and compiled into articles next week :)

Posted on 2020-06-17 18:27:18
Oteph

Great! Im looking forward to that!

Posted on 2020-06-18 10:31:57