Puget Systems print logo
https://www.pugetsystems.com
Read this article at https://www.pugetsystems.com/guides/1458
Article Thumbnail

RealityCapture Benchmark

Written on May 3, 2019 by William George
Share:

CapturingReality Logo (all rights to this image belong to CapturingReality, makers of RealityCapture)

Here at Puget Systems, we have put together a benchmark utility for RealityCapture which measures system performance by running two small projects - a 3D Model and a 3D Map - and tracking the time taken to process each step. This benchmark is freely available to download, below, though running it requires an existing installation and valid license of RealityCapture on a 64-bit Windows computer.

Standard RC Benchmark

Download from MediaFire

Short test - 10 to 30 minutes on most systems

Extended RC Benchmark

Download from MediaFire

Longer test - 1 to 4 hours on most systems

Be sure to check out the full list of benchmarks we have available!

How to Use the RealityCapture Benchmark

Using this benchmark is quite simple, though you do need to have RealityCapture installed and logged in prior to starting.

  1. Download the RealityCapture Benchmark file by clicking on the blue button above
  2. Once downloaded, unzip "Puget Systems RealityCapture Benchmark.zip" to a location where you have write permission
  3. After it has fully extracted, open the Puget Systems RealityCapture Benchmark folder
  4. Run the RealityCaptureBenchmark.exe file
  5. Wait for it to detect the version of RealityCapture that is installed
  6. At the intro screen, read the description and then click on Start to begin
  7. Avoid using the mouse or keyboard during the benchmark execution - changing focus at the wrong time will interfere with it
  8. When the benchmark is complete, a summary screen will be displayed with the time (in seconds) for processing each step
  9. That information will also be saved to a results file in the same folder and uploaded to our website for comparison with others

You can compare the results from running this benchmark on your system to those we show in our RealityCapture articles.

Update Log

5-3-2019: Initial public release
5-23-2019: Update to benchmark, fixing a bug that could cause the script to get stuck locating photos when opening a project file
10-29-2019: Update to benchmark, fixing how settings for log file and cache clearing are configured on first run in newer versions of RC
01-09-2020: Update to benchmark, updating to support latest version (1.0.3.10393 - older versions will not work) and changing how settings are handled (saves current settings to a file, changes them as needed for the benchmark, then restores them at the end); added extended benchmark version for longer test runs
01-24-2020: Minor fixes to improve reliability of the benchmark, particularly the extended version
08-14-2020: Minor updates to support RealityCapture 1.1 Blaze (likely will no longer work with older versions)
10-13-2020: Minor updates to how the benchmark handles RealityCapture not being activated (should be more user friendly)
11-16-2020: Minor bug fix
12-07-2020: Minor updates to support RealityCapture 1.1.1 (likely will no longer work with older versions)
02-02-2021: Changed name to PugetBench for RealityCapture, changed host to MediaFire, and added uploading of results to our website

Tags: RealityCapture, Benchmark, CPU, GPU, Performance, photogrammetry
marik c

Benchmark Started at 05:47 on 05-21-2019
RealityCapture Version: 1.0.3.6310
CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
GPU: NVIDIA GeForce RTX 2080 Ti
Project: Rock Model
Align Images: 24.6
Calculate Model: 98.9
Colorize: 20.9
Texture: 371.7
Total: 516.1 seconds
Project: School Map
Align Images: 34.5
Calculate Model: 193.4
Colorize: 29.3
Texture: 204.8
Ortho Projection: 140.2
Total: 602.2 seconds
Benchmark Completed at 06:10

Posted on 2019-05-21 19:57:28

Thank you for sharing your results! :)

Posted on 2019-05-21 20:01:29
Milos Lukac

Hi Villliam
Would it be possible that you make some sort of Google DOC where can people post the results, so you get much bigger test base and result page ?

If YES i can post it in our 3D scanning forum + few other with approx 50k+ user base...

Posted on 2019-06-02 12:04:21

At some point I would like to build a section on our website where folks could upload results and compare them, but that is probably a long way off at this point. I could see a Google doc as a stop-gap measure until such a portal was ready... I will give that some more consideration.

Posted on 2019-06-03 16:37:16
Lorenz

Hi, I created a Google Form to collect the benchmark data and I'm working now on the automatic evaluation of them. Can I post the link to the form here? and in the forum of the different communities? All the data will be published later when I nicely organized it an will be public available. Is that fine? @William contact me otherwise on Twitter

Posted on 2019-06-17 08:58:17

I am fine with that, but just FYI Milos Lukac has also talked about setting up something similar. Anyone is welcome to use our benchmark tests & results, as long as you comply with the licensing terms... which basically just ask that you don't change the benchmark and that you cite us as the source and link to us. Should be pretty straightforward, but feel free to ask questions here, email me (william [at] pugetsystems [dot] com) or message me on Twitter (@PSWilliamGeorge).

Posted on 2019-06-17 16:37:48
Lorenz

Hi William, Thank you for the answer! Then I will post the Link here. Didn't ask Milos you for setting up a Google Docs? :D
With the citing I do for sure! Thank you very much!

Posted on 2019-06-17 20:23:38
marik c

Benchmark Started at 13:02 on 05-21-2019
RealityCapture Version: 1.0.3.6310
CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
GPU: NVIDIA GeForce RTX 2080 Ti
GPU: NVIDIA GeForce GTX 1080 Ti
Project: Rock Model
Align Images: 12.8
Calculate Model: 44.4
Colorize: 15
Texture: 376.7
Total: 448.9 seconds
Project: School Map
Align Images: 17.8
Calculate Model: 80.3
Colorize: 19
Texture: 199.6
Ortho Projection: 139.4
Total: 456.1 seconds
Benchmark Completed at 13:22

Posted on 2019-05-21 20:25:23

Nice - so it looks like adding the second GPU substantially sped up image alignment, model calculation, and colorization - but not making the texture or ortho projection. That is good to know ahead of my planned multi-GPU testing :)

Posted on 2019-05-22 20:05:17

I'm starting to do multi-GPU testing in RealityCapture, and when going from one RTX 2080 Ti to two I did not see as big of an improvement in performance as you got in the results above. I only got a 5-10% reduction in processing time, while your results look like 10-20% or maybe even a little more.

Is there somewhere in RealityCapture's settings that I need to ensure both GPUs are enabled / selected? Or anything else you know of that I could be overlooking?

Posted on 2019-05-28 19:49:13
marik c

You need to enable the 2nd GPU in the reconstruction settings, but I think that's the only place such an option shows up strangely enough. I'd keep an eye on performance monitor CUDA to make sure it runs on both cards at the same time. My CPU is also at 5ghz all cores if that helps with overhead,

EDIT: Also I am kind of "cheating" extra pcie lanes. I have the full x16 for the 2080ti, and then up to 4x lanes from the PCH as the 1080ti is plugged in egpu thunderbolt style. RC seems to like very fast nvme A LOT though, so I'm not sure if competing with those PCH lanes is good or bad for me. I'll try a few more weird setups and post results.

Posted on 2019-05-29 01:52:05

Okay, I did enable that one setting for the second GPU - I just wasn't sure if there were other places it needed to be turned on as well, etc. Thank you for confirming :)

Thinking through the PCI-E lanes and such, the system I have been using would be x8 / x8 with two video cards installed, both coming from the CPU, while the NVMe is x4 coming (I think) from the PCH. The CPU isn't quite as fast as yours, topping out at 4.7GHz all-core (i9 9900K at stock speeds). Maybe that extra headroom lets the GPUs flex more? Or maybe somehow the 1080 Ti is behaving differently / better than a second 2080 Ti? Hmm, I'll do some more testing and see what I can find.

Posted on 2019-05-29 16:04:56
marik c

Ok silly me, I did not clear the RC's cache between runs which is what caused the performance jumps. It seems an egpu actually slows the benchmark probably due to overhead:

Benchmark Started at 09:14 on 05-29-2019
RealityCapture Version: 1.0.3.6310
CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz (5ghz)
GPU: NVIDIA GeForce RTX 2080 Ti (internal x16 to cpu)

GPU: NVIDIA GeForce GTX 1080 Ti (thunderbolt x4 to pch)

Project: Rock Model
Align Images: 24.7
Calculate Model: 93
Colorize: 21.2
Texture: 379.7
Total: 518.6 seconds
Project: School Map
Align Images: 34.7
Calculate Model: 171.5
Colorize: 29.6
Texture: 210
Ortho Projection: 167.2
Total: 613 seconds
Benchmark Completed at 09:37

RC's performance is still a mystery to many (if not all?!). I am pretty confident that the bulk of gpu impact is seen during the model reconstruction, however even then it is not much until you get to large models in highest quality. I notice during calculate model the gpu's CUDA work shoots up to 100%, stays there until the step is around halfway done, then the gpu does some copy work and the latter half is done all by the CPU.

I think to test multi GPU benefit you would need to bias a benchmark solely towards the model reconstruction, on high quality, with settings tuned for higher quality still. The CUDA portion of the work is done so quickly, there has to be a lot of it to do to benefit from multi gpu.

Posted on 2019-05-29 19:34:35

Fascinating... though now you have me worried, since I wasn't aware that RC cached data that wasn't cleared by simply exiting the program and restarting it (which should happen between every processing step in the benchmark). How is the cache manually cleared in RC? I should do some testing to make sure my own results are not being impacted by that :/

Speaking of my own testing, internally I do use a couple of larger image sets that wouldn't be feasible to include in the public benchmark (too large and too time consuming). They are still run at the same quality settings as everything in the public tests, just with many more photos. If you are interested, check out the article I published yesterday and feel free to leave feedback :)

https://www.pugetsystems.co...

Posted on 2019-05-29 20:05:08
marik c

Great article, makes the mystery deeper for me; so RC doesn't care much for extra cpu cores, uses CUDA for only a few steps in the process, and it barely even uses RAM! Yet it is hard to deny the performance is exceptional.

I did optimize my RC in the past so the caching may be off by default. It's in Workflow > Settings > Clear Cache. Also since a lot of work is done out of memory, I noticed a good speed boost by setting my cache location to a fast nvme that is external from OS.

Posted on 2019-05-30 05:39:14

Hmm, it looks like the "Clear cache on exit" option is set to False by default... I wonder if that may have impacted my results at all. I'm going to do a quick single-pass through the full RC benchmark with that on False and then again with it on True, and see if that makes any difference. If it does, I may need to redo the testing for this article :/

Posted on 2019-05-31 17:12:40

Okay, here are results from runs with "Clear cache on exit" off (false) and then on (true):

OFF
-----System Specs-----
Computer Name: DESKTOP-10G9R51
CPU: Intel(R) Core(TM) i9-9980XE CPU @ 3.00GHz
GPU: NVIDIA GeForce RTX 2080 Ti (26.21.14.3039)
RAM: 128GB
Storage: NVMe Samsung SSD 960 SCSI Disk Device (C:)
OS Version: Windows 10 Pro (Version 1809)
----------
RealityCapture 1.0.3.6310,Rock Model,Align Images,14.4,seconds
RealityCapture 1.0.3.6310,Rock Model,Calculate Model,132.4,seconds
RealityCapture 1.0.3.6310,Rock Model,Colorize,11.6,seconds
RealityCapture 1.0.3.6310,Rock Model,Texture,117,seconds
RealityCapture 1.0.3.6310,Rock Model,Total,275.4,seconds
RealityCapture 1.0.3.6310,School Map,Align Images,15.6,seconds
RealityCapture 1.0.3.6310,School Map,Calculate Model,236.3,seconds
RealityCapture 1.0.3.6310,School Map,Colorize,17.9,seconds
RealityCapture 1.0.3.6310,School Map,Texture,166.2,seconds
RealityCapture 1.0.3.6310,School Map,Ortho Projection,46.5,seconds
RealityCapture 1.0.3.6310,School Map,Total,482.5,seconds
RealityCapture 1.0.3.6310,School Model,Align Images,99.4,seconds
RealityCapture 1.0.3.6310,School Model,Calculate Model,1324.8,seconds
RealityCapture 1.0.3.6310,School Model,Colorize,99.9,seconds
RealityCapture 1.0.3.6310,School Model,Texture,618.5,seconds
RealityCapture 1.0.3.6310,School Model,Total,2142.6,seconds
RealityCapture 1.0.3.6310,Park Map,Align Images,315.8,seconds
RealityCapture 1.0.3.6310,Park Map,Calculate Model,2571,seconds
RealityCapture 1.0.3.6310,Park Map,Colorize,240.8,seconds
RealityCapture 1.0.3.6310,Park Map,Texture,1638.8,seconds
RealityCapture 1.0.3.6310,Park Map,Ortho Projection,166.9,seconds
RealityCapture 1.0.3.6310,Park Map,Total,4933.3,seconds

ON (and manually cleared the cache prior to starting, just to be safe)
-----System Specs-----
Computer Name: DESKTOP-10G9R51
CPU: Intel(R) Core(TM) i9-9980XE CPU @ 3.00GHz
GPU: NVIDIA GeForce RTX 2080 Ti (26.21.14.3039)
RAM: 128GB
Storage: NVMe Samsung SSD 960 SCSI Disk Device (C:)
OS Version: Windows 10 Pro (Version 1809)
----------
RealityCapture 1.0.3.6310,Rock Model,Align Images,15.3,seconds
RealityCapture 1.0.3.6310,Rock Model,Calculate Model,133.4,seconds
RealityCapture 1.0.3.6310,Rock Model,Colorize,11.6,seconds
RealityCapture 1.0.3.6310,Rock Model,Texture,117.4,seconds
RealityCapture 1.0.3.6310,Rock Model,Total,277.7,seconds
RealityCapture 1.0.3.6310,School Map,Align Images,15.7,seconds
RealityCapture 1.0.3.6310,School Map,Calculate Model,238.4,seconds
RealityCapture 1.0.3.6310,School Map,Colorize,17.8,seconds
RealityCapture 1.0.3.6310,School Map,Texture,166.7,seconds
RealityCapture 1.0.3.6310,School Map,Ortho Projection,46,seconds
RealityCapture 1.0.3.6310,School Map,Total,484.6,seconds
RealityCapture 1.0.3.6310,School Model,Align Images,95,seconds
RealityCapture 1.0.3.6310,School Model,Calculate Model,1318.4,seconds
RealityCapture 1.0.3.6310,School Model,Colorize,99,seconds
RealityCapture 1.0.3.6310,School Model,Texture,620,seconds
RealityCapture 1.0.3.6310,School Model,Total,2132.4,seconds
RealityCapture 1.0.3.6310,Park Map,Align Images,305,seconds
RealityCapture 1.0.3.6310,Park Map,Calculate Model,2565.7,seconds
RealityCapture 1.0.3.6310,Park Map,Colorize,239.3,seconds
RealityCapture 1.0.3.6310,Park Map,Texture,1641.6,seconds
RealityCapture 1.0.3.6310,Park Map,Ortho Projection,166.2,seconds
RealityCapture 1.0.3.6310,Park Map,Total,4917.8,seconds

So basically the same performance both ways, within less than one percent (margin of error). It looks like, at least for my testing here, having that setting in place did not matter. Whew! Glad I don't have to re-test everything :)

Posted on 2019-05-31 22:07:56
marik c

Interesting, and your cache location is set to a persistent disk (not a ram drive)? For me if I run the benchmark twice in a row with cache saving I get the performance boosts as seen earlier, however clearing cache makes the next run results the same as my original single card run.

Posted on 2019-05-31 22:28:48

I didn't alter the location of the cache as far as I can recall, and it looks like it is set to the Windows temp folder.

OOOHHH! I think I know what is going on. Did you move your cache? I'd bet $20 that you moved it from its default location in the Windows temp folder :)

See, I didn't realize there were settings in RealityCapture for cache... but I did notice that it was writing files to the temp folder, so in my benchmark it deletes the folder that RC makes in there at the end of every run. So I am manually clearing the cache, in effect, but only if the user hasn't moved it. Since you did (I assume) move yours, that is why it is impacting the performance if the cache is not cleared between runs from within RC.

Posted on 2019-05-31 22:45:00
marik c

Indeed, I moved my cache to a nvme that is dedicated just for that purpose.

Posted on 2019-05-31 23:38:40

Okay, that all makes sense now :)

I will try and see if I can automate a better way of clearing the cache regardless of its location, so that it makes for a more fair comparison when other people run the benchmark. Thank you so much for your input on this!

Posted on 2019-06-01 04:09:31
Milos Lukac

Have you done the testing even with the W10 scheduler "fix" on the AMD system ? that can dramatically improve the speed..

https://hexus.net/tech/news...

https://www.phoronix.com/sc...

Posted on 2019-06-02 12:15:14

In my recent CPU performance article (https://www.pugetsystems.co... ) I was using AMD's Ryzen Master software and made sure that Dynamic Local Mode was enabled, and all Windows updates were applied, but I was *not* using any additional 3rd-party software to try and improve the scheduling on those chips. Given that higher core count / slower clock speed chips from both AMD and Intel did poorly in RealityCapture, I don't think that improving the behavior of Windows' scheduler would help much. RC seems to really like high clock speed without so much concern for core count.

Posted on 2019-06-03 16:17:55
Milos Lukac

Try it, as i send you some additional info on what is going on there

https://bitsum.com/portfoli...

Posted on 2019-06-06 10:56:50

Unfortunately, it doesn't look like CorePrio helps in photogrammetry applications: https://www.pugetsystems.co...

Posted on 2019-06-11 22:33:12
Milos Lukac

I was in the DEV of RC from ( Not anymore ) the beginning so if you need expertise on how to set up benchmark and etc feel free to PM me over Facebook, i send you PM there already today...

Posted on 2019-06-02 12:39:27

I have updated my benchmark tool to enable the "Clear cache on exit" option within RealityCapture, rather than relying on manually deleting the RC subfolder in the Windows temp folder. Before I put it in the public benchmark .zip file, would you be able to give it a try and make sure it sets that option properly and that performance is (fairly) consistent between runs? If so, drop me an email at william [at] pugetsystems [dot] com and I can send you the .exe file as either an attachment or a Google Drive link.

Posted on 2019-06-10 21:37:27

I've been waiting forever for some benchmarks for RC. I'm curious to know if the GPU is the primary difference in the results shown here or if it's the Intel chip. Here's my results from 2 tests:

Benchmark Started at 10:18 on 05-30-2019
RealityCapture Version: 1.0.3.6310
CPU: AMD Ryzen 7 1700X Eight-Core Processor @ 3.5GHz
GPU: NVIDIA GeForce GTX 1070
Project: Rock Model
Align Images: 30.5
Calculate Model: 168.9
Colorize: 14.7
Texture: 139.7
Total: 353.8 seconds
Project: School Map
Align Images: 38.5
Calculate Model: 347.8
Colorize: 22.3
Texture: 191.2
Ortho Projection: 9.4
Total: 609.2 seconds
Benchmark Completed at 10:39

Benchmark Started at 10:38 on 05-31-2019

RealityCapture Version: 1.0.3.6310
CPU: AMD Ryzen 7 1700X Eight-Core Processor @3.8GHz
GPU: NVIDIA GeForce GTX 1070
Project: Rock Model
Align Images: 29.9
Calculate Model: 165.6
Colorize: 14
Texture: 137.1
Total: 346.6 seconds
Project: School Map
Align Images: 38.8
Calculate Model: 320.2
Colorize: 21.2
Texture: 188.5
Ortho Projection: 9.3
Total: 578 seconds
Benchmark Completed at 10:58

Posted on 2019-06-03 21:46:19

We are putting out some articles looking at CPU and GPU performance using this benchmark (and a couple of larger image sets too, which were too big / time consuming to include in this public test). The first article is out now, looking at some CPUs, and another looking at GPUs will be coming later this week:

https://www.pugetsystems.co...

Also, just a side note, I have observed variance of up to about 10% between benchmark runs on the same hardware - so I usually do 2 or 3 runs and take the fastest result for my articles. However, another reader also helped me uncover the fact that if you have moved the RC cache away from its default location (in the Windows temp folder) then my benchmark does not properly clear the cache between runs, so any after the first run will be artificially faster. I am hoping to address that in the next update.

Posted on 2019-06-03 21:52:00

I saw the same results when moving to the slower clock speed, read the discussion and these results are after clearing the cache. Thx!

Posted on 2019-06-03 21:54:00
aya:Tinkering-man:100%UnOffici

Benchmark Started at 23:43 on 06-06-2019
RealityCapture Version: 1.0.3.6310
CPU: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
GPU: NVIDIA GeForce GTX 1080
Project: Rock Model
Align Images: 39.2
Calculate Model: 760.6
Colorize: 17.3
Texture: 44.2
Total: 861.3 seconds
Project: School Map
Align Images: 59.3
Calculate Model: 2197.6
Colorize: 25.2
Texture: 87.9
Ortho Projection: 9.3
Total: 2379.3 seconds
Benchmark Completed at 00:43

Posted on 2019-06-06 15:48:01
Lorenz

Hi, what about RAM configuration, motherboard, strorage type etc. It would be really nice to include them into the bemchmakr for further investigation.
What do you think?

Posted on 2019-06-10 14:26:52

As I have gotten to better understand RealityCapture, I do think that perhaps recording storage and RAM would be nice... but it gets tricky. With storage in particular, the most applicable parts would be the drive where the projects / images are kept and then also where the cache is located. Here in our Labs department, it is generally all done on a single, very fast NMVe SSD... but in the real world, I bet a number of people have multiple drives in their workstations - and I am not sure how to accurately determine which drives (if there are multiple) are being used for which roles. Without that info, simply having a list of drives would not be very informative.

RAM would be a bit easier to record, though. I am planning on doing some testing soon across different amounts of RAM, and if I do see any difference between various amounts of memory (with all other things being equal) then I will likely add that to the system data that is recorded. If it makes no difference, perhaps I'll skip over it. I guess RAM speed could also be a factor, in some systems... hmm...

I am very much open to suggestions, though, so if you think there are easier ways of recording this info or some aspect of it that I am not considering then please let me know :)

Posted on 2019-06-10 15:43:17
Lorenz

That with the different storage configuration is a good point and probably also really hard to evaluate. Esp. depends also on the free disk space, the defragmentation level etc. So a speedtest upfront would be needed anyway.

But for the RAM it would be definitely interesting what is more beneficial; more RAM or faster and what's the balance of the two w.r.t. efficiency and price (better get more slower RAM vs less fast RAM) - so really looking forward to you RAM test!

And probably you totaly aware of this:https://www.hwinfo.com/down... has a library for in-depth pc components assessment, so if possible would be cool to include this! - esp. for the folks which have there CPU OC ;D

Posted on 2019-06-14 12:16:16
Lorenz

Hi All! For consolidating the results of the benchmark of Puget Systems (! thank you for that !!) I created a google docs and there you can post your results. I will try to make a summary public as fast as possible and if a critical amount submit their results ;D - All the data will be public asap!

Thank you very much!! https://forms.gle/9AAndxszt...

Posted on 2019-06-17 20:26:36
Nino Skupnjak

Little older system :)

Benchmark Started at 09:19 on 06-26-2019
RealityCapture Version: 1.0.3.6310
CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
GPU: Intel(R) HD Graphics 4600 (GPU is actually 1060 6gb)

Project: Rock Model
Align Images: 38
Calculate Model: 746.4
Colorize: 20.5
Texture: 233.8
Total: 1038.7 seconds
Project: School Map
Align Images: 46.5
Calculate Model: 1805.6
Colorize: 29.8
Texture: 305.1
Ortho Projection: 13.9
Total: 2200.9 seconds
Benchmark Completed at 10:20

Posted on 2019-06-26 08:44:38

Thank you for sharing your results :)

Posted on 2019-07-01 16:52:24
Adam

Can you clarify whether the benchmark resets Alignment (and other) settings when it runs the projects? i.e. number of alignment features etc, that may impact the time it takes to align? (possibly the other stages too). Launching the Rock Model project independently for example seems to suggest that the alignment settings it is using to record a time in seconds were the ones I had previously used for a project with considerably high settings.

Posted on 2019-11-21 12:41:59

The only settings the benchmark changes, as far as I can recall, are making sure the log file is enabled and that the cache is cleared on exit (to keep subsequent benchmark runs from being artificially faster).

To be honest, I am not sure what all other settings (there are a lot!) could impact performance, and I didn't want to be changing settings that might mess up what users have configured on purpose. In the next version of the benchmark, I would like to find a better way to handle this... maybe exporting or saving the current settings, then loading in my own (or resetting them to default if that is easier), and then restoring the user's settings again afterward. If anyone knows of an easy way to do that sort of thing within RealityCapture, I would love to hear about it :)

Posted on 2019-11-21 17:59:09
Adam

The only reason I twigged the settings was because my Alignment time was 2-3x higher than other users, which seemed a bit off. However, my alignment settings from a previous project were exceptionally high vs. the defaults, so unless users running the benchmark without 'factory resetting' this will at the very least influence each Alignment stage benchmark. I know when you launch RC you can hold shift + click the launcher/shortcut and then choose at the option menu to reset its options - which would be a way for users to baseline each benchmark, but I don't know how to preserve or export settings unfortunately!

Posted on 2019-11-22 16:38:57

Okay, I'll do some research and see if there is any way to save or export settings. If not, perhaps I'll just put a notice in the start-up screen for the benchmark advising users to reset the options before running it. Thank you for bringing this to my attention :)

Posted on 2019-11-22 16:58:57

So after some more digging, it looks like there isn't just a file where the preferences are saved that I could copy. There is an option for exporting and importing global settings, however, and I think that may work. I am testing some adjustments to my script now to see if it can properly export existing user settings, then import our defaults, run the benchmark, and finally import the saved user settings at the end.

Posted on 2019-12-03 19:04:30

Hi Adam, I was wondering if you'd be open to testing out an updated version of our benchmark? It implements something I described in another recent comment on here: instead of changing settings (and missing some important ones, as you noticed) it now exports the existing global settings and then imports a pre-configured settings file, runs the benchmark, and then restores the original settings. I've tested it here, but before replacing the older version for everyone I'd like to have a couple people try it in the field.

If you are open to doing so, here is the download link from Google Drive:

https://drive.google.com/fi...

Anyone else reading this is welcome to try too, and once I hear back from a couple people (assuming it works / behaves for them) I'll switch the main download link over to this version.

Posted on 2019-12-23 23:08:40
Scott Lee

Anyone else having problems with this tool? I am running Version 1.0.3.10393 CLI

It hangs up after a few seconds here:

https://uploads.disquscdn.c...

Posted on 2020-02-11 14:17:21

Hmm, this benchmark has been updated for version 1.0.3.10393 - but I developed it (and do my testing) on the PPI version. Maybe something behaves differently? I wouldn't think so this early in the test, though.

To give you some background, the first thing that the script does it try to save your existing global settings (so they aren't overwritten) and then load in a pre-set preferences file for use during the benchmark. To do so, since much of the RealityCapture UI cannot be directly addressed by button name, it presses Alt to put the focus in the menu system at the top, and then Tab several times until the Settings button is highlighted (toward the right side of the menu). It then selects that button, and uses some crazy logic to find and access the menu which that button opens in the bottom left corner of the screen. It then navigates through to Export the global settings, followed by Importing the pre-configured Puget settings file... and then continues on from there with the actual processing of the benchmark.

If this is happening just a couple of seconds in, then something in that initial set of keystrokes that is supposed to access the Settings menu seems be going amiss. I wonder if the number of options in that top menu are different in the PPI and CLI versions? That could explain it: if the set number of Tab key presses is causing it to miss the Settings button and instead end up at the main menu button, for example. Could you maybe take a screenshot of RealityCapture itself, running but without the benchmark or any project open? I can then compare that to what I see here.

Posted on 2020-02-11 17:04:08
Lorenz

hi - i have a similar issure with the newest version of 1.2.0.16813 RC. I also have the CLI version, but it deactivated it once, and switched back to the PPI version. No difference. Could you help me do figure it out why it doesn't work? I added the GUI from the two versions and don't see any difference... https://uploads.disquscdn.c... https://uploads.disquscdn.c...
PPI is the version with the open Benchmark-script. How long should the initial startup phase be? Run everything from a NVMe SSD

Posted on 2021-08-19 11:43:20
Mark Egan

Benchmark stops at RC Activation screen after it selects "Change Application Owner" under License settings. Using latest version of RC.

Posted on 2020-08-25 08:51:34

Can you provide any more information? Was it the normal or extended version of the benchmark?

That may not matter, actually... as I think about this, there shouldn't be any changing of license setting stuff anywhere in my scripts. For it to work, though, RC does need to already be logged in to your account before you start the benchmark.

I just tested the benchmark again on RC version 1.1.0.12901, and it seems to work just fine... and from what I can tell it looks like that is still the latest version from their website.

Posted on 2020-08-25 19:15:35
MHaibt

Hi there, I would love to send my benchmarks but the benchmarktool checks for activation too quickly. RC validates activation just milliseconds after the tool sends out its error message. Please correct this bug so I can post my benchmarks with a RC-VM

All the Best and many Thanks!
Max

Posted on 2020-10-11 15:37:21

Hi Max! I will look into lengthening that step and post back here if / when I am able to change it :)

Posted on 2020-10-12 17:11:56
MHaibt

Thanks for quick reply, I will wait patiently.

Posted on 2020-10-12 18:28:38

Hi Max - I adjusted the public benchmark so that instead of closing out if RC is not activated (or seems not to be - if activation is still in progress) it will instead give you a message box asking to activate, and then when you do so and click OK it will continue the benchmark from there. That should resolve the issue you were having, but before I upload the fix could you give it a try? This link will download just the new benchmark .exe file, which you can copy into the benchmark folder (replacing the original). Please run it and let me know if it works better for you :)

https://drive.google.com/fi...

Posted on 2020-10-13 21:01:56
Mhaibt

Hm I cannot see much changes, I wait till RC activates and then I hit OK again, but then it hangs at 'Adjusting Settings' 0%.

Posted on 2020-10-15 17:20:00

How long are you leaving it on Adjusting Settings? It takes a minute or two. Also, can you confirm what version of RealityCapture you are using? This benchmark is only confirmed to work with the latest version (1.1.0.12901). If you are on an older version, that is likely the problem.

Posted on 2020-10-15 17:22:56
Mhaibt

Hi William, I tried it again several times, I downloaded the new exe - but it gets stuck after Activation was registered and I click Okay. Something blinks in the toolbar but 'Adjusting Settings' 0% stays at.

Posted on 2020-10-16 05:29:54

It sounds to me like you may be using an older version of RealityCapture. Between versions 1.0.3xxxx and 1.1.0.12901 (the latest) I had to change something because the position of the Settings button in the toolbar changed slightly. What you are describing sounds exactly like what I would expect if the script isn't going to the right place to open Settings.

Posted on 2020-10-16 05:41:28
Mhaibt

https://uploads.disquscdn.c...

Posted on 2020-10-16 09:55:30

Okay, so you've got the right version... but it looks like you might have a traditional license for it as well? I only have access to the PPI "license", so maybe there is something materially different about how the menu system behaves when you have a full license instead. That could easily explain what is going on here.

What *should* happen, just to give you a rough idea, is that the benchmark starts up, launches RealityCapture, and then once any login stuff is out of the way it should open the Settings menu and then export your current settings (so you don't lose them) and import a pre-defined settings file. This process takes a few minutes, mostly because of a really weird way I had to go about managing the controls since most of the RC user interface cannot be directly accessed via scripts (buttons are not identified in normal ways, for example). So it will look like it is doing nothing for up to 1-2 minutes at a time, but it really is working behind the scenes. I know some folks have stopped the benchmark prematurely in the past because they thought it was stuck (I've even done this a few times myself!) while it was, in fact, still working.

But if this is a difference in the UI when logged in with a real license (instead of just using PPI) then I am not going to be able to do anything to help, unfortunately :/

Posted on 2020-10-16 16:55:38
staus

Hey, just wanna add my few cents to this issue. I have the same problem and have tried on numerous PC's now.
It doesn't matter that I'm logged in in advance, the script seems to click the license button and ask for a new login and then instantly hangs at "Adjusting Settings" even before I get to type in my information. I use PPI too.

I'm gonna record a short clip of the steps when I get home to my desktop.

Posted on 2020-11-10 15:30:40

Thank you for adding your experience! A video clip will be *super* helpful, that is a great idea :)

I just tried again myself, on my home workstation (been working from home a lot lately, as I bet a lot of people are). Installed RC 1.1, logged in as Anonymous, then closed it out. Downloaded, unzipped, and ran the public benchmark (Standard, not Extended). Once I clicked on Start it opened RC, there was no log in window / prompt, and then the processing began as usual. This is how it *should* behave, but obviously it isn't working as expected for everyone... so I'll keep an eye on the comments here to see your clip and try to make more sense of it then. Thanks again!

Posted on 2020-11-10 20:56:09
staus

Thank you for taking the time to look into it. It will be really helpful to have a benchmarker.

I did a recording where I first launch RC and log in, then close it and run the Standard Benchmark:
https://www.dropbox.com/s/t...

Hope it helps

Posted on 2020-11-10 23:42:57

Yes, thank you - that does help! I can see where the problem is occurring, I'll just need to try and figure out what is causing it.

The issue, which you can see starting around timestamp 1:30, is that the script should be opening the Global Settings - in order to save your existing settings and then load in a pre-set for the benchmark. However, instead of opening Global Settings it is opening the section before that, which is Licenses... and that is what is prompting the login screen to come up. Somehow the script is ending up on the right part of Settings when I run it, but the wrong part for (at least some of) our readers. I will try and figure out why that is happening.

Posted on 2020-11-10 23:55:02
staus

Great! Glad I could be of help!

Posted on 2020-11-11 09:04:50

I changed the key commands in the script to simply press "down" one more time, to get to the Global Settings properly, and this didn't seem to break things on my end - so I was wondering if you could try it out on your system? I haven't updated the main downloads yet, but here is a link directly to just the new benchmark exe file; simply download it and put it in the benchmark folder (replacing the old one) and let me know if it works properly for you now :)

https://drive.google.com/fi...

Posted on 2020-11-12 19:26:53
staus

It works perfectly :) Thanks!

Benchmark Started at 17:18 on 11-14-2020
RealityCapture Version: 1.1.0.12901
CPU: Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz
GPU: NVIDIA GeForce RTX 3080
Project: Rock Model
Align Images: 22.7
Calculate Model: 88.8
Colorize: 9.5
Texture: 67.7
Total: 188.7 seconds
Project: School Map
Align Images: 29.6
Calculate Model: 187.1
Colorize: 13.3
Texture: 80
Ortho Projection: 48.8
Total: 358.8 seconds
Benchmark Completed at 17:37

Posted on 2020-11-14 17:55:13

Great, I'm glad that worked! Both of the benchmark links in the article have now been updated to use that fixed file :)

Posted on 2020-11-17 00:10:28
Dimitrios Vogiatzis

Hello William,

I have the same problem with Staus script hangs up at the adjusting settings.

https://prnt.sc/wgx0zx

But I cannot download the linked file from google.

Can you help please?

Posted on 2021-01-05 17:42:46

That file I had linked was integrated into the benchmark after the update it contained proved effective (which is why it cannot be downloaded separately anymore).

I can try and look into this further. May I ask what version of RealityCapture you are running?

Posted on 2021-01-05 18:17:56
Dimitrios Vogiatzis

Hello, thanks for looking into this.

The version I have 1.1.1.14258 RC PPI

Posted on 2021-01-05 19:10:44

Hmm, I just tried that version and our Standard benchmark here on my home system - and it worked. However, that Adjusting Settings part does take a couple of minutes... can you try again, and leave it going for at least 5 minutes there? And if it doesn't work, take a screenshot of what the whole RC window looks like at that point? Thanks :)

Posted on 2021-01-06 03:16:59
Dimitrios Vogiatzis

Hello,

I uninstalled and installed again the last version. Rebooted uninstalled CUDA / Drivers and installed them again.

I run the script and left computer there for 20+ min and nothing happened.

Here is the screenshot. https://prnt.sc/wk7kfw

Posted on 2021-01-10 11:11:56

Thank you! It looks like the script is opening the Settings menu properly, but failing to go quite as far down that menu as it needs to (it stopped at Licenses instead of Global Settings). I thought I resolved that in the last version of the benchmark, at the start of December... hmm.

When you said that you installed the latest version, I assume that is referring to RealityCapture? If so, when (roughly) did you download the benchmark itself? If you have a version from before December 7th, that would be the problem. If you downloaded it after that, then I'll need to look at the script again and try to see why this is still happening for some people :/

Posted on 2021-01-11 19:22:10
Dimitrios Vogiatzis

I cant really remember because I had this from before Christmas... I will download it now and try again. I will update asap. Thanks

Posted on 2021-01-11 21:08:24
Dimitrios Vogiatzis

I tried again with the latest files and nothing changed. Same problem...

Posted on 2021-01-12 00:33:32

Okay, I really don't know what is causing this - and it is made more complicated by the fact that I cannot replicate the problem on my test systems - but I made a small modification to see if I could work around it. Can you try downloading this updated version of the benchmark exe file? It is just one file, not all the projects and stuff. Please put it wherever the existing benchmark files are located, replacing the old version, and let me know if it helps :)

https://drive.google.com/fi...

Posted on 2021-01-14 00:13:24