Puget Systems Benchmark Tool for Pix4DWritten on May 15, 2018 by William George
Here at Puget Systems, one of our primary goals is to make sure that our customers end up with a fast, reliable workstation that is perfectly tailored to their unique workflow. The main way we do this is by benchmarking a wide range of hardware across popular software - then publishing the results and using them to create recommended systems for each application.
However, while this ensures we are selling the right hardware, it does not give our customers a great idea of how much faster a new workstation would be compared to their existing system. To address this issue for Pix4D, we have created a tool that walks users through the same sort of testing we do internally.
How to run the benchmark
Simply download and then run the executable linked to above. It will pop up a series of windows with instructions about what image sets to download, where to find them, and what settings to use when setting up each project in Pix4D. Once the projects have been processed, the tool will analyze the log files and display the results. The log files and results are also saved for future reference.
Depending on the hardware in your system, the process of project creation and processing involved in this benchmark should take between 15 and 60 minutes... possibly even longer on old or low-spec systems. It is not recommended to use your computer for anything else during the benchmark process since other activity would slow down Pix4D and lead to longer calculation times.
This benchmark was designed with Pix4Dmapper Pro 4.2.25, on Windows 10, but it should work with future versions as well - at least as long as the 3D Map and 3D Model templates remain the same, along with the log file formatting. Comparing results across different versions could also be used to measure optimizations made to Pix4D itself, but would not be fair for comparing different hardware.
In order to follow along with the instructions in this tool, you will need to have Pix4Dmapper Pro installed and logged in. If you are new to Pix4D, you can sign up for a 15-day trial on their website which will allow you to run this benchmark.
How to interpret the results
Our benchmark utility walks through the normal steps of creating and processing image sets with two of Pix4D's built-in templates: 3D Model and 3D Map. Fairly small photo sets are used, to keep the calculation times reasonable while still ensuring the results can be extrapolated to larger projects.
The several tasks involved in turning still images into 3D representations are combined by Pix4D into two steps for models or three steps for maps. This benchmark utility pulls the time for each step from the log file that Pix4D creates, displaying those results and recording them to a file for later use. Both the individual steps and the total time are useful when comparing the performance of different systems.
Times shown are in seconds, with lower numbers (shorter processing times) being better. We have not implemented any sort of scoring system at this point since it is fairly simple to just compare the times between different benchmark runs. This way we are also not artificially placing more or less weight on different steps. Please note that both the CPU and GPU in a system affect Pix4D performance.
Sample results for comparison
Here are sample results from workstations built here at Puget Systems in the first few weeks of our using this benchmark internally:
|System Specs||Eagle Model Step 1||Eagle Model Step 2||Eagle Model Total||Building Map Step 1||Building Map Step 2||Building Map Step 3||Building Map Total|
|Core i7 8700K +
GTX 1050 Ti 4GB
|Core i7 8700K +
GTX 1080 8GB
|Core i9 7960X +
GTX 1080 Ti 11GB
|2 Xeon E5-2690v4 + Titan Xp 12GB||148||317||465||183||256||341||780|
Beta Version 2018.05
First release. Manual project creation and processing with automated results analysis. Uses Eagle and Building image sets.