PugetBench for Creators User Guide

This user guide includes details about how to use PugetBench command-line tools as well as our data collection policy, update logs, and more.

Download PugetBench for Creators

Quickly Jump To: Data Collection • Troubleshooting • CLI Guide • Sharing Assets • Update Log • Known Issues • Gallery

PugetBench for Creators is designed to test a range of Content Creation workflows and is ideal for end users, hardware reviewers, press, system integrators, and hardware manufacturers who want to examine the performance of different hardware in real-world workflows. Our benchmarks are easy to install and run, and have licensing options to support command-line automation and help prevent uploading NDA or confidential results.

Each benchmark has a low run-to-run margin of error and is highly repeatable. Results are presented as an Overall Score, as well as a breakdown of performance for individual tests and groups of tests.

Data Collection

When uploading a result to our public database, basic information on hardware specifications and application versions is recorded. All data is completely anonymous, and no personally identifiable information is collected. An example set of raw data is available below:

{
  "results": [
    {
      "test": "Overall Score (Standard)",
      "result": "100",
      "major": 2,
      "setting": "",
      "unit": "",
      "preset": "Standard"
    },
-----Repeated test results truncated-----
  ],
  "benchmark_version_major": 1,
  "benchmark_version_minor": "0.0",
  "benchmark_version_patch": 0,
  "benchmark_version_identifier": "",
  "application_version_major": 24,
  "application_version_minor": 0,
  "benchmark_name": "PugetBench for Premiere Pro",
  "application_name": "Premiere Pro",
  "specs_formatted": [
    {
    "type": "CPU",
    "manufacturer": "AuthenticAMD",
    "name": "AMD Ryzen 7 7700X 8-Core Processor",
    "cores": 8
  },
  {
    "type": "GPU",
    "manufacturer": "Advanced Micro Devices, Inc.",
    "name": "AMD Radeon Graphics",
    "driver": "31.0.23013.1023"
  },
  {
    "type": "GPU",
    "manufacturer": "NVIDIA",
    "name": "NVIDIA GeForce RTX 4080",
    "driver": "31.0.15.2824"
  },
  {
    "type": "Memory",
    "manufacturer": "Kingston",
    "capacity_value": 32,
    "capacity_unit": "GB",
    "speed_value": 5200,
    "speed_unit": "MHz"
  },
  {
    "type": "Memory",
    "manufacturer": "Kingston",
    "capacity_value": 32,
    "capacity_unit": "GB",
    "speed_value": 5200,
    "speed_unit": "MHz"
  },
  {
    "type": "OS",
    "manufacturer": "Microsoft Corporation",
    "name": "Microsoft Windows 11 Pro",
    "version": "22621"
  },
  {
    "type": "System",
    "manufacturer": "ASUS",
    "name": "System Product Name"
  },
  {
    "type": "Motherboard",
    "manufacturer": "ASUSTeK COMPUTER INC.",
    "name": "ProArt X670E-CREATOR WIFI",
    "bios": "1602"
  },
  {
    "type": "Engine",
    "name": "Mercury Playback Engine",
    "value": "CUDA"
  }
  ]
}

Troubleshooting

If you have any issues with the benchmark not properly running, first verify that your system meets the System Requirements (Photoshop | Premiere Pro | After Effects | DaVinci Resolve). After verifying that your system and configuration meet the requirements, there are a number of troubleshooting steps you can try:

• Check the list of Known Issues
• If no application is shown in the drop-down, you most likely do not have a version installed that is supported by the benchmark
• Do not attempt to use the system while the benchmark is running. Defocusing the application window can break the benchmark in many situations.
• Ensure that your OS and application you are testing were installed in English.
• Disable Windows Defender or any real-time virus scanning programs.
• If a test reports that it is taking longer than expected and the base application has not crashed or hung, you can adjust the default timeout in the benchmark settings (gear icon on the bottom-left of the benchmark UI). The default is 300 seconds, or 5 minutes.
• Clear the application preferences
  • Adobe: Follow Adobe’s guides (Photoshop | Premiere Pro | After Effects) or use our Adobe CC Preference Backup/Restore/Clear utility (Windows only).
  • DaVinci Resolve: Deleting the “DaVinci Resolve” folder in “%appdata%\Blackmagic Design”
• Adobe: delete the Media Cache files from the following locations:
  • Windows: C:\Users\[USERNAME]\AppData\Roaming\Adobe\Common
  • MacOS: ~/Library/Application Support/Adobe/Common
• Delete the benchmark assets, then re-download. These assets are stored in the following folder:
  • Windows: C:\Users\[USERNAME]\AppData\Local\com.puget.benchmark\assets
  • MacOS: ~/Library/Application Support/com.puget.benchmark/assets
• Check for a temp folder, and delete it if it is present
  • Windows: C:\Users\[USERNAME]\AppData\Local\com.puget.benchmark\temp
  • MacOS: ~/Library/Application Support/com.puget.benchmark/temp
• Photoshop
  • If you are getting an error about a PSD being too large, check that “Disable Compression of PSD and PSB Files” is unchecked in “Edit -> Preferences -> File Handling”

Lower-than-expected performance can be caused by a number of factors, but the most common are:

• Unknowingly comparing scores to different benchmark and/or application versions.
• Outdated BIOS/drivers/software
• Running background processes (Antivirus, Windows Updates, etc.)
• Windows power profile (High Performance is recommended for the best results)
• Adobe application settings are not correct (GPU acceleration not enabled, etc.)
• Corrupt application preferences (semi-common after changing GPU brands)

Benchmarks can also fail due to failing hardware, OS corruption, and other system instability issues.

If you have gone through the above steps and are still having issues, you are welcome to fill out a bug report. Please note that official support is only available if you have purchased a benchmark license. Be sure to include a copy of any log files that are generated by the benchmark and, ideally, a screenshot of the issue in question. Log files can be found in the following locations:

Windows:

• C:\Users\[USERNAME]\AppData\Local\com.puget.benchmark\logs
• C:\Users\[USERNAME]\AppData\Roaming\com.puget.benchmark\logs

MacOS:

• ~/Library/Application Support/com.puget.benchmark/logs
• ~/Library/Logs/com.puget.benchmark

Command Line (CLI) Guide

The PugetBench for Creators application is installed by default in the following locations:

• Windows: C:\Program Files\PugetBench for Creators\PugetBench for Creators.exe
• MacOS: /Applications/PugetBench for Creators.app/Contents/MacOS/PugetBench for Creators

Common CLI Arguments

PugetBench offers a robust amount of control when running via the command line. Use “–help” to get a full list of available arguments, but the most common command line parameters for PugetBench are:

--help

--license

--app

--app_version

--benchmark_version

--preset

--run_count

--rerun_count

--upload_results

Additional CLI arguments for setting custom app/asset paths and more are also available. Use the –help argument for a full list.

CLI Examples

Activate your benchmark license:

--license "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX"

Run the 1.0.0 Premiere Pro Standard benchmark on Premiere Pro 23.5 and upload the results:

--app "Premiere Pro" --app_version 23.5 --benchmark_version 1.0.0 --preset "Standard" --upload_results

Run the 1.0.0 Photoshop Standard benchmark on Photoshop 24.0.0 with 2 loops, retrying 3 times in the case of an error/application crash, and copy the results to the current folder (Windows):

--app photoshop --app_version 24.0.0 --benchmark_version 1.0.0 --preset "Standard" --run_count 2 --rerun_count 3 --copy_log "%~dp0copy_log.csv"

Sharing Assets Between Systems

If you are utilizing the CLI functionality, you may also want to avoid needing to download the benchmark assets onto multiple systems. To do this, follow these steps:

1. On one system, install the PugetBench for Creators application and download the assets for each benchmark you plan to run.
2. Open the benchmark log folder by going to “Settings”, and clicking on “Reveal Logs”
3. Copy the “assets” and “benchmarks” to an external drive, NAS, etc.
4. These two folders can be copied to any other system with the PugetBench for Creators application installed. They are also Windows/MacOS agnostic, although the path will be different for each OS.

Application Update Log

Update log for the desktop app. Updates for individual benchmarks are recorded on each benchmark’s information page.

Version 1.3.20

• Added preliminary support for DaVinci Resolve 20 Beta. Future updates to the beta may require changes to the Creators app.
• When using the –run_count CLI argument or custom test slider, the benchmark now removes the Resolve database between runs. This was necessary to fix an issue with the Magic Mask tests breaking on multiple loops due to a caching bug.
• Improved error handling when reading a locked DaVinci Resolve log file.

Version 1.3.18

• Added fix for After Effects beta to account for a bug where the “Debug Database.txt” could be in a different location than normal.

Version 1.3.17

• Added support for the PugetBench for After Effects benchmark
  • Ae benchmark requires After Effects version 25.2 (currently in beta) due to features required by the 3D tests.
• Improved application launch time by vastly reducing the time it takes to complete the “Loading Benchmarks…” step by approximately 5x.
• Application and benchmark versions now default to release versions first. If you want to test with a beta version, you must set that in the GUI dropdown or CLI arguments.
• Added CLI support for running the benchmark on applications installed in non-default locations. Run with –help for more information.
• Added CLI support for custom path to assets. Run with –help for more information.
• Preliminary support for Snapdragon devices for Photoshop, Premiere Pro, and DaVinci Resolve.
• Fixed a bug that prevented the benchmark from starting if you updated the host application between benchmark runs.
• Fixed a bug that allowed systems with generic NDA hardware to sometimes upload results to the public database.
• Added additional character compatibility when gathering specs to improve multi-language support. Note: using a non-English language is NOT fully supported.

Version 1.2.22

• Adjusted scoring for increased accuracy. These changes should maintain Overall Scores within the margin of error while improving precision, particularly for systems with lower results.
  • Overall Scores are now calculated from the original, unrounded sub-score results, such as “134.643256482” rather than “135.”
  • Sub-scores are now rounded to a fixed number of decimal places when logged, ensuring that each has at least 3 significant digits. Examples:
    • 5 ➝ 5.23
    • 68 ➝ 67.6
    • 135 ➝ 135 (no change)
    • 2685 ➝ 2685 (no change)
  • Please note that the lower the sub-score, the larger the run-to-run variation tends to be. While the rounding adjustment will improve accuracy, we recommend running multiple benchmark passes on lower-powered systems and using the average when possible.

Version 1.2.21

• Added support for DaVinci Resolve (Studio) 19

Version 1.2.20

• Added support for PugetBench for DaVinci Resolve.
• Added helper links alongside the “Benchmark Preset” and “Benchmark Version” dropdowns to take the user to the relevant page explaining the differences between the available options.
• Added a Creators Application Version and Benchmark Version compatibility checker to ensure that the selected benchmark will work properly with the installed Creators application.
• Fixed a bug with Windows specs gathering that would cause the benchmark to hang if a class doesn’t exist.
• Added “–hardware_encoding_mode” CLI argument for DaVinci Resolve to allow the manual selection of the H.264/HEVC encoding mode (AMD/Intel/NVIDIA/Metal)
• If local .csv logging is enabled via the GUI or CLI, do not open the online results page if result uploading is enabled.
• Misc. bug fixes.

Version 1.1.7

• Bug fix for activating benchmark licenses via CLI using the –license argument

Version 1.1.5

• Back-end overhaul of system spec formatting.
  • This will make adding new specs easier in the future and help us filter NDA results.
  • The format change will also resolve some existing bugs from unusual hardware configurations.
• Added client-side blacklist for common early NDA/pre-release hardware values.
  • If pre-release hardware is detected, a banner is shown and result uploading is disabled.
  • The Overall Score is still displayed in the UI after a benchmark run.
  • Full results can still be logged to a local .csv file when using a license.
• Offline activation and lockdown modes added for enterprise users.
  • For enterprise (site license) users only.
  • Includes offline activation and the ability to lock result uploading and/or benchmark downloading in the UI.
  • Even if uploading is blocked in the UI, running the benchmark via the CLI with the –upload_results argument will override the setting and upload results.
• Added a notification banner to notify the user when a newer version of the PugetBench for Creators application is available.
• CLI updates
  • –app_version now only requires the major application version. I.E. “24” rather than “24.2.0”. Using more specific version numbers is still supported.
  • –app_version and –benchmark_version arguments are now optional. If not filled in, the benchmark will default to using the latest version found on the system. Note that if you have beta builds of Photoshop or Premiere Pro installed, it will attempt to use those as they are newer than the release versions. However, beta builds are not officially supported and may have issues.
• Bug Fixes
  • Fixed several issues with system spec gathering that could cause the benchmark to halt before generating the final score.
  • The Adobe extensions folder will now be generated automatically if it is missing. This resolves a bug on MacOS for users who have never installed an Adobe plugin.
  • Fixed Premiere Pro beta detection. However, beta versions are not officially supported since Adobe may make changes that break the benchmark.
  • Miscellaneous UI and back-end bug fixes.

Version 1.0.5

• Fix for low results in Photoshop benchmark due to improper logging of the fastest result across multiple runs.
• Fix for Premiere Pro benchmark sometimes not generating scores.

Version 1.0.2

• Bug fix to allow non-licensed application versions to run the “Extended” Premiere Pro benchmark preset.

Version 1.0.1

• Fix for an intermittent asset download issue.
• Fix to allow non-licensed users to run Premiere Pro Extended preset.

Version 1.0.0

• Initial Release
• Support for Premiere Pro and Photoshop. Other benchmarks are coming soon!
• Individual benchmark licenses are no longer supported. If you have an app-specific license, please contact [email protected] for a free upgrade.

Known Issues

Known issues and possible workarounds for the latest version of PugetBench for Creators are listed below:

Premiere Pro

• Benchmark hangs on the “Processing – 4K ARRIRAW” test in Premiere Pro 25.2
  • Cause: For Premiere Pro 25.2, there is a bug specifically with existing projects using ARRIRAW media. This results in the benchmark hanging when trying to run the “Processing – 4K ARRIRAW” test, which is part of the Extended preset.
  • Adobe is working on a larger fix for this issue that should come in future versions, but the workaround is to delete the Media Cache files from the following locations:
    • Mac: /Users/<username>/Library/Application Support/Adobe/Common
    • Windows: \Users\<username>\AppData\Roaming\Adobe\Common
  • This should only need to be done once, and only on systems that have previously run our benchmark on Premiere Pro 25.0 or 25.1
• “Error compiling movie” message when running the ARRIRAW processing test on systems with multiple NVIDIA GPUs.
  • Cause: This is a Premiere Pro bug in version 23.1 and newer. T
  • Fix: No permanent fix. The workaround until Adobe releases a patch is to disable or remove all but the primary GPU.

After Effects

• Error during tracking tests of “unable to obtain dynamiclink server connection”
  • Cause: Corrupt configuration files for Dynamic Link
  • Fix: Delete “dynamiclinkmanager” folder from “%appdata%\Adobe”, then launch After Effects manually at least once to automatically re-generate the folder and configuration files. If you have custom settings for how much RAM to reserve for other applications, be sure to re-set those in Edit -> Preferences -> Memory & Performance.

Gallery

Previous Next

System Image

×

Open Full Resolution

Open Full Resolution

Open Full Resolution

Open Full Resolution

Open Full Resolution

Open Full Resolution

Open Full Resolution

Previous Next