PugetBench for Creators User Guide

This user guide includes details about how to use PugetBench command-line tools as well as our data collection policy, update logs, and more.

Download PugetBench for Creators

Quickly Jump To: Data Collection • Troubleshooting • CLI Guide • Update Log • Known Issues • Gallery

PugetBench for Creators is designed to test a range of Content Creation workflows and is ideal for end users, hardware reviewers, press, system integrators, and hardware manufacturers who want to examine the performance of different hardware in real-world workflows. Our benchmarks are easy to install and run, and have licensing options to support command-line automation and help prevent uploading NDA or confidential results.

Each benchmark has a low run-to-run margin of error and is highly repeatable. Results are presented as an Overall Score, as well as a breakdown of performance for individual tests and groups of tests.

Data Collection

All data is completely anonymous. No personally identifiable information is collected – only basic information on hardware specifications and application versions is recorded. An example set of raw data is available below:

{
  "results": [
    {
      "test": "Overall Score (Standard)",
      "result": "100",
      "major": 2,
      "setting": "",
      "unit": "",
      "preset": "Standard"
    },
-----Repeated test results truncated-----
  ],
  "benchmark_version_major": 1,
  "benchmark_version_minor": "0.0",
  "benchmark_version_patch": 0,
  "benchmark_version_identifier": "",
  "application_version_major": 24,
  "application_version_minor": 0,
  "benchmark_name": "PugetBench for Premiere Pro",
  "application_name": "Premiere Pro",
  "specs_formatted": [
    {
    "type": "CPU",
    "manufacturer": "AuthenticAMD",
    "name": "AMD Ryzen 7 7700X 8-Core Processor",
    "cores": 8
  },
  {
    "type": "GPU",
    "manufacturer": "Advanced Micro Devices, Inc.",
    "name": "AMD Radeon Graphics",
    "driver": "31.0.23013.1023"
  },
  {
    "type": "GPU",
    "manufacturer": "NVIDIA",
    "name": "NVIDIA GeForce RTX 4080",
    "driver": "31.0.15.2824"
  },
  {
    "type": "Memory",
    "manufacturer": "Kingston",
    "capacity_value": 32,
    "capacity_unit": "GB",
    "speed_value": 5200,
    "speed_unit": "MHz"
  },
  {
    "type": "Memory",
    "manufacturer": "Kingston",
    "capacity_value": 32,
    "capacity_unit": "GB",
    "speed_value": 5200,
    "speed_unit": "MHz"
  },
  {
    "type": "OS",
    "manufacturer": "Microsoft Corporation",
    "name": "Microsoft Windows 11 Pro",
    "version": "22621"
  },
  {
    "type": "System",
    "manufacturer": "ASUS",
    "name": "System Product Name"
  },
  {
    "type": "Motherboard",
    "manufacturer": "ASUSTeK COMPUTER INC.",
    "name": "ProArt X670E-CREATOR WIFI",
    "bios": "1602"
  },
  {
    "type": "Engine",
    "name": "Mercury Playback Engine",
    "value": "CUDA"
  }
  ]
}

Troubleshooting

If you have any issues with the benchmark not properly running, first verify that your system meets the System Requirements. If there are no issues with your system or software, there are a number of troubleshooting steps you can try:

• If no application is shown in the drop-down, you most likely do not have a version installed that is supported by the benchmark
• If you are getting an error about a PSD being too large, check that “Disable Compression of PSD and PSB Files” is unchecked in “Edit -> Preferences -> File Handling”
• Do not attempt to use the system while the benchmark is running. Defocusing the application window can break the benchmark in many situations.
• Clear the application preferences as found in Adobe’s guide or by using our Adobe CC Preference Backup/Restore/Clear utility (Windows only).
• Disable Windows Defender or any real-time virus scanning programs.
• Ensure that your OS and the application you are testing are set to English.

Lower than expected performance can be caused by a number of factors, but the most common are:

• Unknowingly comparing scores to different benchmark and/or application versions.
• Outdated BIOS/drivers/software
• Running background processes (Antivirus, Windows Updates, etc.)
• Windows power profile (High Performance is recommended for the best results)
• Adobe application settings are not correct (GPU acceleration not enabled, etc.)

If you continue to have issues, please contact [email protected] – ideally with a screenshot of the problem and a copy of any log files that are generated by the benchmark. The log files can be found in the following locations:

Windows:

• C:\Users\[USERNAME]\AppData\Local\com.puget.benchmark\logs
• C:\Users\[USERNAME]\AppData\Roaming\com.puget.benchmark\logs

MacOS:

• ~/Library/Application Support/com.puget.benchmark/logs
• ~/Library/Logs/com.puget.benchmark

Command Line (CLI) Guide

The PugetBench for Creators application is installed by default in the following locations:

• Windows: C:\Program Files\PugetBench for Creators\PugetBench for Creators.exe
• MacOS: /Applications/PugetBench for Creators.app/Contents/MacOS/PugetBench for Creators

Common CLI Arguments

Here are the most common command line parameters for PugetBench:

--help

--license

--app

--app_version

--benchmark_version

--preset

--upload_results

CLI Examples

Activate your benchmark license

--license "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX"

Run the 0.98.6 Premiere Pro Standard benchmark on Premiere Pro 23.5 and upload the results

--app "Premiere Pro" --app_version 23.5 --benchmark_version 0.98.6 --preset "Standard" --upload_results

Run the 0.98.1 Photoshop Standard benchmark on Photoshop 24.0.0 with 2 loops, retrying 3 times in the case of an error/application crash, and copy the results to the current folder (Windows)

--app photoshop --app_version 24.0.0 --benchmark_version 0.98.1 --preset "Standard" --run_count 2 --rerun_count 3 --copy_log "%~dp0copy_log.csv"

Sharing Assets Between Systems

If you are utilizing the CLI functionality, you may also want to avoid needing to download the benchmark assets onto multiple systems. To do this, follow these steps:

1. On one system, install the PugetBench for Creators application and download the assets for each benchmark you plan to run.
2. Open the benchmark log folder by going to “Settings”, and clicking on “Reveal Logs”
3. Copy the “assets” and “benchmarks” to an external drive, NAS, etc.
4. These two folders can be copied to any other system with the PugetBench for Creators application installed. They are also Windows/MacOS agnostic, although the path will be different for each OS.

Application Update Log

Update log for the desktop app. Updates for individual benchmarks are recorded on each benchmark’s information page.

Version 1.1.7

• Bug fix for activating benchmark licenses via CLI using the –license argument

Version 1.1.5

• Back-end overhaul of system spec formatting.
  • This will make adding new specs easier in the future and help us filter NDA results.
  • The format change will also resolve some existing bugs from unusual hardware configurations.
• Added client-side blacklist for common early NDA/pre-release hardware values.
  • If pre-release hardware is detected, a banner is shown and result uploading is disabled.
  • The Overall Score is still displayed in the UI after a benchmark run.
  • Full results can still be logged to a local .csv file when using a license.
• Offline activation and lockdown modes added for enterprise users.
  • For enterprise (site license) users only.
  • Includes offline activation and the ability to lock result uploading and/or benchmark downloading in the UI.
  • Even if uploading is blocked in the UI, running the benchmark via the CLI with the –upload_results argument will override the setting and upload results.
• Added a notification banner to notify the user when a newer version of the PugetBench for Creators application is available.
• CLI updates
  • –app_version now only requires the major application version. I.E. “24” rather than “24.2.0”. Using more specific version numbers is still supported.
  • –app_version and –benchmark_version arguments are now optional. If not filled in, the benchmark will default to using the latest version found on the system. Note that if you have beta builds of Photoshop or Premiere Pro installed, it will attempt to use those as they are newer than the release versions. However, beta builds are not officially supported and may have issues.
• Bug Fixes
  • Fixed several issues with system spec gathering that could cause the benchmark to halt before generating the final score.
  • The Adobe extensions folder will now be generated automatically if it is missing. This resolves a bug on MacOS for users who have never installed an Adobe plugin.
  • Fixed Premiere Pro beta detection. However, beta versions are not officially supported since Adobe may make changes that break the benchmark.
  • Miscellaneous UI and back-end bug fixes.

Version 1.0.5

• Fix for low results in Photoshop benchmark due to improper logging of the fastest result across multiple runs.
• Fix for Premiere Pro benchmark sometimes not generating scores.

Version 1.0.2

• Bug fix to allow non-licensed application versions to run the “Extended” Premiere Pro benchmark preset.

Version 1.0.1

• Fix for an intermittent asset download issue.
• Fix to allow non-licensed users to run Premiere Pro Extended preset.

Version 1.0.0

• Initial Release
• Support for Premiere Pro and Photoshop. Other benchmarks are coming soon!
• Individual benchmark licenses are no longer supported. If you have an app-specific license, please contact [email protected] for a free upgrade.

Known Issues

Known issues and possible workarounds for the latest version of PugetBench for Creators are listed below:

• “Error compiling movie” error when running the ARRIRAW processing test on systems with multiple NVIDIA GPUs. This is a Premiere Pro bug in version 23.1 and newer. The workaround until Adobe releases a fix is to disable or remove all but the primary GPU.

Gallery

Previous Next

System Image

×

Open Full Resolution

Open Full Resolution

Open Full Resolution

Open Full Resolution

Previous Next