Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/1519
Article Thumbnail

Puget Systems Adobe Premiere Pro CC Benchmark

Written on July 8, 2019 by Matt Bach
Share:

At Puget Systems, one of our primary goals is to make sure that our customers end up with a fast, reliable workstation that is perfectly tailored to their unique workflow. The main way we do this is by benchmarking a wide range of hardware in Premiere Pro (and numerous other software packages) that we then publish in our ongoing series of hardware articles.

However, while this ensures we are selling the right hardware, it does not give our customers a great idea of how much faster a new workstation would be compared to their existing system. To address this issue, we are making our internal Pr benchmark available for public download so that anyone can compare their own computer to our latest Premiere Pro hardware articles. Even better, this benchmark is compatible with both Windows and Mac-based systems!

Puget Systems Premiere Pro Benchmark

Note that we have our benchmark divided into three separate downloads. The Benchmark Extension is the actual utility that runs the benchmark itself and needs to be installed before use. Currently this is a stand-alone extension, but we may make it available on Adobe Exchange in the future.

The other two downloads contain the test media and project file. If you are only looking to benchmark general performance in Premiere Pro, the "Standard" test files is all you need and include 59.94FPS H.264, ProRes 422, and RED footage. However, if you want to test every codec available in our benchmark, there is an optional 16GB download containing all the other footage used in our benchmark.

If you want to run the full benchmark, note that you actually need to download both sets of test files and combine them together. The Premiere Pro and Photoshop project files are identical in both downloads so it does not matter which you overwrite.

If you are interested in how your system performs in other creative applications, we are working on making our benchmarks available to the public. At the moment, Puget Systems has the following creator benchmarks available:

View all Puget Systems Benchmarks

Looking for a Premiere Pro Workstation?

Puget Systems offers a range of workstations that are tailor-made for your unique workflow. Our goal is to provide the most effective and reliable system possible so you can concentrate on your work and not worry about your computer.

Configure a System!

How to run the benchmark

After downloading the benchmark and media test files, unzip them to a convenient location. We recommend placing the media test files on the same drive that you store your active project media as storage performance can affect the results. However, be sure that they stay on a local drive as network-based drives will not always work properly.

Next, install the benchmark extension by running either the "Install_Benchmark.exe" or "Mac_Install_Benchmark.command" files in the benchmark folder depending on your operating system.

With the benchmark installed, open the "Benchmark_Project.prproj" project file and follow the instructions shown in the "Benchmark Instructions" composition including starting the extension under "Window->Extensions->Puget Systems Pr Benchmark".

Puget Systems Premiere Pro benchmark settings

Note that while a there are no configuration options that need to be set, there are a few things you need to ensure are done before running the benchmark depending on your OS.

  • Windows: make sure that the program window is in the upper-right quadrant of the Premiere Pro application. We need to use an external app that clicks in this are in order to change the playback resolution.
  • MacOS: you need to go to "Security & Privacy" and add Premiere Pro to the Accessibility list. In addition, a few seconds or minutes after the benchmark starts, a popup will come up that asks you to allow Premiere Pro access System Events. Again, this is so that we can change the playback resolution (via the top menu bar).

Once these settings are taken care of, you can run one of three benchmark presets or individually benchmark any of the test media. The three presets test the following:

  • Standard: Tests 4K H.264 150Mbps 8-bit (59.94FPS), 4K ProRes 422 (59.94FPS) and 4K RED (59.94FPS) media in addition to the 4K Heavy GPU Effects (59.94FPS) and 4K Heavy CPU Effects (59.94FPS) tests
  • Full: Includes all the 29.97FPS and 59.94FPS media and GPU/CPU Effects tests (Note: you will need both sets of test media to run this test)

When the benchmark is complete, it will give you an "Overall Score" for the type of test you ran. A log file is generated in the benchmark folder that includes these scores as well as the FPS for each individual task. You can compare your scores to those in our Premiere Pro hardware articles - just make sure they used the same benchmark version!

What is tested?

Our Premiere Pro benchmark looks at both live playback and export performance with a wide range of codecs at 4K and 8K resolutions as well as at 29.97 and 59.94 FPS. In addition, there are dedicated "Heavy GPU Effects" and "Heavy CPU Effects" sequences that are designed to individually stress the GPU and CPU beyond what a typical Premiere Pro user will do.

Our test media consists of the following resolutions and codecs:

Test Media (29.97 & 59.94 FPS)
4K H.264 150mbps 10-bit (29.97FPS) 4K H.264 150mbps 8-bit (59.94FPS)
4K ProRes 422 4K ProRes 4444
4K RED 4K CinemaRaw Light
8K H.265 100mbps 8K RED

For each type of test media, we look at two test sequences:

  1. Lumetri Color - 3 clips in series with a Lumetri Color effect applied
  2. MultiCam - 12 clips across four tracks in a multicam sequence. Playback is tested in "Multi-Camera" display mode.

In addition, we have a "Heavy GPU Effects" sequence using:

  • 10 ProRes 422 clips in series
  • Cross Dissolve, Dip to Black, Dip to White, and Wipe transitions between clips
  • Lumetri Color and VR De-Noise effect applied to half the clips
  • Lumetri Color, Ultra Key, Sharpen, Gaussian Blur, Basic 3D, VR Digital Glitch, and Directional Blur effects applied to the other half of the clips

and a "CPU Heavy Effects" sequence using:

  • 10 ProRes 422 clips in series
  • Additive Dissolve, Page Peel, Center Split, and Band Wipe transitions between clips
  • All clips have two of the following effects applied: Lens Flare, Dust & Scratches, Noise, Lighting Effects, and Magnify

These tests are all used for live playback performance testing at Full and Half playback resolution. All except the MultiCam sequence are also used to test export performance using the "Youtube 2160p 4K Ultra HD" preset (H.264, 4K, 40mbps) as well as exporting to 4K ProRes 422HQ 8-bpc.

Compatibility

This benchmark is designed for Premiere Pro CC 2019 and should run on virtually any Windows or Mac-based system with at least 16GB of system memory and 4GB of VRAM.

Note that this benchmark is still in BETA. Plug-ins and customized preferences in particular may prevent the benchmark from running properly. If you run into any issues, we recommend resetting your preferences to default - be sure to make a backup first!

How does the scoring work?

The scoring system used in our benchmark is based on the performance relative to the FPS of the test media. If the media is 29.97FPS and the system plays it back at 29.97FPS, that results in a score of "100". Likewise, if it can only play back at half the FPS, the score would be "50". Scores for Live Playback cannot go above 100 (since you cannot play at faster than the media's FPS), although scores for Exporting can go well above 100 depending on the performance of the system.

The Live Playback and Export scores are all combined into the "Standard", and "Full" Overall scores depending on the benchmark preset you chose.

Update Log

Version 0.3 BETA

  • Replaced H.264 media with footage straight from a Panasonic GH5 (Thanks Neil Purcell!)
  • Replaced "Quick" preset with "Standard" preset. Changed tested media to 59.94FPS instead of 29.97FPS.
    • This was done with the intention of having a benchmark preset that is geared towards measuring the overall performance of a system in as little time as possible.
  • Separated media files into two groups: a "Standard" set and a "Full" set to make the download much more manageable for people just doing the Standard preset.
  • Due to the change in the test media, the overall scores are not interchangeable with previous benchmark versions

Version 0.2 BETA

  • First release.

Looking for an Premiere Pro Workstation?

Puget Systems offers a range of workstations that are tailor-made for your unique workflow. Our goal is to provide the most effective and reliable system possible so you can concentrate on your work and not worry about your computer.

Configure a System!

Tags: Premiere Pro, Benchmark, Public, Download
Hwgeek

Can't wait for your review for Ryzen7/9 and 3950X in future, from other reviews I saw- looks like the 3900X will be your favorite CPU- better then 9900K and even challenging 7980XE/9980XE even in ADOBE :-).
https://nl.hardware.info/re...

Posted on 2019-07-09 07:31:19
Spale Max

Results are completely off.
I mean 9700k is faster than 9900k in your review, is there anything else to add?

Posted on 2019-07-09 16:06:59
Hwgeek

IDK, here another test in Pro apps- 3900X competing with 9980XE and 2950X!
https://techgage.com/articl...
So the 3950X will completely destroy Intels X299 linup in pro apps.

Posted on 2019-07-10 08:44:20
simonxliu

Adobe software isn't well optimized to use multiple threads and loves single threaded performance. There are plenty of lightly threaded workloads out there that actually benefit from disabling Hyper Threading(a 9700K is just a 9900K without HT) since it increases processing latency and you only start to see benefits from Hyper Threading when you have more simultaneous threads. Games, which also often aren't well optimized to use multiple threads, sometimes perform better on a 9700K(or even just a 9900K with HT manually disabled) compared to a 9900K.

Posted on 2019-07-12 13:32:34
Yariv Nachshon

A new review with Old CPUs ? Seems like the new Ryzen is all what people care for.

Posted on 2019-07-09 08:53:39

Do you mean the sample results? This is just the benchmark information/download page, not a hardware review. Those are just there to give people an idea of normal scores until we have articles published using this benchmark (like our Ryzen 3rd generation first look articles that are up)

Posted on 2019-07-09 15:32:08
Yariv Nachshon

Yes. i understand it but there is no option for it in the sample results list at all :-(
Even mentioning is with a score of zero will make it more interesting to re-examine :-)

https://www.pugetsystems.co...

Posted on 2019-07-09 16:59:42
RDarvan

Would it be possible to add a 1080p files test to the benchmark?
for slightly more modest configs...

Posted on 2019-07-10 07:38:49

We have gone back and forth on that, and ended up deciding against it for a couple of reasons:
1) 1080p is just too easy for the kinds of systems we are typically testing ourselves. If anything can do our playback tests at full FPS, then there is no measurable difference beyond exporting
2) The relative difference between hardware tends to be very similar regardless of whether you are testing 1080p or 4K media. So even if you do just work with 1080p footage, the 4K results are still pretty valid (even if they don't tell you exactly what FPS you could expect)
3) We already have way too many tests

In many ways, #3 is the main limiting factor. Nothing is really stopping us from adding 1080p, but it comes down to benchmark time vs reward.

Posted on 2019-07-10 16:48:58
RDarvan

Thank you for the explanation!

Posted on 2019-07-11 01:31:43
Mefist

1080 Ti?

Posted on 2019-07-10 07:56:43
MacherTV_Peter

Great article, but where is the Ryzen 3900x Premiere Pro review? :D jk.

Posted on 2019-07-10 14:16:41
Matthew Fitz

Very generous of you to share these publicly. I am keenly interested in when the light-room benchmark becomes a public beta also as was previously mentioned. Eagerly waiting to see how Ryzen 3000 takes to Photoshop and Light-room.

Posted on 2019-07-11 02:09:18
Mark Harris

Thanks for sharing this!

Posted on 2019-07-11 19:22:49
Matt

Just downloaded it and was able to restore the archive file. When opening the Premiere project it seems there is a missing .psd file? Benchmark Setup.psd

Posted on 2019-07-12 19:03:37

Are you on Mac or PC? So far, it seems like PC can unzip it just fine , which is very odd.

Fortunately, that PSD isn't really necessary. It is just the setup instructions which are listed on this page on the "How to run the benchmark" section. Or just look at https://www.pugetsystems.co... . Delete it, mark as offline, whatever. It won't affect the benchmark at all.

Posted on 2019-07-12 19:10:07
Matt

On a mac. , I'll give it a run through, thanks.

Posted on 2019-07-12 19:28:11
Matt

It looks like the restore is missing more files than I thought. https://uploads.disquscdn.c... https://uploads.disquscdn.c...

Posted on 2019-07-12 19:41:57
Matt

the bottom list is what was unzipped.

Posted on 2019-07-12 19:42:20

So weird that it works fine on PC... I already have a plan in place to split up the zip across two files since most people aren't going to actually run the full benchmark. Hopefully that will fix it.

I'm actually out of the office right now, but I'll try to get a new zip file(s) uploaded early next week.

Posted on 2019-07-12 20:03:41
Matt

Thanks, I just wanted to let you all know, have a great weekend!

Posted on 2019-07-12 20:06:35
Ann Bens

https://uploads.disquscdn.c...

Unzipping the movie files gives a Headers Error

Resulting in not loading up in Pr. during the Quick test

Posted on 2019-07-12 22:20:36
Scott Moore

I've tried running the benchmark several times (and selecting several different options) and nothing seems to work. I get the "copying media" notification and then nothing else happens. Logs seem to point to an error with copying some of the media https://uploads.disquscdn.c...

Posted on 2019-07-16 21:24:45

We're uploading a new set of benchmark files right now since it looks like there was some corruption in the original zip file. Check the download links in a few hours - that should resolve the issue.

Posted on 2019-07-16 21:28:04
Scott Moore

Sweet, thanks Matt! will keep an eye out for it

Posted on 2019-07-16 21:43:39
Scott Moore

Hey Matt, got it working now!
I don't think the download/zip was my issue as the new download ran into the same "some media not copied" error.
I moved the project folder to my system drive, it was previously on my media drive which is a network drive, and that seemed to get it working correctly.
Not sure if that was in any of the instructions or not but might be helpful for problem solving going forward?
Thanks!

Posted on 2019-07-17 03:18:27

Oh, interesting, I didn't even think about network drives. It should work fine if it is mounted to a drive letter, although I'm not completely sure how it would handle it on Mac. We're mostly using the Premiere Pro SDK, and I'm not confident the file management in it can handle things like that.

Thanks for the heads up, I'll at the very least update the instructions to mention this!

Posted on 2019-07-17 16:34:34
elis

Hello.
WHe runnning 4k CinemaRaw Light in MC there's some Error retrieving Frame instances that would make Premiere crash. What could be the reason?

Posted on 2019-07-24 06:51:21

I have never seen that issue before. If you just manually import that footage and make a sequence, does the issue still occur? If it does, that means it is a problem with your system or Premiere Pro install, not the benchmark

Posted on 2019-07-24 17:10:25
Petrolhead

Project cant be loaded even though I have CC 2019 v13.01 :(

Posted on 2019-07-24 15:05:16

There actually isn't anything in the benchmark project except the instructions (which are on this page as well). You can just make a new project with the same name in that folder and the benchmark should run just fine.

Posted on 2019-07-24 17:11:09
Daniel Toman

A standardized Premiere benchmark is something I've wanted for a long time, thank you! Planning to try it soon to measure the impact of various system upgrades. How will it handle OpenCL vs Metal vs CUDA vs Software Only rendering?

Posted on 2019-07-30 18:50:43

Right now, the benchmark automatically sets the Mercury Playback engine, so manually changing it won't affect anything. We have to do this since it will often default down to "Software Only" mode if you open the project with a different GPU than what the project was created with (which will be the case for most people). I believe the priority goes CUDA, OpenCL (since OpenCL is faster overall than Metal on Mac), Metal, then it errors if it can only do Software Only since some of the tests require CUDA/OpenCL/Metal.

In a future version, we may add a configuration .ini file where you can specific which you want to use for the times when you want to test this kind of thing, but on this initial version we are more concerned about it working at all and being accurate than anything else.

Posted on 2019-08-05 20:48:50
Mishka981

Can you elaborate more on the system used with the 9900K? Running the same with 64 GB of Ram and a 2070. Both of which you have articles stating anything more provides little performance gain. Numbers are good, but you are pulling 20% higher scores on all benchmarks. Are you overclocking the CPU, or pulling performance gains out any other way?

Posted on 2019-08-10 04:33:58

We don't overclock the CPU at all, and don't use overclocked video cards unless there is no other option (this is sometimes the case with AMD GPUs). Usually, it is one of three things that cause lower than expected performance in benchmarks:

1) You are using a different version of drivers, Windows updates, motherboard BIOS, or other software/firmware than we did
2) You have applications running in the background that is eating into performance (we do our testing on very "clean" OS installs)
3) You may be hitting thermal or power limits. Might be inadequate cooling for the CPU/GPU/motherboard, or your motherboard/PSU isn't able to maintain the power states necessary to keep the CPU at the highest Turbo Boost speeds that should be possible. That doesn't necessarily mean a bad mobo/PSU, but not all boards are created equally and some have better power management than others.

You might want to do a few synthetic CPU and GPU-specific benchmarks to make sure you are hitting the performance levels you should be. Geekbench is pretty good for CPU (download at https://www.geekbench.com/ and compare scores at https://browser.geekbench.c.... For the GPU, OctaneBench is great (download at https://render.otoy.com/oct... compare scores at https://render.otoy.com/oct.... Cinebench is also good if you want to look into CPU performance a bit more - they just don't have a result browser so you will need to find reviews that use it.

Posted on 2019-08-12 18:07:33
Mishka981

Much obliged on your input. I have run Cinebench, but run into the exact situation you describe finding similar systems for comparison. Will look into the other benchmarking applications you reference.

I suspect background apps may be robbing a bit of power. Will disable them and rerun with premiere set to high priority for a comparison to a “clean OS” or at least as clean as it can be without wiping my system.

Keep up the great work, it’s great to see you guys sharing so much info with the masses!

Posted on 2019-08-13 15:50:04
Jim Leonard

Hi Matt, I left a longer message on another thread, but wanted to ask you what encoding parameters you used when creating the h.264/h.265 files in the benchmark -- there's something wrong with them and they don't play smoothly on the Premiere timeline, even on a system with Intel quicksync extensions working. When I make a new timeline with them, they play via software only, and don't play smoothly. If I re-encode them with identical size/framerate/depth/bitrate using Adobe Media Encoder, the resulting output is played via hardware decoding and plays fine. I'm using an I7-8700k with two GTX 1080 TIs.

Because premiere can't play these media files using hardware decoding, I'm not sure they're fair to use in the benchmark. Even if you wanted the benchmark to be software decoding only, there's something else wrong; they play choppy even though CPU usage is barely above 15%, which leads me to believe that the Long GOP is simply too long and not representative of footage produced by actual cameras. Looking forward to your thoughts on this.

Posted on 2019-08-14 02:11:13
Chris Holzmann

Hello! I always get
"Failed to install, status = -403!"
when i try to install the Benchmark.

I'm on 10.14.6, i have premiere pro cc 2019 installed
Any suggestions?
Thank you!

Posted on 2019-08-24 06:07:58
Chris Holzmann

Hello! I always get
Failed to install, status = -403!
when i try to install the Benchmark.

I'm on 10.14.6, i have premiere pro cc 2019 installed

Any suggestions?
Thank you!

Posted on 2019-08-24 06:13:54
Brandon Yen

Hi, when I try to run the benchmark it says "Error: Media Missing" and lists out the video files. I am running the standard benchmark, have downloaded the first two links, unzipped them, installed the benchmark, and opened the project. Is there something I'm doing wrong?

Posted on 2019-09-01 23:32:53

I just uploaded a new zip file with the benchmark media uploading right now that should resolve the issue. If you re-download "Download Part 2: Standard Test Media & Project Files (3GB)" it should work now.

Thanks for pointing out the problem!

Posted on 2019-09-03 18:12:23
Brandon Yen

Thanks, will report back!

Posted on 2019-09-04 04:05:08
Brandon Yen

It works, thanks!

Posted on 2019-09-04 17:17:06
Роман Михайлишин

Is any version for test Premiere pro CC 2017?

Posted on 2019-09-05 07:33:00
Mr. Leidy

I have a new build with a i9-9900k, 64 gb of ram, z390 Designare Motherboard, 2070 Super graphics card, and SSD hard drives, and I can't seem to get above 60 on the live playback portion of your benchmark. Do you have any suggestions why they may be? I saw in another one of your articles you had the 9900k benchmarked around 80. Thanks!

Posted on 2019-09-21 23:10:06

Just to let you guys know that your project folder can't be renamed! I accidentally downloaded 2 zips and deleted the first one. So the 2nd one has this (1)... It broke the extension resulting in file not found.

Posted on 2019-10-14 18:20:03
Lukas Messner

Hello, if I want to start the benchmark I get always the error that CUDA/OpenCL/Metal is not available on my system, even though I can set it on OpenCL acceleration in preferences. Drivers and PP are up to date. I have an i5 6500 and rx570 graphics card. Any suggestions what it might be?

Posted on 2019-11-09 17:19:35
Nebukad

Unfortunately, I have the same problem. "CUDA/OpenCL/Metal is not available on your system" all the time, although I have a supported Nvidia card. The log file always shows, that only the software renderer is being used:

15:22:5 - Premiere Pro Version: 13.1.5x47
15:22:5 - Premiere Pro Bencmark: V0.3 BETA
15:22:6 - Log Name Puget Systems Pr Benchmark V0.3 BETA (Pr CC 13.1.5x47),
15:22:6 - Changing playback resolution just so that the allow prompt comes up early in the benchmark process
15:22:6 - Running from Benchmark_Project.prproj
15:22:17 - Renderer set to SOFTWARE Only (No GPU Accel)

Any fix or help would be great! Thank you very much in advance!

Posted on 2019-11-19 20:39:59
Daniel Pape

The MAC installation is not working. The command runs and it says successful but the extension is not available once Premiere opens.
Mac 10.15.1, Premiere Pro 2020

Posted on 2019-11-13 20:36:42

We haven't updated it for Premiere Pro 2020 quite yet. We should have a new version up in a couple of weeks that will support 2020.

Posted on 2019-11-13 20:42:56
Daniel Pape

Worked well on PC. Ran the standard test and results were generated. Attempting a 2nd time on Mac. First run made it through the 4K effects tests but gave an error, something like, "playback is taking longer than expected" prior to generating final overall scores.

Posted on 2019-11-13 22:55:25
Daniel Pape

Mac running Premiere Pro 2020 completed successfully on the 2nd attempt, after a restart of system and Premiere.

Posted on 2019-11-13 23:16:56
Daniel Pape

FIXED: had to run the following command chmod u+x
On both: "Mac_Install_Benchmark.command" and "/ExManCmd_mac/Contents/MacOS/ExManCmd"

Posted on 2019-11-13 20:43:41

Oh, fun. Surprised a bit that the benchmark itself if working with PR 2020, but good to know that we are going to have to adjust the plugin installer to meet whatever new security stuff MacOS has implemented.

Thanks for the update!

Posted on 2019-11-13 20:47:00