Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/1126
Article Thumbnail

DaVinci Resolve 14 Performance: Windows vs Linux

Written on March 16, 2018 by Matt Bach
Share:

Introduction

Linux is often considered the operating system of choice for power users due to its light footprint and open nature but to those without prior Linux experience it can be frustrating to get working properly. Even something as seemingly simple as installing drivers or software can be quite difficult for those that do not already know their way around Linux. And even after the system is up and running, software or driver updates can potentially break the OS if they are not installed in the proper manner.

The question is, does DaVinci Resolve actually see enough of a performance benefit in Linux to compensate for its high learning curve? To find out, we decided to see how DaVinci Resolve performs both in Windows 10 Pro and in CentOS 7.4 which is currently one of two Linux distributions officially supported by Blackmagic (Red Hat being the other).

DaVinci Resolve Windows 10 vs CentOS

Our testing involves a large amount of data (828 data points across 138 different tests) and if you do not find dozens of charts appealing, feel free to jump right to the conclusion section.

Test Hardware & Methodology

While our test platform (shown in the chart below) is a pretty standard DaVinci Resolve configuration we often use, there are two things in particular we want to point out. The first is that we opted to use a Core i9 CPU with the X299 chipset rather than a Intel Xeon W or Dual Intel Xeon SP setup such as those used in our recent DaVinci Resolve 14 GPU Scaling: Core i9 vs Xeon W vs Dual Xeon SP article. Those platforms would allow us to test with up to four GPUs, but unfortunately they simply don't play well with CentOS at the moment. We are working on determining what is necessary to get CentOS stable on them, but for now Core i9/X299 is the fastest modern platform for DaVinci Resolve we offer that is also stable with CentOS.

The second is that we used a completely fresh install of CentOS 7.4 rather than the CentOS 7.3 ISO provided by Blackmagic. We actually tried that ISO initially, but we were seeing extremely low performance - often half of what we achieved in Windows 10. We tried updating the kernel and drivers to see if that would resolve the issue, but we ran into major issues with the OS becoming unstable. So instead, we did a fresh install using CentOS 7.4 and installed the video and other drivers ourselves.

All of our test footage is downloaded or transcoded from media that is publicly available. This was done so that anyone can repeat our testing in order to both verify our findings and to see how their current computer stacks up to the latest hardware available. To test each type of footage, we used three different "levels" of grading. The lowest level is simply a basic correction using the color wheels plus 4 Power Window nodes that include motion tracking. The next level up is the same adjustments but with the addition of 3 OpenFX nodes: Lens Flare, Tilt-Shift Blur, and Sharpen. The final level has all of the previous nodes plus one TNR node.

Performance was measured in the Color tab using the built-in FPS counter. After playback was started, we waited 15 seconds for the FPS to stabilize then recorded the lowest FPS number over the next 15 seconds. This method allowed us to achieve highly consistent and replicable results.

For all the RAW footage we tested (CinemaDNG, ARRIRAW, and RED), we not only tested with the RAW decode quality set to "Full Res." but we also tested at "Half Res." ("Half Res. Good" for the RED footage). In our recent articles we saw major differences in both raw FPS but also GPU scaling when using different decode qualities so this is something we are highly interested in.

The footage used in our testing is shown below with links to where you can download it yourself:

Codec Resolution FPS Camera Clip Name Source
ProRes 422 HQ 3840x2160 24 fps Ursa Mini 4K City Train Station Blackmagic Design
Production Camera 4K Update
ProRes 4444 3840x2160 59.94 fps Canon C200 Untitled00024199 4K Shooters
Canon C200 Raw Footage Workflow
CinemaDNG 4608x2592 24 fps Ursa Mini 4K Interior Office Blackmagic Design
[Direct Download]
ARRIRAW 6560x3100 23.976 fps ALEXA 65 A003C025
(Open Gate spherical)
ARRI
ALEXA Sample Footage
RED 3840x2160
(11:1)
23.976 fps EPIC DRAGON A016_C001_02073O_001 RED
Sample R3D Files
RED 4096x2304
(7:1)
29.97 fps RED ONE MYSTERIUM A004_C186_011278_001 RED
Sample R3D Files
RED 6144x3160
(12:1)
23.976 fps EPIC DRAGON A007_C115_07181B_001 RED
Sample R3D Files
RED 6144x3077
(7:1)
23.976 fps WEAPON 6K S005_L001_0220LI_001 RED
Sample R3D Files
RED 8192x4096
(12:1)
23.976 fps WEAPON 8K S35 S002_C074_02065Z_001 RED
Sample R3D Files
RED 8192x4320
(9:1)
25 fps WEAPON 8K S35 B001_C096_0902AP_001 RED
Sample R3D Files
RED 8192x4320
(7:1)
23.976 fps EPIC-W 8K S35 S002_C074_02065Z_001 RED
Sample R3D Files
DNxHR HQ
8-bit
3940x2160 29.97 fps Transcoded from RED A004_C186_011278_001
DNxHR HQ
8-bit
6144x3160 23.976 fps Transcoded from RED A007_C115_07181B_001
DNxHR HQ
8-bit
8192x4320 25 fps Transcoded from RED B001_C096_0902AP_001

While this is by no means every codec available, we do feel that this covers a wide range of footage that many users work with on a daily basis. In the future we may cut down on the number of RED clips and replace them with something like XAVC-S or AVCHD but for now we really wanted to see how the different compression levels impact performance.

4K Media - Live Playback FPS (RAW DATA)

[Click Here] to skip ahead to analysis section

4K DNxHR HQ

4K ProRes 422 HQ

4K ProRes 4444

4K RED 11:1 (Full Res.)

4K RED 11:1 (Half Res.)

4K RED 7:1 (Full Res.)

4K RED 7:1 (Half Res.)

4K CinemaDNG (Full Res.)

4K CinemaDNG (Half Res.)

4K Media - Live Playback FPS (Analysis)

Since our 4K testing contains almost over 300 data points across six different codecs, it can be difficult to pull meaningful conclusions from the raw data. If you tend to use just one of the codecs we tested, we highly recommend looking at just that result, but for a more general take on how DaVinci Resolve performs with each OS we decided to average the results from each test.


Starting with relatively simple color grading using the color wheels and 4 Power Windows, we saw only minor performance differences between Windows 10 and CentOS. Windows has a slight lead (especially when using three GPUs) which is entirely due to the performance when using ProRes 4444. One thing we want to note is that this is the only codec we are using that has a higher 59.94 FPS media and each OS was easily able to maintain >30 FPS with any GPU combination. So while Windows technically has a slight lead here we would really call it a wash.


Adding 3 OpenFX effects, we start to see a bit more of a difference between the two operating systems. Unfortunately, it is a bit of a mixed bag. With the GTX 1080 Ti GPUs, CentOS was a tiny bit faster with one and two GPUs, but Windows 10 took the lead when using three cards. With the Titan V, Windows 10 was about 1 FPS faster on average with a single GPU but no faster with multiple cards. The difference was all 1 FPS or less, however, so once again we would really call this a tie.


Adding TNR, we get our first look at what some may consider to be a real difference in performance. It was only with a single Titan V, but surprisingly Windows 10 came out on top being a bit more than 2 FPS or about 14% faster than CentOS.

6K Media - Live Playback FPS (RAW DATA)

[Click Here] to skip ahead to analysis section

6K DNxHR HQ

6K RED 12:1 (Full Res.)

6K RED 12:1 (Half Res.)

6K RED 7:1 (Full Res.)

6K RED 7:1 (Half Res.)

6K ARRIRAW (Full Res.)

6K ARRIRAW (Half Res.)

6K Media - Live Playback FPS (Analysis)

Our 6K testing is not quite as extensive as our 4K testing, but it still contains over 250 data points across four different codecs. If you tend to use just one of the codecs we tested, we highly recommend looking at just that data but for a more general take on the performance difference between Windows 10 and CentOS we again decided to average the results from each test.


Starting with relatively simple color grading using the color wheels and 4 Power Windows, there is not much to discuss. With this level of grading, we simply were running at full FPS (or very near to it) with every OS and GPU combination we tested.


Adding 3 OpenFX effects, we see a small performance difference with a single Titan V GPU with Windows 10 coming up slightly on top by just over 1 FPS.


Adding TNR is where things start to get really interesting. On average, Windows 10 has a small lead with a single GPU but CentOS ends up being a good amount faster with two or three GPUs. However, there is a lot not shown in this averaged chart. If you look at the individual results, you will notice that the benefit from CentOS mostly comes from the RED 7: and RED 12:1 tests where we used "Full Res." decode quality. The main reason for this is that for those tests, we saw little to no benefit to using multiple GPUs in Windows 10. This is something we noticed in our recent GPU scaling article and appears to be an issue many are reporting when using "Full Res." decode quality with RED footage on the latest versions of DaVinci Resolve.

Overall for this test, Windows has the edge with a single GPU but CentOS is certainly faster on average - especially if you want to use "Full Res." decode quality with RED footage.

8K Media - Live Playback FPS (RAW DATA)

[Click Here] to skip ahead to analysis section

8K DNxHR HQ

8K RED 12:1 (Full Res.)

8K RED 12:1 (Half Res.)

8K RED 9:1 (Full Res.)

8K RED 9:1 (Half Res.)

8K RED 7:1 (Full Res.)

8K RED 7:1 (Half Res.)

8K Media - Live Playback FPS (Analysis)

Once again, if you tend to use just one of the codecs we tested we highly recommend looking at just that data. However, for a more general take on GPU scaling in DaVinci Resolve on each platform we again decided to average the results from each test.


Like the previous sections, the results with relatively simple color grading is not all that interesting. Windows 10 is up to 3-4 FPS faster with a single GPU when using RED 9:1 footage at "Full Res." decode, but other than that there is little difference between the two operating systems.


Adding 3 OpenFX effects actually results in even less difference between Windows 10 and CentOS. Windows did have a bit of an issue with three GPUs in the RED 12:1 "Full Res." decode test, but other than that you would be hard pressed to see a difference between the two operating systems.


With TNR added, the results are actually much more confusing than the chart above shows. On average the two operating systems are about on par with a slight benefit coming from Windows when using a single Titan V. If you drill into the raw results, however, it gets pretty messy.

Just like in the 6K tests, Windows again has a bit of an issue with RED media using "Full Res." decode and at times is up to 6 FPS slower than CentOS. You would think that makes CentOS the clear winner, but strangely CentOS had similar issues - just with "Half Res." decode quality instead. The difference was not quite as dramatic (which is why CentOS comes out slightly on top with multiple GPUs on average), but Windows 10 was often 3-4 FPS faster than CentOS with "Half Res." decode quality.

Conclusion

We went into our testing expecting DaVinci Resolve to perform slightly faster in CentOS 7.4 than in Windows 10 and while there are certain times this is true, the overall reality is a bit more complicated. The main place the two operating systems perform differently is when doing heavy grading with RED footage at different decode qualities. If you use multiple OpenFX and things like TNR, then CentOS is currently better if you work with RED media and want to use "Full Res." decode quality. On the other hand, Windows 10 is overall better if you turn it down to "Half Res." decode quality. Given the fact that especially at 6K and 8K resolutions "Full Res." will often end up giving you around half the FPS of "Half Res." regardless of the hardware, we feel that having better performance at "Half Res." is actually more important for real-world workflows. "Full Res." decode may be ideal, but if you have to choose between working at 9 FPS or 18 FPS you are most likely going to turn down the decode quality in favor of the much higher FPS.

Because of this, we opted to base our final conclusions on the performance when using "Half Res." decode quality with RAW media. As we've stated before, however, if there is a particular codec you use that we tested, we highly recommend looking at the raw results rather than the overall averaged chart below.

DaVinci Resolve 14.3 Linux vs Windows Benchmark

 

For most workloads (shown in the smaller two charts), CentOS and Windows 10 are really about on par. Windows 10 may have a slight lead, but it is small enough to not matter. With things like multiple OpenFX and TNR, however, we definitely saw a performance benefit from using Windows 10 over CentOS in some situations. It was mostly a wash with the GTX 1080 Ti, but with a single Titan V we saw on average about 2.4 FPS higher performance in Windows 10. With two Titan V GPUs we also saw a small difference, but that was only by less than a single FPS.

If you are curious about what the overall results would be if we only used "Full Res." decode quality with the RAW media instead, those results are also available in the charts below:

From a pure performance standpoint, we surprisingly have to give a slight edge to Windows 10. Frankly, however, the difference is small enough that you probably shouldn't base your OS selection on performance alone. If you already have a firm reason for using Linux, then your decision is likely already made for you. However, for those that have not used it before we do want to point out some major considerations that you should take into account if you are considering Linux:

1: Linux has a high learning curve

If you have not used Linux much in the past, be prepared for a lot of Googling and banging your head against the wall. Even something as seemingly simple as installing software usually requires the use of the terminal (similar to Window's command line) and commands that may look like a random string of characters. In addition, if drivers are not installed in the proper manner you can run into issues as serious as the OS no longer booting if you run software updates. The nice thing with Linux is that it's open nature means almost anything can be fixed but we highly recommend only considering Linux if you or someone on your IT staff has extensive experience with it.

2: DaVinci Resolve has very limited audio support in Linux

If you want to use the audio built into your system or even an external DAC, you are most likely going to be out of luck. At the moment, DaVinci Resolve is only able to output audio in Linux through something like a Blackmagic DeckLink or Intensity card. This will send the audio through HDMI or SDI, however, so you also need a grading monitor that supports audio out - hopefully in the form of an audio jack so you can use speakers or a headset rather than any built-in monitor speakers. We do want to note that even with a DeckLink Studio 4K we have had some audio issues in Resolve. At times, the audio would be just fine in the Edit tab, but in the Color tab it was garbled and filled with static. So if audio is something you need, Linux is likely not the right OS for you.

3: Not all related software works in Linux

In particular, we want to point out that Adobe applications like Premiere Pro, After Effects, and Photoshop do not work natively in Linux. Due to this, be sure to double check that any applications you use in addition to DaVinci Resolve support Linux.

4: Not all hardware has Linux support

This could be anything from external devices to hardware integrated into your motherboard. As an example, the Thunderbolt 3 controller built into the X299 system we used in this testing does not currently work in CentOS and actually causes the OS to hang for 10-15 minutes during startup.

This may sound overly critical of Linux, but the truth is that while Linux is not at all bad and in some situations is completely the right choice, for most DaVinci Resolve users Windows is the better option. If there was a significant performance benefit it may be different, but given that Windows is actually slightly better for performance overall, Linux and CentOS in particular is something we are not going to recommend for most of our customers at this time.

Tags: DaVinci Resolve, Windows 10, CentOS, Linux
ralphtweety

In the realm of hair splitting I also noticed another trend that comes out in your research: It’s small, but, I think it’s important to note that, in general, the more video cards you added to each of many of your tests, the faster the Linux OS tended to perform when compared to Windows 10. Not in all cases, but more often than not. I’ve long been suspicious of this when comparing DaVinci performance on any of the three OSes it supports. My mind-set is: When using Resolve I want the OS platform that best utilizes the most video cards and I still think Linux tends to pull ahead in this regard. In the work flows I’m connected with, rendering, as final output from Resolve, is another key issue where we want to make sure we’ve got the best machinery for shortening flattening times.

Posted on 2018-03-27 12:24:54
mtelesha

One big advantage for Linux over Windows in video editing is just the ability to customize your system to exactly what you need. When I just need to only do video editing without the need of other Window dependent hardware I prefer Linux because I know it is stable and setup exactly as I want it and it will not change. Windows updates can totally cost me hours to get things back to where they were before. So for sheer efficiency I prefer Linux's update system and customization.

Posted on 2018-03-27 14:47:38

I think you hit the nail on the head for the biggest single reason to use Linux. The only downside is that it takes a lot of experience to really get things configured in a proper manner. Plus, if you want to update to the latest version of Resolve you probably should undo some of it so you can also update things like the video drivers and maybe the kernel as well.

I think what it really comes down to is that if you have a solid reason to use Linux and have the knowledge/experience to do things properly, Linux can be an excellent choice. If you don't have both of those things, however, it will probably end up costing you much more in terms of time and frustration than using Windows. We have a good amount of experience with Linux, but I can tell you that getting it up and running for this article was quite a hassle. Using the latest hardware always means more work when it comes to Linux, but I was surprised at how difficult it was to even get things like audio working properly. All the more reason to completely lock down the OS once you get it working like you want though!

Posted on 2018-03-27 16:55:02
Zé Cotinha

Windows brings you everything chewed and easy to configure and customize.
I do not know why losing time using linux when all benchmark around the world show that Windows 10 has no competitor.
Even Apple left the ship with the PCs for rendering.

If Linux were really good as a CUSTOMER, some company would have invested in it to create a robust system to fight Windows.

Linux on Android is a complete disaster and is the only true example of what a famous Linux distribution would be like for the casual public. Totally insecure and buggy.

Posted on 2018-04-15 14:20:25
glassworks

Interesting and most big film post/vfx companies use Linux.
Actually I believe all of those big ones. The funny thing is I started with Unix/Irix in the 90' and later Red Hat and I don't even know how Windows Os looks like. Bizarre but true. Cheers!

Posted on 2018-04-17 16:02:53
Zé Cotinha

For what the hell are you going to customize a client operating system?
This is bullshit, because even Windows 10 Home / Pro is customizable because I can choose which programs start with Windows and I can also choose the services that can start with it.

Even with Benchmark showing the superiority of Windows to Linux, Linux fans close their eyes not to face the truth.

They waste time customizing Linux, when Windows 10 already has everything ready to run any program.
Not to mention that DirectX has a library and performance superior to Vulkan / OpenGL.

Posted on 2018-04-15 14:15:51
mtelesha

Slow down and breath. Yes I can customize my Linux desktop more then anything I can do on my Windows 10 system. I have different file managers and I can right click and trans-code my videos and audio super easy. I use Dolphin (KDE file manager and I also use Ranger which is where I have custom scripts that just run my trans-coding. Heck I have a right click where I run a racket script that builds my folders for me I can do all of that on Windows BUT it isn't a simple shortcut nor a simple right-click.

Sometimes I use a traditional desktop with KDE but more and more I use a tiled Window Managers i3 and I use a keyboard 99% of the time. I have my scripts (I write most in Racket these days and not so much Python) and execute them automatically. This isn't what most people want but it works for me. |

I am also a rare breed that never hated Microsoft or Windows and saved all my rage against OS X and Apple hardware. For video editing I have no idea how they do it on laptops.

Posted on 2018-04-16 17:23:01
Zé Cotinha

If you really care about your files it seems to be for professional use. I would be using Windows 10 Pro with OneDrive for backups.

Using scripts seems like nostalgia to me and not a professional method.
Disk management for Windows 10 Pro or 10 WorkStation is light years ahead of any other linux distribution.
If you have a PC with multiple SSDs or HDDs, Windows 10 would certainly manage much better and make a cloud backup with greater security.

If you do not have the patience to learn to mess with Windows 10 and the new tools, call a professional who has specific knowledge for Windows.

Posted on 2018-04-17 15:14:39
Aurélien Pierre

Not a professional method ? You kidding right ? Is that not professional to write a script once and have it run for years without you worring ? The whole concept of scripting is to make the computer work for you automagically. Scripting is enhancing the OS to fit your particular use. If it's not professional, I don't know what it is.

Posted on 2018-05-29 03:14:30
Aurélien Pierre

Ahahah ok, customization here doesn't mean adjusting the desktop effects and background paper. It means scripting the shit out of it and automatize every bit. Benchmarks show what you make them show : on raw performance Linux and Windows are usually in a tie, but on general overhead you can optimize a Linux distro to remove any unwanted piece of code and make your OS footprint much smaller.

You know, Linux trusts 99.99 % of the supercomputers market shares, and 80 % of the server market for a reason. And as a sysadmin, I much prefer to deal with bugs on Linux than on Windows (and I have had every Windows since 95, and every Ubuntu since 8.10).

Posted on 2018-05-29 03:12:09
Richard Staller

Yep, Linux is the go to solution for large production companies that have millions of dollars to invest in IT support and custom software. It's not a solution at the moment for the SMB creative shop because of its technical support requirements but more importantly due to a lack of support by the major creative video and 3D application companies. And this is from a veteran Linux user. Right now because of Apple's willful abdication from the pro marketplace sadly Windows 10 is the only game in town for SMB graphic shops.

Posted on 2018-10-03 11:52:11
GV100

https://nvidianews.nvidia.c...

https://www.nvidia.com/cont...

Can you please benchmark the new Nvidia Quadro GV100?

Posted on 2018-03-27 17:46:47

We have a GV100 at our booth at GTC right now, but when we get everything back from the show I may do a bit of benchmarking with it. The problem with Quadro cards in general is that they are really expensive. The GV100 should be a bit faster than the Titan V based on the paper specs, but even if the performance is excellent, it is going to be really hard to justify the cost in something like DaVinci Resolve where you can simply use a couple GTX or Titan cards to get higher performance at a much lower cost.

Posted on 2018-03-27 17:54:32
Resolve

https://www.blackmagicdesig...

https://www.blackmagicdesig...

http://forum.blackmagicdesi...

Can you please test DaVince Resolve 15 once it's out of beta?

Posted on 2018-04-11 09:58:04

We definitely will be! If I had to guess though, it will probably be a while before it is out of beta. But once it goes live, we will certainly be taking a very close look at it.

Posted on 2018-04-11 14:27:15
Zé Cotinha

I find the dishonesty of Linux users incredible. Since Windows 7 Microsoft has absolutely outgrown all Linux client systems. And yet people keep spreading nonsense about Windows.
Even rubbing in the face the greater performance of Windows in the face of Linux, the boring gossip continues talking nonsense.

The guy loses time learning to mess with the Linux client but does not know how to set up a Windows 10 to do Updates only when he can or wants to do.
Not to mention human stupidity in installing any .EXE on Windows with viruses. That's why Microsoft created the Windows 10 Store to bring together secure programs for these lay users who only install crap on their Windows.

Posted on 2018-04-15 14:12:45
mtelesha

Once again, I use Windows 10 all the time and I have never been a Windows hater (I hate Apple hardware and OS since the 80s). I think you have left the realm of constructive discussion. I have no idea where this Linux user hate is coming from and why you seem to think I am engaging in anything dishonest here? Linux has been a preference for many different types of users for years.

How am I or other Linux users being dishonest if anything when we express our opinions and our personal experiences. If you think the .exe without viruses is beyond a typical Linux user I think again you lost the realm of constructive criticism. Also I am guessing you might not have never really programmed. About everything you do in a Resolve is really advance scripts and most other gui programs are a string of buttons and scripts. Scripting I use in Windows 10 and yes I even use PowerShell and run my Racket directory scripts. It's just not as elegant as Linux.

Linux in the video editing world has now been going on close to two decades. Pixar has used Linux for years and Linux is a good choice for certain people. https://www.digitaltrends.c... Pixar released this software first in Linux due to that is what they use.

Posted on 2018-04-18 20:16:05

Dang you almost had me considering ditching windows on my ryzen 7 machine and giving resolve on linux a go, until the audio thing. oi.

Posted on 2018-08-24 19:16:15
jahid65

Native audio works in DR 15. No need of extra hardware.

Posted on 2018-08-25 14:50:06

WOW. Bye windows? I might give it a shot soon. Thank you.

Posted on 2018-08-28 03:12:39

OMG ::dies:: i guess i'm trying it.

Posted on 2018-08-29 17:08:06
R. Adam Berk

I think you guys are missing something worth considering that could be measurably impacting the results of many of your Resolve benchmarks, and making those results something other than "real world".

I suspect that many of these motherboards do not actually have the ability to support more than any two pci-e cards at full lane-width/speed. This would mean that if you're testing a "real world" Resolve configuration that includes at bare minimum a Decklink card and 1xGPU, your performance measurements might be taking a hit after adding the 2nd/3rd/4th GPU that's being caused by an environmental factor that you guys are not taking into account, or at least failing to mention in these blog posts. Most "real world" systems found in the wild would more often than not be found with at least an additional storage controller, or ethernet card (10gb/40gb/etc) if not both, in addition to the Decklink and GPU(s).

I have a very strong suspicion that when you guys add more than 1x GPU, you are actually cutting the lane-count that the cards are connected at down to half or maybe even less. This could certainly be a major cause for some of your benchmark results that make less than perfect sense- like when you find 2xGPU's to be faster than 3xGPU's, etc.

There is a very good reason why the blackmagic configuration guides have always included very very very specific instructions on which systems/motherboards to use, and which pci-e slots to put each type of card into. There is also a very good reason why most "real world" Resolve systems you'll find in LA/NY/SF/etc include some type of pci-e expansion chassis if they are using more than 2x GPU's. There is a very good reason why none of the blackmagic configuration guides, except maybe the most recent one, have ever included any supported configurations that host more than 2x GPU's on a single motherboard without a pci-e expansion system involved.

Posted on 2018-10-14 05:35:24