As I discussed last week, I wanted to make the benchmark more modular so I could only test the CPU or GPU as needed, or both. I settled on radio buttons. This gives me three states to work on, CPU, GPU, or Both. I did some test scripts to see if I could capture the states, and sure enough, it worked great.
This is the version I'm going with for now. Keeping it simple.
Image
Then I translated it to the full script with 8 tests, 6 CPU, 2 GPU. The first script launches a UI allowing me to choose any of the three options, CPU, GPU, or Both. Clicking the start button would launch the “master” script. This has “If-Then” statements around each test. If the correct state was chosen, it would run that test. And it didn’t work. No matter what radio button was checked, it would alway run the same set of tests. I double and triple check the logic. Everything is correct. I go back to my test scripts, and they work fine. I comb through these scripts line by line for two days, rewriting them over and over, plugging bits into the test script where things appear to work on a small scale.
I decided to take a break, get some fresh air, and even take a second shower that day. When I came back, I read through the script one more time, and I saw it. “On Rdo_1 changed stat do” This looks right except I needed to be referencing a changed state, not stat. All that work, all that stress, just because I left off an E. At least now it's working. Or, at least the logic behind choosing which tests to run is working.
All the individual tests are running correctly, and reporting results to a text document just fine, except for one. The always problematic Viewport FPS test. Because of the issues I noted last week, I have kept it as the last test to be run. All the other tests run, but then when it gets to that test, nothing happens. If I run that script by itself, it works just fine. It plays the 1000 frame animation and reports back the FPS. However, when that script is called from another script, nothing happens.
To help diagnose I put in a couple simple “Print” commands. One at the beginning of the script, and one at the end, just so I can see if the script is running at all, and if so, where it is breaking. Rerun the benchmark. Both print commands print as expected, but no animation plays, and no result is given. This is beyond baffling. I spend a couple more days digging through documentation and forum posts trying to see why it works in some instances and not others. Was it the callback function? Can you not have a script launch another script that has a callback? Really beginning to feel I’m just spinning my wheels on this specific test.
Then I see Autodesk has launched a new version of 3ds Max. it is now 3ds Max 2021, where everything I had been working on was for 3ds Max 2020. Looking through what is new, there is a lot worth upgrading for. They have finally, FINALLY, updated the viewport to be physically based by default. They made significant changes to saving, especially on large files like in my benchmark, made a whole new texture baking tool and more. Since I wasn’t finished yet, and probably need to redo the whole Viewport FPS test, I figured I might as well update and make the benchmark for 2021.
A couple scenes needed to be updated to match the new configuration. Thankfully they also updated their scene conversion tool, so this was a fairly easy process. Once everything was updated, I held my breath and ran my script. I assumed several things would be broken. Much to my surprise, everything worked. Even the problematic Viewport FPS test that was not working in 2020. I have no idea what happened,or why it is working, but it is working.
This is the output from three different passes showing the correct test were performed or skipped.
Image
At this point, I have a few things that need to be cleaned up, like making sure there are no local paths being referenced. Then I’ll begin packaging everything up in a way our installer can use and seeing how to deploy this to all of our test beds. I’m getting pretty excited to see how this benchmark runs on a variety of hardware.
As always, please feel free to leave a comment below, or on twitter, and be sure to subscribe to be notified when more posts go up.