AMD & NVIDIA GPU VR Performance: Project Cars

If you like racing games then it is likely that you are familiar with Project Cars. This racing game can push a system to its limits when you turn up the visuals, and that is just what we are doing today while using our HTC Vive VR headset. And this is certainly a game we will add to the VR Leaderboard.


How We Got Our Results

Our regular readers know that we are not "benchmark" guys and like to validate PC gaming hardware by actually using it for gaming. I know, call us crazy...and some folks still do. Today’s results reflect "almost" exactly that, real world gameplay. I changed things up a bit for Project Cars.

Project Cars allows you to set up races on specific race courses, with the car of your choice, with up to twenty competitors, and this is exactly what we did. I drove a Boss 302 Mustang up a 3 mile run on a California highway in sunny weather. Rainy weather effects in Project Cars can greatly impact performance so our runthrough is far from a "worst case scenario." This race takes a little over 2 minutes.

Article Image

I mention above that this is "almost" real world gameplay. As you might expect, getting comparable runthroughs in a driving game is extremely difficult. I went back and compared GPU Render Time data from live runthroughs with GPU Render Time data from the recorded runthrough. What I found was that the recorded demo returned results are extremely comparable to the original gameplay. The recording was generally 1 tenth (0.1) to 2 tenths (0.2) of a millisecond faster in GPU Render Time. This can probably be attributed to the replay not having the full information overlay on-screen. Basically a recorded race is representative of realworld gameplay. Given that we could repeat the recording multiple times with performance data being near identical to the original gameplay I decided to use this model this time.

Every Graph is the "Same"

Our data collected here is probably as close to "the same" as anything we have done in our VR Review series yet. The one big difference between our runthrough and what happens while you are actually driving is that I try my best to keep the HMD statically positioned through the entire race. Of course while you are driving, you are looking side to side, and this may also attribute to our recorded race being slightly faster to render as well. The camera position is above the roof-line of our Mustang as pictured above.

Every VR Framerate is the Same, 45 or 90, unless You Count Reprojection

You have likely heard this already. VR headsets, and I will be talking particularly about Vive technology here, although Rift does the "same thing" when looking at the big picture, all run at 90 frames per second. Your first thought is probably, "Why?" This simple answer is that if they do not run at 90fps, you will get motion sickness, and blow chunks. Seriously. But why would you not just run at 89fps instead of dropping to 45fps? Good question. The answer is tearing. Tearing frames is totally unacceptable in a VR game, whereas we may put up with it without consequence on the desktop. So there are some "V-sync" things at work with these new head-mounted displays that leaves us with 90fps, or 45fps if the hardware can’t keep up, and 45fps leaves you with lunch on your shoes, so something had to be done.

Both Vive and Rift have technologies to combat the dreaded 45fps. (I have been told by several people that if you get drunk before you play, that 45fps is a lot more acceptable to users, but I have not tried this. Please send me mail.) Rift’s is Timewarp, and Vive’s is Reprojection. For the use of this article we are going to call these technologies the "same," however these are not. But they do solve the same common problem in much the same manner.

When your Vive is not able to keep a 90fps pace while you are playing a VR game, Reprojection technology kicks in and tells your hardware to render at 45fps and let Steam VR tech fill in every other frame. Pretty slick eh? So in effect, your eyes are still seeing 90fps and therefore you should not vomit on your shoes. The caveat is that Reprojection technologies do have some drawbacks as artifacting can be present in your game. I think Reprojection for the Vive is likely to be a game to game thing. Remember how I said "Reprojection technologies" just above there? That is because there are several ways of doing Reprojection, so your mileage may vary from game to game. Today we are talking about Raw Data and overall, I could tell the game was a lot snappier when my Vive was not in Reprojection, but honestly, I could play for long periods of time with it in Reprojection and it did not bother me. So 90fps is better than Reprojection, and Reprojection is better than 45fps...even though Reprojection is running at 45fps as far as your GPU is concerned but the Vive hardware makes it look as much like 90fps as it can.

In our graphing, and I will remind you of this as we move along, when you see the red line at 1, that mean NO Reprojection is happening. When you see the red line at 2, that means that the system is using Reprojection. You could say Reprojection is "bad," but it is certainly not as bad as 45fps. Trust me. We will call Reprojection "not ideal" instead of give it a bad rap as it is surely there to help us out.

Dropped Frames

On our graphs you will also see a yellow line, this tells you how many dropped frames there were. There will be some argument about how this is shown, because when I show you 200 dropped frame across an X axis that has 50,000 data points on it, we simply cannot draw the lines small enough to actually represent how small of an event this really is. So on some graphs, you will see a lot of yellow and think, "holy crap, this GPU was sucking bad" when it comes to rendering VR frames. Well, the way it is graphed can make it look much, much, much worse than it is. This is why we give you the actual number of frames dropped along with how many actual frames that were drawn. And of course we will discuss that as well.

The Meat, the Taters, the Milliseconds

We have saved the best for last and that is GPU Render Time shown in milliseconds. Since we know that our hardware has to render frames at either 45fps, but hopefully 90fps, how do we really measure what the hell is going on with our GPU. We have examined this from a few ways, and the VR purist will probably tell you that the VR experience must not allow the system to go into Reprojection mode with the Vive, or Timewarp with the Rift. And that would probably be correct, but the fact of the matter is that is simply not going to happen at this early stage in the VR game, if ever. The person a step down from that would probably tell you that Reprojection, with minimal dropped frames is acceptable from an experience point of view. I am probably in this group right now. The next person down the stack would likely tell you, "Turn off Reprojection/Timewarp and ride it out like nature intended." That person is likely not the one you want to listen to in my experience. Seriously, different folks react to these technologies differently, so there is not telling what you might find acceptable gameplay. The only way to do that is to figure it out for yourself.

At 90 frames per second your GPU must render a new frame every 11.1 milliseconds. You will see our graphing has hash lines at 5.55ms, 11.1ms, up to 22.2ms. If the GPU takes more than 11.1ms for a couple of concurrent frames, it turns on Reprojection like we talked about above. If you do not have Reprojection turned on, it will cut your display rendering to 45Hz, since frame tearing is totally unacceptable (I have never seen tearing on a VR headset, but I just guess it is horrible.), then you puke.

But really 11.1ms is not the real number either. Once the Vive hardware and Steam VR software see that you are getting close to busting 90% of 11.1ms, the software starts having the hardware make changes. My explanations here have already gotten too long, but there are 1000s of pages of documentation that you can wade through to figure out for yourself online that will very likely put a lot of things about VR into perspective. I would highly suggest watching this lecture given by Alex Vlachos of Valve on Advanced VR Rendering Performance as it will give you a lot of insight as to what we have discussed here if you need more information.

So to sum it up.....we want 90 frames per second always. To get 90 frames per second always, our GPU must be able to render that frame in less than 11.1 milliseconds. If our GPU does not render a frame in less than 11.1 milliseconds, it is going to turn on Reprojection in the Vive, and Timewarp in the Rift. If the GPU is still having trouble keeping up, it is going to start dropping a lot of frames. And Reprojection with dropped frames is BAD.