Wednesday October 20, 2010

Benchmark Wars

You guys know we shitcanned benchmarks years ago because we found those unreliable as indicators of gaming performance and GPU cross-comparisons. While we have moved on, the GPU companies have not. Yesterday we got an email from NVIDIA about a new H.A.W.X. 2 benchmark and the company encouraged us to use it on upcoming reviews. Today we got an email from AMD asking us not to use it on upcoming products.

It has come to our attention that you may have received an early build of a benchmark based on the upcoming Ubisoft title H.A.W.X. 2. I'm sure you are fully aware that the timing of this benchmark is not coincidental and is an attempt by our competitor to negatively influence your reviews of the AMD Radeonآ™ HD 6800 series products. We suggest you do not use this benchmark at present as it has known issues with its implementation of DirectXآ® 11 tessellation and does not serve as a useful indicator of performance for the AMD Radeonآ™ HD 6800 series. A quick comparison of the performance data in H.A.W.X. 2, with tessellation on, and that of other games/benchmarks will demonstrate how unrepresentative H.A.W.X. 2 performance is of real world performance.

AMD has demonstrated to Ubisoft tessellation performance improvements that benefit all GPUs, but the developer has chosen not to implement them in the preview benchmark. For that reason, we are working on a driver-based solution in time for the final release of the game that improves performance without sacrificing image quality. In the meantime we recommend you hold off using the benchmark as it will not provide a useful measure of performance relative to other DirectXآ® 11 games using tessellation.

That last paragraph is fairly damning in my opinion and it raises some big questions. Is Ubisoft in NVIDIA's pocket and pushing technology while ignoring the company with the largest DX11 market share? Is NVIDIA pushing a benchmark of a yet-to-be released game that is somewhat broken on its competitors cards? We know NVIDIA's current GPU has more Tessellation power than AMD's latest, but we have yet to see it make a difference in anything besides a benchmark. I think this shows NVIDIA grasping at straws and I don't think it has anything up its sleeve that AMD does not already have as well; like refinements in TSMC's 40nm process.