- Tuesday , October 23, 2001
- Kyle Bennett
Optimizing or Cheating Radeon 8500 Drivers
This is a follow-up to our Radeon 8500 Review, where we found some interesting points about the drivers that we did discuss with you. This is simply another look at that issue with a bit more information that will hopefully raise the question, "Is ATi cheating at benchmarks or simply making their drivers for all those Quake 3 players out there?"
Now this all starts out with a game that we are all familiar with. Not only have many of us played it, but most likely the target market that ATi wants to sell the Radeon 8500 to is very familiar with it. Quake 3 Arena is not only a staple to many online gamers, it is also the de facto 3D benchmark in the world at this time. Yes there are others out there, but chances are even your Mom knows what Quake 3 is. Yes, even guys that write computer hardware magazine articles and have never played a first person shooter know how to run Q3 benchmarks...and they use them.
What we think we have found out is that ATi has written their most recent public driver set with Quake 3 Arena in mind. How is that you ask? Well, it seems that there are Quake 3 specific instructions in the Windows 2000 Radeon 8500 drivers version 5.13.01.3276 that are available as the latest driver download from the ATi site.
Now what exactly does "specific instructions" mean? It is our opinion that ATi has written their driver to identify when you are running Quake 3 Arena and give you better frame rates when benchmarking and playing the game. Herein lies the moral dilemma that many of you will not see as a dilemma at all, while others do. Is it wrong for ATi to deliver drivers that give better frame rates in a specific application? Should they be optimizing for the game engine instead? There certainly are other games out there based on the Quake 3 engine that could benefit from it. Now I am not going to lead a crusade here, but rather go about giving you the facts and allowing you to make up your own minds.
We are going to be using some drivers and applications that were put together by readers and friends. We will not be distributing these files as I am pretty sure it is in violation of the ATi EULA. So don't ask us for the files please.
This is the application we used to show you the benchmarks at the end of our Radeon 8500 Review. Here is the complete description of the app.
The reason for doing this is to isolate drivers that are built to recognize the string "Quake" in order to score higher on the Quake3 benchmark.
Here are the steps:
1. Copy the Quackifier.exe program to the Quake3 folder where the Quake3.exe is located.
2. Launch the Quackifier.exe program. The Quack3.exe program will be created automatically in a couple of seconds.
Run Quack3 w/ the same settings as Quake3. Assuming that the driver is not using the app. recognition cheat your results should be the same as the results for Quake3.exe.
These are our original results from the review.
Now what you are seeing here is a comparison of the Frame Rates before and after using the Quackifier. The top three scores are post-Quack, the bottom, pre-Quack. You see frame rate increases of up to 18%, which in this day and age is a pretty big jump.
Now when we posted this last week we not only got tons of feedback, we got a lot of help also. We immediately started getting feedback from programmers with skills far beyond mine, looking deeper into the drivers.
Nick was the first to send us information and this is his email in full, minus the pleasantries.
I downloaded driver version 5.13.01.3276 for w2k off of ATI's website and extracted the files to where my Sun box could see them. There is a useful little command on Unix called "strings" which searches for text in binary file, I assume you are familiar with the command "grep" which only prints out lines that match. I did some hunting with this and found the following:
strings atioglxx.dl_ | grep -i uak
This suggests that there is at least the text "uake" in there. This apparently being the ATI OpenGL driver from the file name, it looks a bit suspicious to me!
Well, we posted Nick's email and it spawned some more activity. Hywel was considerate enough to send a screen shot with his mail!
Hi guys - I saw your article mentioning someone had found "uake" in the ATI Open GL driver for the 8500 - this is definitely Quake - I did the same as him and used my editor which can view binary files to find the string. Here is the direct reference to Quake 3 near the end of the file - this is from the WinXP version. This picture shows the hex dump and ASCII text interpretation.
The Quake3 reference stands out like a sore thumb. Especially with a big red box around it eh?
Then just when we feeling at ease about our decision to publish the Quack benchmarks we got this mail from Joe MacKay.
The Zip file includes two files, ATIOGLXX.DLL and ATIOGLXX.OLD, the .dll file is the modified file while the .old file is the original, you should just be able to replace your .dll file with the modified one, and see if any changes occur! If you need the entire driver install set let me know I can send that to you as well.
It is possible these drivers wont work at all, however with my past experiences w/ hex editing you are usually OK if you modify ASCII strings and keep the number of characters the same, which I did... If you need or want more information just contact me using the below information...
When I got this I about fell out of my chair. There is no doubt that we have the greatest community in the world when it comes to us hardware geeks helping each other out!
So off we went with drivers in hand and a new install of Windows 2000 Pro on our AthlonXP 1800+ test system with an MSI KT266A mainboard. We ran all of our numbers again, on the fresh install. First we installed the ATi drivers along with SP2 and DX8.0a and then proceeded to install Quake 3 Arena. We patched our version to 1.17 and outside of that, did not do anything else to alter the install. We set all system settings and game options to the highest possible configurations and use the resolutions noted. Here is what we got for benchmarks running Demo001. Frame rates were run four times and averaged.
1024x768 - 157.88
1280x1024 - 129.70
1600x1200 - 94.95
We then applied our Quackifier patch and ran the Quack.exe file to launch the game under the same exact settings. Please keep in mind that these results are NOT directly comparable to the ones graphed above.
1024x768 - 143.93
1280x1024 - 111.85
1600x1200 - 81.25
Then we formatted our hard drive and did a fresh install of the OS along with SP2, DX8.0a and Quake 3 Arena and the v1.17 patch...again. We installed our ATi drivers and replaced the ATIOGXX.DLL file with the modified DLL that we were supplied with above by Joe. So to recap, we are running a normal version of Quake 3 with a modified ATi driver. Notice that our Quake.exe scores now look almost identical to our Quack.exe when the "uake" in the ATIOGLXX.DLL has been changed to "uack".
1024x768 - 143.75
1280x1024 - 112.28
1600x1200 - 81.18
Lastly, we were holding the best for last. We took our modified "Quack" drivers and used them with our Quack.exe version of Quake 3. Sadly, this did NOT produce the conclusive results we were wishing for. We expected to see results identical to the Quake.exe/Non-modded driver scores Running Quack with the above drivers simply gave us the same "exact" numbers that we did above when using Quake3.exe and the Quack drivers.
Several of you have requested screen shots of the Quake and Quack screens to compare. Here is a 2MB file that has two .tga files, 24bit non-compressed screenshots that you can look at side by side. There are some obvious differences in the quality of the screenshots if you care to examine them closely.
The Facts As We See Them:
It certainly seems to us here at [H]ardOCP that ATi has in fact included application specific instructions in their version 5.13.01.3276 Win2K drivers that make Quake 3 arena benchmarks faster by up to over 15%.
The Real Question:
Does this really make a difference?
There are those that will argue that ATi is in fact cheating at the benchmark game by producing and distributing drivers that are aimed at increasing scores in what is the most widely used 3D benchmark in the world. The other side of that argument is that ATi is simply building a driver set that is aimed at giving Quake 3 Arena players the smoothest gaming experience that ATi can deliver.
We are not going to interject a lot of opinion on this matter except to say this. We think that ATi should be producing Radeon drivers that are 3D engine specific and not game specific. Especially when the one targeted game is a widely used benchmark that people trust.
As we mentioned we are not out to start an anti-ATi crusade over this matter as the Radeon 8500 already has a great battle in front of it in the battle for market share. Still we think it is important in situations like this that you are armed with this type of information when evaluating your next VidCard purchase. ATi engineers were asked last week if ATi drivers used any game specific instructions and we were told "No."
In closing we would like to say that all the same testing was run on the latest set of NVIDIA DetonatorXP drivers without any of the same issues. Your thoughts and opinions are always welcome HERE with us as well.
3D Center in Germany has done an awesome job in looking at the visual quality of Quake 3 Arena when using the ATi drivers we have looked at here.
Also, ATi has given a statement pertaining to this issue.
John B. Challinor II APR - Director, Public Relations at ATI Technologies Inc. "ATI optimizes its drivers on many different levels, including the application level, the game engine level, the API level, and the operating system level. That is, some optimizations work only on specific games, while others work only on specific game engines or only on specific operating systems. In the case of Quake III and Quake III Arena, we were able to achieve certain optimizations specifically for that game, as we do for other popular games. Our engineering team is committed to providing the best visual experience for the user in all applications, with the optimal combination of high frame rates, image quality, and stability."