• Welcome to Overclockers Forums! Join us to reply in threads, receive reduced ads, and to customize your site experience!

Crysis Benchmarking Tool

Overclockers is supported by our readers. When you click a link to make a purchase, we may earn a commission. Learn More.

PhysX

Member
Joined
Nov 27, 2006
Location
Calgary
Benchmarking Tool for Crysis.

Executive Summary:
The tool provides a robust front end to benchmarking Crysis. Provides ability to queue up many runs and will provide detailed results for each test as well as an overall summary with accurate averages. The first and foremost goals were simplicity and reliability in the tool and in the benchmark scores. Works with retail and demo.

How To Use:
Simply queue up several different runs mixing settings, resolutions, antialiasing levels and click "Begin benchmark Run".
You can add a frame number where screenshots can be made on every frame for comparison. Use Negative numbers to make a screenshot on every N frame (ie: -399). Hit view button to to view them.
For advanced users you can add custom cvars or import a cfg file that will run globally on all tests.
There is no installer/uninstaller, just unzip and double click to use.

benchmarktoolresized2yo9.gif


Notable Features:
  • - Run a variety of tests all at one time.
    - Choose windowed, 32/64 bit, DX9/DX10 tests.
    - Logs setting details for each test run plus provides a summary breakdown (with averages if 3 or more loops).
    - Supports custom demos.
    - Allows pause/resume by hitting spacebar, between runs.
    - Auto-save all benchmark runs in Autosave folder.
    - Allows Copy/Paste from results window.
    - Save and import a queue of runs for ease of retesting.
    - Allows image quality comparisons by taking a screenshot on each run at certain frame(s).
How It Works:
It doesn't change or alter any game files. Except it uses the same log file that the Crysis timedemo does to log the results, that is the only game file altered in any way, although it's just a benchmark log file. There are no batch files used, this is strictly application controlled. The tool will go through the queue one by one. Between test runs you will have an option to pause for 3 seconds (visible with countdown timer) just hit the spacebar to pause and it highlights the test it will run next if resumed. A Summary is provided at the end for reference and details are provided prior. If 3 or more demo loops are set for each run, then the summary will provide an average of all loops except loop 1. For averages the first test loop is ignored since it is cacheing assets on the first loop. If only 2 runs are selected, the averages shown are not 'overall' averages but 'last' average, meaning it will provide the average from loop 2, again ignoring loop 1 wherever possible. All Completed demo runs are saved in the autosave folder by date/time for convenience.
It will detect and notify you if you have any configuration files that may skew benchmark results. Uses same test settings as the batch files, so scores are completely 100% compatible.

The tool provides zero overhead during tests. There is no program logic operating in the tool while a testing run is going, it is only waiting for a signal back from the operating system that the demo loops are done, then it proceeds to start the next run. CPU spikes will be seen between test runs but this doesn't effect the runs at all, since it is essentially asleep during runs. In most cases the tool will find where the game or demo is installed, if you want to specify a path for some reason then go to File/Set Game Path (like if switching between full release and demo).

Limitations:
There are currently a few limitations with the first release that will be added to later.
- Low/Medium/High/VeryHigh global settings can be mixed and matched in the queue as desired but only one custom configuration (mixing individual quality settings) can be mixed into the queue. In other words many custom runs can be added to the queue but it will only use the same custom settings for each run. The custom quality benchmark run will be run with the quality settings that are currently selected at the time that the 'Begin Benchmark' button is pressed.
- For any advanced optional CVARS you add, they will be global to all tests and not on a per test run basis.
- Currently while many resolutions are provided, not all may be supported by your monitor.
- DX10 option will show up in Vista if you don't have DX10 hardware. Choosing DX10 will revert to using DX9, although the log output will show DX10 erroneously. I do not have directx API logic in the tool yet to differentiate HW.
- Vsynch currently disabled as there were issues with it working in demo, will be re-enabled in soon update.

Plans:
I plan to remove the current limitations listed above in a near future release. I also am planning advanced testing options to show video/RAM memory usage as well as per core CPU usage.
Will add DX API support for better resolution detection options.
I also plan to have an advanced tweaking section that I will add to the tool for people that want to test the performance of their tweaks.
Possible reports/graphs and exporting options depending on the community desire for it.

Download and Requirements:
Only tested on XP and Vista 32/64, the tool detects them appropriately.
There is no installer/uninstaller, just unzip and double click to use.
Requires .NET framework. You probably have it installed but if problems arise in running the tool it try these downloads from Microsoft:
x86 .NET framework redist
x64 .NET framework redist

You can download the tool here at these links:
Filefront original release 800k
Rapidshare original release 800k

Enjoy!

Feedback would be appreciated to make it a better tool!
Ask questions and post your results here if you like.

Here is one of my benchmark runs:
Q6600 @ 3 GHZ
8800 GTS 640 OC (580/1456/975)

11/13/2007 7:41:55 AM - Vista 64 (DX10 game default)
Run #1- 1024x768 AA=No AA, 32 bit test, Quality: High ~~ Overall Average FPS: 45.23
Run #2- 1024x768 AA=4x, 32 bit test, Quality: High ~~ Overall Average FPS: 36.725
Run #3- 1280x1024 AA=No AA, 32 bit test, Quality: High ~~ Overall Average FPS: 32.65
Run #4- 1280x1024 AA=4x, 32 bit test, Quality: High ~~ Overall Average FPS: 25.98
Run #5- 1600x1200 AA=No AA, 32 bit test, Quality: High ~~ Overall Average FPS: 24.37
Run #6- 1600x1200 AA=4x, 32 bit test, Quality: High ~~ Overall Average FPS: 15.96
Run #7- 1024x768 AA=No AA, 32 bit test, Quality: VeryHigh ~~ Overall Average FPS: 25.07
Run #8- 1280x1024 AA=No AA, 32 bit test, Quality: VeryHigh ~~ Overall Average FPS: 17.465
Run #9- 1600x1200 AA=No AA, 32 bit test, Quality: VeryHigh ~~ Overall Average FPS: 12.705


Props to MadBoris for that :)
 
Excellent writeup and thanks for this PhysX :beer:...you should post this under "General GPU" section as well, will be very valuable to compare latest and older video card and give members indication what to buy for Crysis type games
 
Back