Questions and Answers:
-
What is SPECcpu2000 anyway?
SPECcpu2000 is a set of 26 applications taken from different
disciplines intended to provide a cross section of the
basic computing capabilities of a vendors hardware
and optimizing compiler.
There are 12 that are fixed point arithmetic, most similar to
commercial workloads like databases, webserving, etc.
There are 14 floating point applications where the CPU time is
heavily influenced by floating point calculations which are indicative
of what you might find in the solver portion of an application like
NASTRAN or ANSYS.
The CINT2000 and CFP2000 benchmark suites are usually run separately.
The metrics for CINT2000 are SPECint_base2000 and SPECint2000.
The metrics for CFP2000 are SPECfp_base2000 and SPECfp2000.
Back to FAQ
-
What does SPECcpu2000 measure?
SPECcpu2000 attempts to measure the "raw" CPU power
of the Central Electronic Complex (CEC).
It doesn't involve disk or network I/O.
All data is held in memory or in some cases, entirely
in cache.
This is analogous to the horsepower rating for your automobile
engine.
Your engine may have a 500hp rating, while mine has a paltry 250hp rating.
However, this doesn't mean your car will go twice as fast as mine or
even 0 to 60 in half the time!
What if your "car" weighs 5 times what mine weighs, for instance.
Using any one metric is very dangerous.
Besides,
out on the road, the top speed of both cars is regulated by lots of other
influences, like speed limits, the police, other traffic, road conditions,
etc.
Just having the "horsepower" to go 200+ mph, doesn't mean you will
ever
be able to get anywhere close to that speed, no less
sustain that speed!
The same holds true for CPU "horsepower" ratings. They are important, but
only a small part of the story.
For the rest of the story, you need to look to "higher level" benchmarks
like SPECjbb2000, SPECweb99, TPC-C, and many, many others.
With higher level benchmarks you start bringing in more components
of the whole computer system, and start to get a true sense of how the
machines will perform.
That is why SPEC has so many different "application" level benchmarks
in the first place.
Back to FAQ
-
Why is SPECcpu2000 important to you?
Without the "raw" horsepower rating of a machine, how would we know
who has the fastest processor?
Does it matter?
It does if the rest of the system takes advantage of it.
It doesn't if some other system bottleneck causes the equivalent
of a "regulator" which the CPU's performance can not overcome.
Back to FAQ
-
Which metric should I use, "base" or "peak?
First let me digress about: Feedback Directed Optimization(FDO).
FDO is a highly sophisticated compiler technology which is generally
used only by software vendors who develop
large server applications
(like a database, webserver, etc).
The computer code is compiled, and automatically augmented with trace
points, allowing for the capture of dynamic logic flow through the application.
The code is then re-compiled with the benefit of the "logic flow analysis",
branch history and locality of reference information.
It is optimally restructured to handle data similar to the
data it was "trained" with.
The performance gains from this procedure are quite good, but it is not
an easy thing to setup, and most application programmers would
not go through the trouble. Most programmers don't even turn "basic"
optimization on!
Both "base" and "peak" SPECcpu metrics allow for the use of feedback
directed optimization.
It was not always allowed and was somewhat controversial when adopted.
The "pro" side is that you should let your compiler do all that it can
and the "con" side is that the "normal user" will never use it,
so it is not "representative" enough.
Both of these answers are correct, to some degree.
base: is defined as a set of 4 compiler options (at most 4)
that can be
applied equally across all 26 runs of the same type (C, C++, FORTRAN).
This would imply these options are rather robust and couldn't be
"tailored" or "patched" just for this benchmark.
peak: allows any and all of the compiler options that you
have at your disposal to be used to squeak out the last
few milli-seconds from each run.
It's really difficult to get the last few milli-seconds without the help
of and the home phone numbers of the vendor's compiler group!
For the record, I only use SPECint_base2000 and SPECfp_base2000
numbers for my modeling and forecasting work!
Back to FAQ
-
Should I buy a machine based on SPECcpu2000 numbers alone?
Of course not!
This is clearly an "IQ" Question.
There are a whole set of other SPEC Benchmarks (as well as
other industry standard benchmarks) to look at.
Find one close to your "application" workload and try and
use that!
Back to FAQ |
Back to top
|
|
|
A.
Machine Tested:
Sun Microsystems Enterprise 420R
All Results quoted or displayed in this section have been
reviewed and accepted
by the OSG CPU Subcommittee of the
Standard Performance Evaluation Corporation (SPEC®).
A hyper-link is provided on each result to the
"SPEC full disclosure report"
which is located on the
OSG CPU2000 Web-site.
|
Back to Topics
a. SPECint2000 Results generated with
"Generally Available" Compilers
Table 1.
"Generally Available" code is that which has the full faith and support
of the vendor company. This code is production ready and recommended
by the vendor. For full details of the levels of code run, please
see the "SPEC full disclosure" for each run.
Back to Topics
b. SPECfp2000 Results generated with "Generally Available" Compilers
Table 2.
"Generally Available" code is that which has the full faith and support
of the vendor company. This code is production ready and recommended
by the vendor. For full details of the levels of code run, please
see the "SPEC full disclosure" for each run.
Back to Topics
c. Tricks/Tips/Cautions:
- Benchmark tests run for a long time.
Just in case you thought you were going to knock out a CPU2000
integer and floating point run (both base and peak) pretty
quickly, think again!
The four measurements described above, after they were completely
tuned and working
on the E420R, took about 36 wall clock hours to run!
Yeah...that is what I said! One set of runs!
- No I/O during Benchmark
Absolutely no I/O here. This is a CPU benchmark!
- Rate Tests
If you have an n-way CPU, run n copies of the benchmark on the
machine.
Assuming there isn't something major wrong with the machine, you
will get close to "n" times the one-way number.
I may be missing something, but I don't have a real fondness for
the "rate" benchmark.
Back to Topics
d. Net/Net - The performance you should Expect!
Think of this as a "best case performance" for your comparison.
I would only use the
"Base" number as my metric, and be a bit cautious with that since SPEC
allows feedback directed optimization (FDO).
If you are not going to compile with
Feedback Directed Optimization, expect lower numbers!
I use the SPECint_base2000 numbers in my personal
models which project SPECweb99 and SPECjbb2000 performance. Along with some
other "fudge" factors, my models are pretty accurate, so CPU measurements are
VERY important, but must be used carefully.
Back to Topics |
Back to top
|
|
