CAD Tech News (#118)

Herrera on Hardware: Hardware Benchmarks for CAD, Part 2 — Uses of SPEC and ISV Benchmarks

Benchmark relevance will vary depending on what application you use and whether you're focusing on graphics throughput or whole-system performance.

By Alex Herrera

In the first part of this article, we discussed reasons why benchmarks remain the best performance evaluation tools, and explored some of their limitations. In this part, we'll explain more about benchmark offerings from SPEC and their best uses, and we'll touch on those from independent software vendors (ISVs) as well.

Assessing 3D Graphics Performance and GPUs

The SPECviewperf benchmark from Standard Performance Evaluation Corporation (SPEC) works by running through a series of canned 3D viewsets selected to be reasonably representative of the content and usage of designers and engineers running popular workstation applications. Several viewsets were extracted from digital media entertainment applications, and a couple from geoscience and medicine. But CAD gets the most prominent billing in SPECviewperf, as the most recent version (13) steps through the drawing, zooming, panning, and rotating of canned viewsets from SOLIDWORKS, Creo, Siemens NX, and CATIA. The content is representative of those applications' usage, but it's characteristic of other CAD packages as well, as a glance at the viewsets illustrates.

SPECviewperf 13 viewsets that were pulled from CATIA, Siemens NX, and SOLIDWORKS — but are certainly typical of project data from other applications as well.

The SPECviewperf benchmark intentionally focuses the stress on the graphics processing unit (GPU), rather than the system as a whole. Other system resources aren't idle, of course, while running Viewperf — for example, the operating system (OS) and some Viewperf application overhead will consume CPU cycles — but for the most part, it's the GPU hardware and driver that are taking the brunt of the computing stress. In some cases, the central processing unit (CPU) might be the bottleneck. In others it might be input/output (I/O), and in either case you may end up realizing that that GPU you were so happy with running Viewperf is either a non-issue or overkill, depending on how the rest of the system functions. As such, Viewperf shines when the goal is to compare GPUs rather than complete workstations.

So unless you're solely focused on choosing a GPU (either to configure in a new machine or upgrade an existing one), you'll want to get past measures that leave the rest of the system unstressed and look at a whole-system benchmark. That means a benchmark that can best simulate and measure how a complete CAD application with typical workloads will run.

Whole System Performance: Combining Heavy-Duty CAD Computation, I/O, and 3D Graphics

There are many general-purpose, third-party PC benchmarks available, such as PassMark. But while they aren't likely to favor one hardware vendor over another, they also won't necessarily be focused on the type of computing and 3D graphics that CAD users typically stress.

SPECapc and ISV-supplied or endorsed benchmarks. Ideally, a user who spends the bulk of the day running one mission-critical application could turn to a benchmark that runs that very application, and issues a bunch of the same types of tasks that the user regularly performs. That's precisely the idea behind SPEC's suite of SPECapc benchmarks, which run popular CAD-utilized applications including ones for 3ds Max, PTC Creo, Siemens NX, and SOLIDWORKS.

What makes SPECapc so powerful is that, unlike other benchmark alternatives, they execute the actual application, executing a sequence of fixed tasks that SPEC (or the ISV) has selected as typical and indicative of users' real-world workloads for that specific application. As such, the SPECapc benchmarks stress all critical system components — not just the GPU, but CPU, memory, and storage as well — providing a better indication of how the whole system should behave when running your application. The caveat with SPECapc is that the application in question must be installed with a valid license (although trial applications may work). Also, be sure to pay attention to the software versions for both the SPECapc benchmark and the application it runs, as SPEC refreshes these benchmarks periodically, although not on any specific schedule.

Similar in use and approach to SPECapc, though neither independent nor broadly available, are performance tests that some ISVs may make available to their users. A popular example is the SOLIDWORKS Performance Test that Dassault Systèmes makes available for SOLIDWORKS users. Read more »

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Alex Herrera is a consultant focusing on high-performance graphics and workstations.

WHAT’S NEW FROM CADALYST

CAD Manager Column: The State of CAD Management at the Start of a New Decade
Conferring with hundreds of CAD managers and vendors at Autodesk University provides perspective on which issues are most pressing for the profession as we enter 2020. Read more »

Graebert Unveils CAD SaaS Road Map, New BIM Features
According to the Berlin-based CAD developer, the software-as-a-service transformation will likely take more than a decade, but the time to start investing in cloud solutions is now. Read more »

Viewpoint: Why BIM Models Serve Designers Better Than Contractors
With BIM, the design stage is privileged over the execution stage — and tools that require specialized knowledge of BIM technology and operating skills are limiting the use of BIM in construction. Read more »

Autodesk Creates Construction Cloud to Connect Scattered Teams
New solution promises to help team members collaborate effectively — whether they're in an office or the field — and to enable data to flow from design to fabrication to the job site. Read more »

In nTop Platform 2.0, New Toolkits Aim to Make Advanced Capabilities More Accessible
Prepackaged workflows are intended to help users quickly grasp and apply the most advanced aspects of the computational modeling software, nTopology reports. Read more »