Benchmarks Reveal Common Struggles in Product Development Processes

For manufacturers on a mission of self-improvement, Tata Technologies creates baselines of comparison for evaluating maturity in digital engineering, product lifecycle management (PLM) analytics, and more.

In business, it can be difficult to determine just how well you’re doing, and what could be done better. Between the black-and-white poles of spectacular success and unmitigated failure lies a vast spectrum of grayer status indicators: Is your company using its software tools effectively? Are your 3D modeling workflows optimally productive, your data management processes logical and efficient? How do your operations stack up to those of your competitors?

Tata Technologies, which provides engineering and product development IT services to manufacturers, uses benchmarks to help its clients find answers to questions like these. The company developed its benchmarks in partnership with product lifecycle management (PLM) consulting firm CIMdata, with the goal of finding where its customers had the most pain (and therefore needed the most help from Tata, naturally). But there was also a nobler motive: “To help our customers focus their minds,” said Kevin Power, business development manager for Tata. Many clients couldn’t clearly define what they wanted from a PLM system, didn’t know where to start with PLM implementation, or couldn’t identify where things were going wrong with their existing PLM tools — they just knew that weren’t getting the value and benefits they expected. Questioning these customers about their processes, and comparing the answers to an aggregate of their peers, helped provide clarity and structure. “Most people don’t necessarily implement all the capabilities of a PLM system … [the benchmark] helps identify which are most beneficial to them,” Power explained.

Today, Tata offers benchmark assessments for PLM analytics and support, machining, digital factory, simulation, and digital engineering. They are part of Tata’s PLM Analytics tool suite, which also comprises a “health check” survey that explores how employees view the PLM system; a financial impact analysis tool, which helps predict return on investment (ROI) and is “useful for trying to justify expenditure on a PLM product,” Power said; and a roadmap that details implementation, functionality, cost, a ramp-up plan, etc. “Some customers have PLM systems [before they begin this process]; it doesn’t matter, because all these tools are software-agnostic,” Power noted.

Most benchmark customers are interested in PLM, Power reported; with the next most common being digital engineering — “that’s all about CAD, how you develop a 3D model,” he said. Each benchmark examines 17 essential facets, or “pillars,” of a mature system of tools and methods. Tata selected them “based on our experience of what are the areas our customers struggle with,” said Power. The company also drew on feedback from its instructors, as “they deal with customers who are using these tools on a day-to-day basis,” he noted.


Tata Technologies' PLM benchmark includes 17 pillars, grouped into data authoring and data management categories.


Pain Points in Common

The pillars will likely be familiar to most product developers as points of concern; the Digital Engineering benchmark, for example, comprises such elements as 3D CAD standards, automated drawing generation, design for manufacturing, and intellectual property protection. “If you’re using 3D CAD, you’ve got these problems no matter what you design,” Power said.

And while every company has its own unique combination of goals, needs, and challenges, aggregated benchmark data reveals that some problems are nearly universal. Power believes the following issues cause the most consistent grief: 1. Disconnected data. Whether it’s drawings or documents, Power sees “a plethora of disconnected data,” with customers wasting time locating it, syncing it, and duplicating what already exists. “[The lack of a] single source of truth is the number-one problem that a lot of companies have,” he reported. 2. Undefined workflows. A lack of properly detailed and repeatable processes for obtaining approvals, say, or verifying that designs are correct before they’re released to manufacturing, plagues many. “A lot of companies struggle with that, they don’t have a defined system,” Power noted. Related to undefined workflows is uncontrolled change management: “You can’t just willy-nilly make a change and stick it into manufacturing,” he scolded. 3. Problematic project management. No one’s happy when tasks aren’t being completed in order and on time. This issue often arises from the use of disparate software tools; for example, “Microsoft Project is [a commonly used tool] … but it’s disconnected from everything else that you’ve got,” said Power.

A Comparison to PLM Peers

To start the benchmark process, a client company gathers a small group of personnel who represent a cross section of managers, decision makers, and software users from various departments (engineering, project management, and IT, for example). Tata benchmark experts then lead the group through questions organized according to the 17 pillars. For example, does the company start from scratch with every modeling project? “A lot of design organizations reinvent the wheel — they should have a library of reusable content [instead],” Power urged.

Sometimes there is disagreement among the representatives, but in Power’s experience, the various group members are usually on the same page about how their company is operating. “A lot of people are fully aware of their shortcomings … and also the reverse: They’re really proud of what they do right,” he observed.

The collected answers are evaluated to determine maturity, and also compared to aggregate statistics compiled from many other companies’ benchmarks — but not to any individually identifiable results. “That’s a little bit of a minefield, we’ve got to be careful of our customers’ confidentiality,” Power said. “The average of all [companies’ feedback], that’s the information we give to our customers.”

Tata’s customers are mainly manufacturing companies that design their own products. Most are part of the automotive supply chain, aerospace suppliers, or industrial machinery makers, and sizes range from $50 or 100 million in annual revenue to billion-dollar behemoths.

Keeping Up and Checking In

According to Power, most of Tata’s customers fall into one of two buckets:

1. They decided they had to have a unique PLM system, worked hard to implement it, and got some benefits out of it.
2. They are “stuck in the early 2000s, and use tools that really haven’t improved since then” — relying on spreadsheets and e-mail for data management and approvals, for example.

“The PLM technology has gotten better and better in the last 10–15 years,” said Power. “Every product development company should be using PLM.” Many still aren’t, however, and Power isn’t certain as to why they’re not embracing the advances in technology. “Some companies are making a profit, doing OK, growing their business a little bit, so [their reasoning may be,] ‘Let’s not mess with stuff that’s not broken,’” he posited. It may take a moment of crisis — “a competitor gets better, we’ve fallen behind” — to change their thinking and finally force action on the technology front.

Although most treat it as a one-time event, benchmarking should be ongoing so companies can keep tabs on where the technology is heading and whether they are keeping up, Power believes. “Yes, that happens at higher levels of planning in the organization, but for some reason, that activity doesn’t [regularly] flow down to the PLM arena,” Power observed. A repeat of the benchmarking process every three to five years would be ideal, he said.