Build a CAD Proving Ground, Part 223 May, 2012 By: Robert Green
Before you get started, take the time to prepare your infrastructure and user team for successful testing.
In the previous installment of the CAD Manager's Newsletter, I advocated the creation of a CAD "proving ground" to assist with the debugging and implementation of new CAD software. If you haven't had a chance to read through that column, you may want to do so now to gain the proper context for this installment.
To continue our discussion, I'll pass along some tips and suggested task sequence to help you select your test pilots and get your proving ground started. Here goes.
Prepare the Infrastructure
First things first: You have a lot of work to do before you start testing your new software. Your test pilot users will expect to encounter glitches and problems, but the goal is to get the software/network infrastructure working as well as possible before they begin their task.
I always recommend that you test and verify software in your proving ground with the same network systems, directories, licensing, etc., that you will use in production mode. You'll debug not only the software function, but the network technology as well. It makes sense to get it all done at the same time, right?
To set up the proving ground installation, you'll need to do (at minimum) the following:
- Acquire software licenses and ensure that network licensing works.
- Create required network folders and set correct permissions.
- Organize standards type files (templates, families, content libraries, etc.).
- Outline filing procedures, or install EDM/PDM (engineering data management or product data management) control software.
- Create basic instruction documents for your "test pilots" (super users who will put the software through its paces).
When should you perform these tasks? Prior to any software testing.
Why? You wouldn't test an airplane before you built a runway and a hanger, so don't test software before you have the appropriate infrastructure ready! Don’t waste time — either yours or your users’ — by rushing to test before you’re actually ready.
Assign Your Test Pilots
Remember that you need great test pilots to debug your software — the success of your proving ground depends on it. As we discussed in the previous installment, your "top guns" should be eager to learn new software, keep their cool when problems arise, and stick with the task until completion. It's also essential that they can clearly communicate their experiences with the software.
Now you must assign the most appropriate test pilot(s) for the task at hand. If you're debugging civil software, assign your best civil engineer test pilot; for mechanical analysis, choose your best mechanical engineer; for productivity and speed enhancement, you'll need your most detail-focused production operators; and so on. The key is to assign your test pilots to jobs where their piloting skills can be put to the best use.
When should you assign your pilots? Just prior to testing, but far enough in advance to clear the assignment with their day-to-day manager.
Why? Without the right pilots testing the software, you won't learn as much — and your proving ground won't deliver its highest value.
Debrief Your Pilots Early and Often
As your test pilots evaluate your new software in the proving ground, be ready to work with them, understand the problems they encounter and support them as they undertake their learning curve so that you can learn from their experiences. To do so, always perform the following steps:
- Record any problems your pilots have.
- Ask what symptoms they notice when errors occur (and what actions trigger or precede those errors).
- Have them describe what is confusing, and what works well.
- Ask them what they would do to make the software more user-friendly.
- Note how they would explain their experience to other users.
Everything I recommend here helps you learn from the test pilots' experience in a way that will not only help you debug the software, but also to create training materials for the users who will use the software in production later. This interviewing process is called debriefing in test pilot environments, and is key to getting the greatest value from your proving ground. So don't let your test pilots report that "this is great" or "this software stinks" — encourage them explain what they experienced in specific terms.
When should you interview your pilots? After each significant work session, or whenever the test pilot feels you should know something they encountered.
Why? To create better-configured software that can be taught to new users more quickly and easily.
Iterate and Improve the Proving Ground
Now that your test pilots have given you their feedback, it is time for you to adjust your software, document their experiences (for the trainings to come), fix anything that appears broken, and otherwise tweak the proving ground so that the next test pilot flight will be smoother and more successful.
After you've done everything you can to improve the software, you'll be ready for another test pilot mission and interviewing session. The repeated cycle of testing and fixing will continue until your test pilots report that the software is ready to go.
When should you do this? As soon as possible after each round of test pilot testing — ideally, immediately after.
Why? To work through the problems while all the information is fresh in the test pilots' minds, and yours. The sooner you tackle the problems, the greater the chance of finding a solution.
How often should you iterate? Whenever you have an inspiration or idea that you believe will make the software easier to work with. The more you iterate and test, the better the software will be when production implementation goes forward.
As the proving ground starts working, don't forget to advertise your results to your senior management team and users. After all, how will anyone know what a great job you're doing if you don't tell them?
When should you advertise? Whenever you achieve a milestone or unexpectedly great result.
Why? So management and users can see that change is on the way, and prepare for it!
I hope this "proving ground" series has given you some ideas about how you can utilize the human resources inside your company to make software innovation and implementation go more smoothly. This approach works in all manner of technology development, so there's no reason we CAD managers can't apply the concept of a proving ground in our offices as well.
If you have any ideas that would enhance the proving ground concept, I'd love to hear them. Until next time.