Innovation Leaders, Are You Paying Attention to These Five Levels of Data?

colored-pyramidIf your answer isn’t a resounding yes, it’s time to change the way you measure, analyze, and report on your innovation projects.

We tend to think of innovation in images: a bulb lights up, gears turn in someone’s head, a plant emerges from the soil, particles and strands of light float futuristically above outstretched hands. (If you doubt it, Google “innovation” and look at the images that pop up to illustrate the process.) That’s because the concepts that makeup innovation—inspiration, creativity, transformation—are hard to quantify in dry facts and numbers. Yet, in the real world of boardrooms and finance sheets, quantify them we must.

Yes, as unromantic as it may sound, successful innovation thrives on data. Collected and used correctly, data can indicate the strength and performance of your innovation projects, and ensure that you are taking the proper steps to carry them out successfully. But too many innovation leaders focus on the wrong kind of data.

Unfortunately, many innovation projects fall short because leaders focus on the impact measures alone, showing new products, cost savings, less waste, improved productivity, or improved customer satisfaction. Trouble is, these measures will change when your project is implemented. What’s needed instead is a balanced set of data that contains financial and nonfinancial measures, as well as qualitative and quantitative data that’s believable, realistic, and accurate.

We point to our own ROI Methodology, laid out in our new book The Value of Innovation: Knowing, Proving, and Showing the Value of Innovation and Creativity. Based on design thinking, the methodology helps innovation professionals generate and assess the five levels of data needed to show the results CEOs are looking for.

For the most part, the current systems of measuring and evaluating inno­vation projects fall short on providing the proper system for accountability, process improvement, and results-generation. Any comprehensive data system must allow opportunities to collect data throughout project implementation rather than waiting until it has been fully completed—perhaps only to find out it never worked from the beginning.

There are five levels of data you should collect, consider, and report on during any innovation project. These include reaction, learning, application and implementation, impact, and ROI. Keep reading to learn about each level: 

LEVEL 1: Reaction

Many current models of measurement omit reaction, because they assume that those involved in innovation are involved because they want to be. This may be a false assumption. It’s important to collect reaction data and make sure that people involved in a project—including stakeholders—see it as important to the organization and their individual success and relevant to current needs.

Reaction marks the beginning of the project’s value stream, and at this level, it should be perceived by all as useful, helpful, relevant, important, and appropriate.

LEVEL 2: Learning

For every innovation project, there is a learning component. In some cases—such as projects for new technology, new systems, and new processes—this component is substantial. For others—such as a new policy or new procedure—learning may be a small part of the process, but it’s still necessary to ensure successful execution. In either case, measurement of learning is essential to success. Measures at this level focus on skills, knowledge, capacity, competencies, confidence, and networking contacts.

Learning is critical, and we must measure it. Learning measurements ensure that the knowledge, skills, and competen­cies are there, with the confidence to make it work and the contacts to make it successful. 

LEVEL 3: Application and Implementation

This level of data measures the extent to which the project is properly applied and implemented. At this level, data collection involves measures such as the extent of use of information, task completion, success with use, and actions completed. Data collection also requires the examination of barriers and enablers to successful application. This level provides a pic­ture of how well the system supports the successful transfer of knowledge, skills, and attitude changes.

Executives invest in innovation, which is an input to the process, and expect success at the impact level. Unfortunately, projects often break down at the lower levels. Although level 3 is one of the most important data categories, this is the level at which most implementation breaks down.

LEVEL 4: Impact

Impact data helps you understand the business and organizational consequences of the innovation project. Here, data is collected that especially attracts the sponsor and other executives. This level shows the new products and improvements in output, productivity, revenue, quality, time, cost, efficiencies, and customer satisfaction con­nected with the project.

These are tangible impacts, which can be converted to money for the ROI calculation. There are also intangible impacts, which usually include image, brand, stress, teamwork, collaboration, and alliances—impacts that are important but maybe not easily converted to money.

LEVEL 5: Return on Investment

The fifth and most important level of data is ROI. This data shows the monetary benefits of the impact measures compared with the cost of the project. This value is typically stated in three ways.

One is the benefit-cost ratio, which is the monetary benefits from the innovation project divided by the cost of the innovation. Benefit-cost analysis has been around for a long time and is very meaningful to many executives, particularly those in nonprofits, governments, and NGOs. Next, there’s the ROI, expressed as a percentage, which is the net ben­efits divided by the cost times 100. The net benefits are the monetary ben­efits minus the project costs. Finally, the payback period is another possibility. This is basically a calculation of how long it takes to get the money back from this invest­ment.

This level of measurement requires two important steps: First, the impact data must be converted to mon­etary values; second, the cost of the project must be captured.

ROI measures keep the CFO and the CEO happy. As Warren Buffett says, the ROI is a way to keep score. And it’s the ultimate accountability. For most executives, it shows the efficient use of funds. Just getting the impact is one thing, but seeing how this could be achieved with less cost is another. The higher your ROI, the more efficient the use of the funds. 

When used correctly, the five levels of data can go a long way toward getting budgets approved and, sometimes, even enhanced.

Data is extremely important. Simply paying attention to the five levels can give you the insight you need to keep your current innovation projects aligned with your goals and fueled with improvements along the way. That’s why it amazes us to see how often innovation professionals neglect data. It’s the language spoken by the decision makers and the purse-string holders—the more fluent you become in speaking it too, the more successful you will be.

Jack & Patti Phillips

Jack & Patti Phillips

Jack is the Chairman of ROI Institute, the leading provider of services for measurement, evaluation, metrics, and analytics. A world-renowned expert on measurement and evaluation, he provides consulting services for more than half of the Fortune 100 companies and workshops for major conference providers worldwide. Jack is a former bank president and has authored or edited more than 100 books. Patti is President and CEO of ROI Institute. She serves as faculty for the UN System Staff College in Turin, Italy, and the Escuela Bancaria y Comercial in Mexico City, Mexico. A former electric utility executive, Patti is author, coauthor, or editor of over 75 books and dozens of articles on the topic of measurement, evaluation, and ROI.

No Replies to "Innovation Leaders, Are You Paying Attention to These Five Levels of Data?"