The Dangers of Measuring Performance
“How do you recommend I measure the individual performance of my engineers?”
It was late, and I had an hour drive ahead of me. I had no energy left to answer more questions.
But this young engineering manager — who asked the question, and stayed behind after my presentation — had me hooked.
It was if he had read my mind, dredged up my biggest management gripe, and handed it back to me in the form of a question.
Re-energized, I asked, “First of all, why would you even want to do that?”
The young manager replied, “Well, I have just been promoted to manage a team of engineers, who were previously my peers, in fact. I want to make sure that I am doing a good job by them. I want to be able to measure each of them so I can get a baseline for evaluating their performance later on. I want to make sure everyone is pulling their weight.”
Seems perfectly reasonable, right? Too bad it’s completely dead wrong.
“Well,” I replied, as I heaved my laptop bag onto my cramped shoulders. “Why don’t you walk me to the parking lot, and describe for me what you’re doing now. I’ll give you my honest feedback.”
I left him with three concrete practices he could apply to his team right away. But in order for those to make sense, we first needed a quick look at the core challenges companies like his are facing.
We Are All Software Companies Now
Marc Andreessen famously quipped that “software is eating the world”. He was referring to the increasing reliance on technology, in nearly every industry, not just software companies.
The rise of SaaS, cloud and big data, AI and machine learning, and the ubiquity of mobile devices with easy access to high-speed Internet represent both a threat and an opportunity to traditional companies, particularly those established before the Internet revolution of the last few decades. These companies now find an imperative to either “act like a software company,” or be crushed by their competitors.
Yet they encounter significant barriers to achieving change from within. Their very maturity exerts an enormous gravitational pull on new initiatives, making it difficult for internal innovators to reach “escape velocity” with their projects.
There are three main barriers to innovation and flexibility within these companies that I regularly encounter:
Organizational structures that prevent collaboration.
Outdated management theories deeply ingrained in the company culture.
Legal or regulatory rigidity that discourages risk-taking.
The third must be handled at the executive level. But any manager leading a product development project can have a profound impact on the first two.
Traditional Management Theory is Broken
Source: details
Two thought leaders from the early twentieth century, Frederick Winslow Taylor and Henry Gantt, had a profound and lasting impact on management orthodoxy. They were among those that formed the intellectual base for management theory during the age of US industrial production. Unfortunately, their continued influence on management practice in the information age is not doing us any favors.
Taylor is credited as the father of scientific management, specifically, time and motion studies. As one of the first management consultants, Taylor revolutionized production by applying observation and measurement to menial tasks in a production system. By breaking production down into discrete tasks and measuring their average completion time, Taylor was able to make more accurate estimates on the length of projects. He also used this method to help the employers improve the efficiency business processes by tweaking or rearranging individual process steps.
It’s largely because of Taylor’s influence that employers focus on utilization as a metric of employee efficiency. It made sense in a word of pre-mechanized menial labor. But that’s not the world we live in now.
Around the same period, Gantt developed the infamous Gantt chart for project management. The Gantt chart, which you’ve almost certainly seen (MS Project, anyone?) is a framework for visualizing all of the resources and tasks required to complete a project.
While the Gantt chart was developed for physical production and assembly jobs, it is not particularly useful for knowledge work. It rewards or punishes based on individual tasks of employees, in order to fine-tune production processes, and it completely ignores the dependencies between parallel tasks.
Again, such focus is wholly inappropriate in the digital age where products are built by teams in collaborative, semi-autonomous units.
Deming and The New Management Science
One of the earliest critics of measuring individual performance was W. Edwards Deming. Deming recognized and highlighted the dangers of over-focus on the “individual” in his writings and talks from as early as the 1950s.
He was emphatic that the quality of output of any production system was due, overwhelmingly, to the configuration and organization of that system, rather than any individual worker’s performance. Thus it was inappropriate, thought Deming, to reward or punish individual workers for performance.
Deming frequently took aim directly at the practice of individual performance-based annual bonuses, as one aspect of this over-focus on the individual.
Deming shared his thoughts on causality in a complex system in this 1994 interview in Industry Week (published the year after his death):
“Chances are good, almost overwhelming, that what happened, happened as aconsequence of the system that he works in, not from his own efforts. In other words, performance cannot be measured. You only measure the combined effect of the system and his efforts. You cannot untangle the two.”
Indeed.
In his collaboration with Japanese scientists and engineers during the post-WWII reconstruction efforts, empowerment of line workers in making their own improvements on the shop floor as a team was a foundational component to so-called “Just-in-Time” production.
The well-known Toyota Production System — which counts among its descendants both the Agile and Lean Startup methodologies — is one of the clear outcomes of this broader collaboration of Japanese industry with Deming. This innovative new philosophy would eventually find its way back to the US through exchanges between US auto manufacturers and executives from Toyota in the 1980s, and finally popularized here as Lean.
Thus, if Deming was such an influential figure in not only the creation of Just-in-Time manufacturing in Japan and Lean Manufacturing in the USA, but also ultimately both Lean Startup and Agile Software Development, maybe we should listen to his warnings on why we shouldn’t measure individual performance.
“I was thinking I could track the time it takes each engineer to finish his assigned tasks in our project tracking system,” said the young engineering manager, as we walked toward my car. “That would tell me who is faster, and who is struggling to get their work done.”
I could tell by the tone in his voice and the look in his eyes that the young engineering manager had nothing but the best of intentions. He wanted to do well in his new position. And further, he clearly had respect for the engineers reporting to him (he’d just been one of them), and genuinely wanted to provide guidance and leadership to any member of his team who might be struggling to keep up.
“That would be a terrible idea for two reasons”, I said. “First, measuring cycle time on individual tasks tells you almost nothing about the quality of workmanship on the line. Second, you are incentivizing individual work over teamwork.”
Knowledge Work Requires New Approaches
Despite the overwhelming success of the Toyota Production System and Lean Manufacturing worldwide, the adoption of Lean practices into knowledge work in large organizations has been remarkably slow. I strongly suspect that at least one reason is that the influence of Taylor and Gantt is still very strong in management teachings.
In the past, management tended to measure product development employees on things like quality, speed, cost, and revenue targets. In today’s world, the only skills that really matter are creativity, problem solving, and collaboration. All of the menial, administrative, and piece work tasks of the 20th century — even at the white collar level — are becoming automated much faster than you probably appreciate.
As the digital transformation wave continues to radically transform companies, two parallel and complementary phenomena appear to be operating in duet.
First, companies are slowly shifting from a state where most employees have an operational or project mindset to a customer-facing or product mindset. In parallel, tasks that are not directly related to the core process of producing products and servicing customers are either outsourced to specialized firms, or converted into an automated process entirely.
Thus, while there is certainly still some work performed by individuals that is not yet automated, it is usually lower-skilled work than the core value-adding product development work. The overwhelming majority of high-value product development work is done by teams.
If you are leading a company that is experiencing a transition to a product and customer focus, it is highly likely that the most valuable activities going on in your company right now are not performed by individuals, but rather collaborative and creative work achieved by groups operating in concert. It would thus make no sense to try and measure, much less evaluate, individuals within these product teams.
This is more or less what I said next to the young manager, as I loaded my jacket and laptop bag into my car.
“Look, you want everyone on the team working effectively together, right? Then you need to develop your incentive and evaluation structure around the team, not the individual,” I said.
Here is what I told him to do:
Measure the throughput of work, not the activity of your people. Organize your work into a queue (backlog, to do list, whatever) in order of importance to the business. The only metric that really matters is the speed at which your team can deliver that work to customers.
Pair or group your people around one work item at at time. Let them focus on collaboratively building, testing, and shipping that work item before taking on any new work from the queue. Moreover, let them pick their own pairs or groups (maybe with a little hand-holding at first).
Empower the team to improve the process themselves. Get everyone together regularly to have retrospectives on what’s working and what’s not. Let the team surface issues and suggest ideas that they think will help.
Your job is to help prioritize based on the needs of the business, remove impediments that slow down the team, and to report progress back to the business. That’s pretty much it. Add to that some career path and mentorship guidance, and you’re already a better manager than most out there.
Driving home, I thought to myself, “what a good question. I should really turn that conversation into a blog post some day.”
