Gaming the Metrics

In this episode of Beneficial Intelligence, I discuss gaming the metrics. We measure things to be able to manage them. But when we start using metrics to reward individual employees and teams, people will start gaming them. Newton's third law for business says that for every system the organization implements, the employees will implement an equal and opposite workaround that negates the system.  

Amazon is managing a huge workforce of delivery drivers. To ensure they drive safely, they require drivers to be logged in to a mobile phone app. The app uses the accelerometer to measure acceleration, braking, and other parameters and gives each driver a score. But because Amazon is also ruthlessly pushing their small subcontractors to deliver a lot of packages very quickly, the delivery companies have started instructing their drivers to game the metrics. Drivers say they are instructed to drive very carefully for the first two hours each day to achieve a good score. After that, they are also instructed to put their phones into airplane mode and drive like the devil for the rest of their 10-hour shift to achieve the number of deliveries required. 

Andy Grove, who used to be the CEO of Intel back when they were successful, was known for understanding productivity. He formulated the rule that for every metric, there should be another ‘paired’ metric that addresses the adverse consequences of the first.  

As an IT leader, getting your measurements right is one of the most important parts of managing your IT organization. If your metrics are used in any to praise or blame individuals and groups, you can be sure people will try to optimize for them. If you are not carefully establishing paired metrics, you can be sure your metrics are being gamed.

Beneficial Intelligence is a weekly podcast with stories and pragmatic advice for CIOs, CTOs, and other IT leaders. To get in touch, please contact me at sten@vesterli.com