Individual tests cannot predict optimal teams

February 15, 2024
Contact:
Concept illustration of a business team collaborating in an office. Image credit: Nicole Smith, made with Midjourney

When hiring people to be on teams, many organizations believe the best team must include the “best” individuals.

Organizations construct IQ tests, pose scenarios, assign scores to applicants or develop criteria to identify the “best.”

These tests, however, may be a bad idea, according to a recent study by Scott Page of the University of Michigan and Lu Hong of Loyola University Chicago. They looked at when it makes sense to compose teams by testing individuals.

The researchers’ theoretical solution has two parts. First, they show that if a test is used, it should examine how a team composed of clones of that person would perform. That’s a different criterion than how effective this person is on their own. Second, the test should examine how a team of clones of that person would perform only for rare types of tasks. Those tasks satisfy a “replace the worst team performer” rule, which is the team improves if the worst performer is replaced, like on a relay team.

But a coding task likely does not.

“Someone adept at finding bugs would rank poorly using the team of clones’ criteria,” said Page, the John Seely Brown Distinguished University Professor of Complexity, Social Science, and Management. “Yet, if you replaced them with someone who wrote better code but could not find bugs, the team would likely perform worse.”

In a series of examples, Page and Hong show that tests do not exist for complex tasks.

Page sees two takeaways from the research. First, asking how a team of clones of this person would perform might be as important as evaluating how well people do on their own. Second, selecting a team based on individual performance is likely not a good rule.

“Organizations have to be more deliberate about how they hire people and make admissions decisions,” he said.

Hiring or admitting the highest scorers on a test—even if it were a perfect test of ability—is rarely an optimal approach, the researchers said. You want to build a team with a constellation of skills, experiences, knowledge and perspectives.

“Firms are making a logical mistake if they just hire the highest scorers on some test or criterions,” Page said.

The study appeared in the journal Theory and Decision.