Can Organizations Really Predict Employee Success Before You Hire Them?

Posted on June 29, 2022

Categories:

Daniel Stein, PhDPower skills/Soft skillsPredictive HiringQuality of HireTalent Models/Personas

Social Share:

Organizations use tools like Searchlight and track metrics like Quality of Hire because they want to hire the right people for the right roles. Academic research has found that top performing employees deliver up to 400% more productivity than average ones. And if in a role that fits their working styles and passions, not only will these employees thrive, they’ll also produce more value for an organization.

But any predictions about employee performance without validation of post-hire outcomes is just that, a prediction. Which is why today so many Heads of Talent and HR leaders are eager to better understand hiring metrics across their talent ecosystem. Better insight into this data and proper analysis can close the predictive talent loop with that much anticipated validation piece (answering that question, “Are we getting it right?”).

Searchlight has predictive, concurrent, and incremental validity, meaning our metrics from pre-hire and within a new hire’s first 60 days accurately predict first year outcomes. Our testing into this produced two main results. First, Searchlight’s specific approach and metrics are valid ways of predicting employee success. Second, looking more broadly, it is entirely possible to accurately predict which employees will go on to excel at a particular company and role during the hiring process. With the right combination of people science research, data gathering and data analytics, it’s possible to take a more scientific approach to hiring predictively – with serious benefits on both sides.   

To explain this in more detail, let’s examine how we tested Searchlight’s metrics for validity together with one of our customers (a 750+ employee high-tech company with a $5B+ valuation).

Testing for Validity

In the science of measurement, “validity” is the degree to which a metric or assessment is true. We tested two metrics for validity: the Searchlight Score (the predicted effectiveness of an applicant in a given role, measured pre-hire) and our Quality of Hire metric (the impact that the new hire has on the organization, measured post-hire). We did this by comparing Searchlight’s predictions about candidates to our customer’s own employee performance reviews for candidates they hired using Searchlight technology.

We performed three specific tests to evaluate validity:

  1. We checked if the results from an applicant’s pre-hire Searchlight Score statistically predict their job performance (measured by their performance reviews). This is called predictive validity. If the Searchlight Score estimated that an employee would perform well, and they did perform well, the Searchlight score has predictive validity.
  2. We checked if the results from an applicant’s pre-hire Searchlight Score statistically predict their job performance (measured by their performance reviews)—above and beyond the effect of interview scorecards on job performance. This is called incremental validity. If the Searchlight Score estimated that an employee would perform well, and they did perform well even controlling for other known predictors of job performance (e.g., interview scorecards), the Searchlight score is incrementally valid.
  3. We checked if Searchlight’s post-hire Quality of Hire metric aligns with our customer’s job performance reviews. This is called concurrent validity (testing against a technically sound independent evaluation). If Searchlight’s Quality of Hire metric matched what our customer’s independent performance reviews found, the Quality of Hire metric has concurrent validity. 

We found a statistically significant, positive correlation in all three cases (between the Searchlight Score and our customer’s performance reviews, the Searchlight Score and our customer’s performance reviews controlling for interview scorecards, and between Searchlight’s Quality of Hire metric and our customer’s performance reviews). When a correlation is statistically significant, we can conclude the relationship between the two variables (e.g., Searchlight Score and post-hire job performance ) is unlikely due to chance. This indicates that the Searchlight Score (collected pre-hire) can accurately predict post-hire job performance and the Searchlight Quality of Hire metric is concurrently valid. Both the Searchlight Score and Searchlight Quality of Hire Score are valid ways to predict an employee’s current or future success.   

How Important is Predicting Employee Success? 

Not only do these metrics predict employees who will go on to thrive, they’re also really good at it. Based on prior research which quantitatively reviewed over 100 years of academic research, Searchlight Score (Pearson r = .24) is more substantial than or on par with these commonly used applicant screening / interviewing tools:

Employees who scored in the top 20% of the Searchlight Score had post-hire job performance 15% stronger than those who scored in the bottom 20%. To put this in a business context, consider the value of an Account Executive who sells 15% more than the average, or a developer that writes 15% more code. This type of predictive modeling (which is based on behavioral traits as well as some of the factors listed above like years of experience) is a highly effective way to hire future rock star employees. 

This analysis also found that Quality of Hire is a highly valid metric. In fact, the relationship between employee performance reviews and Quality of Hire (Pearson r = .31) is stronger than other measures that have been the focus of significant company investment in the past decade, such as the relationship between employee commitment and tenure

We further found that employees in the top 20% of Searchlight’s Quality of Hire metric performed 22% better than those in the bottom 20% of Searchlight’s Quality of Hire calculation. Not only does Quality of Hire matter, but it’s possible to measure it accurately (which many companies have struggled with). This opens the door to building Quality of Hire into HR and Talent Brand processes. Heads of Talent and Heads of People wanting to promote strong employee performance and retention should consider this approach.

Connecting pre-hire and post-hire data is what we do at Searchlight. We allow people leaders and those responsible for talent acquisition to holistically understand candidates and predict performance, and then measure the validity of that hire once in a new role. 

Daniel Stein, PhD

People Data Scientist As a People Data Scientist at Searchlight, Dan combines subject matter expertise in organizational psychology with data science methods to uncover actionable people insights backed by science. Dan has a Ph.D. in organizational behavior from the University of California, Berkeley, and his research has been published in Harvard Business Review and Scientific American.

Download our Guides