Home
HR Analytics and Data-Driven HR
Utility Analysis Revised: The ROI...

Utility Analysis Revised: The ROI Of Better HR Selection Methods

Written by Rene Klep

5 minutes read

In a previous article, I have explained and advocated the use utility analysis for estimating the financial return of learning interventions. As highlighted in this previous article, utility analysis refers to a specific tool designed to estimate the institutional gain or loss anticipated to a company from various HR interventions designed to enhance the value of the workforce (Sturman, 2003). Next to determining the effects of training interventions, utility analysis can also be used to assess the (financial) impact of selection decisions. In this article, I want to share another personal experience in which I used utility analysis to show the added value (or Return on Investment) of using a more sophisticated selection device for a specific group of employees.

Patent Attorneys at Philips

A number of years ago I was working as an HR manager for Philips Intellectual Property & Standards. Philips has continually been investing in new technologies for over 100 years. This has allowed the company to create a substantial portfolio of intellectual property (IP). One of the key jobs in this department is the role of a “Patent Attorney”. Patent Attorneys are specialists in intellectual property, trademarks, design rights, copyright laws, and patents. They represent researchers or companies in the patent application process, assist with patent infringement and act as litigators to protect their client’s rights to an invention. Compared to others roles I had been dealing with before, this role had some unique characteristics:

Philips invested substantially in training with the aim to grow (junior) engineers in a number of years towards the role of Patent Attorney (they needed to qualify for a Dutch and European examination). Associated with this was the notion that Philips had to be very selective in picking out those candidates who were able to fulfill this role. In more psychometric terms: the selection ratio was relatively low.
Most patent attorneys stayed in the company for a long time (high tenure).
The job was well-paid compared to other professionals with an academic degree (scarcity on the market).

The Utility Analysis Equation

Why are these latter characteristics of the role important? These variables play an important role in determining the cost and benefit of a selection procedure. These parameters are incorporated in the utility formula (called the Brogden-Cronbach-Gleser formula), which is depicted below.

In its basis form the formula is:

If we apply this formula for assessing the utility of selections decisions the parameter “dt” is not used as it is about the performance improvement of an existing population (it is used to determine the impact of training interventions like we saw in my previous article). If we express the formula above in words, we might conclude the following:

The more applicants (N) selected using a particular selection procedure, the greater the total utility
A higher mean predictor score (Zx) will lead to a higher utility. Likewise, a smaller selection ratio will lead to higher utility. The calculation of Zx can be derived from the selection ratio (the number of people to be hired, divided by the total number of total applicants), or
Selection procedures with higher validity will have greater utility than procedure with lower validity (r)
Jobs that have a larger SDy, either because they are more complex or because they have more impact on organizational results, will benefit more from the use of valid selection procedures than jobs with a lower SDy. In the previous article, I have explained the meaning of the SDy-parameter being the added value in dollars/euros of an employee who performs one standard deviation above average (that is, compared to the average performer)
Utility will be higher, when the cost is lower.

In sum: according to the Brogden-Chronbach-Gleser model, the utility of a selection system is the portion of the total dollar/euro value added by those selected that is due to that particular selection system.

Applying the formula to Patent Attorneys

When I became HR manager in Philips Intellectual Property & Standard, the department was already using a combination of structured interviews and psychological testing (combining intelligence and personality tests). However, some managers were having doubts about the added value of the psychological testing in the procedure, mainly because of the additional costs associated with this choice.

With the formula above we were able to show the added value of the psychological testing despite the fact the cost per applicant were much higher.

So how can we make calculations?

Based on Cascio and Boudreau’s book “Investing in People” (2010), the Society for Human Resource Management (SHRM) has developed software to make quick calculations to compare the outcomes of different numbers of parameters in our utility formula.

For our example of Patent Attorneys, we compared the utility of a procedure without psychological testing (Existing, E) and a procedure with psychological testing (Replacement, R). In our calculation we assumed two parameters being different:

The validity (r) of the replacement procedure (R) is 0,4 compared to 0,2 of the existing procedure (E). This assumption is based on psychometric literature, which indicates higher (predictive) validities for selection procedures combining several psychological tests.
The cost of the replacement procedure (R) is 2000 euro per applicant, the cost of E is 500 euro per applicant.

For sake of comparability, other parameters were kept the same (selection ratio, tenure, SD of Job Performance and number of selected employees)

Now we can better understand why the low selection ratio, the long tenure and the relatively high pay (affecting the SDy value) of Patent Attorneys mentioned earlier are so important. The outcome of the formula shows that investing in a good selection device is way more important for these roles than for many other kinds of jobs.

If we look at the outcome below the replacement procedure (with psychological testing) show a “per-year gain” in utility per selectee of 7,735 euro!

enhanced selection cronbach model results

Of course, you can have a long debate about which figures to put in the tool but for me that is not the essence of this method. It is not about finding or filling in the correct number, it’s more about showing the relative differences between alternatives based on realistic assumptions. If you combine the findings in literature about validity of selection devices, standard deviations of job performance, and cost estimates of selection procedures in your own company, you are able to make convincing business cases.

Subscribe to our weekly newsletter to stay up-to-date with the latest HR news, trends, and resources.

Weekly update

Stay up-to-date with the latest news, trends, and resources in HR

Rene Klep

René Klep studied Industrial Psychology at Tilburg University and is lecturer Human Resource Management at Avans University of Applied Sciences in ‘s-Hertogenbosch. Next to this he is one of the instructors of the AIHR-course “Strategic HR metrics”. Before his current position René worked as senior HR manager in Belden, Philips and Stork.