Dear hiring managers and recruiters of all the companies I have interviewed at,
I have a confession to make.
Actually, almost everyone who has interviewed at your company, my colleagues, and myself have a confession to make.
We all cheated on your interviews.
When you interviewed each one of us, we already looked on Glassdoor to see if anyone has posted interview answers. That means the exact questions that you spent so much time designing, we found those answers online, memorized them and aced your interviews.
When I attended UC Berkeley’s Computer Science program, one of the most infamous professors known for his impossibly difficult projects, Paul Hilfinger suggested that students should “when in doubt steal code.” I thought this also applied to answers on interviews and so did everyone else.
But how does candidates reading answers to interviews make interviews more biased? It almost seems impossible that it could contribute, but unfortunately posting answers online actually plays a strong role. By leveraging answers on Glassdoor, hiring managers are pressured to make interviews less structured to avoid people looking up the answers and nailing interviews.
The Right Gut Feeling at the Wrong Time
Hiring managers, for good reason, prefer to have unstructured interviews. This is where managers can interview candidates by asking a random set of questions and probe if they find one area more interesting than another. Usually this is to test how well someone understands a subject and also to see if there is synergy. Hands down, this is the most common approach for testing qualified candidates, but unfortunately, unstructured interviews create more problems than solutions. According to Iris Bohnet of Harvard Business Review “should not be your evaluation tool of choice; they are fraught with bias and irrelevant information.”
Humans by nature have bais, so it is obvious that having unstructured interviews will lead to bias. On the other hand, how effective are unstructured interviews compared to other interview techniques? If they are not very effective, many managers should try another technique.
Back in 1998, two researchers, Frank Schmidt of Iowa University and John Hunter of Michigan State University wanted to answer that very question. However, they wanted to take it a step further and look at all of the current interview techniques and see how well each correlated to job performance. The two professors put together one of the most thorough studies covering an incredible 85 years that now sits as cornerstone research for hiring practices.
Their findings can be summarized in the chart below:
The Bad, Good and the Ugly
Reviewing the chart, unstructured interviews have a .38 validatory factor. In other words, if someone does well in an unstructured interview that correlates 38% of the time that they will perform well in the workplace.
Unstructured interviews not only lack in terms of determining job performance, but they are preached to avoid at the core of recruitment bias.
Fortunately, there are much cheaper and easier ways to implement interviews that are also way less biased.
There are a few interview techniques that perform better than unstructured interviews. For example:
Structured interviews, where each candidate is asked the same questions and the results are verified horizontally to reduce basis, outperforms unstructured interviews by about 13%.
Other companies utilize work samples projects to see how well a candidate performs with on the job skills.
Lastly, some companies have also tried general mental ability tests like IQ tests, however there is much controversy surrounding them so they are not as widely used.
Overall, the results from the three previous interview types according to Schmidt and Hunters perform much better than unstructured interviews, however, while they are not perfect, they tend to be much lower in bias.
On a comical note, how well someone writes or even their age has zero correlation to job performance. Even more surprising, reference checks rate more poorly than unstructured interviews.
But how does this relate to Glassdoor and cheating?
Technology Bites Back
On Glassdoor, there is a section where people that have interviewed at companies can crowd source their interview questions and answers.
For example, here is the technology company Box and an interview question that was asked during an interview in February 2019:
A candidate who interviews as a software engineer at Box can now practice these questions that they find on Glassdoor and eventually memorize the answers
Here is where Glassdoor encourages bais
Unfortunately, some of the best tools to help remove interview bias such as structured interviews, general mental ability tests, work sample tests, as were previously discussed, can all have their questions and answers posted online. This means that the data that the company is collecting to compare candidates are being distorted by Glassdoor-conscious candidate. Myself being one of them.
Some of the most reliable and less biased questions can be posted on Glassdoor distorting the interview data.
As a result many hiring managers are left with two choices:
1) either interview candidates more quickly to avoid them posting in time
2) make interviews less structured to avoid candidates unable to leverage memorized answers.
Each option has a caveat:
With batching interviews, the hiring managers and recruiters need to work with each candidates schedule, risking losing that candidate to other companies.
The other option, making the interviews less structured, where employers encourage bias and are less reliable than the techniques.
A Game of Cat and Mouse
Essentially a cat and mouse game arises in this situation where firms have to change interview questions faster than candidates can post the answers online. This cat and mouse game puts pressure on the Human Resources department and managers on how to effectively recruit while keeping bais to a minimum.
I am not implying what Glassdoor is doing is wrong, but rather, with every new technology that improves our circumstances, there is a caveat.
Glassdoor has brought a new challenge to the workplace where technology’s convenience is challenging human equality.
Jeff Butler Internationally respected speaker and consultant, Jeff Butler helps bridge generational gaps between Millennials and companies looking for their talent and patronage. Butler has quickly built his reputation as a memorable presenter with tangible solutions for attracting, retaining, and engaging Millennials as employees and customers. Within just the past three years, he has spoken at two TEDx events and multiple Fortune 500 companies such as Google, Amazon, and LinkedIn.