Fits and Starts

Note: Podcasts on Leading Smart are produced for the ear and designed to be heard. If you are able, I strongly encourage you to listen to the audio, which includes emotion and emphasis that's not on the page. While the text is carefully transcribed, it may contain errors. Please check the corresponding audio before quoting in print.

On Glassdoor, the employee’s review site, there’s the article “Five interview horror stories that will make you cringe.” The online Horror News Network loves a film called “Job Interview” where the process goes violently wrong. TV and movies are filled with tales of a terrified job candidate applying for a job with a miserable, overbearing boss.

Whenever there are stories about the hiring process, it always seems to be from the perspective of the job applicant. The manager has all the power and is in complete control. The poor applicant is coerced, manipulated, and toyed with like a ball of yarn in the paws of a playful cat.

Yet anyone who has hired people knows this is but half the story. Hiring can be nerve-wracking from both sides of the interview desk. While applicants worry about standing out in the crowd and making a great first impression, managers too have plenty to fret over.

What if no one applies for the position? Or heaven forbid, what if too many apply? A pile of 100 applications can be nearly as terrifying as an empty inbox. And imagine if you’re Amazon, or Google, or Microsoft and your pile is measured in the thousands. As we’ve discussed in the last few episodes, winnowing that tidal wave down to a manageable list is a difficult challenge. Whatever process you use is likely to be clumsy and prone to misfires.

Even if you do manage to cull the list to a reasonable size as you screen it down for interviews, the difficulty hardly ends there. You need to make sure your interview process is robust and effective. That the interviewers are good at the task and have judgement that you can rely on. That the process will reveal enough upon which to base a decision. And that when you’re done, the choice is clear.

For many hiring managers, the most challenging task of all is choosing the best of the lot, finding that perfect fit. Especially with brainpower workers, where the measurements for excellence are vague and often purely subjective. It can be easy to fall into “I like them” as a reason to select a candidate. If you’re thoughtful and honest however, this is precisely the moment you should be worried. Perhaps you like them because they look like you, have experiences like you, or even simply approach problems like you. It can be difficult to wean oneself from the comfort of hiring clones of yourself or your team.

The harsh reality is that the best time to doubt your choice is when the fit feels perfect. And that’s what this is all about.

This is Leading Smart, the show about Managing in the Brainpower Age. It’s a field guide to the joys and challenges of leading and working in the modern workplace. I’m Chris Williams, your guide to the stories and ideas that I hope will inspire you to be a better leader in the world of knowledge work.

In this podcast, we’ll take a look at how people meet the challenge of managing smart people in this Brainpower Age. Each episode, we’ll explore everyday problems and provide practical tools you can use to be a better and smarter leader.

This is the fourth of our episodes looking at the challenge of hiring brainpower workers. In this episode we’ll look how the right fit often isn’t. This is Episode Nine: Fits and Starts.

A year or so ago, it was reported that Amazon had scrapped an internal program that used artificial intelligence to help them hire. I appreciate both why they tried the experiment and why it failed.

Amazon is inundated with people wanting to work there. This is not unique to them, the internet makes it almost trivial for people to spam the world with their resume. Whether it’s Google, Facebook, Tesla, or Microsoft, being a “hot” company in the news is guaranteed to result in a tsunami of applications.

As we discussed in previous episodes, organizations have tried a range of ways to separate the wheat from the chaff. Whether it’s online tests or other hoops for candidates to jump through, they work hard to reduce their pile of applications to some reasonable size. In effect, these are the hiring versions of the CAPTCHA algorithms that present you with pictures and ask you to find all the stop lights in the photos. But rather than finding bots, they’re hoping to stem the wave of resumes and make some sense of it all.

As a tech company with lots of programmers, Amazon had worked for years to apply some algorithmic smarts to this problem. This latest attempt used sophisticated artificial intelligence methods to filter out some applicants and advance others. It stumbled in a way that highlights the dangers that lurk for everyone hiring brainpower workers.

Artificial intelligence is the broad term for the goal of making computers behave in a sentient manner. The field of AI offers many different approaches to the challenge. Especially popular today, and the method Amazon used, is the branch called Machine Learning. What this entails is building a large network of computer modules that mimic neurons in the brain, called a neural net. They begin as a large interconnected and untrained blob, much like a newborn baby. Then, after teaching the network to understand the problem, it can be offered new inputs and get very impressive results.

Imagine for example, you want a neural network to recognize images of dogs. In the training process, you run a huge number of pictures through the program and tell it the expected result. You show it a tree, say “no”. A dog, say “yes”. A cow, “no”. Another dog, “yes”. Human, “no”. And so on. You vary the size and type of dog, the length of the hair or tail, its color or age. You also vary the views in the image: sometimes face on, sometimes a side view. You give it difficult edge cases, like a hairless dog or one with only three legs. It’s hard work, and it takes a great deal of both human and computer time.

Eventually, after hundreds of thousands, perhaps millions of images, you have a very smart neural network. You can show it an image it hasn’t seen, and it can identify the dog in it with impressive accuracy.

Neural networks are amazing, and they are used in a broad range of data analysis situations. Factories use them to isolate defects. Businesses use them to find sales trends. Advertisers use them to seek out potential customers. Law enforcement agencies use them to identify threats.

But neural networks have two main issues. The first is that they are largely black boxes. That is, no one has programmed in any rules, the system has just “learned” how to identify dogs. It’s not clear or well documented how it does this. And if it makes a mistake, you can’t go into the code and change it, there are no bugs to fix. You just have to show it more images, and say “no, that’s a goat”, training the net further. Much like a toddler, if you ask how they know it’s a dog, the answer is often, “they just know.”

The other issue with a neural net is that it only knows what it has seen. It can’t extrapolate to new information or new ideas. A new breed is likely to stump it. It doesn’t really understand the anatomy of a dog, it just knows what one looks like. Again like a toddler, an insufficiently trained neural net is likely to point at nearly every four legged creature and call it “doggy.”

Nonetheless, given the promise and broad use of neural nets, it’s easy to see why Amazon would be tempted to try to conquer their application backlog using one.

Amazon trained a net using hundreds of thousands of applications they received, and then said “yes” to the people they’d hired from that pool. I imagine that resulted in an impressive system that could look at a new application and predict whether this person might be hired at Amazon. Probably with confidence well over 90 percent.

What an amazing aid such a system would be in dealing with the thousands of applications they see every month. They built a filter that could reliably isolate the wheat from their seemingly endless of bushels of chaff.

But using machine learning for hiring played right into the two main drawbacks of a neural net. Because it’s a black box, you can’t understand why it rejected some applicants. It not that different from the interviewer who says of a candidate, “I don’t like them. I can’t tell you why, I simply don’t like them.” Any hiring manager who hears that should have red flags waving frantically.

In addition, the machine learning system would also suffer from the other neural net drawback. It would tend identify only people similar to current Amazon employees. And like many tech companies, Amazon’s workforce skews towards young, male, and as we discussed a couple of episodes ago, also quite Asian. Even if they tried to hide identifiable signs of gender, culture, religion, national origin, and so on, the network would behave much like a poorly trained human interviewer. It would prefer applicants similar to those who were already there. The neural net would be just as biased as the hiring manager who says, “I only want to hire people just like me.”

Amazon scrapped their system when it was shown to have a bias against women. But that’s just one example that could have tripped it up. Culture, language, school affiliation, even writing style are all traits that probably might have caused misfires. And there are several others.

In the end, what Amazon discovered was that bias and discrimination can not only be taught to people, but that it can easily, almost inadvertently, be taught to machines.

For years scholars have done experiments to try to identify and remove bias in hiring by hiding or altering personally identifiable information. They’ve changed or hidden the name of the applicant to mask gender or race. They’ve carefully screened and neutralized language that indicates culture or national origin. They’ve tested randomizing a range of characteristics that might be the source of bias for or against particular candidates. All of these experiments have shown that even a simple job application offers a multitude of sources for potential bias. Bias that all too often results in candidates being prematurely removed from consideration.

But no, you say, you’ve been extremely careful. You’ve worked hard to avoid bias in the application review process, and you’re determined to present an open field. And yet, eventually that candidate is going to be sitting across the desk from you. It takes amazing strength of character to not fall victim to your own inherent biases, even subconsciously. To look across the desk and not think about the ways they are different from you or the people on your team.

Welcoming differences is hard work, and some would say it’s risky. We’ll explore the benefits of a diverse workforce for both your organization and its results in a later episode. But from a hiring perspective alone, you’re giving up a ton of very good hires, perhaps the best of them, if you let your biases get in the way. And in a competitive job market, that’s a good way to overlook a lot of great talent.

It’s especially foolish in the market for brainpower workers who, as a group, tend toward the unique and different. It may seem cliché, but some of the best creatives are the ones least likely to go with the flow. The very things you’re hiring them for: their imagination, passion, and drive, are also likely to be the traits that make them different. Hiring only people you like, who seem to be such great fits, can limit you and your organization. Now and in the long term.

So how do you guard against these often subtle, yet pernicious biases? It takes a lot of careful thought and planning, especially at the interview phase.

The most important goal is to include as many voices as possible. Try to make sure you have as diverse an interview loop as you can. Include men and women, young and old, managers and individual contributors, the outspoken and the soft spoken. Try to ensure that you’re including the broadest of your range of perspectives, so that you get a spectrum of opinions on the candidates. Yes, this is an objective, and no you’ll never reach it. You’ll never have enough variety or enough time on your interview loop. But make a concerted and earnest effort to do the best you can.

And please, please, don’t use group interviews. I know it’s tempting, especially as a way to get a number of people a view of the candidate in a short window of time. But a group setting is terrifying for the candidate and it doesn’t let them shine. Also, think back to the worst of your staff meetings and imagine letting a perspective hire in there. The loudest voices often rule, and the conversation gets dragged in their direction. The soft spoken or underrepresented members have to fight to get a chance to be heard. Worse yet, “group think” inevitably sinks in. One member rolls their eyes, others pick it up, and the minority voices get overwhelmed. Before you know it an otherwise outstanding candidate is headed for the door.

Make the time for everyone to meet with the candidate one on one. There, they can set the tone that suits their style, explore the subjects they want to, and take the discussion in the direction they find most productive. And they can begin to establish a relationship with the candidate, the first and most important step to understanding them.

You should rigorously train your team in the skill of the interview. Of course there are all the standard illegal topics: don’t discuss age, religion, marital status, and so on. Those are straightforward and everyone needs to heed them carefully. But with brainpower workers, the interview is often a more subtle affair. We discussed this in the last episode in depth. Make sure to listen carefully to the candidates, and focus on their words, their actions, and their meanings. Avoid judging on appearance, style, flash or the lack thereof. Learn to guard against liking or disliking people for nebulous reasons. Never accept the explanation: “I can’t tell you why, but I just like them” or it’s opposite.

And be careful with humor and colloquialisms, which often don’t translate to other cultures. As a person with an admittedly edgy sense of humor, this is an area I personally struggle with. When I interview, I have to consciously check myself on this continuously.

Last of all, teach everyone to remember the applicant won’t speak up. They won’t push back on things that are confusing or offensive, they’ll just retreat. In this sense, the interviewer is in complete control, and it’s up to them to protect and listen to the candidate.

Even if everything goes right, it still comes down to the hiring manager and their judgement. Listen to the individual voices in the interview loop. Carefully understand what each interviewer is feeling about the applicant. If one feels strongly about a candidate — for or against — don’t dismiss it, understand it. If they are advocating for the candidate, seriously consider taking a chance on them. You may well end up with a great hire, and simultaneously earn the respect of the advocate. If they are opposing the candidate, make sure you understand why. Some people have a gift for seeing problems before they arise, and smart people heed those warnings.

Most importantly be sure to base the decision on solid ground. Don’t hire someone because everyone likes them, hire them because they add something of value to the team. Value comes in many forms, often from directions you don’t expect. Smart leaders know that pushing against the tendency toward comfort is a core part of growth. Both as a person and as a team. It’s often the difficult times that make a team stronger, and a discordant voice in the room can provide perspectives that are often overlooked.

It’s usually only the hiring manager who has the vision to see how a candidate can do that. So, it’s up to them to make what, in many cases, seems like a bold choice. But that’s precisely what leadership is all about.

Especially with brainpower workers, thinking outside the box is not only for doing the job, it’s for hiring as well. The best candidate for the job often is the person who doesn’t seem like a match but brings new perspectives and new talents to the group. They’re the one who really fits, and starts off running.

Leading Smart is from me, Chris Williams. You can find out more about the show and discover other resources for leaders at my web site, That’s

Leading Smart could really use your support. Please subscribe and rate the show in your podcast app, every rating helps new people find it. And please share the show with your friends in person, or on social media.

I’d also love your comments and suggestions. Each episode has a page on my web site with a transcript and a place for your feedback. You can also find me on Facebook, Twitter, or LinkedIn, just search for theCLWill. Or you can send me an email to However you choose to engage, I look forward to hearing what you have to say.

That’s it for this episode. In the next episode we’ll continue our look at hiring brainpower workers. Next we’ll look at how to convince that great candidate to come work for you. It’s called “Soft Landing”. I hope you’ll listen. Until then, please remember that each of the several dozen decisions you make today are part of Leading Smart.