How can you tell if a football coach is actually good at his job?

One important thing to remember about Andre Villas-Boas is that he had ridiculously good hair.

You don’t spend a record €15m (£12.9m; $16.3m) to sign a rookie manager away from Porto unless you’re pretty sure you know what you’re getting, and one thing What Chelsea knew for sure, in the heady days of 2011, was that the man with a swirling fox-red side parting looked incredibly handsome as he was tossed into the air during trophy celebrations.

Villas-Boas at Porto in 2010 (Dimitar Dilkoff/AFP via Getty Images)

Hair like that had sexy new ideas: a philosophy, perhaps. He had the kind of ease that might command a press conference, smoking volcanically over the jagged peaks of an unbuttoned collar. But when the 33-year-old prodigy conducted his first interview as the world’s most expensive manager, all the glamor quickly vanished.

“Don’t expect something,” Villas-Boas warned gently, “from one man.”

True to his word, he was fired by March.

From Villas-Boas to Chelsea it could have been a historic mistake if it hadn’t been for all the other coaches who have squandered transfers in recent years alone: ​​​​Marco Rose at Borussia Dortmund (5 million euros in advance for a poor season brilliant) ); Adi Hutter at Borussia Monchengladbach (7.5 million euros, ditto); Julian Nagelsmann at Bayern Munich (25 million euros for 19 months); Graham Potter at Chelsea (let’s not talk about it). They were the cream of the crop, club coaches couldn’t afford to wait, yet in their new jobs they had the life span of a bunch of bruised bananas.

How do we know if a manager is good? The question seems almost too obvious to ask – anyone in the pub will be happy to explain it out loud to you over a pint – but professional organizations with millions at stake sniff it out every year. Apparently the answer isn’t great hair. They can’t even be trophies, since they are practically only available to managers who already belong to the best clubs. If the study of emerging coaches can be called a science, it remains largely theoretical.

“We’ve been working with football clubs and leagues, actually, on what predicts a coach’s success and it’s very, very difficult,” says Omar Chaudhuri of sports consultancy 21st Group. “There are very few strong predictors.”

Everyone loves a winner, so it makes sense that employers start by looking for coaching talent towards the top of the league. But we also know that in the highly unequal world of European football, wage costs are the fate of most teams, regardless of who fills the technical area. The managers we admire most are those who find ways to give their all.

To identify these top performing players, we can start by modeling the relationship between team strength and success using crowdsourced “market values” from Transfermarkt, which are a decent prosecutor for the quality of the players when you don’t have the salary at hand. We’ll average this season’s numbers with last season’s numbers where available to give coaches some credit for player development, and then weight the numbers by minutes played to account for absences.

Performance-wise, we will use a 70/30 mix of expected goal difference without penalties and actual goal difference, which captures the strength of the team quite well and places more emphasis on the parts of the game that coaches probably have some influence over (creating and denying scoring chances) than the parts they probably don’t (finishing, saving shots, successfully pressuring for penalties by making the VAR rectangle thing with your fingers).

The results are surprising. Over the last seven seasons in Europe’s top leagues, our simple player quality model can explain around 80% of teams’ success.

But what about the remaining 20%? Who should get the credit?

When we look at the outliers in the graph above, it seems fair to say that Gian Piero Gasperini’s freewheeling style helped elevate Atalanta’s mid-budget team to a Champions League contender a few years ago, and the entire platoon of interim coaches and guys who oversaw Schalke’s disastrous 2020-21 season probably weren’t that good at their jobs. Perhaps performance versus team value is a fair measure of what a coach brings to the table.

Reassuringly, the list of this season’s top teams on adjusted-to-expected goal difference is a veritable who’s who of coaching legends and the hottest up-and-coming managers.

Xabi Alonso has rejected offers from Bayern Munich and Liverpool to stay at German champions Bayer Leverkusen, while Brighton’s Roberto De Zerbi, who none other than Pep Guardiola called “one of the most influential managers of the last 20 years”, remains a strong contender for both jobs.

In Catalonia, Barcelona have set their sights on Michel from Girona. Sebastian Hoeness, Paulo Fonseca, Thiago Motta and Will Still have legions of admirers, and perhaps we should all pay more attention to what Eric Roy cooked up in Brest.

So it’s like this: have we found the not-so-secret formula for finding Europe’s next top manager?

Well, wait a second.

An important feature for a good sports statistics is stability, or how much it varies from season to season. If last year’s performance can’t predict next year’s performance because the number is too context sensitive, you probably don’t want to make it the sole basis for costly hiring decisions.

By this standard, our manager’s metrics are a failure. For coaches who change jobs, there is no correlation between the previous year’s performance above or below expectations at the old club and the first season at the new club. While the added goal difference seemed quite effective in identifying this season’s most interesting managers, it has no predictive value for new hires.

When Chelsea spent £21.5 million to sign Graham Potter, they were coming off one of the best performances by any manager in the last seven years: in 2020-21 and 2021-22, Brighton finished 22 and 13 goals adjusted better than expected. His seven months in London didn’t go so well.

Brighton, meanwhile, signed Roberto De Zerbi even though his last season at Sassuolo was average compared to the team’s value. He’d had a good season the year before, and a respectable spell outside the top five leagues at Shakhtar Donetsk in between, but nothing to suggest that his first season at Brighton would be the fourth best out of hundreds in our data set .

What could explain the difference between these two very different hiring stories? Perhaps there is a clue in the way Tony Bloom, Brighton’s famously analytical owner, explained his process. “I am confident,” he said of De Zerbi’s arrival, “his style and tactical approach will fit perfectly into our existing squad.”

De Zerbi (facing the camera) and Potter in 2022 (Bryn Lennon/Getty Images)

Smart clubs don’t just hire successful managers in the hope that they possess an innate knowledge of how to win. They are careful to adapt the manager’s tactics to the players they already have, knowing that changing their style will cost them time and money.

“I don’t want to have to replace 15 players or something like that in two years,” says one veteran analytics consultant, who requested anonymity to protect client relationships. “Because then it becomes a project of just cycling through players and hoping things work out.”

Not all clubs are as attentive to this step as Brighton. Chaudhuri explains that research often starts with a “performance piece” to determine whether managers are making the most of their current squad, but “then you have a playing style piece, which clubs generally tend to be quite vague about how they want to play. They say, “We want games to be attractive and exciting,” whatever that means. And then you say, ‘Okay, tell us what you think it looks like.’”

The other consultant agrees. “I had this meeting yesterday, I gave five candidates, like, ‘What do you think of these five?'” he says. “And he said, ‘Well, I like these four.’ But I said, ‘One of these four is actually not the style you said you wanted.’”

Figuring out which managers have exceeded expectations is the easy part. You can watch their players throw them in the air during a trophy celebration and imagine your club doing the same next season. But success, in itself, is fickle. It also tends to be expensive. The right question is not “How do we know if a manager is good?” but “How do we know if a coach will be right for this group of players?”

The secret ingredient to hiring the right coach is style, and not just the one that comes with really good hair.

(Header photo: Lars Baron/Getty Images)

The Athletic recently profiled six of European football’s most innovative emerging managers.