Two years ago, Ritchie Bros. Auctioneers faced an interesting dilemma: The Vancouver-based company, which organizes auctions for industrial equipment, was accumulating massive amounts of information on its customers and the items it was listing for sale, but it had no one on staff who could really dive deep and make sense of it all. The company wanted to take a more data-driven approach to pricing, because it needed a better idea of what the market would pay for equipment. When structuring a deal for a used oilfield drill for an upcoming auction in Texas, for example, staff look at what similar equipment sold for in that area in the recent past and use that as a guide. But final sale prices depend on other factors, too. The quality of used equipment can vary wildly, and the resource sector that often buys these products is volatile, meaning demand can fluctuate without warning. In order to develop a more accurate, responsive pricing model, Ritchie Bros. needed help. “The problems we were having were too complex for the skills we had,” says Jeremy Coughlin, director of business intelligence.
So Coughlin went on a hunt for data scientists: experts who can derive insights from large, complex data sets. He checked LinkedIn, scoured online forums and attended industry networking events. He experienced first-hand something that a growing number of Canadian companies are now learning: Data experts are in short supply. Many businesses are struggling to find talent, even as more people enter the field. The number of data professionals in Canada—people employed as statisticians, mathematicians and actuaries—has increased by 48% over the past five years, making it the fastest-growing job category in the country. And demand isn’t letting up. A survey conducted last year by IDC found 53% of large Canadian organizations said lack of talent was the biggest impediment to successful completion of big data projects.
Consulting firm McKinsey and Co. estimates the U.S. currently faces a shortage of up to 190,000 people with analytical expertise and 1.5 million managers with the skills to understand and act on what big data can reveal. Universities are jumping on the data trend and attempting to alleviate the talent squeeze by introducing programs to train a new generation of data scientists. “There are not many programs right now, and the output of students is very low,” says Uwe Glässer, a professor of computer science at Simon Fraser University, which recently launched a data science program. Until there are more qualified grads in the field, companies will have to seek out talent.
Coughlin’s six-month search ended at a university, but not with a student: Instead, he hired a professor who taught business intelligence and data mining at the University of British Columbia. Ritchie Bros. now has a small team of data experts, and one of their duties is to develop a more predictive approach to pricing. When the company auctions that oilfield drill, for example, the goal is for its pricing model to forecast demand in the near future based on different factors, such as the price of oil, leaving Ritchie Bros. less vulnerable to market surprises.
Many of the data scientists employed today have jumped ship from the academic world; they’re among the relative few who know what to do with the massive, unstructured sets of data the world is now awash in. IBM estimates we create 2.5 quintillion bytes of data every day, and because of the quantity of data and the sophisticated algorithms required to decipher it, run-of-the-mill business analysts can’t cope with it effectively.
When Steve Woods needed to hire a data scientist last year, he knew it would be hard to find someone and that potential hires were unlikely to be posting resumés and scouring job boards. Woods is the co-founder of a Toronto-based startup called Nudge, which is developing a software platform to help business professionals keep track of their weak ties, meaning people they don’t keep in touch with regularly. The software can, for example, recommend that you send a specific news article to a particular contact in order to restart a conversation. The underlying software has to sift through and make sense of a lot of information—your relationship to your contacts, what their interests are, what you’ve talked about in the past and what’s happening in the world—so it can recommend what to talk about and when. Woods knew he needed a data scientist to make it all work. “I think that was probably the third conversation in terms of folks to hire for the team,” he says.
He was fortunate enough to be fairly well-connected in the Canadian tech world and started asking around. Eventually he found Zoe Katsimitsoulia, who had a PhD in computational biology from the University of Oxford and was working at another Toronto company. Katsimitsoulia says she joined Nudge because it sounded like an interesting challenge. “There’s a lot more immediate feedback, and that’s pretty rewarding,” she adds. “In academia, you can end up doing research for years.”
Indeed, Nudge’s experience shows how companies can woo data scientists, beyond throwing money at them. (Salaries are rising fast, likely as companies fight for talent. According to Statistics Canada, wages jumped by 38% since 2009.) Data scientists typically like to tackle big challenges, and businesses often provide an opportunity to do so. They also like to see their work tied to business results so they know they’re having an impact.
That’s how Indigo had success in hiring its small team of data scientists. “We do have to make sure the type of work they do is really compelling,” says Sumit Oberai, the company’s chief information officer. For the data team, the main focus is on improving recommendations for customers. The company collects data on online purchases and merges this information with customers’ in-store purchases, if they regularly use their loyalty cards. The data scientists are constantly fine-tuning the algorithms that analyze this information to provide personal book recommendations.
The work has revealed a few surprising insights, such as a segment of book buyers who enjoy both romance and sports fiction. Recommendations drive about 2% of sales today, according to Oberai, but Indigo hopes to boost that number as more customers use its loyalty program and mobile app. The company is increasingly crunching data to determine the optimal product mix at individual stores and the ideal locations to set up new outlets (it relied on data to figure out the best place to open its first American Girl doll boutique last year). “Even five years ago, that would have been done predominantly intuitively,” Oberai says.
Despite the talent grab, not every company is aware of what data scientists can do for them. Chris Bildfell graduated in 2013 with a PhD in astronomy and astrophysics from the University of Victoria and had his heart set on staying in academia. But with the slim job prospects, he eventually applied to a company called Mobify, which makes a mobile shopping platform for retailers. Mobify wasn’t even looking for a data scientist; it had posted an opening for a junior analyst. “I knew the job wasn’t really me, but I just wanted to get my foot in the door and show them what I could do,” he says. Bildfell essentially created the role of data scientist for himself and now analyzes huge piles of information to uncover insights for Mobify. For example, he wrote an algorithm that predicts the page you’re likely to visit next on a retailer’s mobile website in order to preload it. He also crunches data to see how the company’s servers are operating, to deliver analytics to customers and even to check how efficiently Mobify hires new staff.
The role of the data scientist is ultimately a big one, and so most organizations are breaking the job down into different components, says Paul Lovell, vice-president of professional services at SAS Canada. For example, a company may employ people who understand the IT and logistical aspects of data management, and others who figure out the business applications for its reams of information, allowing data scientists to focus purely on the analytical aspects. “There are people who can do it all,” Lovell says, “but they’re as rare as hen’s teeth.” Bildfell says it’s also possible to train certain employees to be data scientists. In his view, they need to be masters of statistics, have some programming experience and possess novel problem-solving skills. “Someone who has two of these skills can improve the third and get from a junior position to a full data scientist role,” he says.
Meanwhile, post-secondary institutions are providing a more formal path to the job. In addition to SFU’s new program, Carleton University has established a collaborative master’s program in data science and others, like Ryerson University and the University of Toronto, have launched certificate programs for managers and executives.
The push to beef up training suggests the shortage won’t last forever, but anyone looking to enter the field in the near-term can expect to find jobs waiting, says Oberai. “This will be an absolutely booming industry for the next 10 years.”
This story originally appeared at CanadianBusiness.com.
