AI skills crisis may lead to wasted investments and stifled innovation, research shows

Survey confirms rising importance of relevant experience and non-degree credentials in closing AI and data science skills gaps

Urgent action is needed to tackle an artificial intelligence (AI) skills crisis that is already stifling US productivity and innovation, new research has found. Published by analytics leader SAS, How to Solve the Data Science Skills Shortage is a report based on a survey of decision makers from major US firms spanning nine sectors, including banking, insurance, government and retail.

Fortune Business Insights projects the global artificial intelligence market to grow from $387 billion in 2022 to nearly $1.4 trillion by 2029.[1] Correspondingly, AI and machine learning are top investment priorities over the next one to two years, according to 43% of SAS survey respondents. That is well ahead of data technology stalwarts such as data visualization (25%), data analytics (22%) and big data (17%).

But there is a massive red flag. Sixty-three percent of respondents also claim their largest skills shortages are in AI and machine learning.

No easy answers on how to bridge the skills gap

Without the talent, these increased investments in artificial intelligence and machine learning could be wasted, leading to financial losses and unrealized opportunities. Survey respondents are planning different tacks to address skills gaps, but they cite several challenges.

Three-quarters of respondents want to train and upskill existing staff, compared to 64% who want to recruit new talent. Training and upskilling may prove more cost-effective compared to hiring and using contractors – but the study highlighted barriers such as lack of time and motivation, and belief that senior management may be worried about people taking their skills elsewhere.

Given how fierce the war for talent has become, salary is another sticking point. Companies may have little choice but to pay ever-higher salaries, and recruitment and contractor costs to secure the skills they need. The estimated total pay for a data scientist now stands at around $122,000 in the US[2], and this may not be sustainable for many organizations.

Universities still important, but companies looking beyond degrees

An April study by Indeed found that 67% of large companies surveyed would consider dropping their degree requirements.[3] The findings of the How to Solve the Data Science Skills Shortage report mirror this trend.

Respondents largely want to work with academic institutions to recruit data talent directly, but understand that relying on graduates alone will not fill vacancies fast enough. And what they are seeking from potential hires is not necessarily a four-year degree. The survey indicates:

• Employers are more likely to consider case studies and project work (74%) and other relevant training (71%) over a degree.
• Industry-recognized certifications, from an external provider including tech vendors companies, are deemed as relevant (54%) as degrees.
• Just as valuable is participation in hackathons and data challenges (53%), which demonstrate technical, problem-solving and team-working skills.
• In fact, only 54% listed a degree as a method to evaluate potential employees.

Building data science talent now

Dr. Sally Eaves, AI expert, author and speaker who contributed to the How to Solve the Data Science Skills Shortage report, said: “Businesses cannot rely solely on graduates or continue the poaching merry-go-round. The good news is employers have already begun to recognize the value of on-the-job training and other certifications as stated in the report.” 

The report outlines three recommendations to address the data science skills gap:

• Consolidate diverse AI and analytics tools around modern, open, multi-language tools which will increase data science productivity and empower end users to do basic analytics tasks, allowing data scientists to focus on core tasks. By democratizing analytics, more people can join the field.
• Increase upskilling and cross-skilling of the existing workforce, including people from non-technical backgrounds. Encourage a diverse range of certifications, including training courses from software tools vendors.
• Create a learning environment and culture where employees are empowered and encouraged to grow their skills. This can include anything from allowing employees to take time out to complete online training courses to hackathon participation to setting up in-house data science academies.

“There is no single approach – but a combination of expanding mid-career training including to those currently in non-technology roles, equipping people with the right tools for the job and growing the data science community will start to see that skills gap narrow,” said Eaves. “Together, they could significantly increase the supply of talent, and create good-quality, satisfying jobs that benefit individuals, organizations and the wider economy.”

Learn more about how SAS can help an organization build analytics skills to gain a competitive edge.

Methodology

SAS commissioned a survey of 72 US-based decision-makers in organizations spanning nine sectors, including banking, insurance, government and retail. Each worked for organizations with more than 1,000 employees; some had more than 100,000. The vast majority were in technical roles, including data science and data analytics, and just under a quarter were in HR and talent management. The survey was also carried out in the UK and Ireland, and conducted by Coleman Parkes.

[1] https://www.fortunebusinessinsights.com/industry-reports/artificial-intelligence-market-100114

[2] https://www.glassdoor.com/Salaries/data-scientist-salary-SRCH_KO0,14.htm

[3] https://www.indeed.com/lead/report-how-covid-19-pandemic-changed-recruiting?hl=en&co=US