Are Commercial Labs Stealing Academia’s AI Thunder?
Commercial research labs run by Google Research, DeepMind, and OpenAI are taking central stage in the artificial intelligence era. The eye-popping achievements of these massively-funded AI labs are constantly producing headlines in tech journals and even mainstream media.
Much of academia however remains skeptical of all the attention commercial labs are getting. Many university professors have argued that industry labs results can reflect run-of-the mill research efforts that bring little if any scientific insight. Moreover, they are concerned that academia is becoming less attractive to fresh graduates and young researchers, particularly as industry labs’ starting salaries dwarf those offered to academics. Commercial labs also boast a higher research-to-production conversion. These factors have led to a scarcity of academics to drive innovation and cultivate the next generation.
DeepMind AlphaFold rooted in academia
During the 2018 Conference on Neural Information Processing Systems (NeurIPS) held last December in Montréal, UK research company DeepMind presented AlphaFold, an AI system that topped the Critical Assessment of Structure Prediction (CASP) competition. This competition is regarded as the Olympics of virtual protein-folding, a research area that aims to predict the 3D structure of a protein based on its genetic sequence data.
Regarded as one of the grandest biochemistry challenges of the last 50 years, solving protein folding can produce a better understanding of proteins and enable scientists to change their function for the good of our bodies — for example in treating diseases caused by misfolded proteins, such as Alzheimer’s, Parkinson’s, Huntington’s and cystic fibrosis.
DeepMind often announces its AI breakthroughs at high-profile AI conferences, particularly NeurIPS (formerly NIPS). At NIPS 2013, DeepMind introduced the first deep reinforcement learning model that could play seven Atari 2006 games, surpassing human performance on three of them. At NIPS 2017, DeepMind Founder and CEO Demis Hassabis announced that its Go-master AI computer AlphaZero had achieved a superhuman level of play in Chess and Shogi within 24 hours.
DeepMind’s progress on protein-folding was undeniably impressive: AlphaFold outscored the second-ranked team by 20 points, achieving what CASP called “unprecedented progress in the ability of computational methods to predict protein structure.”
However, a few professors Synced spoke with at NeurIPS 2018 downplayed DeepMind’s achievement as “non-innovative.” Duke University Professor Yiran Chen told Synced “Many academics jumped out and said that the recent DeepMind protein-folding research brings no innovation in theory. It’s just applying existing techniques on a new application scenario.”
What also concerns many academics is the manner in which DeepMind’s announcement was phrased and the ripple effects it produced. One scholar who asked not to be identified told Synced the protein-folding news release did not mention that the work was built on foundations previously laid out in academia: “The news release kind of misled the public to believe that DeepMind invented its own deep learning algorithm.”
Popular protein folding techniques usually consist of two major steps: Learn the relationship between the shape of a protein molecule and its amino acid sequence; and build a 3D model of a protein from that information. Most research efforts over the past several years — including AlphaFold — aim at improving the accuracy of the first step.
A major advance emerged in January 2017 when Toyota Technological Institute at Chicago Professor Jinbo Xu developed what many believe to be the first experimentally-viable deep learning method in the field. Xu applied deep convolutional residual neural networks to predict which amino acid residues are in contact, ie contact prediction.
Xu’s work prompted many protein-folding research groups to reimplement his algorithm, and produced superior performance compared to previous methods. In the CASP13 competition, many participating teams’ predictors (especially contact predictors) utilized deep convolutional neural networks, including DeepMind’s AlphaFold. Considering most of these teams had nowhere near DeepMind’s engineering and computing resources, DeepMind’s success was not entirely unexpected.
Since DeepMind hasn’t yet published a paper on AlphaFold, the question of whether their method is a result of fundamental scientific insight or superb engineering remains unanswered. Harvard Medical School Department Fellow Mohammed AlQuraishi suggests AlphaFold is a combination of both — as DeepMind used deep learning to predict the distances between pairs of amino acids instead of residue contact.
Prof. Xu first proposed extending deep convolutional residual neural networks to distance prediction in 2017, and advanced the idea in his 2018 paper Protein threading using residue co-variation and deep learning. A month before DeepMind announced its results, Prof. Xu presented his latest research on distance-based protein folding in the paper Distance-based Protein Folding Powered by Deep Learning, published on the preprint platform Biorxiv and accepted by top computational biology conference RECOMB 2019.
Which career path: academia or industry?
A typical new computer science graduate is faced with two possible career paths: join an academic faculty and pursue fundamental research or proof-of-concept investigations; or become an industry researcher and focus on product or service improvements.
The distinction between the two groups has recently blurred, as DeepMind, Google Research and Microsoft Research labs are increasingly extending their interest to frontier research. Google for example had 25 papers accepted at NIPS 2015 and the number increased to 107 in 2018 — accounting for 10.5 percent of the total papers accepted by the conference.
Tesla AI Director Andrej Karpathy wrote in his blog “back when I started my PhD (~2011), industry research was not as prevalent. It was common to see in Graphics (e.g. Adobe / Disney / etc), but not as much in AI / Machine Learning. A lot of that has changed and from purely subjective observation, the industry involvement has increased dramatically.”
The AI community has witnessed a one-way brain drain from academia to industry. Admittedly there are many advantages to being an industry researcher. Says Director of Statistical Machine Learning at AIG Yuanyuan Liu: “Industry researchers are focusing on specific productions hence have much narrower and long-term interest into specific areas. Their research directions are connected to the real world, research-to-product conversion rate is relatively high, and have large amount of valuable data to work on.”
As mentioned, another strong motivation for graduates to choose industry is higher salaries. AI specialists with little or no industry experience can take in between US$300,000 and US$500,000 a year in salary and stock, the New York Times has reported. According to the DeepMind’s annual financial filings in Britain, the company paid its 400 employees a total of US$138 million in 2016. That translates to an average of US$345,000 per employee, including researchers and other staff. In comparison, an average computer science professor earns less than US$90,000 a year, according to Glassdoor.
Tech giants are not only targeting recent computer science grads, they have also set their sights on renowned university professors. Two of the three 2018 Turing Award Laureates — Geoffrey Hinton and Yann LeCun — have been working for Google and Facebook respectively since 2013.
The only 2018 Turing Award Honoree to remain in academia is Yoshua Bengio, who is a full professor at the Université de Montréal Department of Computer Science and Operations Research. Bengio recently told Synced that talent migration from academia to industry was even worse a few years ago.
“There are very few senior researchers of my generation that are still working for universities and doing deep learning research. Even among the generation of my graduate students twenty years ago or ten years ago, there are not that many people,” said Bengio.
Aside from the money, there are also other drawbacks to conducting AI research in an academic environment. Yuxi Li, PhD from University of Alberta and Founder of Attain.ai, told Synced that although we are in the era of big data and deep learning, the computational power for some advanced research projects may be insufficient at universities. Also, research collaborators in industrial research labs are usually more experienced than PhD students at universities. Professors also need to spend time writing research funding applications and, of course, teaching.
However, freed from strong delivery pressures or commercialization-oriented research restrictions, academia presents an ideal environment for blue-sky research to spawn significant breakthroughs.
In a blog post, CMU Professor Simon DeDeo opined on the appeal of Google Brain or other industry labs for research scientists: “Google can beat University of Kansas for the sole reason that they can hire ten times more graduate students per researcher. The difference, of course, is that a graduate student at UK has the chance to do something intellectually significant. Not true at GR (Google Research).”
“If you want, at some point in your flourishing career, with your mind and your soul, to join the two-thousand year old parade of intellectual progress, you are not going to do it at Google. Certainly not at Facebook.”
Emerging trends in academia-industry relationship
Despite the apparent conflict of interests between academia and industry, recent trends suggest the two worlds are moving toward collaborations and partnerships to complement each other. Companies like Google, Facebook, Microsoft and Amazon are scaling up their support to help academic research groups facilitate fundamental research with more funding, data and compute resources.
CMU Assistant Professor Zack Lipton told Synced that there are now many excellent collaborations between industry and academia. “A number of research labs have been funding faculty research and PhD student support, some outright fund students, and we have some experimental programs popping up where researchers in industry return to PhD with full financial support from their employers.”
Also noteworthy is that many university professors are now being offered senior researcher positions at industry labs that allow them to continue teaching at universities.
For instance, 2019 Computer Pioneer Award Recipient Jitendra Malik is the Arthur J. Chick Professor of Electrical Engineering and Computer Sciences at the University of California, Berkeley. He is also leading Facebook AI Research out of Menlo Park. Last year, US chip giant Nvidia established a new AI research lab in Toronto led by respected computer scientist Sanja Fidler, who has also continued in her role as an assistant professor in the department of computer science at the University of Toronto.
A main motivation for top researchers to stay in academia could be the young talented students in that environment, says Lipton. “It’s exciting to take on a brilliant but young researcher who has maybe never written a paper, and turn them into a fully-formed independent researcher. And more self-interestedly, PhD students at great schools are tremendously productive and collaborating with them allows you to pursue a more expansive and ambitious research program.”
Professors’ proportioned schedules between academia and industry commitments can vary from extreme 80/20 or 20/80 to an even 50/50 split. Bengio says “the advantage of this kind of formula is that those professors can continue supervising graduate students. So they will probably have less teaching or no teaching at all, but they will still help training the next generation, of course usually have a better salary as well.”
Although opting for industry over academia remains a popular choice, many if not most significant AI achievements have their roots in years of academic work. Clearly, academia needs fresh blood to continue to drive the wheel of innovation. Today we are seeing a more inclusive academic environment that welcomes young graduate students from top schools who want to investigate fundamental problems while also working to make a profound impact on the real world.
Bengio says he is seeing a new influx of young faculty members who have chosen to come back to universities to teach deep learning. “I think it’s still an issue that we lost a lot of people this way, but it’s getting better.”
Journalist: Tony Peng | Editor: Michael Sarazen
2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.
Follow us on Twitter @Synced_Global for daily AI news!
We know you don’t want to miss any stories. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.
