IVADO’s scientific director, Yoshua Bengio, receives the Turing Award

Yoshua Bengio, IVADO’s Scientific Director, professor of Computer Science at UdeM and Scientific Director of Mila, is named a co-recipient of the annual A.M Turing Award, sharing $1 million for revolutionizing artificial intelligence.

Yoshua Bengio is no stranger to accolades. But when the Université de Montréal computer science professor learned that the biggest computing organization in the world had named him a winner of the $1-million A.M. Turing Award, he was stunned.

“The Turing Award is at the top of the IT world’s list of awards and recognitions – it’s considered the Nobel Prize of computing,” Bengio, 55, said of the honour announced today by the New York-based Association for Computing Machinery.

“Needless to say, it was a shock when I heard the news.”

Bengio is a co-recipient of the annual prize with colleagues Geoffrey Hinton of the University of Toronto and Yann LeCun of New York University. They will formally receive the award, named after British mathematician Alan M. Turing and funded by Google, on June 15 at the ACM’s annual convention in San Francisco.

Calling them “fathers of the deep-learning revolution, the organization, which was founded in 1947 and has close to 100,000 members, said Bengio and his colleagues are being recognized “for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.”

Building deep networks

In computer science, the term “neural networks” refers to systems composed of layers of relatively simple computing elements called “neurons” that are simulated in a computer.  These “neurons,” which only loosely resemble the neurons in the human brain, influence one another via “weighted” connections. By changing the “weights” on the connections, it is possible to change the computation performed by the neural network.

Early on in their research, Bengio and his colleagues recognized the importance of building deep networks using many layers – in other words, “deep learning.” In traditional computing, a computer program directs the computer with explicit step-by-step instructions. In deep learning, a subfield of artificial-intelligence research, the computer is not explicitly told how to solve a particular task such as object classification. Instead, it uses a learning algorithm to extract patterns in the data that relate the input data, such as the pixels of an image, to the desired output, such as the label “cat”.

The challenge for researchers has been to develop effective learning algorithms that can modify the weights on the connections in an artificial neural network so that these weights capture the relevant patterns in the data. While the use of artificial neural networks as a tool to help computers recognize patterns and simulate human intelligence had been introduced in the 1980s, by the early 2000s Bengio, Hinton and LeCun, were among a small group who remained committed to this approach.

Skepticism at first

Their efforts to rekindle the AI community’s interest in neural networks were initially met with skepticism. But with the growth of powerful graphics processing unit (GPU) computers, as well as by access to massive datasets, their methodology eventually became the dominant paradigm in their field, resulting in major technological advances in technologies that have since become widely known and used, such as computer vision, speech recognition and machine translation.

“Artificial intelligence is now one of the fastest-growing areas in all of science and one of the most talked-about topics in society,” said ACM president Cherri M. Pancake. “The growth of and interest in AI is due, in no small part, to the recent advances in deep learning for which Bengio, Hinton and LeCun laid the foundation.

“These technologies are used by billions of people. Anyone who has a smartphone in their pocket can tangibly experience advances in natural language processing and computer vision that were not possible just 10 years ago. In addition to the products we use every day, new advances in deep learning have given scientists powerful new tools in areas ranging from medicine to astronomy to materials science.”

For Bengio – an officer of the Order of Canada, fellow of the Royal Society of Canada and winner of the Marie-Victorin Prize – the Turing Award is a kind of vindication for having pursued the neural-network route all these years in the face of skeptics.

“This award for Geoff Hinton, Yann LeCun and me means first and foremost the recognition by the computer science community of the importance of the field of neural networks and deep learning for AI, and therefore of the field that has fascinated us since our research began and that now receives an ultimate seal of approval. Knowing that this was an area that seemed to be frowned upon by the computer elite only a few years ago, it is a very strong message.”

Longtime colleagues

The three scientists are longtime colleagues. LeCun did postdoctoral work under Hinton’s supervision, and beginning in the early 1990s LeCun and Bengio worked together at Bell Labs, when it was still owned by AT&T. Besides influencing each other in their day-to-day research, the three continue to explore the intersection of machine learning with neuroscience and cognitive science, most notably through their joint participation in the Learning in Machines and Brains program, an initiative of CIFAR, formerly known as the Canadian Institute for Advanced Research.

Unlike Hinton, who is a vice-president at Google, and LeCun, a vice-president and chief AI scientist at Facebook, Bengio is first and foremost a professor of Université de Montréal. Though he maintains close ties with the technology industry, he calls UdeM home, and recently helped inaugurate new headquarters in Montreal’s Mile-Ex neighbourhood for Mila, the renowned machine-learning institute he founded and for which he holds the title of scientific director.

“We are exceptionally fortunate in Montreal to have a researcher of this calibre whose ideas are transforming our world,” said Mila CEO Valérie Pisano. “Through his unique approach, notably by bringing together two universities [UdeM and McGill] in the same institute, Yoshua Bengio is helping to build a rich ecosystem here that stimulates both scientific advances and Quebec’s economic growth.”

Added UdeM rector Guy Breton: “The entire Université de Montréal community joins me in congratulating Yoshua Bengio on receiving this award for his outstanding scientific advances,” said UdeM rector Guy Breton. “His achievements are all the more remarkable because Professor Bengio is not only a visionary scientist whose discoveries continue to have an impact on our world, but he is also a formidable teacher, a socially engaged researcher, an excellent popularizer and a tireless advocate of the strengths of Montreal and its community.”

‘No plans for the moment’

It is entirely appropriate that the Turing Award isn’t going to just one scientist, but will be shared among three, said Bengio, adding that he has “no plans for the moment” about what to do with the prize money.

“The community we have helped to create around deep learning does indeed enjoy a deep culture of sharing and collaboration and all three of us have set an example in recent decades,” he said.

Can he imagine a day when a computer will win a Turing, for things it developed itself?
“I don’t think so,” he replied, “just as a car wouldn’t win a prize for going faster than the best driver in the world.”

Three technical breakthroughs of Yoshua Bengio

• Probabilistic models of sequences: In the 1990s, Bengio combined neural networks with probabilistic models of sequences, such as hidden Markov models. These ideas were incorporated into a system used by AT&T/NCR for reading handwritten checks, were considered a pinnacle of neural network research in the 1990s, and modern deep learning speech recognition systems are extending these concepts.

• High-dimensional word embeddings and attention: In 2000, Bengio authored the landmark paper, “A Neural Probabilistic Language Model,” that introduced high-dimension word embeddings as a representation of word meaning. Bengio’s insights had a huge and lasting impact on natural language processing tasks including language translation, question answering, and visual question answering.  His group also introduced a form of attention mechanism which led to breakthroughs in machine translation and form a key component of sequential processing with deep learning.

• Generative adversarial networks: Since 2010, Bengio’s papers on generative deep learning, in particular the Generative Adversarial Networks (GANs) developed with Ian Goodfellow, have spawned a revolution in computer vision and computer graphics. In one fascinating application of this work, computers can actually create original images, reminiscent of the creativity that is considered a hallmark of human intelligence.

About the award and ACM

The A.M. Turing Award was named for Alan M. Turing, the British mathematician who articulated the mathematical foundation and limits of computing, and who was a key contributor to the Allied cryptanalysis of the Enigma cipher during World War II. Since its inception in 1966, the Turing Award has honored the computer scientists and engineers who created the systems and underlying theoretical foundations that have propelled the information technology industry.

ACM, the Association for Computing Machinery, is the world’s largest educational and scientific computing society, uniting computing educators, researchers and professionals to inspire dialogue, share resources and address the field’s challenges. ACM strengthens the computing profession’s collective voice through strong leadership, promotion of the highest standards, and recognition of technical excellence. ACM supports the professional growth of its members by providing opportunities for life-long learning, career development, and professional networking.