The dawn of large neural networks and their transformative impact on AI

The last two years have seen a dramatic growth of neural networks. Models such as Bert Large with its 345 million parameters which were considered huge just three years ago are being dwarfed by modern DNNs which are frequently more than 1000 times bigger. This growth is not driven by accademic vanity of the AI comunity but by their exceptional few shot learning capability. In contrast to their smaller counterparts that require thousands/tens of thousands of training samples to adapt to new tasks large models achieve state of the art performance with just a handful of samples fundamentally transforming the way we do AI. This revolution started in Natural Language Processing but is already transforming countless other disciplines like computer vision, automatic speach recognition, chmical compount modeling or protein modeling where very large unsupervised models are becoming a common practice. In this talk we will dive deeper into the properties of large DNNs and discuss the process of creating them, from data prepartion through distributed training to production deployment.

Adam Grzywaczewski

Adam Grzywaczewski

Senior Deep Learning Data Scientist at NVIDIA

Other sessions from: Learn with the AI Community around the world

Building a Responsible AI strategy

Building a Responsible AI strategy

FATE AI ( Fair, Accountable, Transparent, Ethical) What is Responsible AI?...

Sammy Deprez Sammy Deprez
Hacking Your Vacation: Using Data for Fun

Hacking Your Vacation: Using Data for Fun

Data can be powerful, meaningful, AND fun. Making data-driven decisions can...

Becky Gandillon Becky Gandillon
Machine Learning Frameworks Interoperability

Machine Learning Frameworks Interoperability

To develop mature data science, machine learning, and deep learning applica...

Miguel Martínez Miguel Martínez
Christian Hundt Christian Hundt
Taiwan COVID-19 vax card image recognition by Azure Form Recognizer

Taiwan COVID-19 vax card image recognition by Azure Form Recognizer

Azure Form Recognizer has been wide used for invoices, W-2 form, health i...

Ryan Chung Ryan Chung
Responsible AI

Responsible AI

Nowadays with the development of AI, it became a big part of our lives. It...

Veronika Kolesnikova Veronika Kolesnikova
Troubleshooting real-time business metrics with StarTree ThirdEye

Troubleshooting real-time business metrics with StarTree ThirdEye

StarTree ThirdEye is a source available anomaly detection platform that...

Madhumita Mantri Madhumita Mantri
Suvodeep Pyne Suvodeep Pyne