29. 06. 2023, 13:00

What’s wrong with large language models and what we should be building instead

Video
29. 06. 2023, 13:00

What’s wrong with large language models and what we should be building instead

Video
Prof. dr. Thomas G. Dietterich
Oregon State University, USA
Thomas G. Dietterich is Distinguished Professor Emeritus in the School of Electrical Engineering and Computer Science at Oregon State University, USA. He is one of the pioneers of the field of machine learning and has authored more than 225 refereed publications and two books. His current research topics include robust artificial intelligence, robust human-AI systems, and applications in sustainability.

Large Language Models (LLMs) provide a pre-trained foundation for training many interesting artificial intelligence systems. However, they have many shortcomings. They are expensive to train and to update, their non-linguistic knowledge is poor, they make false and self-contradictory statements, and these statements can be socially and ethically inappropriate. In this talk, I will review these shortcomings and current efforts to address them within the existing LLM framework. I will then argue for a different, more modular architecture that decomposes the functions of existing LLMs and adds several additional components. I believe this alternative can address all of the shortcomings of LLMs. I will speculate about how this modular architecture could be built through a combination of machine learning and engineering.

Prof. dr. Thomas G. Dietterich
Oregon State University, USA
Thomas G. Dietterich is Distinguished Professor Emeritus in the School of Electrical Engineering and Computer Science at Oregon State University, USA. He is one of the pioneers of the field of machine learning and has authored more than 225 refereed publications and two books. His current research topics include robust artificial intelligence, robust human-AI systems, and applications in sustainability.

Last lectures

 

Archive

 

2024
2023
2022
2021
2020
2019

2024

Physics at the edge of chaos

29. 11. 2023

Assoc. prof. dr. Lev Vidmar, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana, Slovenia

2023

Physics at the edge of chaos

29. 11. 2023

Assoc. prof. dr. Lev Vidmar, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana, Slovenia

Machine learning from relational and textual data

23. 03. 2023

prof. dr. Nada Lavrač, Jožef Stefan Institute, Ljubljana, Slovenia and prof. dr. Marko Robnik-Šikonja, Faculty of Computer Science and Informatics, University of Ljubljana, Slovenia

2022

Superdifusion in a Heisenberg magnetic chain
7. 12. 2022
Enej Ilievski, Faculty of Mathematics and Physics, University of Ljubljana

Quantum black holes: an encounter between Hawking and Ramanujan
1. 12. 2022
Atish Dabholkar, International Centre for Theoretical Physics (ICTP), Trieste, Italy

In the depths of the atom
23. 11.2022
Alojz Kodre, Faculty of Mathematics and Physics, University of Ljubljana

Domain-wall dynamics in ferroelectric and relaxor-based ceramics
16. 11. 2022
Tadej Rojac, Jožef Stefan Institute

Usual and unusual states made of quarks
10. 11. 2022
Saša Prelovšek Komelj, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana

Processing matters: materials challenges for the second quarter of the 21st century
26. 10. 2022
Geoff Brennecka, Colorado School of Mines, USA

Geo-inspired ceramic carriers: from design to medical applications
21. 9. 2022
Bogdan Parakhonskiy, Ghent University, Belgium

Innovations in radiation medicine
7. 9. 2022
Thomas Rockwell Mackie, University of Wisconsin, USA

Additive Manufacturing and Liquid Crystal Technologies
15. 6. 2022
Stephen M. Morris, University of Oxford, United Kingdom

Modelling chemical pollution to support a safe and sustainable chemicals management
5. 5. 2022
Peter Fantke, Technical University of Denmark, Denmark

Humanoid Robotics – Understanding Human Performance and Intelligence
24. 3. 2022
Tamim Asfour, Karlsruhe Institute of Technology (KIT), Nemčija

Challenges of the internet of the future: security, connectivity and humanity aspects
23. 3. 2022
Borka Jerman Blažić, Jožef Stefan Institute

What does a black and white picture of atoms tell us?
22. 3. 2022
Goran Dražić, National Institute of Chemistry and Jožef Stefan Institute

Haze Instruments d.o.o., Measurement of aerosol absorption with a photothermal interferometer
21. 3. 2022
Luka Drinovec, University of Nova Gorica, Jožef Stefan Institute

Haze Instruments d.o.o., Light absorption in aerosols affects climate – how do we measure it?
21. 3. 2022
Griša Močnik, University of Nova Gorica, Jožef Stefan Institute

FAIR – the universe in the lab
16. 3. 2022
Paolo Giubellino in Joerg Blaurock, GSI and FAIR, Darmstadt, Germany

From quantum impurities to quantum devices
12. 1. 2022
Rok Žitko, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana

2021

Cavitation – the path from its prevention to its use for destroying viruses and bacteria
15. 12. 2021
Matevž Dular, Faculty of Mechanical Engineering, University of Ljubljana

Multiscale simulations of open molecular systems
24. 11. 2021
Matej Praprotnik, National Institute of Chemistry and Faculty of Mathematics and Physics University of Ljubljana

Why the Nobel Prize for the discovery of the Piezo1 and Piezo2 proteins was awarded
27. 10. 2021
Saša Svetina, Institute of biophysics, Faculty of Medicine, University of Ljubljana in Jožef Stefan Institute

Modeling of the COVID-19 epidemic in Slovenia
12. 05. 2021
Matjaž Leskovar, Jožef Stefan Institute

New research opportunities with free electron lasers
21. 04. 2021
Claudio Masciovecchio, Elettra Sincrotrone Trieste, Italy

X-ray diffraction screening of major protease crystals reveals SARS-CoV-2 inhibitors
16. 04. 2021
Dušan Turk, Jožef Stefan Institute

Optimization of measures to contain the COVID-19 epidemic
14. 04. 2021
Mitja Luštrek, Jožef Stefan Institute

Fullerenes: superconductivity, magnetism and qubits
26. 03. 2021
Denis Arčon, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana

Measurements of cosmic particles of extreme energies at the Pierre Auger Observatory
25. 03. 2021
Andrej Filipčič and Marko Zavrtanik, Jožef Stefan Institute and University of Nova Gorica

Challenges of the development of electronics as a companion of modern research and applications
25. 03. 2021
Janko Petrovčič, Jožef Stefan Institute

Quantum computing with atoms trapped by light
23. 03. 2021
David S. Weiss, Pennsylvania State University, University Park, USA

Ferroelectric ceramics: an overview of the century of research from discovery of the ferroelectric effect to the present day
23. 03. 2021
Barbara Malič, Jožef Stefan Institute and Jožef Stefan International Postgraduate School

Stable isotopes in interdisciplinary research
22. 03. 2021
Nives Ogrinc, Jožef Stefan Institute and Jožef Stefan International Postgraduate School

Archive

 

2024
2023
2022
2021
2020
2019

2024

2023

Physics at the edge of chaos

29. 11. 2023

Assoc. prof. dr. Lev Vidmar, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana, Slovenia

Machine learning from relational and textual data

23. 03. 2023

prof. dr. Nada Lavrač, Jožef Stefan Institute, Ljubljana, Slovenia and prof. dr. Marko Robnik-Šikonja, Faculty of Computer Science and Informatics, University of Ljubljana, Slovenia

2022

Superdifusion in a Heisenberg magnetic chain
7. 12. 2022
Enej Ilievski, Faculty of Mathematics and Physics, University of Ljubljana

Quantum black holes: an encounter between Hawking and Ramanujan
1. 12. 2022
Atish Dabholkar, International Centre for Theoretical Physics (ICTP), Trieste, Italy

In the depths of the atom
23. 11.2022
Alojz Kodre, Faculty of Mathematics and Physics, University of Ljubljana

Domain-wall dynamics in ferroelectric and relaxor-based ceramics
16. 11. 2022
Tadej Rojac, Jožef Stefan Institute

Usual and unusual states made of quarks
10. 11. 2022
Saša Prelovšek Komelj, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana

Processing matters: materials challenges for the second quarter of the 21st century
26. 10. 2022
Geoff Brennecka, Colorado School of Mines, USA

Geo-inspired ceramic carriers: from design to medical applications
21. 9. 2022
Bogdan Parakhonskiy, Ghent University, Belgium

Innovations in radiation medicine
7. 9. 2022
Thomas Rockwell Mackie, University of Wisconsin, USA

Additive Manufacturing and Liquid Crystal Technologies
15. 6. 2022
Stephen M. Morris, University of Oxford, United Kingdom

Modelling chemical pollution to support a safe and sustainable chemicals management
5. 5. 2022
Peter Fantke, Technical University of Denmark, Denmark

Humanoid Robotics – Understanding Human Performance and Intelligence
24. 3. 2022
Tamim Asfour, Karlsruhe Institute of Technology (KIT), Nemčija

Challenges of the internet of the future: security, connectivity and humanity aspects
23. 3. 2022
Borka Jerman Blažić, Jožef Stefan Institute

What does a black and white picture of atoms tell us?
22. 3. 2022
Goran Dražić, National Institute of Chemistry and Jožef Stefan Institute

Haze Instruments d.o.o., Measurement of aerosol absorption with a photothermal interferometer
21. 3. 2022
Luka Drinovec, University of Nova Gorica, Jožef Stefan Institute

Haze Instruments d.o.o., Light absorption in aerosols affects climate – how do we measure it?
21. 3. 2022
Griša Močnik, University of Nova Gorica, Jožef Stefan Institute

FAIR – the universe in the lab
16. 3. 2022
Paolo Giubellino in Joerg Blaurock, GSI and FAIR, Darmstadt, Germany

From quantum impurities to quantum devices
12. 1. 2022
Rok Žitko, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana

2021

Cavitation – the path from its prevention to its use for destroying viruses and bacteria
15. 12. 2021
Matevž Dular, Faculty of Mechanical Engineering, University of Ljubljana

Multiscale simulations of open molecular systems
24. 11. 2021
Matej Praprotnik, National Institute of Chemistry and Faculty of Mathematics and Physics University of Ljubljana

Why the Nobel Prize for the discovery of the Piezo1 and Piezo2 proteins was awarded
27. 10. 2021
Saša Svetina, Institute of biophysics, Faculty of Medicine, University of Ljubljana in Jožef Stefan Institute

Modeling of the COVID-19 epidemic in Slovenia
12. 05. 2021
Matjaž Leskovar, Jožef Stefan Institute

New research opportunities with free electron lasers
21. 04. 2021
Claudio Masciovecchio, Elettra Sincrotrone Trieste, Italy

X-ray diffraction screening of major protease crystals reveals SARS-CoV-2 inhibitors
16. 04. 2021
Dušan Turk, Jožef Stefan Institute

Optimization of measures to contain the COVID-19 epidemic
14. 04. 2021
Mitja Luštrek, Jožef Stefan Institute

Fullerenes: superconductivity, magnetism and qubits
26. 03. 2021
Denis Arčon, Jožef Stefan Institute and Faculty of Mathematics and Physics, University of Ljubljana

Measurements of cosmic particles of extreme energies at the Pierre Auger Observatory
25. 03. 2021
Andrej Filipčič and Marko Zavrtanik, Jožef Stefan Institute and University of Nova Gorica

Challenges of the development of electronics as a companion of modern research and applications
25. 03. 2021
Janko Petrovčič, Jožef Stefan Institute

Quantum computing with atoms trapped by light
23. 03. 2021
David S. Weiss, Pennsylvania State University, University Park, USA

Ferroelectric ceramics: an overview of the century of research from discovery of the ferroelectric effect to the present day
23. 03. 2021
Barbara Malič, Jožef Stefan Institute and Jožef Stefan International Postgraduate School

Stable isotopes in interdisciplinary research
22. 03. 2021
Nives Ogrinc, Jožef Stefan Institute and Jožef Stefan International Postgraduate School