FB 6 Mathematik/Informatik/Physik

Institut für Mathematik


Navigation und Suche der Universität Osnabrück


Hauptinhalt

Topinformationen

Advanced NLP

Block course
8.3546

Dozenten

Beschreibung

The course will provide a historical perspective on deep learning for natural language processing (NLP) and will address recent topics such as Transformers (e.g., BERT and GPT), attention-based models and recent models for dialogue. In addition, we will discuss language acquisition, the cognitive plausibility of AI models, and the extraction of semantic structure from raw text. We will take a look at the current revival of linguistic structure in the deep learning community, either through the analysis of attention patterns in Transformers (according to which linguistic structure is a 'by-product' of neural attention) or through diagnostic classifiers.

We will go through a bit of theory in the first part of every lecture, and proceed with a discussion of
recent literature in the second part, with an active role for students which will introduce papers on the collective reading list and work in groups on short practicals.

Course objectives:
Students will obtain knowledge about the historical and current trends in deep learning-based NLP. They will be able to take a critical look at current literature and will have a rather advanced understanding of the challenges, opportunities and pitfalls of deep learning applied to language. Furthermore, they will have obtained practical knowledge about how to instantiate some of the latest NLP models.

Prerequisites: Basic programming; Deep Learning for NLP or other deep learning background.

Weitere Angaben

Ort: nicht angegeben
Zeiten:
Erster Termin:
Veranstaltungsart: Seminar (Offizielle Lehrveranstaltungen)

Studienbereiche

  • Cognitive Science > Bachelor-Programm
  • Cognitive Science > Master-Programm
  • Cognitive Science > Promotionsprogramm

Past and Forthcoming Events

Publications

  • Asymptotics of a time-bounded cylinder model, with N. Aschenbruck and S. Bussmann, Probability in the Engineering and Informational Sciences, https://doi.org/10.1017/S0269964822000420
  • The method of cumulants for the normal approximation, with S. Jansen and K. Schubert, Probability Surveys 2022, Vol. 19, 185-270, https://doi.org/10.1214/22-PS7
  • Sedentary Random Waypoint, with C. Betken, arXiv:2009.02941
  • The Impact of Bit Errors on Intra-Session Network Coding with Heterogeneous Packet Lengths, with B. Schütz, N. Aschenbruck, S. Bussmann and M. Juhnke-Kubitzke, Proc. of the 45th IEEE LCN Symposium on Emerging Topics in Networking LCN, virtually hosted in Sydney, Australia, Nov. 16–19, 2020.
  • Stationarity for the Small World in Motion Mobility Model, with Nils Aschenbruck, Christian Heiden und Matthias Schwamborn, MSWIM '19: Proceedings of the 22nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, Nov 25-29, 2019, https://doi.org/10.1145/3345768.3355935
  • Crossing Numbers and Stress of Random Graphs, with Markus Chimani and Matthias Reitzner, In Proceedings 26th International Symposium, GD 2018, Barcelona, Spain, 255--268, 2018 available here and for an extended journal version here: https://arxiv.org/abs/1808.07558
  • Fluctuations in a general preferential attachment model via Stein's method, with Carina Betken and Marcel Ortgiese, Random Structures & algorithms, vol.55, no.4, 2019 available here
  • Connection times in large ad-hoc mobile networks, Bernoulli, vol.22, no.4, 2143--2176, 2016 available here
    with Gabriel Faraud, Wolfgang König
  • The random disc thrower problem, Proceedings of the 90th European Study Group Mathematics with Industry, 59-78, 2013  available here with T. van der Aalst, D. Denteneer, M. Hong Duong, R. J. Kang, M. Keane, J. Kool, I. Kryven, T. Meyfroyt, T. Müller, G. Regts, J. Tomczyk
  • Edge fluctuations of eigenvalues of Wigner matrices, High Dimensional Probability VI: the Banff volume, Progress in Probability, vol.66, 261-275, Springer, Basel, 2013 available here
    with Peter Eichelsbacher
  • Moderate deviations for the determinant of Wigner matrices, Dedicated to Friedrich Götze on the occasion of his sixtieth birthday, Limit Theorems in Probability, Statistics and Number Theory, Springer Proceedings in Mathematics & Statistics, vol.42, 253-275, 2013, available here
    with Peter Eichelsbacher
  • Moderate deviations for the eigenvalue counting function of Wigner matrices, ALEA, Lat. Am. J. Probab. Math. Stat. 10 (1), 27-44, 2013, available here
    with Peter Eichelsbacher
  • Moderate deviations via cumulants, Journal of Theor. Probability, 2012, available here
    with Peter Eichelsbacher
  • Moments of recurrence times for Markov chains, Electronic Comm. Probab., 16(28), 296-303, 2011, available here
    with Frank Aurzada, Marcel Ortgiese, Michael Scheutzow
  • Moderate deviations in a random graph and for the spectrum of Bernoulli random matrices, Electronic Journal of Probability, Vol. 14, Paper no. 92, 2636-2656, 2009, available here
    with Peter Eichelsbacher
  • Perpendicular transport of charged particles in slab turbulence: recovery of diffusion for realistic wave-spectra?, Journal of Physics G: Nuclear and Particle Physics, 35, 025202, 2008
    with Andreas Shalchi
  • Velocity correlation functions of charged test particles, Journal of Physics G: Nuclear and Particle Physics, 34, 859, 2007
    with Andreas Shalchi