logo SBA

ETD

Digital archive of theses discussed at the University of Pisa

 

Thesis etd-05022022-184611


Thesis type
Tesi di laurea magistrale
Author
BASTERI, ANDREA
URN
etd-05022022-184611
Thesis title
Quantitative Convergence of Randomly Initialized Wide Deep Neural Networks Towards Gaussian Processes
Department
MATEMATICA
Course of study
MATEMATICA
Supervisors
relatore Trevisan, Dario
Keywords
  • ANN
  • deep learning
  • deep neural networks
  • DNN
  • Gaussian Processes
  • machine learning
  • neural networks
  • numerical experiments
  • optimal transport
  • Wasserstein distance
Graduation session start date
10/06/2022
Availability
Full
Summary
The thesis regards the quantitative convergence of randomly initialized fully connected deep neural networks towards a suitable Gaussian limit. In particular we estimate the 2-Wasserstein distance between the joint distribution of the outputs of a neural network and a suitable Gaussian Process, the "Neural Network Gaussian Process" (NNGP).
We make our estimates with techniques of optimal transport and techniques linked to Gaussian processes, and we obtain that such rate of convergence can be estimated with the inverse of the square root of the numbers of neurons in the layers.
We provide some numerical simulation to show that our estimates are not sharp, so further studies are needed to find the true rate of convergence and to study other properties of this convergence.
File