Tesi etd-02082023-151924 |
Link copiato negli appunti
Tipo di tesi
Tesi di laurea magistrale
Autore
VERZELLA, GIANFRANCO
URN
etd-02082023-151924
Titolo
Sparsity inducing regularization methods for artificial neural networks
Dipartimento
MATEMATICA
Corso di studi
MATEMATICA
Relatori
relatore Prof. Bigi, Giancarlo
relatore Prof. Sardy, Sylvain
correlatore Dott.ssa Ma, Xiaoyu
relatore Prof. Sardy, Sylvain
correlatore Dott.ssa Ma, Xiaoyu
Parole chiave
- artificial neural network
- lasso
- statistical learning
Data inizio appello
24/02/2023
Consultabilità
Non consultabile
Data di rilascio
24/02/2063
Riassunto
Recently scientists are working to make neural networks not only more accurate but also more interpretable, and one approach is to enforce features sparsity, i.e. allow the network to use just few of the available features.
In this thesis, we present some regularization methods for artificial neural networks that are able to induce sparsity.
In particular, we analyze three methods, completely different from each other in the way they induce sparsity.
The first one is Lasso-ANN. This method is driven by a single regularization parameter that is selected using the quantile universal threshold rule.
The second method is LassoNet. This method is driven by two regularization parameters. The interesting feature of LassoNet is the presence of an input-to-output skip-layer connection that allows a feature to have non-zero weights in the first hidden unit only if its skip-layer connection is active.
The third method is called Greedy Elimination Penalized Neural Network. It is driven by three hyperparameters. During each iteration, it selects and drops out, togheter with the weights tied to it, a variable that causes a small change of the loss function.
We report experimental results on a number of datasets to quantify the effectiveness of the methods. In particular,
it is shown that, unlike the other methods, Lasso-ANN follows a phase transition
in the probability of exact support recovery.
In this thesis, we present some regularization methods for artificial neural networks that are able to induce sparsity.
In particular, we analyze three methods, completely different from each other in the way they induce sparsity.
The first one is Lasso-ANN. This method is driven by a single regularization parameter that is selected using the quantile universal threshold rule.
The second method is LassoNet. This method is driven by two regularization parameters. The interesting feature of LassoNet is the presence of an input-to-output skip-layer connection that allows a feature to have non-zero weights in the first hidden unit only if its skip-layer connection is active.
The third method is called Greedy Elimination Penalized Neural Network. It is driven by three hyperparameters. During each iteration, it selects and drops out, togheter with the weights tied to it, a variable that causes a small change of the loss function.
We report experimental results on a number of datasets to quantify the effectiveness of the methods. In particular,
it is shown that, unlike the other methods, Lasso-ANN follows a phase transition
in the probability of exact support recovery.
File
Nome file | Dimensione |
---|---|
La tesi non è consultabile. |