• artificial neural network
• lasso
• statistical learning

Recently scientists are working to make neural networks not only more accurate but also more interpretable, and one approach is to enforce features sparsity, i.e. allow the network to use just few of the available features.
In this thesis, we present some regularization methods for artificial neural networks that are able to induce sparsity.
In particular, we analyze three methods, completely different from each other in the way they induce sparsity.
The first one is Lasso-ANN. This method is driven by a single regularization parameter that is selected using the quantile universal threshold rule.
The second method is LassoNet. This method is driven by two regularization parameters. The interesting feature of LassoNet is the presence of an input-to-output skip-layer connection that allows a feature to have non-zero weights in the first hidden unit only if its skip-layer connection is active.
The third method is called Greedy Elimination Penalized Neural Network. It is driven by three hyperparameters. During each iteration, it selects and drops out, togheter with the weights tied to it, a variable that causes a small change of the loss function.
We report experimental results on a number of datasets to quantify the effectiveness of the methods. In particular,
it is shown that, unlike the other methods, Lasso-ANN follows a phase transition
in the probability of exact support recovery.