• dynamical systems
• machine learning
• neural networks
• recurrent neural networks
• reservoir computing
• statistical learning theory
• statistics
• time series

We are going to articulate our theoretical discussion of this subject as follows: in Chapter 1 we are going to define formally the Reservoir Systems and the Input-Output System we just introduced, together with the spaces they act between and with the so called Echo State Property, which is the key tool in linking the hidden-state paradigm of Reservoir Computers and the Input-Output problems we wish to study.

In Chapter 2 we'll introduce and study the Fading Memory Property, which is a key feature of reservoir systems that is closely linked to the ``good behaviour" of the underlying dynamical systems that governs the Reservoir.

In Chapter 3 we'll deal with the problem of universality, that is that of finding families of Reservoirs that are able to approximate arbitrarily well any given Input-Output function. As we'll see, here it will be key to differentiate between the cases in which the Input is modelled as a stochastic process and that in which is thought as a deterministic feature of the problem. Furthermore, it will be substantially easier to find such families when the Input space is, in some sense, uniformly bounded in time.

In Chapter 4 we'll introduce Time Delay Reservoir Computers, which are a particular class of Reservoir Computers that have exhibited outstanding empirical performances, and for which the bond between the performances and the dynamical properties of the underlying system are especially evident. We'll also introduce the Nonlinear Memory Capacity, which is a technique to better select the dynamical system that controls the Reservoir which has gained a lot of traction lately due to some promising empirical results.

In Chapter 5 we'll study the notion of Differentiability of Reservoir Computers, which is closely linked to both Universality and Time Delay Reservoir Computers where it will once again be evident the strong difference between the stochastic and the deterministic cases.

Finally, in Chapter 6 we'll perform an empirical study of Reservoir Computers, providing a clear benchmark of the TDRC against state-of-the-art RNN techniques on a time series Forecasting task.