ETD

• Criticality
• Deviation from Linearity
• Echo State Network
• Intrinsic Plasticity
• Masked IP
• Memory
• Non Linearity
• Reservoir
• Trimodal

The current work aims to propose a novel method to tune Reservoir and Echo State Networks to the
right point of the memory/non-linearity trade-off, to better accomplish downstream tasks. A geomet-
rical analysis is carried on at first, studying how the t anh space impacts the signal reconstruction
capabilities of RNNs. Then, given the emerged assumptions, a probabilistic solution is designed and
built up, proposing an extension of the Intrinsic Plasticity algorithm, able to fit Gaussian Mixture
Models (GMM). With such a solution we were able to tune the network in a self-supervised way to
approximate any Gaussian-based target distribution and hence to focus on the region of interest of
the t anh space, regulating the degree of desired memory or non-linearity in an unsupervised way.
Four target configurations associated with expected different points of the trade-off were studied: a
Gaussian distribution, a symmetric bimodal distribution, and two hybrid "trimodal" distributions,
combining the previous two. To validate the hypothesis on the impact of the probabilistic optimiza-
tion, experiments were carried out on various tasks with tunable requirements of non-linearity or
memory, confirming the expected behavior and hence the robustness of our solution.