• speech recognition
• sequenze temporali
• reti neurali ricorrenti
• recurrent neural networks
• MFCC
• machine learning
• linear memory networks
• clockwork rnn
• apprendimento automatico
• temporal sequences
• TIMIT
• vanishing gradient

In the context of temporal sequences and Recurrent Neural Networks, the vanishing gradient and the need to discover and memorize long-term dependencies and hierarchical information are actively studied problems, but they may also lead us to create overly-complicated networks. Thus some researchers decided to separate concerns with the purpose of controlling such complexity.
We combined Linear Memory Networks, which conceptually separates functional input-output transformations from memory capabilities, with Clockwork-RNNs, which better memorizes dependencies at different resolutions thanks to dedicated modules.
We call this new model Clockwork Linear Memory Networks (CW-LMNs). We also developed an incremental pretraining algorithm for this model as an extension of the pretraining algorithm available for the memory component of Linear Memory Networks, in which we incrementally add and train a memory module at a time. We show that our model outperforms related models from literature, such as gated networks, in tasks of sequence generation on signals and spoken word recognition and that pretraining algorithms provide better performances, improved training stability and possibly lower training times.