or: Learning Longer Memory In Recurrent Neural Networks

The authors introduce the Structurally-Constrained Recurrent Network (SCRN), which is an RNN with a slowly updated context layer. It is very parameter-efficient compared with other RNN memory architectures.

see comparison