[Paper] Multi-scale dynamics by adjusting the leaking rate to enhance the performance of deep echo state networks

次世代人工知能への応用化に期待 千葉工業大学・基礎生物学研究所・兵庫県立大学などの研究チーム、深層エコーステートネットワークにおける多層化が生み出す多様な時間スケールのダイナミクスが性能向上の鍵

https://www.nibb.ac.jp/press/2024/07/18.html

The deep echo state network (Deep-ESN) architecture, which comprises a multi-layered reservoir layer, exhibits superior performance compared to conventional echo state networks (ESNs) owing to the divergent layer-specific time-scale responses in the Deep-ESN. Although researchers have attempted to use experimental trial-and-error grid searches and Bayesian optimization methods to adjust the hyperparameters, suitable guidelines for setting hyperparameters to adjust the time scale of the dynamics in each layer from the perspective of dynamical characteristics have not been established. In this context, we hypothesized that evaluating the dependence of the multi-time-scale dynamical response on the leaking rate as a typical hyperparameter of the time scale in each neuron would help to achieve a guideline for optimizing the hyperparameters of the Deep-ESN.

Inoue, Nobukawa, et al., Frontiers in Artificial Intelligence, 2024

https://doi.org/10.3389/frai.2024.1397915