Artificial Evolution of Very Small Spiking Neural Network Robust to Noise and Damage for Recognising Temporal Patterns

Biological systems maintain their functionality in presence of noise
and damage (for a more detailed introduction, see [1]). Such robustness can stem from redundancy of elements, but large artificial neural
networks are not necessarily more robust than very small networks. In
order to understand how robustness emerges in spiking neural networks performing simple computational tasks, we have evolved very
small networks of adaptive exponential integrate and fire neurons [2].
The networks consisted of 3 inputs, and up to 4 adaptive exponential
neurons: up to 3 interneurons, and 1 output, with inputs connecting to interneurons, interneurons to output, and recurrent connections allowed between interneurons. The task of the networks was to
produce spike(s) on output when the inputs present signals to the
interneurons in a certain order (Fig. 1). Each signal lasts for 6 ms and
is followed by silence on all inputs for 16 ms. The artificial evolutionary process (in which the structure of the network is encoded in linear
genomes in a way inspired by RNA world; see [1] for details) allows for
addition or removal of connections in the network, changes of synaptic weights, and addition or removal of interneurons [3]. We have
investigated the evolution of such networks in presence of additive
Gaussian noise (SD 2 mV) on membrane potential, and were successful
in obtaining networks who were almost perfect (responding with one
spike to all target subsequences in the output, with hardly any spikes
elsewhere). Our results show that evolved networks were very robust
to perturbation of neuronal parameters. The best network from 50
independent runs was robust to changes of all neuronal parameters
tested, such as effective resting potential (evolved for −70 mV, range
of robustness: [−60, −83] mV), reset potential after the spike (−58;
[−45, −63] mV), spike initiation threshold potential (−50; [−48, −53]
mV), membrane time constant (20; [9, 100] ms), slope factor (2; [1.0,2.7] mV), membrane capacitance (0.2; [0.18, 0.23] nF), subthreshold
adaptation conductance (2; [−10, 17] nS), and spike-initiated adaptation (0; [0, 40] pA). Moreover, we have also observed that the evolved
networks are robust to variation on the length of silences (evolved for
16 mS, robustness range [10, 50] ms) between signals and the length
of signals (6; [5, 7] ms), indicating that such network can maintain their
state.