Abstract
We study two central aspects of information processing in cognitive systems: one is the ability to incorporate fresh information to already learnt models; the other is the “trickling” of information through the many layers of a cognitive processing pipeline. We investigate the extent to which these specific structures of cognitive processing impact their informational optimal limits. To do so, we present mathematical characterisations and low-dimensional numerical examples, which explore formal properties of the Information Bottleneck method: namely, how it relates to successive refinement, and successive coarsening of information.
| Original language | English |
|---|---|
| Pages | 1-8 |
| Number of pages | 8 |
| Publication status | Published - 21 Nov 2022 |
| Event | NeurIPS 2022 Workshop on Information-Theoretic Principles in Cognitive Systems - New Orleans Convention Center, New Orleans, United States Duration: 3 Dec 2022 → 3 Dec 2022 Conference number: 1 https://sites.google.com/view/infocog-neurips2022/home |
Workshop
| Workshop | NeurIPS 2022 Workshop on Information-Theoretic Principles in Cognitive Systems |
|---|---|
| Abbreviated title | NeurIPS 2022 |
| Country/Territory | United States |
| City | New Orleans |
| Period | 3/12/22 → 3/12/22 |
| Internet address |
Keywords
- Incremental learning
- Information Bottleneck
- Successive Refinement