inductive biases (TEST 1)
Published:
Edited: [2025-04-29 Tue 10:16]
Note: this is not a real post (site under construction), but the content comes from my notes nonetheless
A set of assumptions that allows a learning algorithm to learn about structure in data, and as a result generalise well from limited data.
For instance: spatial consistency in the context of learning about objects from images.
From a Bayesian standpoint, the set of prior beliefs about a certain domain or environment on which inference depends.
Inductive biases can be found and/or designed at different levels of abstraction, involving both the algorithmic architecture realizing certain computations and the type of learning involved. For the architecture, we have:
- inductive biases at the micro level (micro-architecture, small-scale structures that have computational consequences, cf. parameter sharing):
- sparsity
- normalization
- attention
- inductive biases at the macro level (macro-architecture)
- inductive biases at the cognitive level