Jaynes's maximum entropy principle

We should be uncertain (adopt a high entropy belief) when we have no information.

Minimizing with respect to Q requires consistency with the generative model (energy) while also maintaining a high posterior entropy.

---

$$F\left[Q,y\right]=\underset{Energy}{\underbrace{-{\mathbb{E}}_{Q(x)}\left[\ln P\left(y,x\right)\right]}}-\underset{Entropy}{\underbrace{H[Q(x)]}}$$

---

Evidence

Supporting

Handle	Title	Summary of relevant evidence	Link
Parr, Pezzulo, Friston 2022	Active Inference Textbook configuration.	p.28: “minimizing with respect to Q requires consistency with the generative model (energy) while also maintaining a high posterior entropy.5 The latter means that, in the absence of data or precise prior beliefs (which only influence the energy term), we should adopt maximally uncertain beliefs about the hidden states of the world, in accord with Jaynes’s maximum entropy principle ( Jaynes 1957). Put simply, we should be uncertain (adopt a high entropy belief) when we have no information. The term energy inherits from statistical physics. Specifically, under a Boltzmann distribution, the average log probability of a system adopting some configuration is inversely proportional to the energy associated with that configuration—that is, the energy required to move the system into this configuration from a baseline

Counter

Handle	Title	Summary of relevant evidence	Link

---

Connections

---

Metadata

---

Sentence

Regular