PHY 28A
Natural Computation and Self-Organization:
The Physics of Information Processing in Complex Systems

Jim Crutchfield
chaos@csc.ucdavis.edu; http://csc.ucdavis.edu/~chaos

Winter
WWW: http://csc.ucdavis.edu/~chaos/courses/poci/

Lecture 14: Entropies

Reading: EIT, Chapters 1 and 2.
Topics:

  1. Review last lecture.
  2. Motivation: Information ≠ Energy
  3. Probability theory review
  4. Information as uncertainty and surprise
  5. Information sources: Ignorance of forces or initial conditions, deterministic chaos, and ...?
  6. Axioms for a measure of information
  7. Entropy function
  8. Convexity
  9. Joint and Conditional Entropy
  10. Mutual information
  11. Examples

Reading for next meeting: EIT, Sec. 5-5.4 and 7-7.7 and Chapter 4.