Physics of Information

Physics 2^{8}A

Syllabus (Winter)

Instructor: Prof. Jim Crutchfield (chaos@ucdavis.edu; http://csc.ucdavis.edu/~chaos)

WWW: http://csc.ucdavis.edu/~chaos/courses/poci/

1 First Lecture: Overview

2 Self-Organization

2.1 Lecture 2: The Big Picture

2.2 Lecture 3: Example Dynamical Systems

2.3 Lecture 4: The Big, Big Picture I

2.4 Lecture 5: The Big, Big Picture II

2.5 Lecture 6: Mechanism of Chaos: Stable Instability

2.6 Lecture 7: Example Chaotic Maps (that you can analyze)

2.7 Lecture 8: Pattern Formation I

2.8 Lecture 9: Pattern Formation II

3 From Determinism to Stochasticity

3.1 Lecture 10: Probability Theory of Dynamical Systems

3.2 Lecture 11: Stochastic Processes

3.3 Lecture 12: Measurement Theory I

3.4 Lecture 13: Measurement Theory II

4 Information Processing

4.1 Lecture 14: Entropies

4.2 Lecture 15: Information in Processes I

4.3 Lecture 16: Information in Processes II

4.4 Lecture 17: Memory in Processes I

4.5 Lecture 18: Memory in Processes II

4.6 Lecture 19: Rate Distortion Theory I

4.7 Lecture 20: Rate Distortion Theory II

2 Self-Organization

2.1 Lecture 2: The Big Picture

2.2 Lecture 3: Example Dynamical Systems

2.3 Lecture 4: The Big, Big Picture I

2.4 Lecture 5: The Big, Big Picture II

2.5 Lecture 6: Mechanism of Chaos: Stable Instability

2.6 Lecture 7: Example Chaotic Maps (that you can analyze)

2.7 Lecture 8: Pattern Formation I

2.8 Lecture 9: Pattern Formation II

3 From Determinism to Stochasticity

3.1 Lecture 10: Probability Theory of Dynamical Systems

3.2 Lecture 11: Stochastic Processes

3.3 Lecture 12: Measurement Theory I

3.4 Lecture 13: Measurement Theory II

4 Information Processing

4.1 Lecture 14: Entropies

4.2 Lecture 15: Information in Processes I

4.3 Lecture 16: Information in Processes II

4.4 Lecture 17: Memory in Processes I

4.5 Lecture 18: Memory in Processes II

4.6 Lecture 19: Rate Distortion Theory I

4.7 Lecture 20: Rate Distortion Theory II

Readings (available via course website):

- Chaos, JP Crutchfield, JD Farmer, NH Packard, RS Shaw, Scientific American 255 (1986) 46–57.
- Odds, Stanislaw Lem, New Yorker 54 (1978) 38–54.

Topics:

- Introduction and motivations
- Physics of Information 256A: Dynamics, Self-Organization, Measurement Theory, Information Theory
- Physics of Computation 256B
- Survey interests, background, and abilities
- Course logistics
- Exams
- CMPy Labs

Reading: Nonlinear Dynamics and Chaos, Strogatz (NDAC), and Course Lecture Notes

Theme: Forms of Randomness, Order, and Intrinsic Instability

- Nonlinear Dynamics:
- Qualitative dynamics
- ODEs and maps
- Bifurcations
- Stability, instability, and chaos
- Quantifying (in)stability

- Pattern-forming systems:
- Instability and stabilization of patterns
- Cellular automata, map lattices, spin systems

Reading: NDAC, Chapters 1 and 2.

Topics:

- Pendulum demo
- Discuss Chaos and Odds readings and homework
- Qualitative dynamics: A geometric view of behavior
- State space
- Flows
- Attractors
- Basins
- Submanifolds
- Concrete, but simple example: One-dimensional flows

Homework: Assign Week 0’s homework today. Everyday unpredictability; see handout or website. Due in one week, but be prepared to discuss at next meeting.

Reading: NDAC, Sections 6.0-6.7, 7.0-7.3, and 9.0-9.4.

Topics:

- Continuous-time ODEs
- 2D flows: Fixed points (Sec. 6.0-6.4)
- 2D flows: Limit cycles (Sec. 7.0-7.3)
- 3D flows: Chaos in Lorenz (Sec. 9.0-9.4)
- Simulation demo

- From continuous to discrete time (Sec. 9.4)
- Poincaré maps and sections
- Lorenz ODE to cusp map
- Rössler ODE to logistic map (pp. 376–379)
- Discrete-time maps

Reading: NDAC, Chapters 3 and 8 and Sec. 10.0-10.4.

Topics:

- Qualitative dynamics: Space of all dynamical systems
- Example: Bifurcations of one-dimensional flows
- Saddle node
- Transcritical
- Pitchfork

- Catastrophe theory
- Catastrophes: Fixed point to fixed point bifurcation
- Example: Cusp Catastrophe
- Catastrophe theory classification of fixed point bifurcations

Homework: Collect Week 0’s, assign Week 1’s today.

Reading: NDAC, Chapters 3 and 8 and Sec. 10.0-10.4.

Topics:

- Bifurcations in discrete-time maps: Logistic map
- Fixed point to limit cycle
- Phenomenon and calculation
- Limit cycle to limit cycle
- Phenomenon and calculation
- Routes to chaos: Period-doubling cascade
- Phenomenon and calculation
- Band-merging
- Periodic windows and intermittency
- Simulation demo

Reading: NDAC, Sec. 12.0-12.3, 9.3, and 10.5.

Topics:

- Chaotic mechanisms: Stretch and fold
- Baker’s map
- Cat map (and stretch demo)
- Henon map: stretch-fold and self-similarity
- Roessler attractor branched manifold
- Dot spreading: Roessler and Lorenz ODEs
- Lyapunov characteristic exponents (LCEs)
- Time to unpredictability
- Dissipation rate
- Attractor LCE classification
- Chaos defined

Homework: Collect Week 1’s, assign Week 2’s today.

Reading: NDAC, Chapter 10.

Topics:

- Shift map
- LCEs for maps
- Tent map
- Logistic map
- LCE view of period-doubling route to chaos
- Period-doubling self-similarity
- Renormalization group analysis of scaling

Reading: Lecture Notes.

Topics:

- Review last lecture.
- Spatially Extended Dynamical Systems
- Synchronous Cellular Automata
- Lattice Maps: Logistic Lattice and Dripping Handrail

Homework: Collect Week 2’s, assign Week 3’s today.

Reading: Lecture Notes.

Topics:

- Review last lecture.
- Asynchronous Cellular Automata
- Spin Systems

Reading: Lecture Notes.

Theme: Stochasticity and Measurement

- Probability Theory of Dynamical Systems
- Stochastic Processes
- Measurement Theory

Reading: Lecture Notes.

Topics:

- Probability theory review
- Dynamical evolution of distributions
- Invariant measures
- Examples

Homework: Collect Week 3’s, assign Week 4’s today.

Reading: Lecture Notes.

Topics:

- Review last lecture.
- Processes
- Markov chains
- Statistical equilibrium
- Hidden Markov models
- Examples: Fair coin, periodic, golden mean, even, and others

Reading: Lecture Notes.

Topics:

- Review last lecture.
- State-space partitioning
- Orbit and sequence spaces
- Good instruments and informative measurements

Homework: Collect Week 4’s, assign Week 5’s today.

Reading: Lecture Notes.

Topics:

- Review last lecture.
- Markov partitions in 1D
- Generating partitions in 1D
- Example: 1D maps
- Generating partitions in 2D
- Example: 2D Cat map

Reading: Elements of Information Theory, Cover and Thomas (EIT), and Computational Mechanics
Reader, JPC (CMR)

Theme: Information, Uncertainty, and Memory

- Entropies
- Communication Channel (and coding theorems)
- Mutual Information and Information metric
- Excess Entropy
- Transient Information
- Connection to Dynamics: Entropy rate and LCEs

Reading: EIT, Chapters 1 and 2.

Topics:

- Motivation: Information ≠ Energy
- Information as uncertainty and surprise
- Information sources: Ignorance of forces or initial conditions, deterministic chaos, and ...?
- Axioms for a measure of information
- Entropy function
- Convexity
- Joint and Conditional Entropy
- Mutual information
- Examples

Homework: Collect Week 5’s, assign Week 6’s today.

Reading: EIT, Sec. 5-5.4 and 8-8.5 and Chapter 4.

Topics:

- Communication channels
- Coding theorems
- Examples

Reading: EIT, Sec. 5-5.4 and 8-8.5 and Chapter 4.

Topics:

- Entropy rates for Markov chains
- Entropies for times series
- Connection to Dynamics: Entropy rate and LCEs

Homework: Collect Week 6’s, assign Week 7’s today.

Reading: CMR article RURO.

Topics:

- Entropy convergence
- Excess entropy
- Examples

Reading: CMR article RURO.

Topics:

- Generalized synchronization
- Transient information
- Examples

Homework: Collect Week 7’s, assign Week 8’s today.

Reading: EIT, Chapter 10.

Topics:

- Rate distortion theory

Reading: EIT, Chapter 10.

Topics:

- Rate distortion theory

Homework: Collect Week 8’s.