ABSTRACT:
The world around us is awash with structure and pattern. We observe it in the cycles of the seasons, the destructive beauty of coherent large-scale atmospheric events, and the petri-dish bound patterns produced by chemical reactions. In response, we incorporate these patterns into predictive models that allow us to forecast natural phenomena, giving us the ability to say that a plane will fly and a certain medical compound will help, instead of harm. This practice is perhaps intrinsic to the conscious mind, but in the information age these models have become an object of mathematical curiosity, tamed by the axiomatic frameworks of probability and automata theory.
This dissertation focuses on processes generated by a class of models known as finite-state hidden Markov models. Despite the relative simplicity of their generators, the information theoretic properties of these processes are still elusive, eluding our attempts to quantify their structure, randomness, and complexity. This dissertation introduces methods to accurately calculate the Shan- non entropy rate (randomness) and statistical complexity dimension (structure) by constructively determining their minimal (though, infinite) set of predictive features.
To do so, we must reframe the stochastic process in the language of random dynamical systems, introducing the set of predictive features as the attractor of a chaotic system. This task accomplished, we turn to the development of a toolset to measure randomness and structure. In the process, we introduce the ambiguity rate, a new intrinsic complexity measure. This quantity is used to measure the rate at which the state space of an infinite model must grow to retain optimal prediction. This quantity is closely related to the fractal dimension of the predictive state set, and we offer a conjectured correction to the Kaplan-Yorke information dimension formula for this class of processes.
To highlight the usefulness of these informational quantities, that otherwise appear rather abstracted from natural systems, we apply these theoretical results to two, rather different, physical domains. The first is to analyze the origin of randomness and structural complexity engendered by quantum measurement. The second is to solve a longstanding problem on exactly determining the thermodynamic functioning of Maxwellian demons, aka information engines. Taken together, we believe the new approach will find even wider use than in these application areas.