Information in Infinitary and Spatial Processes

Computational Mechanics Group Workshop

Complexity Sciences Center, Physics, UC Davis

Davis, CA

December 3-4, 2013

Jim Crutchfield and Dave Feldman, Co-Organizers

DESCRIPTION

The meeting will explore several related open areas of information theory (IT) and computational mechanics (CM).

The first is infinitary processes. How must the constructions and definitions of IT and CM be modified in settings in which the excess entropy E or the statistical complexity C_mu are infinite?  Might the nature of these divergences be better understood by drawing upon ideas from the theory of critical phenomena, such as finite-size scaling and critical exponents?  Aperiodic sequences such as the Thue-Morse sequence, might be an analytically tractable class of systems with which to begin this analysis. Infinite measures also occur in infinite-state and continuous-state processes? Can we adapt IT and CM to analyze these processes and their generators?

The second area concerns spatial processes in more than one dimension. How can one extend E, C_mu, and other measures to higher-dimensional processes? We know that different definitions of E, identical in one dimension, are not the same in two dimensions. It thus seems likely that there will be multiple measures of memory, structure, and synchronization for spatial systems.

GOAL

Our is to develop plans for 3-5 research projects.  These plans should include a statement of the main problem the research will address and specific questions that can be answered that relate to the main problem.  Booth, Colomb, and Williams (The

Craft of Research, 3rd ed, Chicago University Press, 2008) draw a distinction between research questions and problems. The distinction is useful when teaching classes that involve research projects, for example, and Booth et al's taxonomy may be useful to us in the workshop.

In this context, a “problem” is a significant and fairly open-ended issue. For example, “How can information/memory/structure be measured in two-dimensional systems?”, “How are the information processing capacities of a system related to its thermodynamic properties?” or “How might we measure and compare memory and structure in systems with finite memory?”  Part of specifying a research problem often entails asking “The ‘so-what’ question”---why should this research matter to

people outside our research group? 

A “question” is, well, a question---particular enough that it can be answered in the course of our research. For example, “What is the behavior of the excess entropy at the critical point of the 2D Ising model?”, or “Does the rate of divergence of the excess entropy tell us about the computational class of the model that generated the process?” In many cases, questions for us will take the form things to calculate.

The final product will be several 4-6 page project write-ups, including nominal citations. Ideally, these write-ups will propose specific calculations or explorations to carry out, and specific questions to answer. These projects could form the basis of an MS or PhD thesis or could be tackled by teams of workshop participants.

FORMAT

Overall, the meeting consists of short presentations of current knowledge in one of the frontier areas and proposed directions for future work. These will be followed by a brainstorming session and a time for synthesis. In the meeting's last session, participants will group ideas and questions into 3-5 projects.  

More specifically, the first day consists of six 45-minute sessions, structured as follows.

1.  Brief presentation summarizing relevant knowledge and suggesting directions for future work. Fifteen minutes: maximum of 8 slides.

2.  Brainstorming session: The goal is to generate as many ideas as possible in a fairly short amount of time. Fifteen minutes.

3.  Discussion and synthesis: Participants will discuss, evaluate, and synthesize ideas generated during the presentation and

brainstorming session. Fifteen minutes.

In addition to the presenter, assigned to each session will be a moderator and one or two scribes. The moderator's main role is to keep a close eye on the time and make sure we move through all three components in a timely fashion. In addition, the moderator will direct the conversation, making sure that all who wish are able to contribute. The scribe will record all ideas generated during the brainstorming session and the subsequent discussion.

We begin the second day by asking all workshop participants to offer thoughts on possible research project topics. The structure for the rest of the second day will depend on to what extent consensus emerges about project areas. We may continue to discuss projects as a full group; we might also flesh out project goals and descriptions in smaller teams.

TOPICS, PRESENTERS, SCRIBES

Jim Crutchfield will moderate the meeting’s morning presentation sessions and Dave Feldman, the afternoon sessions.

I.  2D Information Theory

Presenter: Dave Feldman

Scribes: Pooneh Mohammadiara, Greg Wimsatt

II.  Countably Infinite Processes

Presenter: Chris Ellison

Scribe: Nix Barnett

III.  Continuous-time Processes

Presenter: Sarah Marzen

Scribe: John Mahoney

IV.  Inferring Infinite-state Processes

Presenter: Chris Strelioff

Scribe: Dowman Varn

V.  Information Engines

Presenter: Alec Boyd

Scribe: Korana Burke

VI.  Spectral Decomposition

Presenter: Paul Riechers

Scribe: Ryan James