Intersection Information based on Common Randomness

Virgil Griffith
Computation and Neural Systems, Caltech, Pasadena, CA 91125

Edwin K. P. Chong
Dept. of Electrical & Computer Engineering
Colorado State University, Fort Collins, CO 80523

Ryan G. James
Computer Science Department, University of Colorado, Boulder, CO 80309

Christopher J. Ellison
Center for Complexity and Collective Computation
University of Wisconsin-Madison, Madison WI 53706

and

James P. Crutchfield
Complexity Sciences Center
Physics Department
University of California at Davis
Davis, CA 95616

ABSTRACT: The introduction of the partial information decomposition generated a flurry of proposals for defining an intersection information that quantifies how much of &dlquo;the same information^drquo; two or more random variables specify about a target random variable. As yet, none is wholly satisfactory. A palatable measure of intersection information would provide a principled way to quantify slippery concepts such as synergy. Here, we introduce an intersection information measure based on the Gacs-Korner common random variable that is the first to satisfy the coveted Target Monotonicity property. Our measure is imperfect, too, but we suggest directions for improvement.


V. Griffith, E. K. P. Chong, R. G. James, C. J. Ellison, and J. P. Crutchfield, “Intersection Information based on Common Randomness”, Entropy 16 (2014) 1985-2000.
[pdf]
Santa Fe Institute Working Paper 13-10-031. arXiv:1310.1538 [cs.IT].