The influence of degree of organization on the ability of Ss to recall lists of syllables immediately after learning was used as a measure in applying the concept of information to the problem of learning. More syllables were correctly recalled from a passage with a lower average rate of information than from a passage with a higher average information rate. The amount of information learned by the Ss was constant when the degree of organization was between (...) 2 and 1.5 bits per syllable. (shrink)

Quantum Information Theory and the Foundations of Quantum Mechanics is a conceptual analysis of one of the most prominent and exciting new areas of physics, providing the first full-length philosophical treatment of quantum information theory and the questions it raises for our understanding of the quantum world. -/- Beginning from a careful, revisionary, analysis of the concepts of information in the everyday and classical information-theory settings, Christopher G. Timpson argues for an ontologically deflationary (...) account of the nature of quantum information. Against what many have supposed, quantum information can be clearly defined (it is not a primitive or vague notion) but it is not part of the material contents of the world. Timpson's account sheds light on the nature of nonlocality and information flow in the presence of entanglement and, in particular, dissolves puzzles surrounding the remarkable process of quantum teleportation. In addition it permits a clear view of what the ontological and methodological lessons provided by quantum information theory are; lessons which bear on the gripping question of what role a concept like information has to play in fundamental physics. Topics discussed include the slogan 'Information is Physical', the prospects for an informational immaterialism (the view that information rather than matter might fundamentally constitute the world), and the status of the Church-Turing hypothesis in light of quantum computation. -/- With a clear grasp of the concept of information in hand, Timpson turns his attention to the pressing question of whether advances in quantum information theory pave the way for the resolution of the traditional conceptual problems of quantum mechanics: the deep problems which loom over measurement, nonlocality and the general nature of quantum ontology. He marks out a number of common pitfalls to be avoided before analysing in detail some concrete proposals, including the radical quantum Bayesian programme of Caves, Fuchs, and Schack. One central moral which is drawn is that, for all the interest that the quantum information-inspired approaches hold, no cheap resolutions to the traditional problems of quantum mechanics are to be had. (shrink)

This book constitutes the refereed proceedings of the International Conference on Spatial Information Theory, COSIT 2005, held in Elliottville, NY, USA in September 2005. The 30 revised full papers presented were carefully reviewed and selected from 82 submissions. The papers are organized in topical sections on vagueness, uncertainty, and gradation; paths and routes; ontologies and semantics; ontologies and spatial relations; spatial reasoning: cognitive maps and spatial reasoning; time, change, and dynamics; landmarks and navigation; geographic information, and spatial (...) behaviour. (shrink)

This paper argues that Information Theoretic Redundancy (ITR) is fundamentally a composite concept that has been continually misinterpreted since the very inception of Information Theory. We view ITR as compounded of true redundancy and partial redundancy. This demarcation of true redundancy illustrates a limiting case phenomenon: the underlying metric (number of alternatives) differs only by degree but the properties of this concept differ in kind from those of partial redundancy. Several other studies are instanced which also imply (...) the composite nature of ITR. We thus provide broadly based but particular support for earlier generalized suggestions that it is the underlying calculus of Information Theory rather than the ill-named concepts themselves that provides something of a unitary language for the description of phenomena. (shrink)

Integrated Information Theory (IIT) identifies consciousness with having a maximum amount of integrated information. But a thing’s having the maximum amount of anything cannot be intrinsic to it, for that depends on how that thing compares to certain other things. IIT’s consciousness, then, is not intrinsic. A mereological argument elaborates this consequence: IIT implies that one physical system can be conscious while a physical duplicate of it is not conscious. Thus, by a common and reasonable conception of (...) intrinsicality, IIT’s consciousness is not intrinsic. It is then argued that to avoid the implication that consciousness is not intrinsic, IIT must abandon its Exclusion Postulate, which prohibits overlapping conscious systems. Indeed, theories of consciousness that attribute consciousness to physical systems, should embrace the view that some conscious systems overlap. A discussion of the admittedly counterintuitive nature of this solution, along with some medical and neuroscientific realities that would seem to support it, is included. (shrink)

Information theory and statistical mechanics II.

– The paper explores first the postulational basis and significance of‘measures of information’in current information theory and their possible relations to physical entropy and Brillouin's‘negentropy’regarded as the negative of entropy. For some purposes, the same pattern or formal structure may be abstracted from both‘entropy’and‘information’. The paper analyzes, in the second place, the mathematical analogies which have been traced between information theory and quantum mechanics and argues that the analogies have but a limited value when (...) we come to grips with the deeper, and yet unsolved, problems of quantum theory. Lastly it is urged that the indiscriminate extensions of statistical and information‐theoretic methods, and of quantum‐mechanical analogies, e.g.‘linguistic duality’, to psycholinguistics, literary and aesthetic Gestalten, can result only in crude exaggerations. Dialectical methodology requires the continual reformulation of all scientific descriptions which have to reckon perpetually with the richness of human materials. (shrink)

A classic source for understanding the connections between information theory and physics, this text was written by one of the giants of 20th-century physics and is appropriate for upper-level undergraduates and graduate students. Topics include the principles of coding, coding problems and solutions, the analysis of signals, a summary of thermodynamics, thermal agitation and Brownian motion, and thermal noise in an electric circuit. A discussion of the negentropy principle of information introduces the author's renowned examination of Maxwell's (...) demon. Concluding chapters explore the associations between information theory, the uncertainty principle, and physical limits of observation, in addition to problems related to computing, organizing information, and inevitable errors. 1962 ed. 81 figures. 14 tables. (shrink)

Integrated Information Theory (IIT) stands out as one of the most promising approaches to scientifically understand the emergence of consciousness. Even if it borrows from the phenomenology of consciousness to derive its axiomatic formulation, IIT does not initially adhere to any particular ontological position. However, its founder leans towards Panpsychism. More recently, Owen has studied the pros and cons of different ontologies as a metaphysical basis for IIT, defending a hylomorphic stance where en-grounding, en-forming relations gain the upper (...) hand in the causal narrative. This paper aims at (1) complementing Owen’s work by invoking additional arguments against physicalist IIT, both reductive and non-reductive; and (2) improving Owen’s characterization of non-physicalist causality through a “nested” version of hylomorphism that avoids the combination problem of Panpsychism. (shrink)

Informational theories of semantic content have been recently gaining prominence in the debate on the notion of mental representation. In this paper we examine new-wave informational theories which have a special focus on cognitive science. In particular, we argue that these theories face four important difficulties: they do not fully solve the problem of error, fall prey to the wrong distality attribution problem, have serious difficulties accounting for ambiguous and redundant representations and fail to deliver a metasemantic theory of (...) representation. Furthermore, we argue that these difficulties derive from their exclusive reliance on the notion of information, so we suggest that pure informational accounts should be complemented with functional approaches. (shrink)

This paper presents Integrated Information Theory (IIT) 4.0. IIT aims to account for the properties of experience in physical (operational) terms. It identifies the essential properties of experience (axioms), infers the necessary and sufficient properties that its substrate must satisfy (postulates), and expresses them in mathematical terms. In principle, the postulates can be applied to any system of units in a state to determine whether it is conscious, to what degree, and in what way. IIT offers a parsimonious (...) explanation of empirical evidence, makes testable predictions, and permits inferences and extrapolations. IIT 4.0 incorporates several developments of the past ten years, including a more accurate translation of axioms into postulates and mathematical expressions, the introduction of a unique measure of intrinsic information that is consistent with the postulates, and an explicit assessment of causal relations. By fully unfolding a system's irreducible cause-effect power, the distinctions and relations specified by a substrate can account for the quality of experience. (shrink)

First comprehensive introduction to information theory explores the work of Shannon, McMillan, Feinstein, and Khinchin. Topics include the entropy concept in probability theory, fundamental theorems, and other subjects. 1957 edition.

Simple hypotheses are intrinsically attractive, and, for this reason, need to be formulated with utmost precision if they are to be testable. Unfortunately, it is hard to see how Phillips & Singer's hypothesis might be unambiguously refuted. Despite this, the authors have provided much evidence consistent with the hypothesis, and have proposed a natural and powerful extension for information theoretic approaches to learning.

Function refers to a broad family of concepts of varying abstractness and range of application, from a many-one mathematical relation of great generality to, for example, highly specialized roles of designed elements in complex machines such as degaussing in a television set, or contributory processes to control mechanisms in complex metabolic pathways, such as the inhibitory function of the appropriate part of the lac-operon on the production of lactase through its action on the genome in the absence of lactose. We (...) would like a language broad enough, neutral enough, but yet powerful enough to cover all such cases, and at the same time to give a framework form explanation both of the family resemblances and differences. General logic and mathematics are too abstract, but more importantly, too broad, whereas other discourses of function, such as the biological and teleological contexts, are too narrow. Information is especially suited since it is mathematically grounded, but also has a wellknown physical interpretation through the Schr dinger/Brillouin Negentropy Principle of Information, and an engineering or design interpretation through Shannon's communication theory. My main focus will be on the functions of autonomous anticipatory systems, but I will try to demonstrate both the connections between this notion of function and the others, especially to dynamical systems with a physical interpretation on the one side and intentional systems on the other. The former are based in concepts like force, energy and work, while the latter involve notions like representation, control and purpose, traditionally, at least in Modern times, on opposite sides of the Cartesian divide. In principle, information can be reduced to energy, but it has the advantage of being more flexible, and easier to apply to higher level phenomena. (shrink)

Information theory and statistical mechanics.

The Integrated Information Theory is a leading scientific theory of consciousness, which implies a kind of panpsychism. In this paper, I consider whether IIT is compatible with a particular kind of panpsychism, known as Russellian panpsychism, which purports to avoid the main problems of both physicalism and dualism. I will first show that if IIT were compatible with Russellian panpsychism, it would contribute to solving Russellian panpsychism’s combination problem, which threatens to show that the view does not (...) avoid the main problems of physicalism and dualism after all. I then show that the theories are not compatible as they currently stand, in view of what I call the coarse-graining problem. After I explain the coarse-graining problem, I will offer two possible solutions, each involving a small modification of IIT. Given either of these modifications, IIT and Russellian panpsychism may be fully compatible after all, and jointly enable significant progress on the mind–body problem. (shrink)

This paper is a criticism of full-information theories of welfare. Such theories unsuccessfully attempt to accommodate an internalist intuition (that one's good depends in some way on one's desires or hypothetical desire) with a rationalist intuition (that only fully-informed desires are relevant to one's good).

Turing machine An idealized computing device attached to a tape, each square of which is capable of holding a symbol. We write a program (a nite binary string) on the tape, and start the machine. If the machine halts with string o written at a designated place on the tape.

To become more useful and efficient in sustaining the Earth's health, economics must undergo a paradigm shift in its thinking. From a humanistic perspective, humans should be the center of everything. However, from the standpoint of physics and the universe, this is not the case. As a species, having a planet among the millions in the universe where humans can survive and thrive is already a great fortune. Through this book, we also try to answer one of our long-standing questions: (...) why has economics learned so little from physics, the oldest and most rigorous academic discipline that studies nature? -/- Learning from the quantum and information theories, we also provide a new definition of value, which is expected to help economists relax assumptions and loosen established economic principles in order to change and evolve. The new definition is also expected to enable social scientists to address newly arising phenomena or events that are not accounted for by existing value systems, such as environmental crises, artificial intelligence (AI), and interdisciplinary information, more proactively and productively. (shrink)

Integrated information theory (IIT) starts from the essential properties of experience (axioms) and translates them into requirements that any physical system must satisfy to be conscious (postulates; see Integrated Information Theory of Consciousness: An Outline, this volume). The postulates of IIT can be seen as a list of requirements for something to exist ‘for itself’, as an intrinsic entity, and thus have relevance for ontology and metaphysics. Some implications of the theory include the divide between (...) intrinsic and extrinsic entities, the incoherence of ontological reductionism, and the intrinsic nature of meaning. (shrink)

Recently it has been shown that the classical “stick and ball” viewpoint of molecules is inconsistent with quantum theory (QT). We suggest an unusual reconciliation: The QT state is not a physical property, but instead reflects our state of knowledge about observable aspects of “reality.” We show how this perspective is nevertheless objective. Applied to molecules, the view permits “structure” to exist only when observable evidence is compatible with this feature. Typically one must replace the a priori model (in (...) particular, the dynamical generator) with one consistent with the evidence. We show that such “structure” is stable in the context of first-order perturbation theory. We also indicate how dynamics can be inferred from scattering data—a process alternative to postulating (field-theoretic) models for “environment.”. (shrink)

Integrated Information Theory (IIT) stands out as one of the most promising theories for dealing with the hard problem of consciousness. Founded on five axioms derived from phenomenology, IIT seeks for the physical substrate of consciousness that complies with such axioms according to the criterion of maximally integrated information (Φ). Eventually, IIT identifies phenomenal consciousness with maximal Φ or, what is the same thing, with the strongest cause-effect power in the system. Among the scholars critical of this (...) theory, some point to the so-called Intrinsicality Problem (IP), namely that consciousness cannot be an intrinsic property of the system because maximal Φ crucially depends on the possible existence of bigger values of Φ if the initial system is appropriately linked to or embedded in larger systems. Although proposals in the recent literature aim to solve the IP by going beyond reductionism and physicalism, none of them tackle the real issue, i.e., the insufficiency of IIT’s causal-metaphysical structure. This papers endeavors to provide a solution to the IP in IIT within a hylomorphist ontology that includes formal causation. Complementing IIT with formal causation provides the theory with a criterion of individuation that solves the IP and, by relaxing identification between maximal Φ and consciousness, it lends a more robust metaphysical structure. To wit, maximal Φ is a necessary but not sufficient condition for the existence of consciousness. (shrink)

The objective of the present edition of this monograph is the same as that of earlier editions, namely, to provide readers with some mathemati cal maturity a rigorous and modern introduction to the ideas and principal theorems of probabilistic information theory. It is not necessary that readers have any prior knowledge whatever of information theory. The rapid development of the subject has had the consequence that any one book can now cover only a fraction of the (...) literature. The latter is often written by engineers for engineers, and the mathematical reader may have some difficulty with it. The mathematician who understands the content and methods of this monograph should be able to read the literature and start on research of his own in a subject of mathematical beauty and interest. The present edition differs from the second in the following: Chapter 6 has been completely replaced by one on arbitrarily varying channels. Chapter 7 has been greatly enlarged. Chapter 8 on semi-continuous channels has been drastically shortened, and Chapter 11 on sequential decoding completely removed. The new Chapters 11-15 consist entirely of material which has been developed only in the last few years. The topics discussed are rate distortion, source coding, multiple access channels, and degraded broadcast channels. Even the specialist will find a new approach in the treatment of these subjects. Many of the proofs are new, more perspicuous, and considerably shorter than the original ones. (shrink)

Tononi has proposed a fundamental theory of consciousness he terms Integrated Information Theory (IIT). IIT purports to explain the quantity of conscious experience by linking it with integrated information: information shared by the system as a whole and quantified by adopting a modified version of Shannon's definition of information. Since the fundamental aspect of IIT is information the theory allows for the multiple realizability of consciousness. While there are several concepts within IIT (...) that need further theoretical development, the main failings of the theory are an absence of a link between conscious experience and awareness and the use of Shannon's limited data based definition of information. These limitations prevent the theory from satisfying Chalmers' principles of structural coherence and organizational invariance which any functionalist theory should obey. It is not clear whether IIT can be repaired without simply reducing the theory to a general statement of computational functionalism. (shrink)

We present a critical discussion of the claim (most forcefully propounded by Chaitin) that algorithmic information theory sheds new light on Godel's first incompleteness theorem.

The causal and simulation theories are often presented as very distinct views about declarative memory, their major difference lying on the causal condition. The causal theory states that remembering involves an accurate representation causally connected to an earlier experience. In the simulation theory, remembering involves an accurate representation generated by a reliable memory process. I investigate how to construe detailed versions of these theories that correctly classify memory errors as misremembering or confabulation. Neither causalists nor simulationists have paid (...) attention to memory-conjunction errors, which is unfortunate because both theories have problems with these cases. The source of the difficulty is the background assumption that an act of remembering has one target. I fix these theories for those cases. The resulting versions are closely related when implemented using tools of information theory, differing only on how memory transmits information about the past. The implementation provides us with insights about the distinction between confabulatory and non-confabulatory memory, where memory-conjunction errors have a privileged position. (shrink)

Integrated Information Theory of Consciousness Integrated Information Theory offers an explanation for the nature and source of consciousness. Initially proposed by Giulio Tononi in 2004, it claims that consciousness is identical to a certain kind of information, the realization of which requires physical, not merely functional, integration, and which can be measured mathematically according … Continue reading Integrated Information Theory of Consciousness →.

There are aspects of privacy theory that are analogous to quantum theory. In particular one can define distillable key and key cost in parallel to distillable entanglement and entanglement cost. We present here classical privacy theory as a particular case of information theory with adversaries, where similar general laws hold as in entanglement theory. We place the result of Renner and Wolf—that intrinsic information is lower bound for key cost—into this general formalism. Then (...) we show that the question of whether intrinsic information is equal to key cost is equivalent to the question of whether Alice and Bob can create a distribution product with Eve using I M bits of secret key. We also propose a natural analogue of relative entropy of entanglement in privacy theory and show that it is equal to the intrinsic information. We also provide a formula analogous to the entanglement of formation for classical distributions. (shrink)

In this essay we discuss recent attempts to analyse the notion of representation, as it is employed in cognitive science, in purely informational terms. In particular, we argue that recent informational theories cannot accommodate the existence of metarepresentations. Since metarepresentations play a central role in the explanation of many cognitive abilities, this is a serious shortcoming of these proposals.

Integrated Information Theory (IIT) is one of the most influential theories of consciousness, mainly due to its claim of mathematically formalizing consciousness in a measurable way. However, the theory, as it is formulated, does not account for contextual observations that are crucial for understanding consciousness. Here we put forth three possible difficulties for its current version, which could be interpreted as a trilemma. Either consciousness is contextual or not. If contextual, either IIT needs revisions to its axioms (...) to include contextuality, or it is inconsistent. If consciousness is not contextual, then IIT faces an empirical challenge. Therefore, we argue that IIT in its current version is inadequate. (shrink)

Communication is an important feature of the living world that mainstream biology fails to adequately deal with. Applying two main disciplines can be contemplated to fill in this gap: semiotics and information theory. Semiotics is a philosophical discipline mainly concerned with meaning; applying it to life already originated in biosemiotics. Information theory is a mathematical discipline coming from engineering which has literal communication as purpose. Biosemiotics and information theory are thus concerned with distinct and (...) complementary possible meanings of the word ‘communication’. Since literal communication needs to be secured so as to enable semantics being communicated, information theory is a necessary prerequisite to biosemiotics. Moreover, heredity is a purely literal communication process of capital importance fully relevant to literal communication, hence to information theory. A short introduction to discrete information theory is proposed, which is centred on the concept of redundancy and its use in order to make sequences resilient to errors. Information theory has been an extremely active and fruitful domain of researches and the motor of the tremendous progress of communication engineering in the last decades. Its possible connections with semantics and linguistics are briefly considered. Its applications to biology are suggested especially as regards error-correcting codes which are mandatory for securing the conservation of genomes. Biology needs information theory so biologists and communication engineers should closely collaborate. (shrink)

We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and wherethey are fundamentally different. We discuss and relate the basicnotions of both theories: Shannon entropy, Kolmogorov complexity, Shannon mutual informationand Kolmogorov (``algorithmic'') mutual information. We explainhow universal coding may be viewed as a middle ground betweenthe two theories. We consider Shannon's rate distortion theory, whichquantifies useful (in a certain sense) information.We use the communication of (...) class='Hi'>information as our guiding motif, and we explain howit relates to sequential question-answer sessions. (shrink)

Searching for information is critical in many situations. In medicine, for instance, careful choice of a diagnostic test can help narrow down the range of plausible diseases that the patient might have. In a probabilistic framework, test selection is often modeled by assuming that people's goal is to reduce uncertainty about possible states of the world. In cognitive science, psychology, and medical decision making, Shannon entropy is the most prominent and most widely used model to formalize probabilistic uncertainty and (...) the reduction thereof. However, a variety of alternative entropy metrics (Hartley, Quadratic, Tsallis, Rényi, and more) are popular in the social and the natural sciences, computer science, and philosophy of science. Particular entropy measures have been predominant in particular research areas, and it is often an open issue whether these divergences emerge from different theoretical and practical goals or are merely due to historical accident. Cutting across disciplinary boundaries, we show that several entropy and entropy reduction measures arise as special cases in a unified formalism, the Sharma–Mittal framework. Using mathematical results, computer simulations, and analyses of published behavioral data, we discuss four key questions: How do various entropy models relate to each other? What insights can be obtained by considering diverse entropy models within a unified framework? What is the psychological plausibility of different entropy models? What new questions and insights for research on human information acquisition follow? Our work provides several new pathways for theoretical and empirical research, reconciling apparently conflicting approaches and empirical findings within a comprehensive and unified information‐theoretic formalism. (shrink)

Backtracking counterfactuals are problem cases for the standard, similarity based, theories of counterfactuals e.g., Lewis. These theories usually need to employ extra-assumptions to deal with those cases. Hiddleston, 632–657, 2005) proposes a causal theory of counterfactuals that, supposedly, deals well with backtracking. The main advantage of the causal theory is that it provides a unified account for backtracking and non-backtracking counterfactuals. In this paper, I present a backtracking counterfactual that is a problem case for Hiddleston’s account. Then I (...) propose an informational theory of counterfactuals, which deals well with this problem case while maintaining the main advantage of Hiddleston’s account. In addition, the informational theory offers a general theory of backtracking that provides clues for the semantics and epistemology of counterfactuals. I propose that backtracking is reasonable when the state of affairs expressed in the antecedent of a counterfactual transmits less information about an event in the past than the actual state of affairs. (shrink)

In this paper I argue that the Integrated Information Theory of Consciousness has an underlying emergentist metaphysics, specifically of a kind that has received minimal attention and we may call functionalist emergentism. I will try to show that in this scientific theory conscious experience is a functional-role property possessed by the whole system, not by their parts, which is dependent on, but also (purportedly) causally powerful over and above, the properties of the parts. However, I will argue (...) that depicting conscious experience as a functional-role emergent property threatens the whole coherence of the theory because, by definition, functional-realizers do all the causal work associated with the instantiation of any functional-role property on any occasion. Hence, to preserve the causal power of consciousness, which, as we will see, is one of the fundamental building blocks of the theory, linked to the assertion of the very existence of consciousness, phenomenal properties should be re-thought as realizers of the relevant informationally integrated causal-roles. (shrink)