Professor of System Science
Portland State University
My research interests are in three areas:
Ordering Genetic Algorithm Genomes With Reconstructability Analysis: Discrete Models
An Overview of Reconstructability Analysis
Reconstructability Analysis with Fourier Transforms
Multi-Level Decomposition of Probabilistic Relations
A Software Architecture for Reconstructability Analysis
Reversible Modified Reconstructability Analysis of Boolean Circuits and Its Quantum Computation
Modified Reconstructability Analysis for Many-Valued Logic Functions
State-Based Reconstructability Analysis
Reconstructability Analysis Detection of Optimal Gene Order in Genetic Algorithms
Directed Extended Dependency Analysis for Data Mining
Ordering Genetic Algorithm Genomes With Reconstructability Analysis
Using Reconstructability Analysis to Select Input Variables for Artificial Neural Networks
Enhancements to Crisp Possibilistic Reconstructability Analysis
An Information Theoretic Methodology for Prestructuring Neural Networks
Wholes and Parts in General Systems Methodology
State-Based Reconstructability Modeling for Decision Analysis
Prestructuring Neural Networks for Pattern Recognition Using Extended Dependency Analysis
Complexity Reduction in State-Based Modeling
Complexity and the Decomposability of Relations
Resolution of Local Inconsistency in Identification
Structure and Dynamics of Cellular Automata
Control Uniqueness in Reconstructability Analysis
Information-Theoretic Mask Analysis of Rainfall Time-Series Data
Set-Theoretic Reconstructability of Elementary Cellular Automata
On Matching ANN Structure to Problem Domain Structure
An Information Theoretic Framework for Exploratory Multivariate Market Segmentation Research
The evolution of altruism: Game theory in multilevel selection and inclusive fitness
What's wrong with inclusive fitness?
Unifying the Theories of Inclusive Fitness and Reciprocal Altruism
Strong Altruism Can Evolve in Randomly Formed Groups
Hamilton's Rule Applied to Reciprocal Altruism
Reconstructability Analysis Detection of Optimal Gene Order in Genetic Algorithms
Altruism, the Prisoner's Dilemma, and the Components of Selection
N-Player Prisoner's Dilemma in Multiple Groups: A Model of Multilevel Selection
Effect of Environmental Structure on Evolutionary Adaptation
Structure and Dynamics of Cellular Automata
Dependence of Adaptability on Environmental Structure in a Simple Evolutionary Model
Global Optimization Studies on the 1-D Phase Problem
Set-Theoretic Reconstructability of Elementary Cellular Automata
Variance and Uncertainty Measures of Population Diversity Dynamics
Diversity Dynamics in Static Resource Models
Dynamics of Diversity in an Evolving Population
Application of the Genetic Algorithm to A Simplified Form of the Phase Problem
Improving Crystallographic Macromolecular Images: The Real-Space Approach
Systems Metaphysics: A Bridge from Science to Religion
A Review of Systems: New Paradigms for the Human Sciences
An Informal Review of the Crisis of Global Capitalism: a letter to George Soros
Complexity Theory and Systems Theory
Towards an Ontology of Problems
Incompleteness, Negation, Hazard: On The Precariousness of Systems
Personal Knowledge and the Inner Sciences
Information, Constraint and Meaning
Some Analogies of Hierarchical Order in Biology and Linguistics
Requisite Variety and the Second Law
The notion that a system (group) does better when it achieves cooperation among its parts (individuals), often against the self-interest of those parts, goes beyond just biological systems undergoing natural selection. It is applicable to hierarchical systems across a variety of fields. The non-zero sum nature of aggregation is general and optimization by subsystems often results in sub-optimization at a higher level. The PD is often used to model such non-zero sum situations. Like Simpson's paradox, the PD involves an anomaly of composition: individually-rational strategies, when aggregated, give a deficient collective outcome.
As Sober and Wilson (1998) have demonstrated, Simpson's paradox (even if not always identified as such) is important in understanding multilevel selection. These authors show (pp. 18-26) that this paradox can be derived from simple fitness functions for altruists and non-altruists in two populations. These functions amount to an n-player PD (see Appendix A), although Sober and Wilson do not call attention to this fact. In this paper and in Fletcher and Zwick (2000), we make the connection between the PD and Simpson's paradox explicit. Our main finding is that Simpson's paradox emerges transiently, but for a wide range of PD parameter values, when a minimal group structure is imposed on an n-player PD. This result is produced in a model which involves an implicit competition between two groups and a simple n-player PD in each. The model is based on only two parameters which correlate with the within-group and between-group selection components in multilevel selection theory.
Reconstructability analysis is a method to determine whether a multivariate relation, defined set- or information-theoretically, is decomposable with or without loss (reduction in constraint) into lower ordinality relations. Set-theoretic reconstructability analysis (SRA) is used to characterize the mappings of elementary cellular automata. The degree of lossless decomposition possible for each mapping is more effective than the lambda parameter (Walker & Ashby, Langton) as a predictor of chaotic dynamics. Complete SRA yields not only the simplest lossless structure but also a vector of losses of all decomposed structures. This vector subsumes lambda, Wuensche's Z parameter, and Walker & Ashby's "fluency" and "memory" parameters within a single framework, and is a strong but still imperfect predictor of the dynamics: less decomposable mappings more commonly produce chaos. The set-theoretic constraint losses are analogous to information distances in information-theoretic reconstructability analysis (IRA). IRA captures the same information as SRA, but allows lambda, fluency, and memory to be explicitly defined.
This paper reports an algorithm for the resolution of local inconsistency in information-theoretic identification. This problem was first pointed out by Klir as an important research area in reconstructability analysis. Local inconsistency commonly arises when an attempt is made to integrate multiple data sources, i.e., contingency tables, which have differing common margins. For example, if one ha)s an AB table and a BC table, the B margins obtained from the two tables may disagree. If the disagreement can be assigned to sampling error, then one can arrive at a compromise B margin, adjust the original AB and BC tables to this new B margin, and then obtain the integrated ABC table by the conventional maximum uncertainty solution.
The problem becomes more complicated when the common margins themselves have common margins. The algorithm is an iterative procedure which handles this complexity by sequentially resolving increasingly higher dimensional inconsistencies. The algorithm is justified theoretically by maximum likelihood arguments. It opens up the possibility of many new applications in information theoretic modeling and forecasting. One such application, involving transportation studies in the Portland area, will be briefly discussed.
I use the label, "complexity theory," for the research program which studies nonlinear dynamics, "complexity," "complex adaptive systems," "artificial life," etc., and whose intellectual Mecca in the United States is the Santa Fe Institute. I use the label, "systems theory," for the research program which crystallized after World War II under the names of "general systems theory" and "cybernetics," and which subsumed such postwar scientific developments as information theory, game theory, feedback control theory, and the beginnings of computer science and artificial intelligence. The central thesis of this paper is that complexity theory is a continuation and revitalization of systems theory. I demonstrate the validity of this assertion in two steps. First, I describe the essential properties of the research program of systems theory, so that the underlying unity in the diverse manifestations of this program is evident. Second, I show that complexity theory shares in these properties, and thus continues this research program. (While complexity theory is systems theory's predominant contemporary manifestation, the "classical" system tradition, more strongly and explicitly rooted in the aspirations and literatures of general systems theory and cybernetics, also continues.) To many people this assertion may be obvious, but from my discussions with researchers in systems theory or complexity theory and from my preliminary encounters with relevant work in the philosophy and sociology of science, this proposition is far from being even widely recognized, not to speak of being generally accepted. The paper makes extensive use of a characterization of systems theory made by Mario Bunge which applies equally well to complexity theory. Bunge described systems theory as an attempt to construct an "exact and scientific metaphysics." The attempt to construct such a metaphysics represents a fundamental rejection of the possibility and desirability of a sharp demarcation separating science and metaphysics. At the very least, metaphysics can serve as a heuristic for science, but systems theory holds out a more radical promise: the recovery of metaphysics via its scientific reconstitution. Such a metaphysics would be less abstract than mathematics but more abstract than the theories of specific scientific disciplines. It would be "stuff-free" (materiality-independent) and only "vicariously" testable. It would represent an attempt to develop a "theory of everything" on an altogether different basis than the way such theories are conceived of in theoretical physics. A systems theoretic TOE, were one available, would genuinely unify the sciences, and not merely offer the illusory unity of a cascade of promised inter-theoretic reductions all the way down to elementary particle physics. Of course, a systems theoretic TOE is not currently available, but ample materials for constructing one are already at hand.
This paper concerns the relationship between the detectable and useful structure in an environment and the degree to which a population can adapt to that environment. We explore the hypothesis that adaptability will depend unimodally on environmental variety, and we measure this component of environmental structure using the information-theoretic uncertainty (Shannon entropy) of detectable environmental conditions. We define adaptability as the degree to which a certain kind of population successfully adapts to a certain kind of environment, and we measure adaptability by comparing a population's size to the size of a non-adapting, but otherwise comparable, population in the same environment. We study the relationship between adaptability and environmental structure in an evolving artificial population of sensorimotor agents that live, reproduce, and die in a variety of environments. We find that adaptability does not show a unimodal dependence on environmental variety alone, although there is justification for preserving our unimodal hypothesis if we consider other aspects of environmental structure. In particular, adaptability depends not just on how much structural information is detectable in the environment, but also on how unambiguous and valuable this information is, i.e., whether the information accurately signals a difference that makes a difference. How best to measure and integrate these other components of environmental structure remains unresolved.
When the reconstructability analysis of a directed system yields a structure in which a generated variable appears in more than one subsystem, information from all of the subsystems can be used in modeling the relationship between generating and generated variables. The conceptualization and procedure proposed here is discussed in relation to Klir's concept of control uniqueness.
The Genetic Algorithm (GA) and Simulated Annealing (SA), two techniques for global optimization, were applied to a reduced (simplified) form of the phase problem (RPP) in computational crystallography. Results were compared with those of "enhanced pair flipping" (EPF), a more elaborate problem-specific algorithm incorporating local and global searches. Not surprisingly, EPF did better than the GA or SA approaches, but the existence of GA and SA techniques more advanced than those used in this study suggest that these techniques still hold promise for phase problem applications. The RPP is, furthermore, an excellent test problem for such global optimization methods.
Set-theoretic reconstructability analysis is used to characterize the structures of the mappings of elementary cellular automata. The minimum complexity structure for each ECA mapping, indexed by parameter sigma , is more effective than the lambda parameter of Langton as a predictor of chaotic dynamics.
This study explores an information-theoretic/log-linear approach to multivariate time series analysis. The method is applied to daily rainfall data (4 sites, 9 years), originally quantitative but here treated as dichotomous. The analysis ascertains which lagged variables are most predictive of future rainfall and how season can be optimally defined as an auxiliary predicting parameter. Call the rainfall variables at the four sites A...D, and collectively, Z, the lagged site variables at t-1, E...H, at t-2, I...L, etc., and the seasonal parameter, S. The best model, reducing the Shannon uncertainty, u(Z), by 22%, is HGFSJK Z, where the independent variables, H through K, are given in the order of their predictive power and S is dichotomous with unequal winter and summer lengths.
Several methods of image reconstruction from projections are treated within a unified formal framework to demonstrate their common features and highlight their particular differences. This is done analytically (ignoring computational factors) for the following techniques: the Convolution method, Algebraic reconstruction, Back-projection and the Fourier-Bessel approach.
We consider the problem of matching domain-specific statistical structure to neural-network (NN) architecture. In past work we have considered this problem in the function approximation context; here we consider the pattern classification con-text. General Systems Methodology tools for finding problem-domain structure suffer exponential scaling of computation with respect to the number of variables considered. Therefore we introduce the use of Extended Dependency Analysis (EDA), which scales only polynomially in the number of variables, for the desired analysis. Based on EDA, we demonstrate a number of NN pre-structuring techniques applicable for building neural classifiers. An example is provided in which EDA results in significant dimension reduction of the input space, as well as capability for direct design of an NN classifier.
Systems theory offers a language in which one might formulate a metaphysics -- or more specifically an ontology -- of problems. This proposal is based upon a conception of systems theory shared by von Bertalanffy, Wiener, Boulding, Rapoport, Ashby, Klir, and others, and expressed succinctly by Bunge, who considered game theory, information theory, feedback control theory, and the like to be attempts to construct an ``exact and scientific metaphysics.''
Our prevailing conceptions of ``problems'' are concretized yet also fragmented and in fact dissolved by the standard reductionist model of science, which cannot provide a general framework for analysis. The idea of a ``systems theory,'' however, suggests the possibility of an abstract and coherent account of the origin and essence of problems. Such an account would constitute a secular theodicy.
This claim is illustrated by examples from game theory, information processing, nonlinear dynamics, optimization, and other areas. It is not that systems theory requires as a matter of deductive necessity that problems exist, but it does reveal the universal and lawful character of many problems which do arise.
We define variance and uncertainty measures of population diversity. Both measures have precise decompositions that we can exploit in analysis of evolutionary dynamics. We discuss how these measures are related and how they can be observed in artificial and natural evolving systems.
We define three information-theoretic methods for measuring genetic diversity and compare the dynamics for these measures in simple evolutionary models consisting of a population of agents living, reproducing, and dying while competing for resources. The models are "static resource models," i.e., the distribution of resources is constant for all time. Simulation of these models shows that (i) focusing the diversity measures on used alleles and loci especially highlights the adaptive dynamics of diversity, and (ii) even though resources are static, the evolving interactions among the agents makes the effective environment for evolution dynamic.
We propose a family of measures of population diversity in evolving system, and observe the dynamics of these quantities in the context of a particular model -a two dimensional world with organisms competing for resources and evolving by changes in their movement strategy. We measure the dependence of diversity upon two model parameters: selection and mutation rate.
An account is offered of the dialectical tensions which afflict systems of widely differing type, "contradictions" which cannot be fully or permanently resolved, and from which follow the lawfulness of both hazard and opportunity.
INTRODUCTION
Mario Bunge (1973) has provided a deep and succinct characterization of systems and cybernetics theories, e.g., information theory, game theory, automata theory, and the like, as attempts to construct an exact and scientific metaphysics. These theories can be considered "metaphysical" in their generality, "exact" in being mathematical, and "scientific" in having a close connection with specific theories in one or more scientific disciplines. This view is fundamentally in close agreement with the goals of general systems theory and/or cybernetics as expressed by Boulding, von Bertalanffy, Wiener, Ashby, and others.
This paper develops the outlines of a metaphysics of "problems," an account of the nature and origin of those difficulties which afflict many different kinds of systems, difficulties which reflect contradictions* intrinsic to being and to becoming which can never be completely resolved. Such difficulties are lawful and ubiquitous. This analysis serves as a necessary corrective to the tendency of systems thought to assume or to overemphasize the stability and internal harmony of systems and to neglect dysfunction, conflict, and change. What is outlined here is an entity-based metaphysics which takes the existence of entities to be intrinsically precarious.
This essay is a synthetic effort, and constraints of space make it impossible to "unpack" the technical and philosophical allusions of the narrative. An expanded version of this paper which details specific connections to the sources listed in the bibliography and to other works in the systems literature will be published elsewhere. The present text seeks to demonstrate that a coherent ontology is implicit in systems-theoretic ideas by casting these ideas into the form of a metaphysical discourse.
* The word "contradiction" is used in its dialectical and not logical meaning, i.e., to denote the coexistence of opposing forces, needs, tendencies, etc. No distinction is made in this paper between "concrete" and "conceptual" systems. Emphasis on the former is intended, and terms which properly belong only to the domain of the latter are used metaphorically.
Despite the familiar and correct disclaimer that information theory (Shannon and Weaver, 1949) does not concern the semantic level of communication, the technical definition of the information nonetheless bears directly and importantly on the subject of meaning. Meaning, at least in one sense of the word, is the recognition of the constraint and is based on isomorphism of structure. Constraint reduces information, yet information is also very substrate of meaning. Meaning is thus the union of the informative and the intelligible (Moles, 1958), the reconciliation of this dialectic opposition being achievable in several different ways.
The ubiquity of hierarchical order is obvious, and the obvious is hard to explain, but a number of workers [1] have suggested the possibility of constructing a theory (or cluster of theories), rooted in such disciplines as thermodynamics, information theory, topology, and logic, which might reveal the underlying unity of a wide variety of branching and multi-level systems. It is the purpose of this paper to contribute to both the empirical and theoretical aspects of this discussion, by examining levels of structure and function in molecular biology and linguistics, and by developing, from parallelisms between these two areas, a hierarchical model of possibly greater generality.
The measurement problem in quantum mechanics has the character of a fundamental incompleteness within that theory similar to the incompleteness of the axiomatic systems in mathematics, discovered and elaborated by Gödel and others. The difficulty of describing the measurement process by the time-dependent Schrödinger equation may reflect the limitations of formal language, and quantum theory may thus require a formalism consisting of two levels of description, one for the dynamics and one for measurement, levels whose relationship resembles that of a calculus and meta-calculus.
In an recent short note, Flondor has alluded to a possible linkage of fuzzy set theory and catastrophe theory. We consider several features of catastrophe theory, namely the properties of discontinuous jumps, hysteresis, and divergence in the "cusp catastrophe," and the role of the bias factor in the butterfly catastrophe", which have affinities to and suggest possible extensions of fuzzy set ideas. Certain functions extensively considered in catastrophe theory lend themselves in some cases to interpretation as membership functions. The use of such functions may be of interest for the characterization of linguistic descriptions which are time-varying and encompass both discrete and fuzzy distinctions.
A discrete multivariate relation, defined set-theoretically, is a subset of a cartesian product of sets which specify the possible values of a number of variables. Where three or more variables are involved, the highest order relation, namely the relation between all the variables, may or may not be decomposable without loss into sets of lower order relations which involve subsets of the variables. In a completely parallel manner, the highest order relation defined information-theoretically, namely the joint probability distribution involving all the variables, may or may not be decomposed without loss into lower-order distributions involving subsets of the variables. Decomposability analysis, also called "reconstructability analysis," is the specification of the losses suffered by all possible decompositions.
The decomposability of relations, defined either set- or information- theoretically, offers a fundamental approach to the idea of "complexity" and bears on all of the themes prominent in both the new and the old "sciences of complexity." Decomposability analysis gives precise meaning to the idea of structure, i.e., to the interrelationship between a whole and its parts, where these are conceived either statically or dynamically. It specifies the structuring and distribution and the amount of information needed to describe complex systems. It is partially predictive of chaotic versus non-chaotic dynamics in discrete dynamic systems. It provides a framework for characterizing processes of integration and differentiation which are involved in the diachronics of self-organization.
For a system described by a relation among qualitative variables (or quantitative variables "binned" into symbolic states), expressed either set-theoretically or as a multivariate joint probability distribution, complexity reduction (compression of representation) is normally achieved by modeling the system with projections of the overall relation. To illustrate, if ABCD is a four variable relation, then models ABC:BCD or AB:BC:CD:DA, specified by two triadic or four dyadic relations, respectively, represent simplifications of the ABCD relation. Simplifications which are lossless are always preferred over the original full relation, while simplifications which lose constraint are still preferred if the reduction of complexity more than compensates for the loss of accuracy.
State-based modeling is an approach introduced by Bush Jones, which significantly enhances the compression power of information-theoretic (probabilistic) models, at the price of significantly expanding the set of models which might be considered. Relation ABCD is modeled not in terms of the projected relations which exist between subsets of the variables but rather in terms of a set of specific states of subsets of the variables, e.g., (Ai,B j,Ck), (Cl,Dm), and (B n ). One might regard such state-based, as opposed to variable-based, models as utilizing an "event"- or "fact"-oriented representation. In the complex systems community, even variable-based decomposition methods are not widely utilized, but these state-based methods are still less widely known. This talk will compare state- and variable-based modeling, and will discuss open questions and research areas posed by this approach.
Reconstructability analysis (RA) decomposes wholes, namely data in the form either of set-theoretic relations or multivariate probability distributions, into parts, namely relations or distributions involving subsets of variables. Data is modeled and compressed by variable-based decomposition, by more general state-based decomposition, or by the use of latent variables. Models, which specify the interdependencies among the variables, are selected to minimize error and complexity.