Hazards of “Emergentism” in Psychology

© Roger K. Thomas 2001

 

 

Roger K. Thomas, Ph.D.

University of Georgia

rkthomas@uga.edu

www.arches.uga.edu/~rkthomas/

 

This manuscript was prepared for the symposium, “Controversial Issues in Psychology: The Role of Emergent Processes,” held at the annual meeting of the Southern Society for Philosophy and Psychology, Louisville, KY, April,1999. There was considerable emphasis in the symposium on the role of "emergents" in animal cognition. The symposium included Duane Rumbaugh, William Hillix, and David Washburn who champion the value of emergents in animal cognition and the present author who takes a considerably more conservative view. Other participants were Robert Burton, a philosopher, and Terence Deacon, a biological anthropologist.

Introduction

Psychologists can be good scientists and do research only at the behavioral level. They need not be immediately concerned with the physico-chemical foundations of behavior. However, psychology cannot be good science if its concepts and theories contradict or are inconsistent with the physico-chemical foundations of behavior.

The concept of emergence appears to be used in two fundamentally different ways in behavioral science. A material reductionist’s use of emergence accepts that emergent behavioral properties or processes are, in principle, reducible to physico-chemical properties and processes at foundational levels, although how the emergence occurs may not be readily obvious from what is currently known about the physico-chemical properties and processes. So, for example, the reductionist accepts that properties of water emerge from the combination of the elements hydrogen and oxygen in accordance with other principles in physics, but when water is reduced, nothing is left but those elements; nothing has been added. The whole is equal to the sum of the parts. An anti-reductionist’s use of emergence accepts or implies that properties or processes may emerge that are not reducible, even in principle, to the fundamental physico-chemical properties and processes. In this sense, something new has been added, for example at the behavioral level, and the whole is not merely greater than the sum of the parts, it is not even traceable to the combinations which may occur on summation.  As will be discussed below, this anti-reductionist use of emergence seems to have a role in psychology that is parallel to the role that vitalism once had in biology.  It is now generally considered that biology had to rid itself of vitalism to enable significant progress to occur.  It is suggested that psychology will develop as a science only after it rids itself of  anti-reductionistic, "emergentism."     

Psychology as a science must not have emergent concepts and theories that deny their theoretical reduction to physico-chemical fundamentals. Most of what some of my symposium colleagues have written about “emergents” appears to be consistent with physico-chemical reduction, but at times they have written things that appear to be consistent with an anti-reductionist use of emergence. One example of an apparent anti-reductionist use occurred when Rumbaugh, Washburn, and Hillix (1996) embraced John Stuart Mill’s “mental chemistry” as a model for their “emergents” and wrote, “Emergent complex ideas had their own distinguishing structures and properties and, hence, were more than just a composite of the simple ideas on which they were based.” (1996, p. 59; emphasis added).  Admittedly, there is sufficient ambiguity among these words, especially “ideas” and “composite” and how they may relate to their physico-chemical fundamentals, but it sounds like something has been added that is not, in principle, reducible to those fundamentals. Later, I will cite other things they have written that appear to be based on their acceptance of an anti-reductionist emergence.

Emergentism

[Addendum...what follows immediately is a somewhat tedious construction of a definition of “emergentism.” Some readers may wish to skip to the end of this section where the constructed definition may be seen.]

What is emergentism? I was surprised to discover (after submitting this paper’s title) that “emergentism” does not appear in any dictionary I have consulted, including the Oxford English Dictionary (Second Edition, 1989) and Webster’s New International Dictionary, Unabridged (Second Edition, 1956). Fortunately, it was reasonable to construct a definition of what I had in mind. What I had in mind, was the analogy: ‘emergentism is to psychology as vitalism is (was?) to biology.’ The construction of a definition for emergentism began with Runes’ (1963) definition of “emergent mentalism.”

Emergent Mentalism...The theory of emergent evolutionism considered as an explanation of the genesis of mind or consciousness in the world. Mind is a novel quality emerging from the non-mental when the latter attains a certain complexity of organization. ( p. 89)

That definition may be viewed as being consistent with either a reductionist’s or an antireductionist’s “emergence.” However, Runes defined emergent mentalism as a special case of “emergent evolutionism,” but he then defined “emergent evolutionism” circularly as “generalization of emergent mentalism.” 

Webster’s (1956) definition of “emergent evolution” defines it reciprocally with “creative evolution,” and with that reciprocity the anticipated hazards of “emergentism” emerged!

emergent evolution...evolution conceived as characterized by the appearance at different levels of new antecedently unpredictable quality of being or modes of relatedness, such as life and consciousness. (p. 837; emphasis added)

creative evolution... evolution conceived as a creative, rather than a mechanically explicable or predictable process. (p. 621; emphasis added)

 Runes’ (1956) definition of “vitalism” is now quoted for its usefulness in conjunction with the analogy mentioned earlier and as a model, with the aid of Webster's definitions, to complete the definition of emergentism.

Vitalism: The doctrine that phenomena of life posses a character sui generis by virtue of which they differ radically from physico-chemical phenomena. The vitalist ascribes the activities of living organisms to the operation of a “vital force”....(p. 333; emphasis added)

Emergentism: The doctrine that mental processes possess a character sui generis by virtue of which they are antecedently unpredictable, are creatively rather than mechanically explained, and are radically different from physico-chemical phenomena.

Vitalism and Emergentism

Because the hazards of emergentism and the lessons for psychology are so closely related to the hazards and lessons of vitalism for biology, it is useful to consider vitalism.  Guyer’s (1931, pp. 23-24) account of vitalism expresses the essence of what I want to emphasize.

Are these characteristics which mark off living from nonliving matter explainable by physics and chemistry and the known laws of matter or is there something else?...Two opposing interpretations have been suggested; one known as vitalism, the other as mechanism. By vitalism is meant a directive tendency beyond the inherent properties of mere molecules or chemical elements which manifests itself in and is peculiar to the living organism....They believe they find evidence of purpose in life-activities and that such activities are inexplicable on the basis of mere physics or chemistry....When it comes to mind, some of them would maintain that mind inserts itself into matter rather than emerges from it.

....admitting that many of the phenomena seen in living things are yet unexplained or are even inexplicable in terms of our present knowledge of chemistry and physics, the mechanist points out that with our advancing knowledge in these fields many of the processes originally claimed by vitalists to be distinctively vital have been shown to be physical or chemical and that continual progress is being made by mechanistic methods....Mechanists believe it is simpler and more accurate to regard life as process or function rather than as a separate essence, and to consider living matter as ordinary matter so arranged as to become a metabolic mechanism....

 The controversy, though changing its form from time to time, has been carried on ever since the days of Aristotle and there seems no prospect of agreement in the near future. The problem may be insoluble. As our knowledge of fundamental life processes has advanced, the vitalist has been forced to abandon one position after another, but there is still such a great unexplained residue of facts relating to the constructive and coordinating processes of living matter that he still has abundant material for argument. As a practical working program, however, it is well to note that the science of biology has advanced mainly as it has been able to explain its phenomena in mechanistic terms, and that there is undoubtedly much yet that can be so explained. To rest content with merely attributing vital phenomena to some sort of "vital principle" is in effect to give up the problem, and such an attitude of mind can lead only to scientific stagnation.

 Summarized below are some central points about vitalism (in italics) as quoted from Guyer’s passages above. Each is followed immediately by its reformulation (in color) to reflect emergentism. 

[Vitalists]...believe they find evidence...in life-activities...that...are inexplicable on the basis of mere physics   or chemistry. 

Emergentists believe they find evidence in mental activities that are inexplicable on the basis of mere physics and chemistry.     

 ...many...processes...claimed by vitalists...have been shown to be physical or chemical...continual progress is being made by mechanistic methods...

Many processes claimed by emergentists have been shown to be physical or chemical, and continual progress is being made by mechanistic methods.

            ...the problem may be insoluble....there is still such a great unexplained residue of facts relating to the constructive and coordinating processes of living matter that...[the vitalist]... still has abundant material for argument.

            The problem may be insoluble. There is still such a great unexplained residue of facts relating to the constructive and coordinating processes of living matter that the emergentist still has abundant material for argument.

            Biology has advanced mainly as it has been able to explain its phenomena in mechanistic terms, and that there is undoubtedly much yet that can be so explained.

            Psychology has advanced mainly as it has been able to explain its phenomena in mechanistic terms, and that there is undoubtedly much yet that can be so explained.

            To rest content with merely attributing vital phenomena to..."vital principle" is in effect to give up the problem, and...[that]...can lead only to scientific stagnation.

            To rest content with merely attributing mental phenomena to irreducible processes is in effect to give up the problem, and that can lead only to scientific stagnation.

Am I ‘jousting against windmills?’ I don’t think so. Examples in psychology can be found in too many issues of too many of its current journals. Any psychologist for whom mind or stress, or anxiety, to cite three common examples, do not reduce to physico-chemical activities of the brain and body is an emergentism emergentist.  A defining characteristic of such writing is the reification of these nonexistent entities.  Such reification is evident when a process is described in ways that suggest that it can be a stimulus, a cause, etc. ('stress caused his ulcers') or a response, an effect, etc. ('she was treated for anxiety').  It is my impression that most psychologists reflect emergentism in what they write (and, apparently, believe).  I do not know whether my symposium colleagues here are such emergentists, but some of the things they have written lead me to wonder. I will cite some examples and, as needed, they can set me straight.

Rumbaugh, Savage-Rumbaugh, and Washburn (1996) wrote the following which, in its larger context, appears to combine their view of “emergents” with the possibility of vitalism.

This research philosophy [Behaviorism] emulated that of physics and chemistry - the “hard” sciences - that enjoyed substantially more respect and prestige than psychology. It was as though psychologists attributed the success of the other sciences to their refutation of life variables, and thus rejected life dimensions from their own theory and methods to achieve “standing” for their science. In doing so, they failed to acknowledge a major error; although the sources of the data for physics and chemistry are lifeless, the very foundation of psychology’s subject material, behavior, is generated only by life - the human and animal life of our world. Thus, the data for psychology must be qualitatively different from the data of physics and chemistry. (p. 114; their emphasis)

 It may be instructive to examine some of that. First, I am not sure what they meant by “the data for psychology must be qualitatively different from the data of physics and chemistry,” but if they meant that psychological data are, in principle, not reducible to physico-chemical data, that is emergentism. I am also unsure what they meant by “the sources of the data for physics and chemistry are lifeless.” If they meant what chemists and physicists are more likely to investigate, surely that is a trivial distinction. Questions regarding the origin of life and what distinguishes living matter from nonliving matter are best understood from the standpoint of data that physicists and biophysicists, chemists and biochemists, and other biologists have provided and not from the standpoint of "vital force" or "vital principle."

For example, precursor molecules for life were formed in a laboratory experiment in 1953 by Stanley Miller using no more than common chemical constituents of the Earth’s atmosphere together with a commonly available energy source, an electric spark to simulate lightning. Other common energy sources such as ultraviolet light and heat were shown to be equally effective (Audesirk & Audesirk, 1993). [Addendum: An updated consideration of Miller's experiments and subsequent developments may be seen in de Duve (1995).] Given the Earth’s chemical constituents, energy sources, and millions of years of conditions conducive for such precursor molecules to form, it takes little imagination to envision how those same resources might lead from precursor molecules to the more complex molecules that could manifest life’s defining processes (e.g., material and energy conversion, responding to environment, homeostasis, growth, self-replication); Audesirk & Audesirk, 1993, p. 1; or as Guyer, 1931, expressed it; “living matter...[is]...ordinary matter so arranged as to become a metabolic mechanism.” p. 24). Geneticists today are on the brink of determining the minimum genome necessary to manifest life’s processes. Thus, it appears that the chemical and physical threshold of life is now definable, it is narrow, it is about to be breached, and it is reducible to its physico-chemical fundamentals.

Material Reductionist Psychology and Emergent Concepts

As a context for understanding how emergentism in psychology might be avoided while preserving the utility of emergent concepts, the following is based on an extended consideration of McCorquodale and Meehl’s (1948) distinction between intervening variables and hypothetical constructs in psychology. The central point is that emergent concepts in psychology, including my colleagues’ “emergents,” with few if any exceptions, are akin to intervening variables (for that matter, so are “respondents” and “operants” with which my colleagues contrasted “emergents”); more precisely, emergents are a kind of superordinate intervening variable (see below). As such, emergents are no more or less special than any other concept or conceptual category that we use and try to understand in psychology.

Figure 1

The simplest case in which to consider the intervening variable in psychology is when the behaving organism is treated as a “black box” (or ellipse) with which there is no need to concern one's self about the physico-chemical foundations of behavior. In psychological experimental research, among the antecedents external to the organism that influence its behavior is the subset that is isolated and manipulated by the experimentalist; these are known in psychological research as independent variables. Among the consequents, the subset that the experimentalist isolates to measure and relate to the independent variables are known as the dependent variables. In an experimental context, an intervening variable is a concept that abstracts or links the observed independent and dependent variables. Strictly speaking, an intervening variable means no more or less than what is provided by the empirical data in a specific situation.  However, as indicated above, too often with intervening variables, reification occurs.

Extending from the limited meaning of intervening variable that McCorquodale and Meehl (1948) described, we can determine that the same name for an intervening variable may be applied in different experiments which have different independent and dependent variables. We can also determined that there are commonalities among those different experiments that make use of the same name for the intervening variable meaningful. It makes sense, then, to consider such sets of specifically defined intervening variables with the same name as representing a superordinate type of intervening variable.

For example, an intervening variable that might be specified independently in several different experimental contexts is “fear.” The antecedent conditions across those experiments where “fear” may be appropriate conceptual term have in common that they represent potential pain, injury, or death (e.g., a poisonous snake, a rabid dog, a pointed gun), and the consequent conditions across those experiments have in common that they are associated with intense bodily responses involving greater-than-usual autonomic and endocrine activation (e.g., tachycardia, hyperventilation, involuntary urination etc.).

 We can observe the antecedent conditions that represent potential pain, injury, or death, and we can observe consequents such as tachycardia, hyperventilation, and involuntary urination, but we can not observe “fear.” Fear is simply a word that was invented (or chosen from historical usage; see below) to specify, link, or summarize the empirical relationships that were observed. In the case of an intervening variable, such as “fear,” we acknowledge that as an observable entity, event, or process, fear does not exist independently of the observed antecedents and consequents; that is, an intervening variable reduces completely to the observed empirical relationships. While this discussion has been in terms of experimental situations, obviously the history of the term “fear” predated the history of experimental psychology. Nevertheless, when analyzed carefully, what most people appear to mean by the everyday use of “fear” is based on similar kinds of antecedent and consequent observations.

In contrast to the intervening variable, there are some instances when a summarizing or linking concept may be useful, and we do believe that an entity, event, or process may exist independently of observable antecedents and consequents. That is, we may believe that the entity, event or process is potentially observable and that it has yet to be discovered or identified. In that case, following McCorquodale and Meehl (1948), it is a hypothetical construct. A historical example of a hypothetical construct was the “gene.” When the concept of the gene was proposed by Johannsen in 1909 (McClearn, 1963) to represent the presumed physico-chemical substrate responsible for the observed manipulations (antecedents) and consequents of plant and animal crossbreeding, no one knew anything about DNA and its associated mechanisms. I don’t know if psychology has any hypothetical constructs as conceptual entities. Perhaps, Lashley’s (1950) “engram,” his term for a presumed physico-chemical substrate for memory, is one.

Figure 2 shows an enhancement of the “black box” model that acknowledges the possibility to investigate and observe (via appropriate technologies) the physico-chemical extensions of the external antecedents and precursors of the external consequents that may occur inside the black box. In addition to specifying the relevant antecedents and consequents external to the boundaries of the organism, one can specify physico-chemical activities of the sensory receptors, along the sensory pathways to the brain, and inside the brain. In principle, one can investigate the physico-chemical processes at all stages associated with sensory processing (antecedents) until one reaches a definable transition to the physico-chemical processes associated with effector activation (muscles and glands) associated with the consequents. To cite an example of the latter, voluntary motor responding as an effector process probably begins at the cerebral cortex. There may also be neural structures and physico-chemical processes that serve functionally as an interface between sensory and effector processing, although one can argue well for the case that the brain only does sensory and effector processing. Historically, such hypothesized interfaces included “association cortex” and other so-called “association areas” of the brain.

Figure 2

Compared to Figure 1, the model here includes three important additional considerations.

1. The organism and its external environment are constantly changing and interacting through time. The antecedents and consequents, both externally and internally, are constantly being updated. Physico-chemical activities associated with ongoing precursor effector processing as well as those associated with the observable external consequents may immediately become part of new external and internal antecedents.

2. Memories for past sensory and effector processing may affect ongoing sensory and effector processing. Memories are constantly affected, updated, and possibly modified by ongoing sensory and effector processing.

3. A role for emergents as conceptual entities or intervening variables has been added.

The complexity and dynamics of the model suggested in Figure 2 may appear forbidding, perhaps as forbidding as the challenges faced by the astrophysicist in the quest to understand the origin, past, present, and future of the universe or of the biologist to determine the origin of life, but as a model to inform and guide the conceptualization of psychological processes, including emergent “mental” processes, it is a model that is most consistent with psychology’s development as a science. There are other formidable obstacles that all sciences face, such as those arising from the implications of Heisenberg’s indeterminacy principle (Heisenberg, 1958), from the fact that scientific observations always involve transformations of the “raw” data (e.g., Bateson, 1972, p. xviii; Hacking, 1981), and from other well known weaknesses and limitations of scientific method.

Nevertheless, such complexities and challenges are practical matters that render attainable scientific knowledge as being probable and subject to revision as opposed to being certain. However, the extremely high probabilities attained, for example, in astronomy, chemistry, genetics, and physics suggest that such practical matters do not diminish the value of adhering to mechanistic and material reductionistic viewpoints as guiding principles in psychological science, the guiding principles that have served the “hard sciences” so well.

Closing Remarks: To Revisit My Colleagues’ Views of Emergents

In their Table 1, Rumbaugh, Washburn, and Hillix (1996) identified 11 parameters with which to compare respondents (Pavlov’s unconditional and conditional responses), operants (Skinner’s responses emitted by organisms that become selected by their consequences), and emergents (“new competencies or patterns of responding that were never specifically reinforced”).  Figure 3 here replicates their Table 1 with two modifications: (a) their columns for respondents and operants have been deleted because, except for parameter F where Rumbaugh et al. said "yes" for all three but with some equivocation for emergents, their responses for both respondents and operants were always the same for each other and were always the opposite of their corresponding responses for emergents; (b) a column was added to show my responses to their emergents.  

So, for example, the way to read the table (if the columns for  respondents and operants had been included here) regarding Parameter A, "A well-defined CS or antecedent," would have been that Rumbaugh et al. (1996) said "yes" for respondents and operants meaning that respondents and operants required a well-defined CS or antecedent and "no" for emergents meaning emergents did not require a well-defined CS or antecedent.  As may be seen in the column added by Thomas who viewed "Emergents as Intervening Variables," Thomas said "yes," that such emergents require a well-defined CS or antecedent, although as may be seen, identification of the CS or antecedent may be difficult.   With this general understanding of how Figure 3 works, the reader may wish to skip to Figure 4 where only the four rows in Figure 3 on which Rumbaugh et al. and I disagree are shown. 

                                                                         Figure 3

        Modified (see below) Version of Table 1 from Rumbaugh, Washburn, & Hillix (1996) 

Parameters

Emergents (Rumbaugh et al.)

Emergents as Intervening Variables (Thomas)

A. well-defined CS or antecedent

No

Yes, but antecedent may be difficult to isolate and observe.

B. acquisition depends upon experience with specific and limited antecedents and consequents.

No

Yes to Aspecific@ but they may be difficult to isolate and observe.

Yes? to Alimited@...depends what Alimited@ means.

C. overt response required and recordable during acquisition.

No - their function may be SILENT

Not necessarily, but should be internally observable.

D. conditionable to CS/SD

No

Yes...if Aconditionable@ means learning...but they may be difficult to isolate and observe.

E. based on histories that emphasize generalized classes of experiences.

Yes

Yes

F. repetition of trials or events important

Yes?

Yes

G. new response modes form

and provide for novel adaptations.

Yes

Yes, but I have a Aproblem@ with the definition of Anovel@ they do not seem to have.

H. appear in novel contexts/problems and transfer tests.

Yes

Yes, but I have a problem with the definition of Anovel@ they do not seem to have.

I. entails syntheses of individually acquired responses

Yes

Yes

J. particularly sensitive to early rearing variables

Yes

No?

K. interactive products of Task X Organismic variables (e.g., brain complexity as per maturation and species.)

Yes

Yes

 

As may be seen in Figure 4, Rumbaugh et al. (1996) and I disagree clearly on three parameters and somewhat on a fourth parameter. We agree generally on the other seven parameters, although I have some misgivings (indicated in column three above) about some of those. To facilitate consideration of our disagreement, Figure 4 shows only the four parameters where we disagree.

                                                                   Figure 4

         Rows Selected From Figure 3 (see explanation above).

Parameters 

 

Emergents  (Rumbaugh et al., 1996)

Emergents As Intervening Variables (Thomas)

A. well-defined CS or antecedent

No

Yes, but antecedent may be difficult to isolate and observe.

B. acquisition depends upon experience with specific and limited antecedents and consequents

No

Yes to Aspecific@ but they may be difficult to isolate and observe. Yes? to Alimited@... depends on what Alimited@ means.

D. conditionable to CS/SD

No

Yes, if Aconditionable@ means learning...but CS/SDmay be difficult to isolate and observe.

J. particularly sensitive to early rearing variables

Yes

No?

Parameter J reflects only a minor disagreement I have with my colleagues, namely, on the importance of early experience for the acquisition of emergents. My response is based simply on the fact that I have trained squirrel monkeys successfully on some of the tasks they identified elsewhere in their article as tasks that are associated with emergents, and as far as I know, there was nothing in my monkeys’ early rearing that might have prepared them for the tasks. Most of the monkeys I used grew to young adulthood in natural jungle habitats, and I fail to see how that experience might be applicable to performance on the tasks in question. On the other hand, it is an empirical question, and I do not reject the possibility that the monkeys had early rearing experience that was relevant.

The basis for our disagreement on parameters A, B, and D is fundamental. For a mechanistic, material reductionist, all intervening variables, including superordinate intervening variables, in principle, have discoverable antecedents and consequents. To say otherwise is to say that their "emergents" cannot be reduced ultimately to physico-chemical properties and processes. Presumably the material reductionistic path to that ultimate physico-chemical reduction will also include reduction to psychological processes (superordinate intervening variables) that characterize precursors to their emergents. In brief, as intervening variables, emergents have antecedents both external and internal to the organism, although identifying them clearly may be an extremely difficult if not practically impossible undertaking. But such difficulty cannot be a barrier to continue the search for precursor psychological processes and their physico-chemical foundations. Repeating an earlier point:

To rest content with merely attributing mental phenomena to irreducible processes is in effect to give up the problem, and that can lead only to scientific stagnation.

It is too soon to give up on the problem of analyzing and reducing emergents in psychological science.

References

Audesirk, G., & Audesirk, T. (1993). Biology: Life on Earth (3rd Ed.). New York: Macmillan.

Bateson, G. (1972). Steps to an Ecology of Mind. New York: Ballantine Books.

de Duve, C. (1995). The beginnings of life on Earth. American Scientist.  Article available at:

                                 http://www.sigmaxi.org/amsci/articles/95articles/cdeduve.html

Guyer, M. F. (1931). Animal Biology. New York: Harper & Row.

Hacking, I. (1981). Do we see through a microscope? Pacific Philosophical Quarterly, 62, 305-322.

Heisenberg, W. (1958). Physics and Philosophy: The Revolution in Modern Science. New York: Harper & Row.

Lashley, K. S. (1950). In search of the engram. Symposium Society for Experimental Biology, No. 4 (pp. 454-482). Cambridge, England: Cambridge University Press.

McClearn, G. E. (1963). The inheritance of behavior. In L. Postman (Ed.), Psychology in the making (pp. 144-252). New York: Alfred A. Knopf.

MacCorquodale, K.,& Meehl, P.E. (1948). On a distinction between hypothetical constructs and intervening variables. Psychological Review, 55, 95-107.

Neilson, W. H. (Editor-in-Chief) (1956). Webster’s New International Dictionary of the English Language (2nd Ed. Unabridged). Springfield, MA: G. & C. Merriam Co, Pub.

Rumbaugh, D. M., Savage-Rumbaugh, S., & Washburn, D. A. (1996). Toward a new outlook on primate learning and behavior: complex learning and emergent processes in comparative perspective. Japanese Psychological Research, 38, 113-125.

Rumbaugh, D. M., Washburn, D. A., & Hillix, W. A. (1996). Respondents, operants, and emergents: Toward an integrated perspective on learning. In K. Pribram & J. King (Eds.), Learning as a Self-Organizing Process (pp. 57-73). Hillsdale, NJ: Lawrence Erlbaum  Associates.

Runes, G. D. (Ed.). (1963). Dictionary of Philosophy (15th Edition Revised). Patterson, NJ: Littlefield, Adams, & Co,

Simpson, J.A., & Weiner, E.S.C. (Prepared by). (1989). The Oxford English Dictionary: Second Edition. Oxford: Clarendon Press.