homeAboutJPSAboutPiagetNewsStudentsmembershippublicationsconferencecontactlinkssearch

INTERNATIONAL OBJECTIVE MEASUREMENT WORKSHOP

March 21-23, 1997, Judd Hall, University of Chicago

Organized Symposium:

'The Measurement of Psychological Development'

Organiser: Trevor Bond, James Cook University, Townsville, Australia

Discussion-Workshop:

Chair: Dr Bill Gray, University of Toledo

Posters:

(related to the analysis of developmental issues):

Paper:


Abstracts

Organized Symposium:

The Measurement of Psychologial Development

Organiser: Trevor Bond, James Cook University, Townsville, Australia

This symposium is an undisguised attempt to enhance communication between Rasch theorists on the one hand, and investigators who use explicitly theory-driven research to address substantive issues in the psychology of human development, on the other. Just as Rasch analysts affirm that meaningful data analysis is based on the principles of measurement theory that are implemented by Rasch analysis, these researchers affirm that meaningful developmental research is based on an in-depth understanding of Piagetian theories of human development. Each of these researchers has found considerable success with the application of Rasch analysis and has presented papers and/or posters at Jean Piaget Society Symposia. Of course, they share with other developmental practicioners common problems about the meaning of core Rasch analysis concepts. However, their specific attempts to investigate and clarify central issues in developmental theory by marrying their empirical investigations with sensitive data analytical techniques raises crucially important questions of interpretation and meaning. Indeed, each investigator presenting here has uncovered a significant important conceptual problem that would benefit from discussion and the advice of Rasch theoreticians and practioners. In opening up this direct dialogue, we intend to clarify how we interpret Rasch concepts in terms of developmental theories. Moreover we seek to understand the extent to which Rasch theorists consider our interpretations to be reasonable in terms of the Rasch model.

(HOME PAGE) (return to top of page)


Developing conceptions of the good: Untangling content and structure with the assistance of Rasch Analysis

Theo Dawson, Graduate School of Education, University of California at Berkeley


Several structural-developmental theorists have called attention to the need to distinguish between content and structure in the conceptualization of stage (for examples, see Commons, Richards, with Ruf, Armstrong-Roche & Bretzius, 1984; Puka, 1991; Rosenberg, 1988). Unfortunately, though most structural developmental stage sequences are derived using structural criteria, the scoring systems that result consist predominantly of content descriptions (for examples, see Armon, 1984; Colby & Kohlberg, 1987; Damon & Hart, 1988; Selman, 1977). This has resulted in a tendency on the part of some researchers to confound stage and content, using the occurence of cultural differences or age inconsistencies in concept usage as evidence that structural change is not a universal feature of human development (Gilligan, 1982; Miller, 1988; Shweder). In the present analysis of 180 semi-structured, probed, clinical interviews of a life-span sample of participants, content and structure have been analysed separately. First, a fine-grained concept analysis was conducted, resulting in the identification of over 600 concepts. This unwieldy pool of data was reduced to a manageable size through a theoretically driven spiral bootstrapping process involving the use of Rasch modeling. Second, stage was assessed using the General Stage Scoring System (GSSS) (Commons, Straughn, Meaney, Johnstone, Weaver, Lichtenbaum, Sonnert, Rodriquez, 1995), a method of assessing stage that does not rely on the particular content of responses. Rasch analysis is used to examine the relationship of concept usage and stage.

(HOME PAGE) (return to top of page)


A theoretical and empirical investigation of misfitting persons and items based on cognitive development

William Gray & Christine Fox, University of Toledo

An assumption of Piaget's approach to psychological development is that development is continuous. Through continuous adaptations the overall pattern of functioning becomes relatively stable and this stability is represented by a sequence of forms (stages) of thought.
Traditionally, this sequence of forms has been assessed by tasks representative of one of the forms, and then evaluating individuals' responses to the tasks, especially responses that are not adequate for successful solving of the tasks. In most replications, extensions, etc., of Piaget's investigations, the focus has been on the participants: their demographic characteristics, can the development of various adaptations be accelerated, etc. However, focusing on the participants and their characteristics, demographic or otherwise, was not the intent or focus of Piaget's perspective. Rather, "over the series of [Piaget's] works which attack intelligence at different points on the growth continuum and focus on different functions [emphasis added by us], the over-all aim has been to trace the development of intelligence as it comes to deal with increasingly complex problems or as it deals with simple problems in increasingly more efficient ways" (Parsons, 1958, p. xi). In essence, Piaget's focus and emphasis was on categorizing different forms of thought according to their hierarchical complexity (Chapman, 1988, p. 33). Obviously, the focus and emphasis of most replications and extensions of Piaget's work was not his focus and emphasis. Recently, there has been a renewed interest in scaling individuals' forms of thought generated when they respond to Piagetian-based tasks. The present study is within the spirit of this renewed interest.

In a recent ex post facto analysis of a data set generated from parallel forms of a written test of operational thought, Gray and Fox (1996) raised several concerns regarding Rasch scaling of the data. Those concerns ranged from the appropriate fit of the data with the Rasch model to the standard psychometric concerns regarding the participants who generated the data (e.g., test anxiety, excessive cautiousness, cheating, illness, distractions, guessing, or fatigue) (Smith, 1992). Although raising a number of issues, Gray and Fox did not pursue an analysis of the misfitting participants nor did they pursue an analysis of the misfitting problems. The present paper will be a theoretical analysis of well understood Piagetian-based data guided by Rasch analysis. Questions to be addressed will include (a) equivalence of structurally identical items, (b) effect of participants' ages on scale scores, (c) effect of number of categories in which responses may be placed, (d) effect of categories in which no participant is placed, (e) effect on scale scores of items after items and/or participants are eliminated from the data, (f) effect on scale scores of participants after items and/or participants are eliminated from the data, (g) effect on scale calibration after items and/or participants are eliminated from the data, etc.

Since Piagetian theory provides a well-established framework for both measuring and interpreting cognitive development, it provides an excellent opportunity to closely examine the effects of measurement disturbances on the interpretation of the underlying variable. Extensive exploration of rating scale use (Lopez, 1996) and misfit analysis (Lopez, 1996; Smith, 1992) can more clearly identify problems with the items, the raters, or the respondents without having to question the theoretical basis of the test. Not only do these analyses aid in the understanding of the measurement of cognitive development, they should also provide a valuable contribution to the ongoing dialogue in the development of more thorough and systematic ways of extracting meaning with the Rasch model (see Lopez, 1996).

(HOME PAGE) (return to top of page)


Comparing décalage and development with cognitive developmental tests

Trevor Bond, School of Education, James Cook University, Australia

Research into formal operational thought using the Rasch model (Bond 1995a,b; Bond and Bunting, 1995) substantiates important aspects of the original theorising of Piaget (Inhelder & Piaget, 1953/58) which was based wholly on qualitative structural analyses of children's responses. Common-person equating of student performances has been used across different tasks to estimate the relative difficulties of tasks measuring the same underlying construct and across repeated measures of the same task to estimate cognitive development over one- and two-year intervals. Estimates of cognitive development do not exceed 0.5 logits per annum (Bond 1996); a result that has been estimated independently in two large research projects (Shayer, 1996 and Lake, 1996). Interestingly, difficulty differences (décalage) between tests of formal thought are as large as 2.0 logits (Bond, 1995a; Bond, 1996), confounding attempts to differentiate development from décalage. The paper canvasses the possiblity of using other Rasch modelling techniques to quantify these fundamental aspects of human cognitive performance.

(HOME PAGE) (return to top of page)


Can qualitative stage characteristics be revealed quantitatively?

Gino Coudé, Gérald Noelting, & Jean-Pierre Rousseau, École de Psychologie, Université Laval
Québec, Canada

The correspondence between Piagetian stages of cognitive development in three tasks as well as the hierarchy of item difficulties were examined. The three tasks, Mixing Juices (MJ), Caskets Task - Text-form (CTT) and Coded Orthogonal Views (COV) were submitted to a sample of 350 subjects aged from 16 years old into adulthood (Brunel et al., 1992; Noelting et al., 1992). Each task concerns a different cognitive domain; the MJ bears on the ratios embodied in mixtures of orange juice and water in varying proportions (Noelting, 1980); the CTT involves propositional logic where affirmation and negation statements are combined (Noelting et al., 1993); while the COV is a spatial task involving the drawing of 3D figures using codes that enable the figure's reconstruction from the subject's production (Gaulin et al., 1984, Noelting et al., 1986). The cognitive developmental levels in each of the tasks had been previously established by Noelting and collaborators (1980, 1995) on the basis of a qualitative analysis derived from a Piagetian framework. A subsequent Rasch analysis provided a helpful representation of item difficulty levels as well as an assesment of fit of items to the underlying cognitive developmental trait. The results reveal a high correspondence between qualitative and quantitative analyses. While some differences in item estimates are found to be statistically significant, the issue of quantifying stage leaps remains to be fully investigated. The items devised in the light of a qualitative framework and previously submitted to other samples of subjects show a good intra-task fit. Item misfits are discussed in relation to the isomorphisms between tasks.

(HOME PAGE) (return to top of page)


Discussion-Workshop:

'The Measurement of Psychological Development--
The Unresolved Issues"

Chair: Dr Bill Gray, University of Toledo

A presentation by Mark Wilson and invited reactions by Ben Wright and Bill Fisher to pre-circulated papers from the Organized Symposium: 'The Measurement of Psychological Development' will set the scene for a discussion on the so-far unresolved issues raised by the papers in that Symposium.

Posters

(related to the analysis of developmental issues):

Using Rasch Analysis to examine the development of socio-moral concepts

Theo L. Dawson, The University of California at Berkeley

In order to trace the development of conceptions good education, I performed a fine-grained analysis of educational concepts present in the transcripts of 180 semi-structured, probed, clinical interviews of a life-span sample of participants. In the initial analysis, over 600 concepts were identified--an unwieldly number, to understate the case. However, due to the fact that the more commonly used concepts tended to span a large age-range, conventional means of collapsing the conceptual categories (such as collapsing infrequently occurring concepts into more general categories) tended to blur developmental differences. This poster traces the theoretically driven spiral bootstrapping process that was used to refine concept categories, thereby reducing the data set to a managable size. The role of Rasch modeling in this process is described.

(HOME PAGE) (return to top of page)


A Saltus analysis of developmental data from the laundry problem task series.

Goodheart, E. A., Harvard University

Dawson, T. L., University of California at Berkeley

Draney, K., University of California at Berkeley

Commons, M. L., Harvard Medical School

We perform a Saltus analysis of cross-sectional developmental data gathered from a group of 36 adults and 37 children to whom we presented the Laundry Problem Task Series (Commons, Miller, and Kuhn, 1982). A Saltus analysis (Wilson, 1989; Wilson & Draney, 1995) is a mixture model extension of the Rasch model first applied to intelligence tests (see Rasch, 1960) and later applied to the study of developmental data. Whereas a traditional Rasch analysis determines the probability of a given subject performing a given item in terms of item difficulty (delta) and subject ability (beta), a Saltus analysis introduces item and subject stage as a third concept. This additional concept (along with the parameter[s] that embody it) can help to determine whether the gapiness and systematic shifts in item misfit present in an earlier two-parameter Rasch analysis can be explained as stage change. Issues of fit and validity will be raised, and the results of the analysis will be discussed in terms of their general implications for developmental research as well as the more specific development of the Laundry Problem instrument.

(HOME PAGE) (return to top of page)


Morality and rule comprehension: Catholic children's cognitive moral development during grade school II: A Rasch analysis

Michael A. Morabito

The present study utilizes Rasch analysis to examine data on grade school children's knowledge of differing social rules. Male and female children from the third and eighth grades were presented with ten item sets, with each set containing three social rule violations. Of the three statements, two were from the same general rule domain, while the third was from a different domain. Children were asked to identify the two like rule violations. It was expected that younger children would be less successful in correctly pairing the two like rules violations. Rasch analysis identified the developmental sequence inherent in these rules and provided detail of the previously indicated age trends in correctly discerning ascriptive and interpersonal rule violations from the other domains.

(HOME PAGE) (return to top of page)


Using common-person equating to estimate décalage across and within specific cognitive contents

Trevor G Bond, James Cook University, Australia

The lack of transfer or generalization of demonstrated cognitive abilities to other content domains is termed décalage in Piagetian theory. While critics regard the construct as a mere post hoc modification, one reasonable interpretation is that horizontal décalage remains a key Piagetian theoretical construct in need of widespread empirical investigation in order that Piagetian theory might be amplified to explain the phenomena more adequately. Smith & Knight (1992) describe the construction of three 16-item variants of the BLOTCH test (Bond's Logical Operations Test in the Context of History) where itemswith a common sense science content taken from the original BLOT (Bond, 1976) were transposed into the context of medieval English history. Rasch analysis of person performances across pairs of the tests by secondary school students in NW England (N=314) allowed differences in test and item difficulties to be estimated. Because each test represents instantiations of the 16 binary operations identified by Piaget in his structural account of formal operational thinking (Inhelder & Piaget, 1955/1958), direct empirical assessment of décalage (see Bond, 1996) between contents (science and history) and within a restricted content domain (BLOTCH I, II and III instantiates the identical 16 logical operations in three differing formats) is possible using common-person equating of parallel test forms.

(HOME PAGE) (return to top of page)


Mathematically Demonstrated Hierarchical Complexity of Tasks and Behavior Development Theory

Michael L. Commons, Department of Psychiatry, Harvard Medical School

Edward J. Trudeau, Department of Mathematics, Harvard University

Many developmental theories posit the existence of stages to explain how acquisition of behavior is sequenced. To find a consistent basis for stages, Commons and Richards (1984a; 1984b) define a minimal formal standard by which different systems of stages may be constructed or compared. Mathematically proving that "hierarchical complexity of tasks" forms a well defined system shows that stages exist independent of developmental theory. To define this system, the General Stage Model describes the axiomatic definition of simple tasks as logically primitive elements of the system. Relations defined on these elements recursively and linearly order them into chains of discriminative operants, each requiring an output. These linear orderings have a common property called the order of hierarchical complexity. Each defines a higher order task, its constituent elements being its prerequisites. Hierarchical task sequences are formed using these definitions.

(HOME PAGE) (return to top of page)


A Saltus Analysis of Developmental Data from The Balance Beam Task Series

Commons, M. L, Harvard Medical School

Goodheart, E. A., Harvard University

Dawson, T. L., University of California at Berkeley

Draney,K., University of California at Berkeley

We perform a Saltus analysis of cross-sectional developmental data gathered from a group of 121 adults and 20 children to whom we presented the Balance Beam Task Series (Commons, Miller, and Kuhn, 1982). A Saltus analysis (Wilson, 1989; Wilson & Draney, 1995) is a mixture model extension of the Rasch model first applied to intelligence tests (see Rasch [1960]) and later applied to the study of developmental data. Whereas a traditional Rasch analysis determines the probability of a given subject performing a given item in terms of item difficulty (delta) and subject ability (beta), a Saltus analysis introduces item and subject stage as a third concept. This additional concept (along with the parameter[s] that embody it) can help to determine whether the gapiness and systematic shifts in item misfit present in an earlier two-parameter Rasch analysis can be explained as stage change. Issues of fit and validity will be raised, and the results of the analysis will be discussed in terms of their general implications for developmental research as well as the more specific development of the Balance Beam instrument.

(HOME PAGE) (return to top of page)


Paper presentation

The Psychophysics of Stage: The General Stage Model analytically measures hierarchical task complexity and the Rasch Model statistically models corresponding stage of performance

Michael L. Commons, Harvard Medical School

Francis A. Richards,

Edward Trudeau, Harvard University

Theo Linda Dawson , University of California at Berkeley

Eric Andrew Goodheart, Harvard University

The assessment of developmental stage has been limited to measures of performance rather than an analysis of task demands. To remedy that lack, a behavior-analytic-compatible core-developmental notion of hierarchical complexity of tasks has been introduced. The hierarchical complexity of tasks and the stage of the response chain that complete them have three unusual properties. (1) If a task sequence requires a chain of responses, some responses in the chain may not only require precursors, but also are more hierarchically complex than others. (2) The higher-order task action is defined in terms of the less complex responses, and organizes them in a non-arbitrary way. (3) An action is at a given stage when it successfully completes a task of a given hierarchical order of complexity. The General Stage Model predicts that performance of items within a hierarchically ordered task sequence will have a corresponding Rash Model generated stage scores. Rasch analysis can be used to relate performance to hierarchical complexity, thereby creating a Psychophysics of stage.

(HOME PAGE) (return to top of page)

 
© Jean Piaget Society  •  Updated: January 30, 2007  •   Comments: webmaster@piaget.org