Purify: Finding Unidimensional Measurement Models
9.1 Introduction
Structural equation models in which each latent variable is measured by several indicators are often called multiple indicator models (e.g. Fig. 9.1).
Fig. 9.1: A Multiple Indicator Model
Let T be the set of latent variables and V the set of measured indicators. Such models can be divided into two parts, the structural model (Fig. 9.2) and the measurement model (Fig. 9.3).
Fig. 9.2: The Structural Model
The graph of the structural model contains the members of T and edges between members of T, and the graph of the measurement model contains all of the variables (T È V) and the edges not in the graph of the structural model.
Fig. 9.3: The Measurement Model
We say a measurement model is pure, or unidimensional if each indicator xi is the cause of no variable, is a direct effect of exactly one latent variable and an error term ei, and for every other error variable ej, ei and ej are uncorrelated.[1] In what follows we use the terms unidimensional and pure interchangeably. An indicator is impure if it is the cause of another variable in T È V, has a correlated error with another variable in T È V, or the effect of some variable in T È V other than the latent it is intended to measure. Note that an indicator is impure relative to a set of variables. For example, in Fig. 9.4(B), x4 is impure relative to set of variables that contains x5 (because x4 is an effect of x5), but not impure relative to a set of variables that does not contain x5.
Fig. 9.4: Pure and Impure Measurement Models
Model A in Fig. 9.4 is a pure measurement model, whereas model B is not. In model B, the indicators x1 and x2 are impure because their error terms are correlated, the indicators x4 and x5 are impure because x5 is a direct cause of x4, and x8 is impure because it measures both T2 and T3.
All parameterizations of a pure measurement model imply a variety of vanishing tetrad constraints on the observed covariances no matter what the structural model may be (Scheines, 1993). Thus testing for unidimensionality can be done before specifying the structural model (Anderson & Gerbing, 1988). The user inputs into Purify an initial pure measurement model and sample data (in the form of a correlation or covariance matrix, or raw data.) Purify tests the initial pure measurement model's fit to the data. If it does not fit the data, then Purify systematically eliminates measured variables from the initial pure measurement model until it is left with a smaller pure measurement model that does fit the data, or a model that can no longer be tested against the data.
For example, suppose that data on X = {x1, x2, ... , x12} are produced by a structural equation model with the impure measurement model shown in Fig. 9.4 B. If these data and the initially specified measurement shown in Fig. 9.4 A are given as input to Purify, it uses tetrad constraints to locate and discard a set of three indicators that are impure in the generating model. The set is not unique, and if any of the sets: {x1, x5, x8}, {x1, x4, x8}, {x2, x5, x8}, or {x2, x4, x8} are removed, the resulting measurement model among the remaining variables is pure. If the set {x1, x5, x8} is discarded, then Purify outputs the pure measurement model in Fig. 9.5.
Fig. 9.5: Purified Measurement Model
In the actual model among only these indicators, all are actually pure.
9.2 Purify's Input and Output
Purify takes as input:
• covariance data or continuous raw data for a set of indicators V, and
• an initial measurement model specified as pure.
It gives as output:
• a pure submodel of the initially specified measurement model in which each latent is measured by at least three indicators, and which fits the data as closely as possible (in the sense that the difference between the set of vanishing tetrad difference entailed by the submodel and those judged to hold in the population is as small as possible).
Pure and Almost Pure
If T is a set of latent variables and V is a set of measured variables, a measurement model is almost pure if each measured variable has a latent parent, each latent variable has at least one measured child, and the graph of the model entails that if xi is an indicator of T, then xi is independent of all other variables in T È V conditional on T. An almost pure measurement model entails the same set of vanishing tetrad differences among the variables in V as does a pure measurement model, so Purify cannot distinguish between an almost pure measurement model and a pure measurement model. For example, Purify cannot distinguish between the two measurement models in Fig. 9.6. Hence it would output the pure measurement model, even if the actual model is only almost pure. If Purify identifies a set of indicators such that all tetrad constraints that are entailed by a pure measurement model for those indicators hold, then at most one indicator per latent can be almost pure and not pure.
Fig. 9.6
Purify attempts to find a submodel of the initially specified model that is pure and fits the data, but this is not always possible because there might not be a unidimensional measurement model with at least three indicators per latent which fits the data. If Purify locates a submodel in which each latent has at least three indicators, and the tetrad constraints implied by such a pure measurement model hold in the population, then under Purify's general assumptions every indicator in such a submodel is guaranteed to be almost pure in the actual model among just those indicators (Scheines, 1993).
9.3 An Example
Suppose we are interested in the relationships between three latent variables: T1, T2, and T3. We believe that T1 has an influence on T2, and T2 on T3, but we are unsure of whether T1 has any influence on T3 that is not mediated by T2. We might build a measurement model for each of the latents, and decide to settle the issue by specifying a full model (Fig. 9.7) and performing a statistical test on b13, the parameter representing the direct dependence of T1 on T3.
Fig. 9.7: Model Specified for Estimating T1 ® T3
If b13 = 0, that is, T1's influence on T3 is entirely mediated by T2, but the actual measurement model is not unidimensional as we have specified in Fig. 9.7, but is instead the one we show in Fig. 9.8, then the estimate of the direct dependence of T3 on T1 would be biased, and we would in many cases mistakenly conclude that there is a direct effect of T1 on T3 not mediated by T2.
Fig. 9.8: True Generating Model
To illustrate this problem and show how Purify can help solve it, we arbitrarily parameterized the model in Fig. 9.8 and generated a pseudo-random multivariate normal sample (n = 2,000), the covariance matrix of which is in the file pure1.dat. When we estimated the model specified in Fig. 9.7 on this data, the overall model did not fit well, but the parameter representing T1 ® T3 was 1.85 times the size of its standard error, which means it is significant at approximately the .1 level. Several options exist at this point. We could use a variety of heuristic procedures[2] for respecifying the model by freeing additional parameters (adding extra edges), or we can use Purify to eliminate impure indicators and then either employ the parameter estimation strategy mentioned previously or use MIMbuild to examine whether T1 and T3 are adjacent. To run Purify we specified our initial guess at a measurement model, here shown in Fig. 9.9 and contained in the file pure.imm.
Fig. 9.9: Initially Specified Measurement Model
We then ran Purify to locate and discard indicators that are actually impure (session 9.1).
Session 9.1: Running Purify
************************************************
For
help, type "help"
Initializing
Data Structures
>input
Input
File: pure1.dat
>input
Input
File: [pure1.dat]pure1.imm
>purify
Output
file: pure1.out
PURIFY:
A
Program to Find Unidimensional Measurement Models
Run
PURIFY interactively (i), or in batch (b) ? [b] <CR>
Reading initial measurement model.
>exit
************************************************
Purify's output for this case (pure1.out) is as follows (Fig. 9.10).
############## pure1.out
###############
PURIFY:
A Program to Find Unidimensional Measurement Models
Initially Specified Measurement Model
Indicators of T1: [ x1 x2 x3 x4 x5 x6 ]
Indicators of T2: [ x7 x8 x9 x10 x11 ]
Indicators of T3: [ x12 x13 x14 x15 x16 ]
INTRA-CONSTRUCT PHASE.
----------------------
T1 --
Original Status: 12 of 45 tetrads fail the Bonferroni test.
Dropped
x1 Without it, 0 of 15 tetrads fail the
Bonferroni test.
T2 --
Original Status: Needs NO
pruning.
0 of 15
tetrads fail the Bonferroni test.
T3 --
Original Status: 1 of 15 tetrads
fail the Bonferroni test.
Dropped
x13 Without it, 0 of 3 tetrads fail the
Bonferroni test.
CROSS-CONSTRUCT PHASE.
----------------------
Iteration 1
102 of 660 tetrads fail the
Bonferroni test.
Dropped
x12 Without it, 60 of 510 tetrads fail
the Bonferroni test.
Iteration 2
60 of 510 tetrads fail the Bonferroni test.
Dropped
x7 Without it, 0 of 333 tetrads fail
the Bonferroni test.
--------------------------------------------------------------
Output Measurement Model
Indicators of T1: [ x2 x3 x4 x5 x6 ]
Indicators of T2: [ x8 x9 x10 x11 ]
Indicators of T3: [ x14 x15 x16 ]
ALL tetrad equations implied by this measurement
model
pass the Bonferroni test.
--------------------------------------------------
}
/graph
T1 x2
T1 x3
T1 x4
T1 x5
T1 x6
T2 x8
T2 x9
T2 x10
T2 x11
T3 x14
T3 x15
T3 x16
############## pure1.out
###############
Fig. 9.10: pure1.out
Purify goes through several stages of indicator pruning, each corresponding to a different type of tetrad constraint. In the INTRA-CONSTRUCT stage, it considers only tetrad constraints among the indicators of a single latent. In the CROSS-CONSTRUCT stage, it considers tetrad constraints among 3 indicators from one latent and 1 from another. Here we ran Purify in batch mode, in which case it terminates a stage when the set of tetrad equations implied by any parameterization of a pure measurement model pass a Bonferroni test. The Bonferroni test, suggested to us in Bollen (1990), is a heuristic statistic for simultaneously testing multiple tetrad equations. See Appendix A
If Purify is run in interactive mode, then the user is asked at each stage whether Purify should continue pruning or terminate. In the case given, Purify's output model is pure and we use it to specify the model in Fig. 9.11 in order to test the same hypothesis concerning T1 ® T3. In this case the overall model fits reasonably well (c2 = 58, d.f. = 51) and b13 is indistinguishable from 0.
Fig. 9.11
Now that we have a pure measurement model, another way to test the hypothesis concerning a direct T1 ® T3 connection is with MIMbuild, which is described in chapter 10.
9.4 Running Purify Interactively
In the preceding example we ran Purify in batch mode, in which case it terminates a pruning stage when none of the tetrad equations tested in the stage fail the Bonferroni test. Although it performs well in simulations, the Bonferroni test is heuristic. Further, like any other statistical decision procedure it is sensitive to outliers. For example, suppose that out of 500 tetrad equations implied by the originally specified measurement model, 200 fail the Bonferroni test. Suppose further that after two indicators are removed only 2 equations fail out of 192. One of these equations might still fail even after seven more indicators are removed. It does not seem worth eliminating several indicators in order to prevent the measurement model from implying a single false tetrad equation. We therefore provide the option of running Purify in interactive mode, in which case a pruning stage can be terminated manually at any time.
Suppose in the graph of Fig. 9.8 we eliminate indicator x16 from the generating and initial measurement model. Data from a different arbitrary parameterization of this submodel of Fig. 9.8 is stored in the file pure1a.dat. Upon feeding this data and a submodel of the initial measurement model in Fig. 9.9 (pure1a.imm) to Purify, we get the output in pure1a.out (Fig. 9.12).
################## pure1a.out
##############
PURIFY:
A
Program to Find Unidimensional Measurement Models
Initially
Specified Measurement Model
Indicators
of T1: [ x1 x2 x3 x4 x5 x6 ]
Indicators
of T2: [ x7 x8 x9 x10 x11 ]
Indicators
of T3: [ x12 x13 x14 x15 ]
INTRA-CONSTRUCT
PHASE.
----------------------
T1 --
Original Status: 11 of 45 tetrads fail the Bonferroni test.
Dropped
x1 Without it, 0 of 15 tetrads fail the
Bonferroni test.
T2 --
Original Status: Needs NO
pruning.
0 of 15
tetrads fail the Bonferroni test.
T3 --
Original Status: Only 4
indicators, needs NO pruning.
0 of 3 tetrads fail the Bonferroni test.
CROSS-CONSTRUCT PHASE.
----------------------
Iteration 1
107 of 660 tetrads fail the
Bonferroni test.
Dropped x12
Without it, 64 of 510 tetrads fail the Bonferroni test.
Iteration 2
64 of 510 tetrads fail the Bonferroni test.
Dropped x7
Without it, 16 of 333 tetrads fail the Bonferroni test.
Iteration 3
16 of 333 tetrads fail the
Bonferroni test.
Dropped
x6 Without it, 3 of 192 tetrads fail
the Bonferroni test.
Iteration 4
3 of 192 tetrads fail the Bonferroni test.
Dropped
x4 Without it, 2 of 114 tetrads fail
the Bonferroni test.
Iteration 5
2 of 114 tetrads fail the Bonferroni test.
Dropped
x8 Without it, 2 of 54 tetrads fail the
Bonferroni test.
Iteration 6
2 of 54 tetrads fail the Bonferroni test.
No
foursomes left
--------------------------------------------------------------
Output Measurement Model
Indicators of T1: [ x2 x3 x5 ]
Indicators of T2: [ x9 x10 x11 ]
Indicators of T3: [ x13 x14 x15 ]
2 out of 81 tetrad equations implied by this
measurement model
fail the Bonferroni test.
--------------------------------------------------
}
################## pure1a.out ##############
Fig. 9.12: pure1a.out
After dropping x6 in iteration 3 of the 3x1 phase, Purify is suffering from diminishing returns. Session 9.2 shows Purify run on the same data and initial measurement model, but interactively.
Session 9.2: Running Purify interactively
************************************************
>input
Input
File: pure1a.dat
>input
Input
File: [pure1a.dat]pure1a.imm
>purify
Output
file: pure1b.out
PURIFY:
A
Program to Find Unidimensional Measurement Models
Run
PURIFY interactively (i), or in batch (b) ? [b] i
Reading
initial measurement model.
T1 --
Original Status: 11 of 45
tetrads fail the Bonferroni test.
Continue? (y/n)
[y] <CR>
Dropped
x1 Without it, 0 of 15 tetrads fail the
Bonferroni test.
T2 --
Original Status: Needs NO
pruning.
0 of 15
tetrads fail the Bonferroni test.
T3 --
Original Status: Only 4 indicators, needs NO pruning.
0 of 3
tetrads fail the Bonferroni test.
CROSS-CONSTRUCT PHASE.
----------------------
Iteration 1
107 of 660 tetrads fail the
Bonferroni test.
Continue? (y/n)
[y] <CR>
Dropped
x12 Without it, 64 of 510 tetrads fail
the Bonferroni test.
Iteration 2
64 of 510 tetrads fail the Bonferroni test.
Continue? (y/n)
[y] <CR>
Dropped
x7 Without it, 16 of 333 tetrads fail the Bonferroni test.
Iteration 3
16 of 333 tetrads fail the Bonferroni test.
Continue? (y/n)
[y] <CR>
Dropped
x6 Without it, 3 of 192 tetrads fail
the Bonferroni test.
Iteration 4
3 of 192 tetrads fail the Bonferroni test.
Continue? (y/n)
[y] n
>exit
************************************************
In this case we answered "n" to Purify's query after iteration 4, leaving us with a larger measurement model that can then be input to MIMbuild or used to estimate the correlations among the latent variables.
If, after Purify's final iteration, a substantial number of tetrad differences fail the Bonferroni test, this is an indication that the output model does not fit the data, and should not be used as input to MIMbuild.
9.5 Syntactic Requirements of Input
Purify's syntactic input requirements are the same as MIMbuild's. Both require that in the measurement model input, each latent variable have at least two indicators, and that every indicator measure some latent. They also prohibit edges from an indicator to a latent. Violating these conditions will prevent Purify or MIMbuild from starting an analysis, but the program will produce error messages. For example, if the measurement model in Fig. 9.13 is input (pure2.dat) it will produce the error messages we show in session 9.3.
Fig. 9.13: Illegal Purify Input in pure2.dat
Session 9.3
************************************************
>input
Input
File: pure2.dat
Converting
covariance matrix to correlation matrix.
>purify
Output
file: pure2.out
PURIFY:
A
Program to Find Unidimensional Measurement Models
Run
PURIFY interactively (i), or in batch (b) ? [b] <CR>
Reading
initial measurement model.
Error
in initial measurement model:
T1
must have at least 2 indicators, but has only 1.
Error
in initial measurement model:
x5
measures more than one latent: T2 and T3
Error
in initial measurement model:
x8
measures no latent.
Please
correct these errors before you run PURIFY.
>exit
************************************************
One can input measurement models that satisfy these conditions but that are not unidimensional. In these cases MIMbuild and Purify will use only that part of the input model that contains a unidimensional measurement model and will ignore the rest. In particular it will ignore:
1. Edges from indicators to latent variables.
2. Edges from indicators to other indicators.
3. Edges from latent variables to other latent variables.
For example, suppose the measurement model in Fig. 9.14 is typed into the file pure3.dat and input to Purify.
Fig. 9.14: The Measurement Model Input
If given this model, Purify will actually use the measurement model shown in Fig. 9.15, and produce the error messages we show in session 9.4.
Fig. 9.15 The Measurement Model Analyzed
Session 9.4
************************************************
>input
Input
File: pure3.dat
Converting
covariance matrix to correlation matrix.
>purify
Output
file: pure3.out
PURIFY:
A
Program to Find Unidimensional Measurement Models
Run
PURIFY interactively (i), or in batch (b) ? [b] <CR>
Reading
initial measurement model.
- Ignoring measured-measured edge: x7 ->
x8
- Ignoring measured-latent edge: x5 -> T3
- Ignoring latent-latent edge: T1 -> T2
>exit
************************************************
9.6 Purify's Assumptions
Purify assumes that the data was generated by a linear structural equation model with normally distributed error terms, and that the population distribution is linearly faithful to model that generated it, that is, each of the tetrad differences that vanish in the population are entailed to vanish for all values of the linear parameters of the model. Purify also assumes that the edges in the initially specified measurement model exist in the generating model. That is, if an indicator x measures (is directly caused by) a latent T in the measurement model you specify initially, then x must actually measure T. Purify does not assume that relationships left out of the initially specified measurement model do not exist in the generating model, however. The assumption that every x actually measure its latent is one that can often be established reliably by substantive knowledge. For example, few would argue that an indicator consisting of some measure of agreement with the statement: "I like to going to work" measures the latent variable: "Job Satisfaction." Purify also assumes that indicators do not cause latent common causes of any of the measured variables, and that the relations among the latent variables are recursive, but it does not assume causal sufficiency among the latents, that is, that there are no unspecified latents that are themselves common causes of specified latents.
9.7 An Empirical Example
In chapter 1 we presented an example from Bollen (1980) in which a number of measures of political democracy were hypothesized as unidimensional indicators of a common feature of societies. Using measures of press freedom (pf), freedom of group opposition (fg), government sanctions (gs), fairness of elections (fe), executive selection (es), and legislature selection (ls), Bollen considered the linear factor model in Fig. 9.16.
Fig. 9.16
Using LISREL, Bollen estimated the model and found that several of the indicators were not pure. Instead of attempting to locate and discard the impure indicators, Bollen elaborated his original model by freeing parameters which represent correlated errors, or other latent common causes of the measured variables (fig. 9.17).
Fig. 9.17
When estimated with EQS, this model has a c2 of 6.009 based on 6 degrees of freedom, with the probability value for the c2 statistic = 0.42218. Although Bollen's final measurement model of democracy fits the data well, it is not unidimensional. To find a unidimensional submodel, we can run Purify on Bollen's original model and data.
Session 9.5
************************************************
>input
Input
File: bollen.dat
>purify
Output
file: bollen.out
PURIFY:
A
Program to Find Unidimensional Measurement Models
Run
PURIFY interactively (i), or in batch (b) ? [b] <CR>
Reading
initial measurement model.
>exit
************************************************
The output is as follows (bollen.out).
############## bollen.out ##########
PURIFY:
A Program to Find Unidimensional Measurement Models
Initially Specified Measurement Model
Indicators of T: [ pf fg gs fe es ls ]
INTRA-CONSTRUCT PHASE.
----------------------
T --
Original Status: 3 of 45 tetrads fail the Bonferroni test.
Dropped
fg Without it, 0 of 15 tetrads fail the
Bonferroni test.
CROSS-CONSTRUCT PHASE.
----------------------
--------------------------------------------------------------
Output Measurement Model
Indicators of T: [ pf gs fe es ls ]
ALL tetrad equations implied by this measurement
model
pass the Bonferroni test.
--------------------------------------------------
}
/graph
T pf
T gs
T fe
T es
T ls
############## bollen.out ##########
Fig. 9.18: bollen.out
Running the resulting model through EQS yields a c2 = 10.103 based on 5 degrees of freedom, with a probability value for the c2 statistic = 0.07236. Such a marginal fit indicates that perhaps Purify left in some marginally impure indicators Purify recognizes an indicator as impure when it occurs in a tetrad difference that is entailed to vanish, but is judged not to vanish in the population. Making it easier to reject a tetrad equation makes it easier to eliminate an impure indicator. Running Purify at a higher significance level causes it to discard more (or at least as many) indicators as running Purify at a lower significance level. Running Purify with a significance level of .2 gives the following output, which is in the file bollen.o2:
################## bollen.o2 #################
T --
Original Status: 6 of 45 tetrads fail the Bonferroni test.
Dropped fg Without it, 1
of 15 tetrads fail the Bonferroni test.
Dropped ls Without it, 0
of 3 tetrads fail the Bonferroni test.
-----------------------------------------------------------
Output Measurement Model
Indicators of T: { pf gs fe es }
################## bollen.o2 #################
Fig. 9.19: bollen.o2
Running the resulting model through EQS yields a c2 = 1.687 based on 2 degrees of freedom, with a probability value for the c2statistic = 0.43013. .
Fig. 9.20: Purified Measurement Model of Democracy
9.8 Simulation Studies
Here we report on simulation studies that give a sense of Purify's reliability on linear, multinormal samples ranging from n = 100 to n = 2,500. In each study we randomly generated causal structures, randomly parameterized them, and then took 10 samples from each population described. We then applied Purify to each sample at three significance levels, automatically recording its performance on a number of dimensions. We began from the following pure measurement model (fig. 9.21), which we will call skeleton 1.
Fig. 9.21: Skeleton 1
We randomly created a causal graph from skeleton 1 in two stages. To build the structural model, for each pair of latent variables Ti and Tj we pseudo-randomly added an edge from Ti to Tj (where i < j) with a probability of 1/3. Thus there were an average of 5 edges in the structural model (e.g., Fig. 9.22).
Fig. 9.22: Randomly Generated Structural Model
We then randomly added connections that created impurities in the measurement model. We gave each indicator xi a 25% chance of measuring some latent besides the one it measures in skeleton 1, and also gave each xi a 25% chance of having some source of covariance with xj besides the latent parents of xi and xj, respectively. The model in Fig. 9.23 is an example of such a generating model.
Fig. 9.23: Generating Model
Purify decides which variables to remove based on a model's Tetrad-score (explained in chapter 8, section 2.) The Tetrad-score is affected by two parameters that the user can control, the significance level and the weight. We have found through simulation tests that the weight does not significantly affect Purify's choices, so we leave it at its default value. The following tests show how reliability varies with significance level and sample size. From each such generating model we pseudo-randomly selected the linear coefficients from a uniform distribution with a lower limit of .5 and an upper limit of 1.5. Each exogenous variable was given a standard normal distribution. We then drew 10 samples at each of sample sizes 100, 250, 500, 1,000, and 2,500. Giving skeleton 1 as the initial measurement model, we ran Purify 3 times on each sample by varying the settings of the significance level. The significance level was one of: {0.01, 0.05, 0.2}.
For each Purify run, we calculated the percentage of pure indicators Purify mistakenly removed (omitted), and the percentage of impure indicators it mistakenly failed to remove (committed). In the two graphs that follow, we plot the sample size on the x-axis and the level of error on the y-axis. The three plots in each graph correspond to three different significance levels used, coded B for big (.2), M for medium (.05), and S for small (.01). Each corresponds to a weight = 1.0.
Fig. 9.24
Fig. 9.25
Because an error of commission means that an impure indicator was left in the measurement model, errors of commission are much worse than errors of omission if the goal is reliable causal inference among the latent variables. As you can see from Fig. 9.24, Purify is quite reliable at sample sizes at or above 250.
In the Monte Carlo simulations all of the assumptions made by Purify were satisfied exactly, so these results should be viewed as upper limits to the reliability of Purify applied to real data.
In the following chapter we extend these simulation studies to illustrate the reliability of Purify and MIMbuild in sequence.