Introduction
Statistical Models and Causality
Statistical models have at least two roles in science. One is to describe the joint probability distribution that governs a population, and another is to model the causal mechanisms that lie behind statistical dependencies. A description of the joint distribution allows us to calculate conditional distributions, which in turn make prediction possible. But as many writers have realized (Holland, 1986; Pratt & Schlaifer, 1988; Robins, 1986; Rubin, 1974, 1977), calculating the distribution of y conditional on x = x, that is, P(y|x=x), is not the same as calculating the distribution of y after we intervene to force x to equal the value x, unless the causal structure connecting x and y is of a certain sort. For example, from a joint distribution over the smoking habits, finger colors, and lung cancer rates of American men we can calculate the conditional probability of a man getting lung cancer given that his fingers have no trace of nicotine stains. But this is clearly not the distribution that we should use to calculate the probability of a man getting lung cancer given that he has just washed his hands until they are unstained. If it were, then the most sensible social policy to combat lung cancer would require everyone to wash their hands regularly with nicotine solvent soap.
In the last decade considerable progress has been made in characterizing the statistical signature of causal structure, and this work has made it possible in some circumstances to estimate from sample data features of the causal structure that govern a population. (See Spirtes, Glymour, and Scheines 1993 and Pearl 1988 for descriptions and brief histories of this research.) TETRAD 3 is a set of computational tools to help with such inferences. The program provides procedures that can be combined with the investigator's substantive knowledge to explore alternative explanations of sample data. All but one of these procedures (the Search procedure) also come with a theoretical guarantee: If general assumptions (explained in more detail in Chap. 2) are satisfied and the substantive assumptions made by the user are correct then the output of the search procedures will be correct as long as the judgments made about certain statistical constraints in the population are correct.
What TETRAD 3 Does
Statistical causal model building has a number of stages. One must select variables to measure and a sample to measure them on, actually gather the data, explore the data and clean it up, specify a model, estimate the model's parameters, test the model's overall fit and perhaps respecify it if the fit is inadequate, and consider other plausible alternative explanations of the data.
In many cases, the standard recommendation for model specification is to employ theory, and where theory is informative and well-supported this is surely what should be done. But if the theoretical knowledge available does not entail a unique model specification, then TETRAD 3 can help search the space of models over which the scientist is uncertain. TETRAD 3 is not meant to replace good scientific intuition with blind computation; there are several ways to constrain the program so that it searches only among models deemed theoretically plausible. TETRAD 3 helps search for statistical models that explain the data and are compatible with user-entered background knowledge.
TETRAD 3 currently handles two sorts of statistical models: recursive linear structural equation models with variables that are jointly normally distributed, and Bayesian networks with discrete variables,[1] which for many purposes, provide an alternative to logistic regression and to log-linear models. We describe each class of models in the next chapter.
For recursive linear structural equation models, TETRAD 3 does not include many of the estimation and testing functions found in standard statistical analysis packages. Once TETRAD 3 suggests a set of models, these models can be submitted to other statistical programs such as LISREL, EQS, or CALIS for estimation and testing. However, for discrete variables we have included an Estimate module, which calculates maximum likelihood estimates of the parameters of a discrete Bayesian network.
An Overview of TETRAD 3
TETRAD 3 has 13 modules, each of which has its own chapter later in this manual. They are:
• Build • Purify • Makemodel
• Estimate • MIMbuild • Monte
• Update • Search •
STATwriter
• Tetrads • Regress •
Cequiv • Gibbs
Build, Purify, MIMbuild and Search are the main inference modules. Using appropriate statistical tests, Build makes a series of decisions about independence and conditional independence relations among measured variables. This information is used to search for a class of models that cannot be distinguished by conditional independence relations alone, and that are compatible with user-entered background knowledge. Purify, MIMbuild, and Search work on normally distributed linear structural equation models with latent variables, and use substantive assumptions provided by the user together with statistical decisions about vanishing correlations and tetrad differences to estimate features of the causal structure.
If the variables modeled are all discrete, then the Estimate module can calculate maximum likelihood estimates of the parameters of a Bayesian network that contains no latent variables, and the Update module can use these estimates to calculate conditional distributions. If the variables are continuous, then Regress will regress one variable on a set of other variables that the program chooses, and Gibbs provides parameter estimates for linear models.
Makemodel and Monte provide a useful Monte Carlo simulation package. Makemodel takes an unparameterized causal structure and, after prompting the user for an interpretation of the structure as either a Bayesian network or a recursive linear structural equation model, parameterizes the structure and writes a fully parameterized statistical model to a TETRAD 3 readable file. Monte can then be used to generate samples of any size from such a model. Cequive takes two directed cyclic graphs as input, and determines whether they are statistically equivalent.
Although TETRAD 3 does not test linear models, it will automatically write out input files to three popular packages that do: LISREL, EQS, and CALIS. The STATwriter module takes a path diagram and covariance or raw data and will automatically write out an input file to the package of your choice.
Problems TETRAD 3 Can Help Solve
If the general modeling assumptions are approximately satisfied, TETRAD 3 can help with the following problems, among others:
• Recognizing when a data set provides little or no information about the processes
that generated it.
• Finding statistically equivalent alternatives to a given model.
• Finding alternatives that explain the data equally well or better than a given model.
• Selecting regressor variables that directly influence an outcome variable.
• Reducing the number of regressors needed for prediction.
• Finding causal models that are not regression models.
• Detecting the existence of latent variables.
• Locating unidimensional measurement models.
• Finding structural equation models with latent variables.
• Respecifying a given linear structural equation model.
• Estimating the reliability of a model specification search.
• Building a Bayesian network for use as an expert system for a data base.
• Using a Bayesian network to classify and predict.
• Generating simulated data from causal models.
Selecting Causal Regressors
If the purpose of a regression analysis is to estimate the individual causal influences each regressor exerts on the outcome, then full regression or best subsets procedures are both theoretically unreliable.[2] Even on population data that perfectly satisfy general distributional assumptions, the solution to a multiple regression equation can include non zero coefficients for variables that are not direct causes of the outcome. Under similar assumptions the Build module can often do better (see Scheines, 1994b). Sometimes, depending upon the true causal structure and which variables have been measured, it can identify which variables are direct causes and which are direct effects. Thus, when the distributional assumptions for linear regression are justified, Build can be used to help select causal regressors, whose precise influence can then be estimated with any regression package.
The following case may help to illustrate some differences between linear regression and the Build procedure, when the two are used for identifying causal regressors. Incoming U.S. military personnel are given batteries of tests that are used for a variety of personnel decisions. The variables in this case are:
afqt composite score
no numerical operations
wk word knowledge
ar arithmetic reasoning
ei electronics information
gs general science
mc mechanical comprehension
mk mathematical knowledge
Through 1987, the afqt variable was calculated by taking a linear combination of no, wk, ar, and other variables not included in this list. In the causal process that generated the sample, individual scores on no, wk, ar and the other variables not represented in this list did actually cause the afqt score assigned to each individual in the sample. Further, afqt is a linear function of no, wk, ar, and of other variables that may be represented by an error term e. Of course, in reality e may be correlated with any or all of the variables in the preceding list, which are also correlated with each other. So a correct causal description of the process that generated the data is given by the equation:
afqt = b1 no + b2 wk + b3 ar + e (1)
where b1, b2, b3 are unknown linear coefficients.
Suppose we did not know which of the tests actually influenced, or were components of, the afqt score, and we wanted to try to determine that information from the sample statistics. Regressing afqt on the other seven test scores gives the following equation, with the standard errors and t-statistics included:
afqt = .242*no
+ 1.175*wk + 1.031*ar - .244*mk
(.022) (.029) (.028) (.028)
10.805 40.486 36.397 -8.667
-.031*ei + .030*mc -.132*gs + e
(.024) (.024) (.029)
-1.301 1.271 -4.583
All of the regressors are significant at the .01 level except ei and mc. The coefficients on mk and gs are significant but negative, as they would be if mk and gs were components of afqt and negatively correlated with the unreported components of afqt, or equivalently with the error term in Equation 1, matters that cannot be decided from the data alone. Thus, without further knowledge, regression allows that mk and gs may influence afqt. We can run the same covariance data through TETRAD 3's Build module with the input file afqt.dat (shown below). The input file, and all files TETRAD 3 handles, are simple text files and can be created in any text editor.
The /covariance section of the input file begins with the sample size, includes a list of the measured variables, and finally lists the lower triangle of the covariance matrix. The /knowledge section specifies that afqt is assumed not to influence the other variables, and so it plays a role analogous to the dependent variable in a regression.
############# afqt.dat ################
/covariance
6224
afqt no wk ar mk ei
mc gs
253.9850
29.6490 51.7649
60.3604 6.2931 41.9670
57.6566 14.5143 16.0226
40.9329
29.3763 18.2701 13.2055
20.6052 40.7386
36.2318 2.1073 22.6958 16.3664 12.1773 63.1039
35.8244 4.4553 17.4155 20.3952 16.4590 35.1981 62.9647
38.2510 5.6151 27.1492 14.7402 14.8442 29.9095 26.6842 48.93
/knowledge
addtemporal
1 no wk ar mk ei mc gs
2 afqt
#############
afqt.dat ################
Running Build is simple and goes as follows (session 1.1).[3]
Session 1.1
******************************************************************
C:\TETRAD\RELEASE>tetrad
TETRAD 3 - Version
1.2 for DOS
by
Peter Spirtes, Richard Scheines,
Christopher Meek, and Clark Glymour
Copyright (C) 1994 by Lawrence Erlbaum Associates
For help, type
"help"
Initializing Data Structures
>input
Input File: afqt.dat
Converting covariance
matrix to correlation matrix.
>output
Output file: afqt.out
>build
Assume latent common
causes? [NO]: yes
Use the exact algorithm?
[YES]: <CR>
>exit
C:\TETRAD\RELEASE>
******************************************************************
Instead of outputting a single causal model, Build outputs a set of causal structures among which it cannot distinguish. In all of these structures, no, wk, and ar and only no, wk, and ar are connected directly to afqt, and the Build procedure correctly and uniquely identified the components of the afqt score. It may be relevant that at the time the original TETRAD study was first done, we had been incorrectly informed that all seven tests are components of afqt, and the TETRAD results prompted us to inquire whether that information was correct.
Building Causal Structures
In some modeling contexts knowing the full causal structure is important, and in these cases the number of theoretically plausible structures is often enormous. Again, the Build module can sometimes help.
A paper by Rodgers and Maranto (1989) considered hypotheses about the causes of academic productivity drawn from sociology, economics, and psychology, and produced a combined model that they claim is theoretically satisfactory and fits the data well.
The data were obtained from solicitations and questionnaires sent to 932 members of the American Psychological Association who obtained doctoral degrees between 1966 and 1976 and were currently working academic psychologists. Equal numbers of male and female psychologists were sampled, and after deleting respondents for a variety of reasons, a sample of 86 men and 76 women was obtained. Forming scales in a way we do not describe, they consider seven variables:
ab ability
sex gender
gpq graduate program quality
pre publications in graduate school
qfj quality of first job
pubs publication rate
cites citation rate
Rodgers and Maranto considered no fewer than six different models obtained from the social science literature, all of which are rejected by an asymptotic c2 test. By various heuristics, they finally obtained a linear structural equation model that fits the data well.
To illustrate what TETRAD 3 can do with this problem, we follow the general assumptions made by Rodgers and Maranto, namely that the variables are linearly related, and that there are no latent common causes of the recorded variables; of course we are in no position to endorse these assumptions for the particular data set. We first create the input file pub.dat, which contains correlation data and time order information specifying that publication rate during and after first job does not cause quality of first job, none of these cause either quality of graduate program or publication in graduate school, and so on.
############# pub.dat ###############
/Covariance
162
ab gpq pre
qfj sex cite
pubs
1.00
.62 1.00
.25 .09
1.00
.16 .28
.07 1.00
-.10 .00 .03
.10 1.00
.29 .25
.34 .37 .13
1.00
.18 .15
.19 .41 .43
.55 1.00
/knowledge
addtemporal
1 sex
2 ab
3 gpq pre
4 qfj
5 pubs
6 cite
############# pub.dat ###############
Having created this simple input file, we use TETRAD 3's Build module to search among the possible linear structural equation models for these measured variables that are consistent with the temporal information given and with statistical patterns the program finds in the data. Because of the temporal information given, in this case the class of models output by Build contains only a single model. We show both TETRAD 3's suggested specification and the Rodgers and Maranto model in the form of a path diagram or directed acyclic graph in Fig. 1.1.
Fig. 1.1
Virtually all of the Rodgers and Maranto model specification is produced instantaneously from the data and common sense knowledge of the domain (the time order of the variables). The edge omitted by Build (ab ® pubs) is associated with one of the three smallest linear coefficients in the estimated model. TETRAD 3's STATwriter module can turn either of these model specifications into an input file for the EQS, LISREL or CALIS programs, which will estimate the free parameters and test the corresponding linear model. An asymptotic c2 test cannot reject either the Rodgers and Maranto model or the TETRAD 3 model.
Detecting the Existence of Latent Variables
Some statistical procedures require a user either to assume that all statistical associations among measured variables are due to unmeasured variables, as in factor analysis, or that none are, as in path analysis. In reality, an investigator rarely knows that either case applies, and probably in many studies both assumptions are false. Instead, the associations among some measured variables are due entirely to a direct influence of one measured variable on another, and other associations are due to unmeasured common causes, and still others are due to both, and the investigator may have very little knowledge of which cases apply to which variables. So far as we know, there exists no conventional statistical package that offers any reliable guidance in deciding whether and how latent variables should be introduced to explain the data. Here we illustrate one of two ways in which TETRAD 3 can do so. The other is discussed in Chap. 5, section 5.
Consider a hypothetical study of the influence of variables thought to be related to smoking on breathing disorders. Suppose the variables measured are:
smo smoking behavior
in income
psmo parent's smoking behavior
cd cilia damage
lung lung disease
hd heart disease
mbd measured breathing dysfunction
From a linear system with normal variates the following correlation matrix was obtained by Monte Carlo simulation for a sample of 10,000 units, and is stored in the file lung.dat:
############# lung.dat ###############
/covariance
10000
hd inc smo psmo cd
lung mbd
1.000
0.00502 1.000
0.00615 0.74285
1.000
-0.01321 0.00673 0.40033
1.000
0.23670 0.48860
0.66530 0.27104 1.000
0.42836 0.43456
0.59130 0.23602 0.88176
1.000
0.75969 0.31147
0.42250 0.16231 0.74575
0.88274 1.000
############# lung.dat ###############
Fig. 1.2: lung.dat
The question is what sort of causal structure generated the data. With this input, and no other information, the relevant part of the output file (lung.out) obtained from Build is shown in Fig. 1.3:
############# lung.out ###############
NOT assuming causal sufficiency
The Partially Oriented Inducing Path Graph (POIPG):
hd <-> cd
hd <-> lung
hd --> mbd
inc o-> smo
psmo o-> smo
smo --> cd
cd --> lung
cd --> mbd
lung --> mbd
############# lung.out
###############
Fig. 1.3: lung.out
Drawn graphically, these connections look like Fig. 1.4:
Fig. 1.4
We will not go into the complete interpretation of this kind of graph here (for more details see Chap. 5) but from the output it is possible to tell that in the hypothetical model that generated the data: smoking is a direct cause of cilia damage and an indirect cause of lung capacity and measured breathing dysfunction, but not a cause (either direct or indirect) of heart disease, income, or parents' smoking habits. Also there are latent common causes of cilia damage and heart disease, and of lung capacity and heart disease.
The covariance data were actually generated from the following linear structure, where Ep and Gen are latent variables (to fit in with the story, call them "Environmental Pollution" and "Genotype" respectively):
hd = 0.987*Ep + 1.352*Gen + e1
smo = 1.354 * inc + 0.726*psmo + e2
cd = 0.606*smo + 0.800*Ep + e3
lung = 1.498*cd + 0.900*Gen + e4
mbd = 1.279*hd + 0.511*cd + 0.934*lun + e5
The error variables and exogenous variables were uncorrelated and had zero means and unit variance. The causal structure of this model can be represented by the graph in Fig. 1.5. Comparing Fig. 1.4 and 1.5, you can see that Build uncovered almost everything about the generating structure except the names of the latent variables.
Fig. 1.5
Of course in many applications the output of the Build module will not be so informative, but it will always be obtained through a procedure that is asymptotically correct given the general assumptions of the program (as long as the judgements it makes about conditional independence are correct).
Finding Unidimensional Measurement Models
If each latent variable in a linear model is measured with a unidimensional measurement model (as in Fig. 1.6 A)[4] then consistent estimates can be obtained for the correlations among the latent variables. If the measurement model is not unidimensional, but its form is known, then consistent estimates of the correlations among the latent variables can still be calculated. However, if the form of the measurement model is not known, then it cannot always be calculated unless the structural relations among the latent variables are known. Since finding or testing these structural relations is usually the purpose of building measurement models, this is circular. One can, however, find unidimensional measurement models without knowing anything about the structural model (Scheines, 1994a). The Purify module helps find unidimensional measurement models by eliminating "impure" indicators.
Fig. 1.6
For example, Bollen (1980) studied whether a number of measures of political democracy were unidimensional indicators of a common feature of societies. Using the following measures:
pf press freedom
fg freedom of group opposition
gs government sanctions
fe fairness of elections
es executive selection
ls legislature selection
he considered the linear unidimensional factor model in Fig. 1.7, where it is understood that for each of the measured variables there is an error term. Bollen estimated this model with LISREL and found that the data reject it.[5] Giving the initial model and the measured covariances to Purify, fg and ls are identified as impure indicators and discarded.
Fig. 1.7
Estimating the resulting unidimensional measurement model (Fig. 1.8) with EQS yields a c2 = 1.687 based on 2 degrees of freedom, with p(c2) = 0.43013.
Fig. 1.8
For more detail on Purify, see chapter 9.
Building Structural Equation Models with Latent Variables
In most research involving latent variables, the real issue is the causal structure among the latent variables themselves. For example, a large organization might want to know the effect of perceived fairness in the workplace on job satisfaction and ultimately on job performance. With unidimensional measurement models, information about the causal structure among the latents can be calculated with the MIMbuild (Multiple Indicator Model Builder) module.[6]
For example, suppose that you had developed the following measurement models of latents interpreted as job satisfaction, job challenge, and spousal support (Fig. 1.9), where the x variables were answers to survey questions.
Fig. 1.9
Suppose that the actual causal structure that generated the data is depicted in Fig. 1.10.
Fig. 1.10
If given covariance data for the measured (x) variables, and the measurement model of Fig. 1.9, MIMbuild would return the following output:
Fig. 1.11
This output represents the set of graphs shown in Fig. 1.12.
Fig. 1.12
See chapter 10 for more detail.
Respecifying Structural Equation Models
In some cases you may specify a model that you believe to be correct but incomplete. If the model is a linear structural equation model, then LISREL and EQS have statistical procedures for guiding respecification.[7] TETRAD 3's Search module performs a similar function, but does not require any parameter estimation and as a result searches through a much bigger portion of the space of model respecifications (Spirtes, Scheines, & Glymour, 1990).
Search takes an initial model specification and adds to it. For example, consider again the Bollen (1980) model of political democracy.
Fig. 1.13
After this model was rejected by the data, Bollen considered variants in which other factors confound his measures. The best model he found is:
Fig. 1.14
This model has a c2 = 6.009 based on 6 degrees of freedom, with p(c2) = 0.42218. Given the same initial model and measured covariances, Search suggests a set of four elaborations, one of which is exactly the model Bollen found, and the others of which score equally well on a c2 test. See chapter 11 for more detail.
Searching for Alternative Causal Models
Before giving a proposed hypothesis any great credence, good scientific practice ought to try to articulate and investigate every serious alternative. A frequent objection to causal models in any discipline is that they are arbitrarily selected without any sound arguments that would exclude alternative explanations of data. That some cherished causal model cannot be rejected statistically is little reason to believe its causal claims: There might be alternatives that also cannot be rejected statistically, but that make contrary causal claims. Published studies may be defective in their general distributional assumptions, in their data collection procedures, or in their assumptions about what is influencing what. TETRAD 3 is meant to be used to help search for and articulate alternative causal explanations under varying background assumptions.
Here is one illustration. In a study published in the American Sociological Review, Timberlake and Williams (1984) claimed that foreign investment in Third World or "peripheral" nations causes the exclusion of various groups from the political process. In other words, foreign investment inhibits democracy. Their empirical case for this claim rests on fitting a linear regression.
They develop measures of political exclusion (po), foreign investment penetration (fi), energy development (en), civil liberties (cv) (measured on an ordered scale from 1 to 7, with lower values indicating greater civil liberties.) The correlations given by Timberlake and Williams for these variables on 72 "non-core" countries are in the file tw.dat:
############# tw.dat ####################
/covariance
72
po fi en cv
1.000
-.175 1.000
-.480 0.330 1.000
0.868 -.391 -.430
1.000
############# tw.dat
####################
Fig. 1.15: tw.dat
An apparent embarrassment to their claim is that political exclusion is negatively correlated with foreign investment; further, foreign investment is negatively correlated with the civil liberties scale (and hence because of their reverse ordering of the civil liberties scale, positively correlated with civil liberties). To defeat this objection, Timberlake and Williams regress po on the other variables on the assumption that the coefficient relating fi to po is a superior measure of fi's causal influence on po than is their simple correlation. A regression on the correlations above yields:
po = .227*fi - .176*en +
.880*cv + e
(.058) (.059) (.060)
3.941 -2.985 14.604
You can see that the crucial coefficient is positive and highly significant. Timberlake and Williams took this as evidence to support the claim that foreign investment causes more political exclusion. They do not explicitly consider any alternative models.
But a regression model is only one among many that might describe the relations among these four variables. To search for alternatives, we again use TETRAD 3's Build module, again without considering whether the linearity and normality assumptions are warranted.[8] Without assuming that all common causes are included in the variables measured, and using a significance level (a) of .05 for its statistical hypothesis tests, Build's output is:
Fig. 1.16: Build output at a = .05
where an "o" means that Build cannot tell if the variables are directly causally connected, or share an unmeasured common cause, or both. Because this structure gives no relation at all between foreign investment and political exclusion (po), we increase a to .15, at which point Build produces the output shown graphically in Fig. 1.17.
Fig. 1.17: Build output at a = .15
The output represents a set of graphs, all of which have certain features in common. The double headed arrow indicates that the variables are connected by an unmeasured common cause but neither has any direct influence on the other. Because none of the connections involving fi have arrowheads directed out of fi, these data indicate that foreign investment is not a cause of any of the other variables. A large number of causal models are consistent with the output in Fig. 1.17, but they share relevant features of the following model, which is one of the simplest represented by the output:
Fig. 1.18: An Alternative Model
This model asserts that en (a measure of economic development) causes both the level of foreign investment and the level of political exclusion. Political exclusion causes the lack of civil liberties, and there is some unmeasured common cause connecting foreign investment and civil liberties (or in other terms, that their errors are correlated). Estimating this model with EQS yields a c2 = .136 with 2 degrees of freedom, with p(c2) = .934. (This p value, like most of the others we give in this manual, should be regarded as an indication of fit for the model rather than as an error probability for the search procedure). We give the coefficients with their standard errors and t-statistics in Fig. 1.19 (subject to the same qualifications as the p value).
Fig. 1.19: Estimated Alternative Model
The signs of the coefficients suggest that the relation between fi and po is negative and mediated by a common cause, contrary to Timberlake and Williams' hypothesis.
We do not mean to suggest that this analysis shows our alternative to be correct. At this small a sample size statistical tests have little power against alternatives, so it is difficult to statistically distinguish between two models even when they are not statistically equivalent. Our point is to show how the Build module can be used to search for plausible alternatives to a given model.
Calculating Equivalent Models
TETRAD 3 can also be used to determine the set of models that are statistically equivalent to a given model.[9] Two causal graphs are statistically equivalent if and only if for any parameterization of one there is a parameterization of the other that describes the same distribution and vice versa. To calculate the set of models equivalent to a graph G over a set of measured variables V:
1) Create a file, e.g., equiv.g, that represents G,[10] and input equiv.g to TETRAD 3.
2) Use Makemodel (chapter 12) to randomly parameterize G and write out a TETRAD 3 readable input file, e.g., equiv.mod, that represents this parameterized model.
3) Input the file equiv.mod to TETRAD 3, and use the Monte Carlo generator (chapter 13) to generate population data and store it in the file equiv.dat.
4) Input the file equiv.dat to TETRAD 3 and run the Build module assuming causal sufficiency with the significance level set to .9.[11]
5) Build's output will be a pattern[12] that represents all the models that are statistically equivalent to G.
For example, suppose that you wanted to calculate the set of equivalent models to the model in Fig. 1.20.
Fig. 1.20: Given Causal Graph
Fig. 1.21 shows a TETRAD 3 readable input file for this causal graph.
############# equiv.g ####################.g;
/graph
x1 x2
x2 x4
x3 x4
############# equiv.g ####################
Fig. 1.21: The input file equiv.g
After following steps 2, 3, and 4 above, Build will produce the pattern in Fig. 1.22.
Fig. 1.22: The Pattern Representing the Graphs Equivalent to equiv.g
This pattern represents a set of statistically equivalent causal graphs, which we show in Fig. 1.23. Patterns, and how to interpret them, are discussed in detail in chapter 5, section 5.4
Fig. 1.23: The Graphs Equivalent to equiv.g
In this case, knowing the set of models equivalent to the one in equiv.g class shows us that the orientation of the causal connection between x1 and x2 cannot be distinguished from just statistical data. Even if some parameterization of the graph in equiv.g passes a statistical test with enormous power against alternatives, there is a parameterization of model b in Fig. 1.23, in which the cause goes the other way, that will pass the test equally well. On the other hand, the causal direction between x3 and x4, and between x2 and x4, is the same in all the graphs equivalent to the one in equiv.g, so a statistical test with sufficient power could confirm that x2 and x3 are causes of x4.
Warnings, Cautions, Caveats
Thousands of investigators in many subjects attempt to extract causal information from uncontrolled or partially controlled observations and to make predictions on the basis of that information. It is so difficult to make such inferences correctly that some critics have concluded that they are impossible in principle.
In some cases if a user applies TETRAD 3 procedures to a data set, the results will be messy, ambiguous, and less informative than the examples used here for purposes of illustration. The user may find any of the following:
1. The output model contains no causal pathways connecting significantly correlated variables.
2. The program gives a great many alternative models.
3. Slight variations in the significance level used produce large variations in the output.
4. The output directly connects every (or almost every) pair of variables.
5. Monte Carlo simulations show the search procedure has a large probability of error associated with the relevant models.
Sometimes when these undesirable outputs are found the program will be at fault-either because the statistical tests it uses internally are not optimal and are insufficiently powerful to make distinctions that could in principle be made reliably, or because the internal algorithms do not find the best compromises among inconsistent statistical decisions. Very often, however, these difficulties indicate that there is something wrong with the general modeling assumptions used and that no reliable inferences can be made from the sample under the modeling assumptions imposed by the user or assumed by the program. The fault may lie in the bad luck of sampling variation, in a sample size that is too small for statistical tests to have much power against important alternatives, in the assumption of linearity, in the assumption of normality, in the use of variables that are aggregates of quite different properties, in erroneous substantive assumptions imposed on the search, or in other problems. In these cases the undesirable outputs are a real guide to truth, just not to the truth hoped for.
We recommend that users test for linearity where it is assumed and where they can. We recommend they transform to make continuous variables as close to normally distributed as possible. We recommend that users test for stability of the output under small variations of the significance level used by the program (the default is .05), that they consider carefully how likely and how important is the assumption that there are no unmeasured latent variables acting, and if the assumption is important and not certain, that they use the Build procedure both with and without that assumption. We recommend that when and if you find a result you like, you use the Monte Carlo facility to test the reliability of the search. We will illustrate one way to do that in chapter 12.
TETRAD 3 is indifferent with respect to cross-validation and sample splitting. It simply analyzes the sample it is given. If you wish to let TETRAD 3 find a candidate set of models on a subset of the sample data, and then test these models on the remaining data, by all means do so.