Class DeterministicClusters

java.lang.Object
edu.cmu.tetrad.data.audit.DeterministicClusters
public final class DeterministicClusters extends Object
Fits and reports the linear equations behind deterministic (and near-deterministic) relations among the continuous variables of a dataset. Used by the audit's determinism-removal flow to show, for each variable a user removes, the fitted equation expressing it in terms of the retained variables, so that removed definitional variables (BMI from height and weight, totals from parts, region aggregates) remain reconstructible and the report carries the relation explicitly.

A variable is judged deterministic when the fraction of its variance left unexplained by a regression on the other continuous variables falls below a threshold: residual variance / marginal variance < threshold, a scale-free criterion. Which member of a deterministic cluster is written on the left-hand side is a representation choice, not a discovery -- every member is an exact function of the rest -- so callers presenting equations should invite the user to identify the DERIVED (definitional) variable where one exists.

Fitting many equations on a wide dataset goes through DeterministicClusters.Fitter, which computes the correlation matrix ONCE and gets every leave-one-out regression at once from a single ridge-regularized precision matrix (the identity b_j = -Ω_ij / Ω_ii, residual fraction 1 / (Ω_ii r_ii)), then screens each support down to the few variables with non-negligible coefficients and refines exactly on that small set. One O(p^3) inversion serves all variables; the per-variable work is a handful of small solves. The previous implementation recomputed the covariance per call and pruned supports by greedy backward elimination (O(p^5) worst case per variable), which was unusable on wide data with many deterministic relations.

Version:
$Id: $Id
Author:
josephramsey
See Also:

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final record 
    DeterministicClusters.Constraint
    One fitted constraint: determined = sum(coefficients * support) + intercept.
    static final class 
    DeterministicClusters.Fitter
    A reusable equation fitter for one dataset: the correlation matrix and its ridge-regularized inverse are computed once, and each equation costs a few small solves.

  • Method Summary

    Modifier and Type
    Method
    Description
    static DeterministicClusters.Constraint
    constraintFor(DataSet data, String varName, double threshold)
    Fits the equation writing the given variable as a function of the other continuous variables of the dataset.
    static List<DeterministicClusters.Constraint>
    find(DataSet data, double threshold)
    Finds deterministic constraints among the continuous variables of the dataset: while some continuous variable has residual fraction below the threshold given the other active continuous variables, its (screened) support is pruned to a minimal set still meeting the threshold, the constraint is recorded, and the variable is removed from the active set -- so multiple and overlapping constraints are each reported once.

    Methods inherited from class Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • find

      public static List<DeterministicClusters.Constraint> find(DataSet data, double threshold)
      Finds deterministic constraints among the continuous variables of the dataset: while some continuous variable has residual fraction below the threshold given the other active continuous variables, its (screened) support is pruned to a minimal set still meeting the threshold, the constraint is recorded, and the variable is removed from the active set -- so multiple and overlapping constraints are each reported once. Discrete variables are ignored.
      Parameters:
      data - The dataset.
      threshold - The residual-fraction threshold; 1e-8 detects exact relations up to rounding.
      Returns:
      The detected constraints, one per independent deterministic relation; empty if none.

    • constraintFor

      public static DeterministicClusters.Constraint constraintFor(DataSet data, String varName, double threshold)
      Fits the equation writing the given variable as a function of the other continuous variables of the dataset. The caller should check fractionResidual against its own threshold to label the equation exact or approximate. For fitting MANY equations on one dataset, construct a DeterministicClusters.Fitter once instead: this convenience method rebuilds the shared matrices on every call.
      Parameters:
      data - The dataset.
      varName - The name of the variable to write on the left-hand side.
      threshold - The exactness threshold used in the pruning rule.
      Returns:
      The fitted constraint, or null if the variable is not a continuous variable of the dataset or fewer than two continuous variables exist.