Class TestwiseCovariance

java.lang.Object
edu.cmu.tetrad.data.missing.TestwiseCovariance
public final class TestwiseCovariance extends Object
Test-wise (family-wise) covariance estimation for components whose local calculations consume a covariance submatrix and a sample size: for a requested column set, the sample covariance matrix is computed over exactly the rows that are complete on those columns, together with the number of such rows. This is the per-family statistic that SemBicScore computes inline; it is factored out here so that other covariance-consuming scores and tests (BGe, the embedded basis-function scores, the degenerate-Gaussian and basis-function likelihood-ratio tests) can offer the same TESTWISE missing-data policy without each re-deriving the row bookkeeping.

Missing values are NaN entries of the underlying matrix. Results are cached per (sorted) column set, so a search that revisits the same family pays for the covariance once. Row subsets supplied by a caller (e.g., from subsampling) bypass the cache.

Every family is estimated on its own subsample, so the collection of family covariances is not in general consistent with any single positive-definite matrix, and a score assembled from them is not the score of any one dataset. This is the standard caveat of test-wise deletion and is unbiased only under MCAR.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final record 
    TestwiseCovariance.Family
    The covariance matrix of a column set over the rows complete on that set, the number of such rows, and the rows themselves.

  • Constructor Summary

    Constructors
    Constructor
    Description
    TestwiseCovariance(Matrix data)
    Constructs the helper over a data matrix.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    clearCache()
    Clears the family cache.
    TestwiseCovariance.Family
    column(int column)
    Returns the sample variance (divisor n - 1) of one column over its non-missing rows, and the count of those rows, as a one-column family.
    TestwiseCovariance.Family
    family(int[] columns)
    Returns the sample covariance matrix (divisor n - 1) of the given columns over the rows complete on all of them, in the order the columns are given, together with the number of such rows.
    TestwiseCovariance.Family
    family(int[] columns, List<Integer> candidateRows)
    As family(int[]), but restricted to the given candidate rows (null means all rows).
    static TestwiseCovariance
    forMatrix(Matrix matrix)
    The shared instance for the given raw data matrix, created on first request.
    int
    getNumColumns()
    Returns the number of columns of the underlying matrix.
    int
    getNumRows()
    Returns the number of rows of the underlying data set.
    static void
    invalidate(Matrix matrix)
    Discards any shared instance for the given matrix.
    ICovarianceMatrix
    pairwiseCovarianceMatrix(List<Node> variables)
    Returns a pairwise-deletion covariance matrix over the given variables: entry (i, j) is the covariance of columns i and j over the rows complete on both, with each diagonal entry computed over the rows complete on that column alone.

    Methods inherited from class Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • TestwiseCovariance

      public TestwiseCovariance(Matrix data)
      Constructs the helper over a data matrix.
      Parameters:
      data - The data matrix (rows are cases, columns are variables); NaN marks a missing entry.

  • Method Details

    • forMatrix

      public static TestwiseCovariance forMatrix(Matrix matrix)
      The shared instance for the given raw data matrix, created on first request. The matrix must not be mutated after analysis begins; if it is, discard the instance with invalidate(Matrix).
      Parameters:
      matrix - The matrix.
      Returns:
      This instance.

    • invalidate

      public static void invalidate(Matrix matrix)
      Discards any shared instance for the given matrix.
      Parameters:
      matrix - The matrix.

    • getNumRows

      public int getNumRows()
      Returns the number of rows of the underlying data set.
      Returns:
      the number of rows.

    • getNumColumns

      public int getNumColumns()
      Returns the number of columns of the underlying matrix.
      Returns:
      This number.

    • family

      public TestwiseCovariance.Family family(int[] columns)
      Returns the sample covariance matrix (divisor n - 1) of the given columns over the rows complete on all of them, in the order the columns are given, together with the number of such rows. Cached per column set.
      Parameters:
      columns - The columns; need not be sorted; must not contain duplicates. The result is indexed in this order.
      Returns:
      The family statistics.

    • family

      public TestwiseCovariance.Family family(int[] columns, List<Integer> candidateRows)
      As family(int[]), but restricted to the given candidate rows (null means all rows). Results for explicit candidate row lists are not cached.
      Parameters:
      columns - The columns.
      candidateRows - The rows to draw from, or null for all rows.
      Returns:
      The family statistics.

    • column

      public TestwiseCovariance.Family column(int column)
      Returns the sample variance (divisor n - 1) of one column over its non-missing rows, and the count of those rows, as a one-column family.
      Parameters:
      column - The column.
      Returns:
      The family statistics for the single column.

    • pairwiseCovarianceMatrix

      public ICovarianceMatrix pairwiseCovarianceMatrix(List<Node> variables)
      Returns a pairwise-deletion covariance matrix over the given variables: entry (i, j) is the covariance of columns i and j over the rows complete on both, with each diagonal entry computed over the rows complete on that column alone. The reported sample size is the smallest pairwise complete count. The result is not guaranteed positive definite; it is intended for screening computations (e.g., pairwise correlation-based basis-column selection) rather than for fitting.
      Parameters:
      variables - The variables, one per column of the underlying matrix.
      Returns:
      The pairwise covariance matrix.

    • clearCache

      public void clearCache()
      Clears the family cache.