Package edu.cmu.tetrad.search.test

Class IndTestFisherZ

java.lang.Object
edu.cmu.tetrad.search.test.IndTestFisherZ
All Implemented Interfaces:
RawMarginalIndependenceTest, IndependenceTest, RowsSettable, EffectiveSampleSizeSettable
public final class IndTestFisherZ extends Object implements IndependenceTest, EffectiveSampleSizeSettable, RowsSettable, RawMarginalIndependenceTest
Fisher's Z CI test with shrinkage (RIDGE/Ledoit–Wolf) and optional pseudoinverse fallback.

  • Constructor Details

    • IndTestFisherZ

      public IndTestFisherZ(DataSet dataSet, double alpha)
      Constructs an independence test using the Fisher Z test statistic. The test evaluates the independence of variables given a dataset and a significance level (alpha). The dataset must be continuous and should not contain missing values for certain operations.
      Parameters:
      dataSet - the dataset used for the independence test; must be continuous
      alpha - the significance level for the Fisher Z test; must be in the range [0, 1]
      Throws:
      IllegalArgumentException - if the dataset is not continuous
      IllegalArgumentException - if the alpha value is not in the range [0, 1]

    • IndTestFisherZ

      public IndTestFisherZ(DataSet dataSet, double alpha, double ridge)
      Constructor for IndTestFisherZ which initializes the independence test using the Fisher Z test with specified parameters.
      Parameters:
      dataSet - the dataset on which the independence test will be performed
      alpha - the significance level for determining independence
      ridge - the ridge parameter used for regularization in the test

    • IndTestFisherZ

      public IndTestFisherZ(Matrix data, List<Node> variables, double alpha)
      Constructs an instance of the IndTestFisherZ class, which is a statistical test for conditional independence based on the Fisher Z-test.
      Parameters:
      data - The data matrix, where rows represent samples and columns represent variables.
      variables - A list of variables corresponding to the columns in the data matrix. The order must match the column order in the data matrix.
      alpha - The significance level (type I error rate) for the Fisher Z-test.

    • IndTestFisherZ

      public IndTestFisherZ(Matrix data, List<Node> variables, double alpha, double ridge)
      Constructor for the IndTestFisherZ class, which performs a Fisher Z independence test with ridge regularization applied to handle issues with covariance matrix inversion.
      Parameters:
      data - The data matrix containing the dataset.
      variables - The list of nodes corresponding to the variables in the dataset.
      alpha - The significance level for the independence test.
      ridge - The ridge parameter for regularization of the covariance matrix.

    • IndTestFisherZ

      public IndTestFisherZ(ICovarianceMatrix covMatrix, double alpha)
      Constructs an instance of IndTestFisherZ using the given covariance matrix and significance level.
      Parameters:
      covMatrix - the covariance matrix used to compute correlations and perform the Fisher's Z test
      alpha - the significance level for independence tests

    • IndTestFisherZ

      public IndTestFisherZ(ICovarianceMatrix covMatrix, double alpha, double ridge)
      Constructor for the IndTestFisherZ class. It initializes the test with a given covariance matrix, significance level, and ridge parameter. This test determines the conditional independence of variables using the Fisher Z test.
      Parameters:
      covMatrix - the covariance matrix used for calculating correlations.
      alpha - the significance level for testing conditional independence.
      ridge - the ridge parameter used for regularization in the shrinkage mode.

  • Method Details

    • indTestSubset

      public IndependenceTest indTestSubset(List<Node> vars)
      Returns a new IndependenceTest instance for a subset of variables. This method verifies that the given variables are part of the original variable set, then creates a submatrix of the covariance matrix and constructs an IndependenceTestFisherZ object for the subset.
      Specified by:
      indTestSubset in interface IndependenceTest
      Parameters:
      vars - A list of variables for which the subset independence test will be created. All variables in this list must be part of the original set of variables.
      Returns:
      An IndependenceTest instance that operates on the given subset of variables.
      Throws:
      IllegalArgumentException - If the provided subset is empty or contains variables not in the original variable set.

    • checkIndependence

      public IndependenceResult checkIndependence(Node x, Node y, Set<Node> z)
      Checks the independence of two nodes given a conditioning set and returns the result.
      Specified by:
      checkIndependence in interface IndependenceTest
      Parameters:
      x - the first node to be tested for independence
      y - the second node to be tested for independence
      z - the set of nodes conditioning the independence test
      Returns:
      an IndependenceResult containing the results of the independence test, including whether x and y are independent given z, the p-value of the test, and other test details
      Throws:
      RuntimeException - if a singular matrix is encountered during computation or if the p-value is undefined

    • getPValue

      public double getPValue(Node x, Node y, Set<Node> z) throws org.apache.commons.math3.linear.SingularMatrixException
      Calculates the p-value for the partial correlation between two nodes conditioned on a set of other nodes.
      Parameters:
      x - the first node involved in the correlation.
      y - the second node involved in the correlation.
      z - the set of conditioning nodes.
      Returns:
      the p-value for the partial correlation between nodes x and y given the conditioning set z.
      Throws:
      org.apache.commons.math3.linear.SingularMatrixException - if the covariance matrix inversion fails during computation.
      IllegalArgumentException - if the degrees of freedom (df) are non-positive.

    • getEffectiveSampleSize

      public int getEffectiveSampleSize()
      Retrieves the effective sample size.
      Specified by:
      getEffectiveSampleSize in interface EffectiveSampleSizeSettable
      Returns:
      the effective sample size as an integer.

    • setEffectiveSampleSize

      public void setEffectiveSampleSize(int effectiveSampleSize)
      Sets the effective sample size. If the provided effective sample size is negative, it will default to the sample size.
      Specified by:
      setEffectiveSampleSize in interface EffectiveSampleSizeSettable
      Parameters:
      effectiveSampleSize - the effective sample size to set; if negative, the sample size will be used instead.

    • getBic

      public double getBic()
      Computes and returns the Bayesian Information Criterion (BIC) value.
      Returns:
      the BIC value as a double calculated based on the effective sample size and the correlation coefficient squared.

    • getAlpha

      public double getAlpha()
      Retrieves the significance level.
      Specified by:
      getAlpha in interface IndependenceTest
      Returns:
      the significance level as a double.

    • setAlpha

      public void setAlpha(double alpha)
      Sets the significance level. Validates that the provided significance level is within the valid range (0.0 to 1.0).
      Specified by:
      setAlpha in interface IndependenceTest
      Parameters:
      alpha - This level.

    • getVariables

      public List<Node> getVariables()
      Retrieves the list of variables.
      Specified by:
      getVariables in interface IndependenceTest
      Returns:
      the list of variables.

    • setVariables

      public void setVariables(List<Node> variables)
      Sets the list of variables for the instance. Validates that the size of the provided variable list matches the current size of the instance's internal variable list, and updates internal state accordingly.
      Parameters:
      variables - the list of variables to set; must match the size of the current variable list
      Throws:
      IllegalArgumentException - if the size of the provided variable list does not match the current variable list size

    • getVariable

      public Node getVariable(String name)
      Retrieves a variable by its name from the internal mapping of variable names to nodes.
      Specified by:
      getVariable in interface IndependenceTest
      Parameters:
      name - the name of the variable to retrieve; must match an existing key in the name map
      Returns:
      the node corresponding to the given name, or null if no such node exists

    • getData

      public DataSet getData()
      Retrieves the data set associated with the current instance.
      Specified by:
      getData in interface IndependenceTest
      Returns:
      the DataSet object representing the data in this instance

    • getCov

      public ICovarianceMatrix getCov()
      Retrieves the covariance matrix used by this instance.
      Specified by:
      getCov in interface IndependenceTest
      Returns:
      an object implementing the ICovarianceMatrix interface, representing the covariance matrix associated with this instance

    • getDataSets

      public List<DataSet> getDataSets()
      Retrieves a list of data sets associated with this instance.
      Specified by:
      getDataSets in interface IndependenceTest
      Returns:
      a list containing the data set associated with this instance

    • getSampleSize

      public int getSampleSize()
      Retrieves the sample size of the data set associated with this instance.
      Specified by:
      getSampleSize in interface IndependenceTest
      Returns:
      the number of rows in the data set, or the sample size of the covariance matrix if no data set is available

    • isVerbose

      public boolean isVerbose()
      Retrieves the verbosity setting for this instance.
      Specified by:
      isVerbose in interface IndependenceTest
      Returns:
      true if verbose output is enabled, false otherwise

    • setVerbose

      public void setVerbose(boolean verbose)
      Sets the verbosity setting for this instance.
      Specified by:
      setVerbose in interface IndependenceTest
      Parameters:
      verbose - True, if so.

    • toString

      public String toString()
      Returns a string representation of this instance, including details about the test configuration.
      Specified by:
      toString in interface IndependenceTest
      Overrides:
      toString in class Object
      Returns:
      a string representation of this instance

    • determines

      public boolean determines(List<Node> z, Node x) throws UnsupportedOperationException
      Determines whether the given set of nodes, z, has a deterministic relationship with the specified node, x. Specifically, this method checks if the covariance matrix derived from z is invertible. If the matrix is singular (non-invertible), it indicates a determinism detected between the nodes.
      Parameters:
      z - the list of nodes to analyze as a potential set of deterministic parents for the node x
      x - the node to check for a deterministic relationship with the set of nodes z
      Returns:
      true if a deterministic relationship exists (when the covariance matrix is singular), false otherwise
      Throws:
      UnsupportedOperationException - if the operation is not supported due to some internal state or configuration

    • getRows

      public List<Integer> getRows()
      Retrieves the list of row indices currently associated with this instance.
      Specified by:
      getRows in interface RowsSettable
      Returns:
      a list of integers representing the row indices

    • setRows

      public void setRows(List<Integer> rows)
      Sets the list of row indices for the instance. Validates the provided list to ensure all elements are non-negative and non-null. Resets internal correlation state if the rows are updated.
      Specified by:
      setRows in interface RowsSettable
      Parameters:
      rows - the list of row indices to set. Each element must be non-null and non-negative. If the provided list is null, the current row list is set to null.
      Throws:
      NullPointerException - if any element in the rows list is null.
      IllegalArgumentException - if any element in the rows list is negative.

    • setLambda

      public void setLambda(double lambda)
      Sets the value of the lambda parameter for this instance. Lambda is used for regularization or other purposes within the class, depending on the context of its implementation.
      Parameters:
      lambda - the value to set for the lambda parameter

    • getRidge

      public double getRidge()
      Retrieves the current value of the ridge parameter associated with this instance.
      Returns:
      the ridge parameter as a double value

    • setRidge

      public void setRidge(double ridge)
      Sets the value of the ridge parameter for this instance. The ridge parameter is commonly used for regularization purposes. The provided value must be non-negative.
      Parameters:
      ridge - the value to set for the ridge parameter; must be greater than or equal to 0
      Throws:
      IllegalArgumentException - if the ridge parameter is negative

    • getShrinkageMode

      public IndTestFisherZ.ShrinkageMode getShrinkageMode()
      Retrieves the current shrinkage mode used by this instance.
      Returns:
      the shrinkage mode, represented as a ShrinkageMode enum value, which indicates the type of regularization or adjustment (if any) applied in computations.

    • setShrinkageMode

      public void setShrinkageMode(IndTestFisherZ.ShrinkageMode mode)
      Sets the shrinkage mode for the instance. The shrinkage mode determines the type of regularization or adjustment applied during calculations, if any. If the provided mode is null, the shrinkage mode defaults to ShrinkageMode.NONE.
      Parameters:
      mode - the shrinkage mode to set, represented as a ShrinkageMode enum value. It can be ShrinkageMode.NONE, ShrinkageMode.RIDGE, or ShrinkageMode.LEDOIT_WOLF.

    • isUsePseudoinverse

      public boolean isUsePseudoinverse()
      Checks whether the pseudoinverse is being used in computations.
      Returns:
      true if the pseudoinverse is enabled, false otherwise

    • setUsePseudoinverse

      public void setUsePseudoinverse(boolean use)
      Sets whether to use the pseudoinverse in computations.
      Parameters:
      use - true to enable pseudoinverse, false to disable

    • getPinvTolerance

      public double getPinvTolerance()
      Gets the tolerance for the pseudoinverse computation.
      Returns:
      the tolerance value

    • setPinvTolerance

      public void setPinvTolerance(double tol)
      Sets the tolerance for the pseudoinverse computation.
      Parameters:
      tol - the tolerance value

    • getLastR

      public double getLastR()
      Gets the last computed partial correlation.
      Returns:
      the last computed partial correlation

    • getRho

      public double getRho()
      Gets the last computed partial correlation.
      Returns:
      the last computed partial correlation

    • computePValue

      public double computePValue(double[] x, double[] y)
      Computes the p-value for the statistical test of independence between two variables.
      Specified by:
      computePValue in interface RawMarginalIndependenceTest
      Parameters:
      x - the array of values representing the first variable
      y - the array of values representing the second variable
      Returns:
      the computed p-value indicating the strength of independence between the two variables