Class IndTestBasisFunctionBlocks

java.lang.Object
edu.cmu.tetrad.search.test.IndTestBasisFunctionBlocks
All Implemented Interfaces:
RawMarginalIndependenceTest, IndependenceTest, RowsSettable, EffectiveSampleSizeSettable
public class IndTestBasisFunctionBlocks extends Object implements IndependenceTest, RawMarginalIndependenceTest, EffectiveSampleSizeSettable, RowsSettable

Basis–function independence test using block–structured Wilks statistics. This class:

  • Applies a per-variable truncated basis expansion (via Embedding) to produce an embedded dataset.
  • Builds a block mapping from each original variable to the list of embedded columns that belong to that variable.
  • Delegates conditional-independence testing to IndTestBlocksWilkes, using a BlockSpec constructed from the embedded dataset, the block mapping, and the original variable nodes.

Subsampling support: This class implements RowsSettable. Whenever setRows(...) is called:

  • If rows == null, the test reverts to using the full embedded dataset.
  • If a non-null list of row indices is provided, a row-subsetted embedded dataset is constructed, and the internal IndTestBlocksWilkes delegate is rebuilt on that subsample.

This guarantees that:

  • The block structure remains identical across subsamples.
  • All independence tests are run on the subsampled data when used inside tools such as the Markov Checker.
  • Node identity is preserved (the same Node instances are passed through to the delegate).

Contract: blocks.get(v) returns the list of embedded column indices corresponding exactly to original variable v, for v = 0, ..., V-1.

  • Constructor Details

    • IndTestBasisFunctionBlocks

      public IndTestBasisFunctionBlocks(DataSet dataSet, int truncationLimit, int basisType)
      Constructs an instance of IndTestBasisFunctionBlocks. This class is designed to perform independence tests based on basis function transformations of the provided dataset, using specified degree and basis type.
      Parameters:
      dataSet - the input dataset, which provides the raw data to be analyzed. Cannot be null.
      truncationLimit - the degree of the basis function transformation. Must be a non-negative integer.
      basisType - the type of basis functions to use for transformations (e.g., polynomial, Fourier, etc.). This value determines the embedding style and configuration specifics.
      Throws:
      IllegalArgumentException - if the raw dataset is null or if the degree is negative.

    • IndTestBasisFunctionBlocks

      public IndTestBasisFunctionBlocks(DataSet dataSet, int truncationLimit, int basisType, boolean adaptiveBasisSelection)
      Constructs an instance of IndTestBasisFunctionBlocks, optionally with adaptive basis selection.
      Parameters:
      dataSet - the input dataset, which provides the raw data to be analyzed. Cannot be null.
      truncationLimit - the degree of the basis function transformation. Must be a non-negative integer.
      basisType - the type of basis functions to use for transformations.
      adaptiveBasisSelection - if true, basis columns beyond the linear term that fail a BIC-crossing screen against every other variable's embedded block are dropped from the blocks before testing, so that the effective truncation is chosen by the data and the test's behavior converges as the truncation limit is raised past the data-supported order. The screen is computed once, from the full-sample embedded correlation matrix, and the same pruned blocks are used for every test and every row subset (so bootstrap subsamples share one fixed block structure). Wilks degrees of freedom shrink accordingly.
      Throws:
      IllegalArgumentException - if the raw dataset is null or if the degree is negative.
      See Also:

    • IndTestBasisFunctionBlocks

      public IndTestBasisFunctionBlocks(DataSet dataSet, int truncationLimit, int basisType, boolean adaptiveBasisSelection, MissingDataSpec spec)
      As the four-argument constructor, with an explicit missing-data specification. On a data set with missing values the supported policies are LISTWISE, TESTWISE and EM_COVARIANCE. In both of the last two the embedding carries NaN in every derived column of a variable wherever that variable is missing, and the adaptive screen (if any) uses pairwise-deletion correlations. Under TESTWISE the delegate blocks test computes each test's correlation matrix over the rows complete on the columns it uses. Under EM_COVARIANCE it estimates one correlation matrix of the embedded columns by EM, under a jointly Gaussian working model, and the spec's effective-sample-size mode sets the sample size unless the caller sets one. As for the basis-function BIC score, this working model ignores the exact relations among a continuous variable's basis columns (the quadratic column is the square of the linear one), so the EM path is offered for comparison, not as the default. A null spec on missing data throws.
      Parameters:
      dataSet - the input dataset.
      truncationLimit - the degree of the basis function transformation.
      basisType - the type of basis functions.
      adaptiveBasisSelection - see the four-argument constructor.
      spec - the missing-data specification, or null.

  • Method Details

    • checkIndependence

      public IndependenceResult checkIndependence(Node x, Node y, Set<Node> z)
      Checks for statistical independence between two given variables (nodes), conditioned on a set of other variables.
      Specified by:
      checkIndependence in interface IndependenceTest
      Parameters:
      x - the first variable (node) to test for independence
      y - the second variable (node) to test for independence
      z - the set of conditioning variables (nodes) for the independence test
      Returns:
      an IndependenceResult containing details about the test result, including whether the variables are independent, the p-value, and the significance level used

    • getVariables

      public List<Node> getVariables()
      Retrieves the list of nodes (variables) associated with this instance. The nodes returned are exactly the ones used internally by the delegate, ensuring identity consistency and preventing "unknown node" issues.
      Specified by:
      getVariables in interface IndependenceTest
      Returns:
      a list of nodes representing the variables

    • getData

      public DataModel getData()
      Retrieves the original dataset represented by this instance.
      Specified by:
      getData in interface IndependenceTest
      Returns:
      the underlying data as a DataModel object

    • isVerbose

      public boolean isVerbose()
      Indicates whether this instance is operating in verbose mode. Verbose mode allows for detailed output or logging to assist in debugging or monitoring execution.
      Specified by:
      isVerbose in interface IndependenceTest
      Returns:
      true if verbose mode is enabled; false otherwise

    • setVerbose

      public void setVerbose(boolean verbose)
      Sets the verbosity mode for this instance. Enabling verbosity allows for more detailed output or logging during execution.
      Specified by:
      setVerbose in interface IndependenceTest
      Parameters:
      verbose - true to enable verbose output, false to disable it

    • computePValue

      public double computePValue(double[] x, double[] y) throws InterruptedException
      Description copied from interface: RawMarginalIndependenceTest
      Computes the p-value for the statistical test of marginal independence between the two given variables represented by the input arrays.
      Specified by:
      computePValue in interface RawMarginalIndependenceTest
      Parameters:
      x - the first variable, represented as an array of doubles
      y - the second variable, represented as an array of doubles
      Returns:
      the computed p-value for the test of marginal independence
      Throws:
      InterruptedException - if the computation is interrupted

    • computePValue

      public double computePValue(double[] x, double[][] Y) throws InterruptedException
      Default multivariate fallback: run BFIT on each column of Y and combine with Fisher. If you later add a true multivariate BFIT, override this to call it directly.
      Specified by:
      computePValue in interface RawMarginalIndependenceTest
      Parameters:
      x - the first variable (scalar), represented as an array of doubles of length n
      Y - the multivariate variable, represented as a 2D array of shape [n][m] (n samples, m variables)
      Returns:
      the computed p-value for the test of independence
      Throws:
      InterruptedException - if the computation is interrupted

    • getAlpha

      public double getAlpha()
      Retrieves the alpha value currently set for this instance. The alpha value is typically used as a threshold or parameter in statistical tests or computations.
      Specified by:
      getAlpha in interface IndependenceTest
      Returns:
      the alpha value, which is a double strictly between 0 and 1

    • setAlpha

      public void setAlpha(double alpha)
      Sets the alpha value used as a threshold or parameter for statistical tests or computations. The alpha value must be within the range (0, 1), exclusive.
      Specified by:
      setAlpha in interface IndependenceTest
      Parameters:
      alpha - the desired alpha value, strictly between 0 and 1
      Throws:
      IllegalArgumentException - if alpha is less than or equal to 0 or greater than or equal to 1

    • getRows

      public List<Integer> getRows()
      Description copied from interface: RowsSettable
      Gets the rows to use for the test. These rows over override testwise deletion if set.
      Specified by:
      getRows in interface RowsSettable
      Returns:
      The rows to use for the test. Can be null.

    • setRows

      public void setRows(List<Integer> rows)
      Description copied from interface: RowsSettable
      Sets the rows to use for the test. This will override testwise deletion.
      Specified by:
      setRows in interface RowsSettable
      Parameters:
      rows - The rows to use for the test. Can be null.

    • getBlocks

      public List<List<Integer>> getBlocks()
      Retrieves the list of blocks, where each block is represented as a list of integers. These blocks may correspond to partitions or groupings derived from the data or configuration.
      Returns:
      a list of blocks, with each block being a list of integers

    • getEffectiveSampleSize

      public int getEffectiveSampleSize()
      Description copied from interface: EffectiveSampleSizeSettable
      Returns the effective sample size.
      Specified by:
      getEffectiveSampleSize in interface EffectiveSampleSizeSettable
      Returns:
      the effective sample size

    • setEffectiveSampleSize

      public void setEffectiveSampleSize(int nEff)
      Sets the effective sample size. The value is remembered and survives subsequent setRows calls.

      Changes from the pre-2026-8 implementation: previously, setRows reset the effective sample size to the row count when it rebuilt the internal delegate, so a caller-set value was silently discarded by any tool that subsamples or shuffles rows -- in particular the Markov Checker, which calls setRows on every run (even with resample fraction 1). The requested value is now stored and re-applied whenever the row set changes. Under a proper row subset, the applied effective sample size is rescaled proportionally: if the caller declares that the full data of N rows carry nEff independent observations' worth of information, a subset of m rows is treated as carrying nEff * m / N (floored at 1). Passing a negative value restores the default (effective sample size = current row count).

      Specified by:
      setEffectiveSampleSize in interface EffectiveSampleSizeSettable
      Parameters:
      nEff - the effective sample size, or a negative value for the default.

    • getMissingValueSupport

      public MissingValueSupport getMissingValueSupport()
      Declares what this test can do, natively, with data containing missing values. The default is MissingValueSupport.NONE, meaning that missingness must be handled upstream (e.g., by imputation, or by supplying an EM-estimated covariance matrix if the test consumes one); tests that handle missing values themselves should override this.

      TESTWISE: constructed with MissingDataSpec.testwise(), each test uses the rows complete on the embedded columns of x, y, and z.

      Specified by:
      getMissingValueSupport in interface IndependenceTest
      Returns:
      This support level.