Class BasisFunctionBicScore

java.lang.Object
edu.cmu.tetrad.search.score.BasisFunctionBicScore
All Implemented Interfaces:
Score
public class BasisFunctionBicScore extends Object implements Score
Calculates the basis function BIC score for a given dataset. This is a generalization of the Degenerate Gaussian score by adding basis functions of the continuous variables and retains the function of the degenerate Gaussian for discrete variables by adding indicator variables per category.

This version uses covariance matrices to calculate likelihoods.

Author:
bryanandrews, josephramsey
See Also:

  • Constructor Details

    • BasisFunctionBicScore

      public BasisFunctionBicScore(DataSet dataSet, int truncationLimit, double lambda)
      Constructs a BasisFunctionBicScore object with the specified parameters.
      Parameters:
      dataSet - the data set on which the score is to be calculated.
      truncationLimit - the truncation limit of the basis.
      lambda - Singularity lambda
      See Also:

    • BasisFunctionBicScore

      public BasisFunctionBicScore(DataSet dataSet, int truncationLimit, double lambda, boolean adaptiveBasisSelection)
      Constructs a BasisFunctionBicScore object with the specified parameters.
      Parameters:
      dataSet - the data set on which the score is to be calculated.
      truncationLimit - the truncation limit of the basis.
      lambda - Singularity lambda
      adaptiveBasisSelection - if true, basis columns beyond the linear term that fail a BIC-crossing screen against every other variable's embedded block are dropped from the embedding before scoring, so that the effective truncation is chosen by the data and the score converges as the truncation limit is raised past the data-supported order. The screen is computed once from the full-sample embedded correlation matrix, independently of any graph, so the pruned embedding is a single fixed embedding and score equivalence is preserved.
      See Also:

    • BasisFunctionBicScore

      public BasisFunctionBicScore(DataSet dataSet, int truncationLimit, double lambda, boolean adaptiveBasisSelection, boolean rankTransform)
      As BasisFunctionBicScore(DataSet, int, double, boolean), with the option to rank-transform continuous variables to [-1, 1] before the Legendre embedding. See Embedding.RANK_TRANSFORM for why this is recommended: it gives the score's LRT an exact chi-square null and removes leverage-driven spurious edges, at the cost of modeling the copula rather than the joint distribution with its marginals.
      Parameters:
      dataSet - the data
      truncationLimit - the truncation limit of the basis
      lambda - the singularity lambda
      adaptiveBasisSelection - see the four-argument constructor
      rankTransform - if true, rank-transform continuous variables before embedding

    • BasisFunctionBicScore

      public BasisFunctionBicScore(DataSet dataSet, int truncationLimit, double lambda, boolean adaptiveBasisSelection, boolean rankTransform, MissingDataSpec spec)
      As BasisFunctionBicScore(DataSet, int, double, boolean, boolean), with an explicit missing-data specification. On a data set with missing values the supported policies are LISTWISE and TESTWISE. Under TESTWISE the embedding carries NaN in every derived column of a variable wherever that variable is missing, and the underlying SemBicScore computes each family's covariance over the rows complete on that family's embedded columns (from the raw embedded columns rather than the correlation matrix; the two differ only by a per-column scale, which contributes the same constant to every parent set of a node and so leaves all score comparisons unchanged). With adaptive basis selection, the screen is computed from pairwise-deletion correlations. A null spec on missing data is treated as FAIL.
      Parameters:
      dataSet - the data
      truncationLimit - the truncation limit of the basis
      lambda - the singularity lambda
      adaptiveBasisSelection - see the four-argument constructor
      rankTransform - if true, rank-transform continuous variables before embedding
      spec - the missing-data specification, or null

    • BasisFunctionBicScore

      public BasisFunctionBicScore(DataSet dataSet, int truncationLimit, double lambda, Map<Integer, List<Integer>> embedding, boolean rankTransform)
      Constructs a BasisFunctionBicScore that scores against an externally supplied embedding - a map from variable index to the embedded columns to use for that variable. The correlation matrix and the underlying SemBicScore are built at full size from this data set's own embedding, exactly as in the other constructors; the supplied map only controls which columns are referenced. This is how a multi-data-set algorithm aiming at a single common model (e.g. IMaGES) shares one basis-column decision across all of its data sets, so that every data set scores the identical parameterization: with per-data-set adaptive selection, different data sets can keep different columns, and the summed score then compares models whose effective embeddings differ across data sets.

      The supplied embedding must be consistent with this data set's own (unpruned) embedding layout: same variable keys, and for each variable a subsequence of its unpruned columns. This holds automatically when the map is derived (e.g. by union) from per-data-set decisions over data sets with the same variables, types, and categories, as Embedding.getEmbeddedData lays such data sets out identically.

      Parameters:
      dataSet - the data set on which the score is to be calculated. Must not be null.
      truncationLimit - the truncation limit of the basis. Must be a positive integer.
      lambda - Singularity lambda. Must be non-negative.
      embedding - the embedding (variable index to embedded columns) to score against. Must not be null.
      rankTransform - whether to apply a rank t ransformation to the unit interval.

  • Method Details

    • adaptivePrunedEmbedding

      public static Map<Integer, List<Integer>> adaptivePrunedEmbedding(DataSet dataSet, int truncationLimit)
      Computes the adaptive basis-column decision for a single data set: the embedding that new BasisFunctionBicScore(dataSet, truncationLimit, lambda, true) would score against. Exposed so that multi-data-set callers can combine per-data-set decisions (e.g. by union) into one common embedding.
      Parameters:
      dataSet - the data set.
      truncationLimit - the truncation limit of the basis.
      Returns:
      the pruned embedding for this data set.

    • fullEmbedding

      public static Map<Integer, List<Integer>> fullEmbedding(DataSet dataSet, int truncationLimit)
      Computes the unpruned embedding layout for a single data set, for callers that need to combine or validate basis-column decisions across data sets.
      Parameters:
      dataSet - the data set.
      truncationLimit - the truncation limit of the basis.
      Returns:
      the full (unpruned) embedding for this data set.

    • getEmbedding

      public Map<Integer, List<Integer>> getEmbedding()
      Returns a copy of the embedding this score is scoring against (variable index to embedded columns).
      Returns:
      a copy of the embedding.

    • localScore

      public double localScore(int i, int... parents)
      Calculates the local score for a given node and its parent nodes.
      Specified by:
      localScore in interface Score
      Parameters:
      i - The index of the node whose score is being calculated.
      parents - The indices for the parent nodes of the given node.
      Returns:
      The calculated local score as a double value.

    • localScoreDiff

      public double localScoreDiff(int x, int y, int[] z)
      Description copied from interface: Score
      Returns the score difference of the graph.
      Specified by:
      localScoreDiff in interface Score
      Parameters:
      x - A node.
      y - TAhe node.
      z - A set of nodes.
      Returns:
      The score difference.

    • getVariables

      public List<Node> getVariables()
      Retrieves the list of nodes representing the variables in the basis function score.
      Specified by:
      getVariables in interface Score
      Returns:
      a list containing the nodes that represent the variables in the basis function score.

    • isEffectEdge

      public boolean isEffectEdge(double bump)
      Determines if the given bump value represents an effect edge.
      Specified by:
      isEffectEdge in interface Score
      Parameters:
      bump - the bump value to be evaluated.
      Returns:
      true if the bump is an effect edge, false otherwise.

    • getSampleSize

      public int getSampleSize()
      Retrieves the sample size from the underlying BIC score component.
      Specified by:
      getSampleSize in interface Score
      Returns:
      the sample size as an integer

    • embeddingBlockSizes

      public int[] embeddingBlockSizes()
      The number of embedded components per variable, in variable order: the basis-function count for a continuous variable, the number of categories minus one for a discrete one (after any adaptive pruning). Adding a parent x to y costs size[y] * size[x] degrees of freedom, so this is the input PenaltyDiscountCalibration.pairDofHistogram(int[]) needs to calibrate the penalty discount for this score.
      Returns:
      One entry per variable.

    • fitNullsByPermutation

      public Map<Integer, PenaltyDiscountCalibration.NullFit> fitNullsByPermutation(DataSet rawData, int samplesPerClass, long seed)
      Estimates, by permutation on the raw data, the null distribution of this score's gain for adding one parent, per degrees-of-freedom class. Re-embeds the data with this score's truncation limit and restricts to the columns this score actually uses (after any adaptive pruning), so the statistic is the one the search sees.

      Use the result with PenaltyDiscountCalibration.penaltyDiscountForFalseDiscoveryRateFitted(Map, int, int, double, double) rather than the exact chi-square version: the embedded components of a variable are not jointly Gaussian, so the LRT's null has the chi-square mean but roughly double the variance, and a chi-square calibration sets the penalty discount far too low for this score.

      Parameters:
      rawData - The data this score was built from.
      samplesPerClass - Null draws per df class; a few hundred is plenty.
      seed - Seed for reproducible calibration.
      Returns:
      Map from df to fitted null.

    • getMaxDegree

      public int getMaxDegree()
      Retrieves the maximum degree from the underlying BIC score component.
      Specified by:
      getMaxDegree in interface Score
      Returns:
      the maximum degree as an integer.

    • toString

      public String toString()
      Returns a string representation of the BasisFunctionBicScore object.
      Specified by:
      toString in interface Score
      Overrides:
      toString in class Object
      Returns:
      A string detailing the degenerate Gaussian score penalty with the penalty discount formatted to two decimal places.

    • getPenaltyDiscount

      public double getPenaltyDiscount()
      The penalty discount in use, which may have been set automatically by the algcomparison wrapper.
      Returns:
      c.

    • setPenaltyDiscount

      public void setPenaltyDiscount(double penaltyDiscount)
      Sets the penalty discount value, which is used to adjust the penalty term in the BIC score calculation.
      Parameters:
      penaltyDiscount - The multiplier on the penalty term for this score.

    • isRankTransform

      public boolean isRankTransform()
      Whether continuous variables were rank-transformed before embedding.
      Returns:
      true if so.

    • setDoOneEquationOnly

      public void setDoOneEquationOnly(boolean doOneEquationOnly)
      When calculation the score for X = <X1 = X, X2, X3,..., Xp> use the equation for X1 only, if true; otherwise, use equations for all of X1, X2,...,Xp.
      Parameters:
      doOneEquationOnly - True if only the equation for X1 is to be used for X = X1,...,Xp. *

    • getMissingValueSupport

      public MissingValueSupport getMissingValueSupport()
      Declares what this score can do, natively, with data containing missing values. The default is MissingValueSupport.NONE, meaning that missingness must be handled upstream (e.g., by imputation, or by supplying an EM-estimated covariance matrix if the score consumes one); scores that handle missing values themselves should override this.

      TESTWISE: constructed with MissingDataSpec.testwise(), each family is scored on the rows complete on that family's embedded columns.

      Specified by:
      getMissingValueSupport in interface Score
      Returns:
      This support level.