Class BlocksUtil
java.lang.Object
edu.cmu.tetrad.search.blocks.BlocksUtil
Utility class for handling operations related to blocks, such as creating block variables, canonicalizing blocks,
ensuring valid indices, and applying various cluster policies. This class includes methods to manipulate and process
blocks and their corresponding data representations within a dataset.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic enumAn enumeration representing different naming modes for assigning names to latent variables in the context of block specifications. -
Method Summary
Modifier and TypeMethodDescriptionstatic BlockSpecapplySingleClusterPolicy(BlockSpec blockSpec, SingletonClusterPolicy policy, double alpha) Applies a single-cluster policy to the provided BlockSpec.canonicalizeBlocks(List<List<Integer>> blocks) Canonicalizes a list of blocks by removing null or empty blocks, sorting the contents of each block, and ensuring the resulting blocks are unique.expandLatents(BlockSpec spec) Expand ranks -> per-latent variables named Lk-1..Lk-r.static BlockSpecgiveGoodLatentNames(BlockSpec spec, Map<String, List<String>> trueClusters, BlocksUtil.NamingMode mode) Assigns meaningful names to latent variables in the provided BlockSpec object based on the given true clusters and the specified naming mode.makeBlockVariables(List<List<Integer>> blocks, DataSet dataSet) Creates a list of block variables based on the provided list of blocks and the dataset.makeDisjointBySize(List<List<Integer>> blocks) Creates a list of disjoint blocks from the provided list of blocks, prioritizing larger blocks first.static BlockSpecmakeDisjointSpec(DataSet ds, List<List<Integer>> blocks) Constructs a BlockSpec object using the provided DataSet and block definitions, ensuring that the blocks are made disjoint by prioritizing larger blocks first.static CovarianceMatrixpooledCovariance(List<DataSet> imputations) Pools several imputations of one data set into a single covariance matrix: the elementwise average of the imputations' covariance matrices, at the imputations' (shared) sample size.static intpooledEffectiveSampleSize(List<DataSet> imputations) Estimates an effective sample size for analyses run on the pooled covariance of several imputations, from the disagreement among the imputations themselves.static BlockSpecConverts a list of block indices and a dataset into a BlockSpec object, ensuring the blocks are canonicalized and generating the appropriate block variables.static BlockSpecConverts a list of blocks, ranks, and a dataset into a BlockSpec object.static voidvalidateBlocks(List<List<Integer>> blocks, DataSet data) Validates the provided list of blocks to ensure that all indices within each block are non-negative, within the range of columns in the given dataset, and not null.
-
Method Details
-
pooledCovariance
Pools several imputations of one data set into a single covariance matrix: the elementwise average of the imputations' covariance matrices, at the imputations' (shared) sample size. This is the multiple-imputation point estimate of the covariance; the between-imputation variance is not carried into it, so tests run on the pooled matrix at the full sample size act as if the pooled covariance had been estimated from complete data. The effective-sample-size parameter of the consuming method is the knob for compensating.The variables of the returned matrix are the first imputation's variable objects, so a
BlockSpecbuilt over the first imputation and blocks discovered from the pooled matrix refer to the same nodes.- Parameters:
imputations- the imputed data sets: the same variables, with the same number of rows. One with the variables in a different order is put in the order of the first.- Returns:
- the averaged covariance matrix at the shared sample size.
- Throws:
IllegalArgumentException- if the list is empty, a member is not continuous tabular data, or the members differ in their variables or row counts.
-
pooledEffectiveSampleSize
Estimates an effective sample size for analyses run on the pooled covariance of several imputations, from the disagreement among the imputations themselves. For each pair of variables, the correlation is Fisher-z transformed in each imputation; its within-imputation variance is 1/(n-3) and its between-imputation variance B is observed, so by Rubin's rules the pooled estimate has the precision of a complete sample of about (n-3)/(1 + (1 + 1/M) B (n-3)) + 3 observations. The SMALLEST of these pairwise sizes is returned: a rank test's precision is limited by its least well determined correlation, and in calibration runs on simulated missing-at-random data the per-pair sizes consistently understated the imputation noise that rank tests react to, so the conservative end of their distribution recovered clusters best at every missingness level tried, with no cost at low missingness. With many variables the minimum leans low (it is a minimum over many noisy ratios), which fails toward less power rather than toward spurious structure.Improper imputers (which do not redraw the imputation model's parameters) understate B, so this estimate leans large; it is nevertheless far closer to the truth than the full n, which treats the pooled covariance as if it had been estimated from complete data.
- Parameters:
imputations- the imputed data sets, as forpooledCovariance(List).- Returns:
- the estimated effective sample size, at least 10 and at most n.
- Throws:
IllegalArgumentException- as forpooledCovariance(List), and if fewer than two imputations are given.
-
makeBlockVariables
Creates a list of block variables based on the provided list of blocks and the dataset. If a block contains a single index, the corresponding variable from the dataset is added to the result. For larger blocks, a new latent variable is created and added to the result.- Parameters:
blocks- a list of lists, where each inner list represents a block of indicesdataSet- the dataset associated with the specified blocks, providing the variables- Returns:
- a list of Node objects representing the block variables, either existing or newly created
-
canonicalizeBlocks
Canonicalizes a list of blocks by removing null or empty blocks, sorting the contents of each block, and ensuring the resulting blocks are unique. The returned list maintains the order of the first occurrence of each unique block.- Parameters:
blocks- a list of lists, where each inner list represents a block of indices to canonicalize- Returns:
- a list of canonicalized blocks that are non-empty, sorted internally, and unique in order
-
validateBlocks
Validates the provided list of blocks to ensure that all indices within each block are non-negative, within the range of columns in the given dataset, and not null. Throws an IllegalArgumentException if any of these conditions are violated.- Parameters:
blocks- a list of lists, where each inner list represents a block of indices to validatedata- the dataset providing the number of columns for range validation
-
toSpec
Converts a list of block indices and a dataset into a BlockSpec object, ensuring the blocks are canonicalized and generating the appropriate block variables.- Parameters:
blocks- a list of lists, where each inner list represents a block of indicesdataSet- the dataset associated with the blocks- Returns:
- a BlockSpec object containing the dataset, canonicalized blocks, and block variables
-
toSpec
Converts a list of blocks, ranks, and a dataset into a BlockSpec object. The blocks are canonicalized to ensure uniformity, and block variables are generated based on the canonicalized blocks and dataset.- Parameters:
blocks- a list of lists, where each inner list represents a block of indicesranks- a list of integers representing the ranks associated with the blocksdataSet- the dataset associated with the blocks, providing the variables for block creation- Returns:
- a BlockSpec object containing the dataset, canonicalized blocks, block variables, and ranks
-
expandLatents
-
makeDisjointBySize
Creates a list of disjoint blocks from the provided list of blocks, prioritizing larger blocks first. Each block is processed to ensure no overlapping indices, and elements within processed blocks are sorted. The resulting list is unmodifiable and contains unique, disjoint, and sorted blocks.- Parameters:
blocks- a list of lists, where each inner list represents a block of indices to be made disjoint- Returns:
- a list of disjoint blocks, where each block is a sorted and unmodifiable list of indices
-
makeDisjointSpec
Constructs a BlockSpec object using the provided DataSet and block definitions, ensuring that the blocks are made disjoint by prioritizing larger blocks first. The resulting BlockSpec includes the dataset, the disjoint blocks, and associated block variables.- Parameters:
ds- the dataset associated with the blocksblocks- a list of lists, where each inner list represents a block of indices- Returns:
- a BlockSpec object containing the dataset, disjoint blocks, and block variables
-
applySingleClusterPolicy
public static BlockSpec applySingleClusterPolicy(BlockSpec blockSpec, SingletonClusterPolicy policy, double alpha) Applies a single-cluster policy to the provided BlockSpec. Depending on the specified policy, the method modifies the blocks, ranks, and variables in the BlockSpec and returns a new BlockSpec object.- Parameters:
blockSpec- the BlockSpec object containing the current block configuration, ranks, and datasetpolicy- the SingletonClusterPolicy to apply, which determines how unused columns or variables are handled (e.g., INCLUDE, EXCLUDE, NOISE_VAR)alpha- a double value representing a parameter used in the computation of ranks- Returns:
- a new BlockSpec object that reflects the changes made according to the specified policy
-
giveGoodLatentNames
public static BlockSpec giveGoodLatentNames(BlockSpec spec, Map<String, List<String>> trueClusters, BlocksUtil.NamingMode mode) Assigns meaningful names to latent variables in the provided BlockSpec object based on the given true clusters and the specified naming mode. This helps in creating more interpretable and user-friendly block specifications.- Parameters:
spec- the BlockSpec object containing the initial latent variable definitionstrueClusters- a map where keys represent cluster names and values are lists of variable names associated with each clustermode- the NamingMode specifying how the latent variables should be named- Returns:
- a BlockSpec object with updated latent variable names based on the true clusters and naming mode
-