Class MissingnessIndicatorAdder

java.lang.Object
edu.cmu.tetrad.search.utils.MissingnessIndicatorAdder
public final class MissingnessIndicatorAdder extends Object
Appends missingness indicators to a dataset. For each selected variable X, an indicator X_missing is added that records, for every case, whether X was missing in that case. The original columns are left exactly as they are, so the substantive variables still carry their missing values and are still handled by whatever missing-data policy the score or test is using.

The point of doing this is that the missingness mechanism becomes part of the graph rather than a nuisance the score has to work around. Edges into an indicator say what the mechanism depends on; edges among indicators say how the losses are coupled across variables. The one thing this cannot recover is self-masking, the edge from X to its own indicator, because X is observed on exactly the cases where the indicator says "observed" and is therefore constant on every case the score can use. That edge is unscorable, not merely hard, and by default it is forbidden in the knowledge this class attaches.

Two conventions matter and are easy to get backwards. The indicator is 1, or category "missing", when the value is absent; this matches the name. And an indicator is only worth adding when it actually varies: a variable with no missing values, or with nothing but missing values, yields a constant column that no score can use and that will make some scores singular. The rate window in the spec is what keeps those out.

Indicators are discrete by default, which requires a score that accepts mixed data, such as the degenerate Gaussian or conditional Gaussian score. Setting discreteIndicators to false emits them as continuous 0/1 columns so that a purely continuous score can be used; that is a linear-probability approximation to a binary variable and should be read as such.

Author:
josephramsey

  • Method Details

    • addMissingnessIndicators

      public static DataSet addMissingnessIndicators(DataSet dataSet)
      Adds missingness indicators using the default spec.
      Parameters:
      dataSet - The original dataset.
      Returns:
      A new dataset with indicators appended.

    • add

      public static MissingnessIndicatorAdder.Result add(DataSet dataSet, MissingnessIndicatorAdder.Spec spec)
      Adds missingness indicators under the given spec.
      Parameters:
      dataSet - The original dataset. Not modified.
      spec - The options to use.
      Returns:
      The augmented dataset plus a record of what was added and what was skipped.