Class LongToWide

java.lang.Object
edu.cmu.tetrad.data.LongToWide
public final class LongToWide extends Object
Reshapes a long-format data set (one row per unit per occasion, e.g., one row per student per assessment) into a wide-format data set (one row per unit, one column per occasion per measured variable). This is the reshaping a causal search needs when the rows of the file are not the units of analysis: searching on the long table would treat each unit's repeated rows as independent cases.

The transform is specified by:

  • row keys: one or more variables whose joint values identify a unit; each distinct combination becomes one output row, in order of first appearance;
  • column key: one discrete variable whose levels become column suffixes;
  • value variables: the variables to spread, by default every variable that is neither a row key nor the column key. Each value variable v yields one output column v.level per level of the column key, of the same type (and, for discrete variables, the same categories) as v.
A (unit, level) combination absent from the long data becomes a missing value in the wide data. A combination present more than once (a second attempt, say) is resolved by the LongToWide.Aggregation policy, which defaults to LongToWide.Aggregation.FAIL so that duplicates are never silently collapsed.

The result carries a findings-only LongToWide.Result.report() (units, levels, per-column fill rates, duplicates resolved) in the manner of the data audit; what to do about sparse columns is a decision for the user.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static enum 
    LongToWide.Aggregation
    How multiple long rows for the same (unit, level) combination are combined into one wide cell.
    static final record 
    LongToWide.Result
    The wide data set, the unit keys in row order (one list of row-key values per wide row), and a findings-only report.

  • Constructor Summary

    Constructors
    Constructor
    Description
    LongToWide(String columnKey, String... rowKeys)
    Constructs a transform with the given row key(s) and column key.

  • Method Summary

    Modifier and Type
    Method
    Description
    LongToWide.Result
    apply(DataSet data)
    Applies the transform.
    List<String>
    columnNames(DataSet data)
    Returns the names the wide columns would have, in order, without building the wide data set: the kept row keys (if any), then for each value variable one name per observed level of the column key.
    LongToWide
    setAggregation(String variable, LongToWide.Aggregation aggregation)
    Sets the aggregation used for a value variable whose (unit, level) combinations are duplicated.
    LongToWide
    setDefaultAggregation(LongToWide.Aggregation aggregation)
    Sets the aggregation used for value variables not given one by setAggregation(String, Aggregation).
    LongToWide
    setIncludeValueName(boolean includeValueName)
    If false, and exactly one value variable is spread, output columns are named by level alone (e.g., Final rather than score.Final).
    LongToWide
    setKeepRowKeys(boolean keepRowKeys)
    If true, the row key variable(s) are kept as the leading column(s) of the wide data set (as discrete variables, one category per observed value).
    LongToWide
    setLevelNames(Map<String,String> levelNames)
    Sets short names for levels of the column key: an output column for level L uses levelNames.get(L) in place of L when present.
    LongToWide
    setSanitizeNames(boolean sanitizeNames)
    If true (the default), output column names have runs of characters other than letters, digits, underscore, period, and hyphen replaced by a single underscore, so that they survive whitespace-delimited contexts such as knowledge files.
    LongToWide
    setSeparator(String separator)
    Sets the separator between a value variable's name and the level suffix.
    LongToWide
    setValueVariables(List<String> names)
    Sets the value variables to spread.
    static Map<String,String>
    suggestShortLevelNames(List<String> levels, int maxLength)
    Suggests short, unique, sanitized names for a set of levels, for the user to edit: each level is sanitized (as by setSanitizeNames(boolean)), split into tokens at underscores, tokens shared by more than half of the levels are dropped when at least three levels are present (so "Mastery" and "Assessment" fall away while "Unit_01" and "ver_A" stay), the remaining tokens are rejoined, and the result is truncated to maxLength characters.

    Methods inherited from class Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • LongToWide

      public LongToWide(String columnKey, String... rowKeys)
      Constructs a transform with the given row key(s) and column key. Value variables default to every other variable of the data set; see setValueVariables(List).
      Parameters:
      columnKey - The name of the discrete variable whose levels become column suffixes.
      rowKeys - The names of the variable(s) identifying a unit.

  • Method Details

    • setValueVariables

      public LongToWide setValueVariables(List<String> names)
      Sets the value variables to spread. An empty list (the default) means every variable that is not a row key or the column key.
      Parameters:
      names - The variable names.
      Returns:
      This transform.

    • setAggregation

      public LongToWide setAggregation(String variable, LongToWide.Aggregation aggregation)
      Sets the aggregation used for a value variable whose (unit, level) combinations are duplicated.
      Parameters:
      variable - The value variable.
      aggregation - The aggregation.
      Returns:
      This transform.

    • setDefaultAggregation

      public LongToWide setDefaultAggregation(LongToWide.Aggregation aggregation)
      Sets the aggregation used for value variables not given one by setAggregation(String, Aggregation). The default is FAIL.
      Parameters:
      aggregation - The aggregation.
      Returns:
      This transform.

    • setSanitizeNames

      public LongToWide setSanitizeNames(boolean sanitizeNames)
      If true (the default), output column names have runs of characters other than letters, digits, underscore, period, and hyphen replaced by a single underscore, so that they survive whitespace-delimited contexts such as knowledge files.
      Parameters:
      sanitizeNames - Whether to sanitize.
      Returns:
      This transform.

    • setKeepRowKeys

      public LongToWide setKeepRowKeys(boolean keepRowKeys)
      If true, the row key variable(s) are kept as the leading column(s) of the wide data set (as discrete variables, one category per observed value). The default is false, since a unit identifier has one category per row and is of no use to a search; the keys are always available from LongToWide.Result.unitKeys().
      Parameters:
      keepRowKeys - Whether to keep the row keys.
      Returns:
      This transform.

    • setLevelNames

      public LongToWide setLevelNames(Map<String,String> levelNames)
      Sets short names for levels of the column key: an output column for level L uses levelNames.get(L) in place of L when present. Levels not in the map keep their own names. See suggestShortLevelNames(List, int) for a starting point.
      Parameters:
      levelNames - Map from level (category string of the column key) to the name to use for it.
      Returns:
      This transform.

    • setIncludeValueName

      public LongToWide setIncludeValueName(boolean includeValueName)
      If false, and exactly one value variable is spread, output columns are named by level alone (e.g., Final rather than score.Final). With more than one value variable the value name is always included, since the level alone would not be unique. The default is true.
      Parameters:
      includeValueName - Whether to prefix the level with the value variable's name.
      Returns:
      This transform.

    • suggestShortLevelNames

      public static Map<String,String> suggestShortLevelNames(List<String> levels, int maxLength)
      Suggests short, unique, sanitized names for a set of levels, for the user to edit: each level is sanitized (as by setSanitizeNames(boolean)), split into tokens at underscores, tokens shared by more than half of the levels are dropped when at least three levels are present (so "Mastery" and "Assessment" fall away while "Unit_01" and "ver_A" stay), the remaining tokens are rejoined, and the result is truncated to maxLength characters. Names made identical by this process are disambiguated with a numeric suffix. This is a heuristic starting point, not a decision: the names should be reviewed.
      Parameters:
      levels - The levels, in the order the columns will be produced.
      maxLength - The maximum length of a suggested name (at least 4).
      Returns:
      A map from level to suggested name, in level order.

    • setSeparator

      public LongToWide setSeparator(String separator)
      Sets the separator between a value variable's name and the level suffix. The default is ".".
      Parameters:
      separator - The separator.
      Returns:
      This transform.

    • columnNames

      public List<String> columnNames(DataSet data)
      Returns the names the wide columns would have, in order, without building the wide data set: the kept row keys (if any), then for each value variable one name per observed level of the column key. This is what apply(DataSet) produces, and it lets an editor preview the outcome of a configuration.
      Parameters:
      data - The long-format data set.
      Returns:
      The wide column names.
      Throws:
      IllegalArgumentException - As apply(DataSet), for an invalid specification.

    • apply

      public LongToWide.Result apply(DataSet data)
      Applies the transform.
      Parameters:
      data - The long-format data set.
      Returns:
      The wide data set with its report.
      Throws:
      IllegalArgumentException - If a named variable is absent, the column key is not discrete, or duplicated combinations exist under the FAIL aggregation.