Class JoinDatasets

java.lang.Object
edu.cmu.tetrad.data.JoinDatasets
public final class JoinDatasets extends Object
Joins two tabular data sets on a key, in the manner of a database join: each row of the left data set is looked up in the right data set by its key value(s), and the right data set's non-key columns are appended to it. This is the operation needed to attach, say, a table of per-student background variables to a wide table with one row per student, when the two tables share an (anonymized) student identifier.

The join is specified by:

  • left keys and right keys: one or more variables on each side, paired by position (the first left key matches the first right key, and so on), so the names need not agree and composite keys such as student-plus-course are allowed;
  • a JoinDatasets.JoinType: JoinDatasets.JoinType.LEFT (the default) keeps every left row, filling the right columns with missing values where there is no match; JoinDatasets.JoinType.INNER keeps only matched rows; JoinDatasets.JoinType.FULL_OUTER additionally appends the unmatched right rows at the end, with the left columns missing;
  • whether a key may repeat on the right (setAllowOneToMany(boolean), off by default). When a right key repeats, each matching left row is emitted once per right row, which multiplies rows and breaks the independence of cases; so by default this is reported as an error rather than done silently. Repeated keys on the left are the ordinary many-to-one lookup and are always allowed.
Key values are matched by their string representation, not by variable type, so an identifier loaded as a discrete variable in one file and as an integer-valued continuous variable in the other still matches: numeric strings are canonicalized so that "1234" and 1234.0 agree. A row whose key is missing (in any component) never matches anything, on either side, as in SQL.

The key is emitted once, as the left data set's key variable(s). Non-key columns of the two tables that share a name are both renamed with suffixes (setLeftSuffix(String), setRightSuffix(String); default "_left" and "_right"). Left row order is preserved; unmatched right rows, if kept, follow in right row order.

The result carries a findings-only JoinDatasets.Result.report() (row and key counts, match rates, missing keys, repeated keys, renamed columns) in the manner of the data audit.

  • Constructor Details

    • JoinDatasets

      public JoinDatasets(List<String> leftKeys, List<String> rightKeys)
      Constructs a join on the given keys, paired by position.
      Parameters:
      leftKeys - The key variable names in the left data set.
      rightKeys - The key variable names in the right data set, the same number as the left keys.

    • JoinDatasets

      public JoinDatasets(String key)
      Constructs a join on a single key with the same name on both sides.
      Parameters:
      key - The key variable name.

  • Method Details

    • setJoinType

      public JoinDatasets setJoinType(JoinDatasets.JoinType joinType)
      Sets the join type (default JoinDatasets.JoinType.LEFT).
      Parameters:
      joinType - The join type.
      Returns:
      This, for chaining.

    • setAllowOneToMany

      public JoinDatasets setAllowOneToMany(boolean allowOneToMany)
      Sets whether a key value may occur in more than one right row (default false). If false, a repeated right key is an error; if true, each matching left row is emitted once per right row.
      Parameters:
      allowOneToMany - Whether to allow repeated right keys.
      Returns:
      This, for chaining.

    • setLeftSuffix

      public JoinDatasets setLeftSuffix(String leftSuffix)
      Sets the suffix appended to a left non-key column whose name collides with a right non-key column (default "_left").
      Parameters:
      leftSuffix - The suffix.
      Returns:
      This, for chaining.

    • setRightSuffix

      public JoinDatasets setRightSuffix(String rightSuffix)
      Sets the suffix appended to a right non-key column whose name collides with a left column (default "_right").
      Parameters:
      rightSuffix - The suffix.
      Returns:
      This, for chaining.

    • apply

      public JoinDatasets.Result apply(DataSet left, DataSet right)
      Applies the join.
      Parameters:
      left - The left (primary) data set.
      right - The right (lookup) data set.
      Returns:
      The joined data set with its report.
      Throws:
      IllegalArgumentException - If a key variable is absent, a column is of an unsupported type, or a right key repeats and one-to-many joins are not allowed.