What is the best way to merge two datasets?

[bowbahdoe]

I am translating the following pandas code

initial = pd.read_excel("Initial.xlsx")
final = pd.read_excel("Final.xlsx")

categories = final[["Tag","Category"]].merge(initial[["Tag","Category"]], on = ["Tag","Category"], how = "outer")

And I don't know what to do to get equivalent semantics.

My first instinct was this

  (let [initial    (first (excel/workbook->datasets "Initial.xlsx"))
        final      (first (excel/workbook->datasets "Final.xlsx"))
        categories (join/left-join "Tag"
                    (dataset/select-columns initial ["Tag" "Category"])
                    (dataset/select-columns final ["Tag" "Category"]))

But I think it misses the mark. There is no "outer" join and the column names don't mesh either. I get "Sheet1.Category" as an extra column

[cnuernber]

Tablecloth offers a full join. I will take a look at merge and see what its semantics are. As a work around the low level hash join returns a lot more information you may be able to use to construct what you want.

[cnuernber]

The tech.v3.dataset.join namespace now includes pd-merge that should match pretty much exactly the semantics you are looking for.

Keep in mind that the metadata on the returned dataset contains maps that tell you how column names are mangled when they are mangled in order be able to map from the old name to the new name.