extracts= option for insert/update/etc

[simonw]

Relates to #42 and #44. I want the ability to extract values out into lookup tables during bulk insert/upsert operations.

db.insert_all(rows, extracts=["species"])

• creates species table for values in the species column

db.insert_all(rows, extracts={"species": "Species"})

• as above but the new table is called Species.

[simonw]

This will play very well with the new ability to specify these arguments in the class constructor, like so:

    table = db.table("trees", extracts={"species_id": "Species"})
    table.insert_all([{
        "id": 1,
        "species_id": "Oak",
    }, {
        "id": 2,
        "species_id": "Oak",
    }, {
        "id": 3,
        "species_id": "Palm",
    }])

[chrishas35]

@simonw any thoughts on allow extracts to specify the lookup column name? If I'm understanding the documentation right, .lookup() allows you to define the "value" column (the documentation uses name), but when you use extracts keyword as part of .insert(), .upsert() etc. the lookup must be done against a column named "value". I have an existing lookup table that I've populated with columns "id" and "name" as opposed to "id" and "value", and seems I can't use extracts=, unless I'm missing something...

Initial thought on how to do this would be to allow the dictionary value to be a tuple of table name column pair... so:

table = db.table("trees", extracts={"species_id": ("Species", "name"})

I haven't dug too much into the existing code yet, but does this make sense? Worth doing?

[simonw]

That seems sensible to me. Opening a ticket for it.