Browse collection object

[bpescador]

I am really struggling to figure out what I've achieved so far with importing. After a record is imported using the DWC importer, the first thing a user needs to do is validate if the fields were correctly parsed to the TW fields The only option given is to click on Browse for those that were successfully imported:

https://sandwich.taxonworks.org/tasks/collection_objects/browse?collection_object_id=1273169

From this page, however, there is little information provided. I don't see what the browse tool is providing. It should provide a clear listing of all data fields for that record.

The tool should allow one to check if the fields imported were mapped. For example, was the determiner and date of determination correctly imported? It is not clear to me in the example above.

Each value should also be hotlinked so if you click on an otu, locality, or determiner, you are taken to that particular page.

If I click on edit object: https://sandwich.taxonworks.org/tasks/accessions/comprehensive?collection_object_id=1273169

I see that the determiner is not included and also the added attributes are not present for editing.

[LocoDelAssembly]

@bpescador not sure what you mean about determiner. In the first link it shows:

Determined	2020-11-30	TaxonDetermination	Pheidole navoatrensis Salata & Fisher, 2020 ✓ by Salata on 2020-11-30 by Salata on 2020-11-30
			* Looks the display is duplicating the determiner

(Also visible in second link in Determinations panel)

As for the hotlinks you mean that for instance if I click on "Imported" link I'm given a manifest of what was imported/re-used and if I click on each I'm redirected to a page specific to that kind of information? (e.g. going to https://sandwich.taxonworks.org/taxon_determinations/1230992 when clicking on determination)

[bpescador]

If the determiner and date is correctly imported, I thought it would fill in the appropriate fields here: https://sandwich.taxonworks.org/tasks/accessions/comprehensive?collection_object_id=1273169

I don't understand where I should look for the data to verify it was correctly imported;

[bpescador]

By hotlink I mean: from the browse window, https://sandwich.taxonworks.org/tasks/collection_objects/browse?collection_object_id=1273169

if I click on by Salata it takes me here: https://sandwich.taxonworks.org/otus/290434

but why not to: https://sandwich.taxonworks.org/taxon_determinations/1230992 [I could not figure out how to get to that page until you provided the link]

[debpaul]

@bpescador I'm (thinking out loud) wondering if a "report" of sorts would be something we might generate that could help. I would say

As a result of your import to TW [on such-and-such date] you have

• XX new collection objects
• XX new collecting events
• XX new people
• XX new OTUs
• etcetera

The report may be able to have links to those objects?

As you (and I) learn TW, it's clear we'll both know more about where to look for our uploaded information, but I can understand what you are looking for. A way to suss out if all has gone well, all data is in and where expected.

[debpaul]

We could have a task: Summarize
Which would count types of objects, and provide links to them.
A sort of summary dashboard of what's in the datastore.

[mjy]

So I typed out a long response and promptly closed the tab.

We have three things:
A - Data represented by DwC incoming
B - Data represented in TW
C - Data represented by DwC outgoing

The current Browse OTU task is 1) a super simple stub, albeit useful according to @tmcelrath, that is really a placeholder in my mind that 2) presents B, with a superficial reference to C. This clearly doesn't meet the needs of @bpescador.

I think @debpaul is precisely right that what we need is a new task that specifically seeks to validate the input. For example you could imagine three columns one each for A, B, C, with visual groups on each column, and links between those groups to help you trace the flow of data between the columns. Tricky, but not impossible.

I'm not as sold on counts to be honest. In some cases, with simple datasets, they may be useful as validators, but in the scale of @bpescador's data they quickly become not particularly informative beyond min/maxes. Those might be useful, but not "guiding", i.e. they won't tell you where to go look for problems.

It's important to remember, I think, that our importer is a new paradigm- you don't have to import everything at once, nobody else has this type of functionality, where you can piecemeal fix this issues, this is huge, and eliminates some of the all/nothing issues. I.e. first it's important to know that the data are in OK, but after that you can tease out the remaining issues, right in the interface, rather than having to go check counts, for example, and re-do everything from scratch.

So, quick counts, super nice, we can make an endpoint for that, but I think @debpaul suggestion for a new task is really the way to go to address the underlying issue.

@LocoDelAssembly what we somewhat need is a metadata map b/w A->B. I haven't looked at your code in depth, but if we had a constant, or hash that did much of this map, then we could perhaps expose that for use in the comparative interface. We may just have to hard code the linkages b/w the 3 columns as well. For example this field in DwC (date) maps to these 6 fields, potentially, in TW.

[bpescador]

I think these are all good suggestions, but I would suggest following GBIF's lead. Compare how GBIF shows how they mapped imported data for the same specimen:

TW: https://sandwich.taxonworks.org/tasks/collection_objects/browse?collection_object_id=1273163

Compare with GBIF: https://www.gbif.org/occurrence/865078905

I think this would also work for the showing what I expected for Browse collection object: