CLB: failure of Project - Sectors report #1007

yroskov · 2022-02-16T16:01:23Z

CLB on PROD
Frontend version: 1bb80f5 February 11, 2022 1:19 PM Backend version: c94dbef February 15, 2022 12:00 AM

I have opened Project - Sectors and typed in "ITIS" awaiting report on ITIS sectors. I have got a white screen (9.45-10.00 am, Champaign).

White screen as a result:

yroskov · 2022-02-16T16:02:24Z

Note: I did re-login. Result is the same.

timrobertson100 · 2022-02-16T16:17:08Z

Thomas is on holiday, so I took a look.

It takes you to https://data.catalogueoflife.org/catalogue/3/sector?limit=100&offset=0&subjectDatasetKey=2144 and the browser gives JS error reading the subject.name.toString()

The API being called is this https://api.catalogueoflife.org/dataset/3/sector?datasetKey=3&limit=100&offset=0&subjectDatasetKey=2144

Looking at the JSON, the only suspicious bit seems to be this (note the broken bit):

{
  "created": "2020-08-25T21:00:51.89555",
  "createdBy": 102,
  "modified": "2022-02-16T15:53:32.92273",
  "modifiedBy": 103,
  "datasetKey": 3,
  "id": 867,
  "target": {
    "id": "57b4cba3-9a7d-493b-b44c-866591658f54",
    "name": "Coleoptera",
    "rank": "order",
    "broken": false,
    "label": "Coleoptera",
    "labelHtml": "Coleoptera"
  },
  "subjectDatasetKey": 2144,
  "subject": {
    "broken": true,
    "label": "null",
    "labelHtml": "null"
  },
  "originalSubjectId": "109217",
  "mode": "attach",
  "syncAttempt": 25,
  "size": 116
},

@mdoering - looks like a bad API response?

timrobertson100 · 2022-02-16T16:23:09Z

If it is not a bad response, then I think this line should read:

                    textToHighlight={
                      _.get(record, "subject.name")
                        ? record.subject.name.toString()
                        : ""

(and perhaps this one too)

mdoering · 2022-02-16T16:29:25Z

Thanks Tim, looking into this. It's an unexpected response. If the subject (or target) of a sector is "broken" this means we cannot link to the source and do not have a working ID at hand. The name should have been stored when the sector was created and should not change - it is actually used to rematch broken sectors and lookup the current id.

Sth must have gone wrong when the sector got created. Hm, it was created in 2020 and only now modified.

mdoering · 2022-02-16T16:33:09Z

looking at the database there is just this one ITIS sector without a subject name, odd.
I will try to manually correct it based on older releases...

col=> select * from sector where dataset_key=3 and subject_dataset_key=2144 and subject_name is null;
-[ RECORD 1 ]-------+-------------------------------------
id                  | 867
dataset_key         | 3
subject_dataset_key | 2144
subject_rank        | 
subject_code        | 
subject_status      | 
target_rank         | ORDER
target_code         | 
mode                | ATTACH
code                | 
sync_attempt        | 25
created_by          | 102
modified_by         | 103
created             | 2020-08-25 21:00:51.89555
modified            | 2022-02-16 15:53:32.92273
subject_id          | 
subject_name        | 
subject_authorship  | 
subject_parent      | 
target_id           | 57b4cba3-9a7d-493b-b44c-866591658f54
target_name         | Coleoptera
target_authorship   | 
note                | 
ranks               | {}
entities            | 
original_subject_id | 109217
placeholder_rank    | 
dataset_attempt     | 48

mdoering · 2022-02-16T16:37:31Z

something happened in january 2021. That was shortly after we went live, so maybe we had some early bugs still.
@yroskov I am reverting to Archostemata, does that make sense?

col=> select dataset_key,original_subject_id,subject_id,subject_rank,subject_code,subject_status,subject_name,subject_authorship,subject_parent, modified from sector where subject_dataset_key=2144 and id=867 order by dataset_key;
 dataset_key | original_subject_id | subject_id | subject_rank | subject_code | subject_status | subject_name | subject_authorship | subject_parent |          modified          
-------------+---------------------+------------+--------------+--------------+----------------+--------------+--------------------+----------------+----------------------------
           3 | 109217              |            |              |              |                |              |                    |                | 2022-02-16 15:53:32.92273
        2237 | 109217              | 109217     | SUBORDER     | ZOOLOGICAL   | ACCEPTED       | Archostemata | Kolbe, 1908        | 109216         | 2020-08-25 21:00:51.89555
        2242 | 109217              | 109217     | SUBORDER     | ZOOLOGICAL   | ACCEPTED       | Archostemata | Kolbe, 1908        | 109216         | 2020-08-25 21:00:51.89555
        2296 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2303 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2315 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2328 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2332 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2344 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2349 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2351 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2366 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
        2368 | 109217              | 109217     |              |              |                |              |                    |                | 2021-01-27 17:01:41.988956
(13 rows)

mdoering · 2022-02-16T16:39:32Z

Archostemata is a suborder of Coleoptera, so looks good

yroskov · 2022-02-16T16:44:43Z

Yes, according to my logs, suborder Archostemata from ITIS was assembled on 2020-08-25

yroskov · 2022-02-16T16:48:47Z

Unfortunately, I cannot confirm that exactly Archostemata is missing among ITIS sectors, because I still have a white window on my screen for ITIS sectors report:
https://data.catalogueoflife.org/catalogue/3/sector?limit=100&offset=0&subjectDatasetKey=2144

yroskov · 2022-02-16T16:51:10Z

Before a white window, "error 403" message flashes for a second.

yroskov · 2022-02-16T16:54:13Z

Re-login failed with 401 error:

mdoering · 2022-02-16T16:58:58Z

Updated the database manually, works for me now

mdoering · 2022-02-16T17:00:03Z

That sector now also shows as broken, let me try to rematch it and monitor what happens.
https://data.catalogueoflife.org/catalogue/3/sector?broken=true&limit=100&offset=0&subjectDatasetKey=2144

yroskov · 2022-02-16T17:02:03Z

I also have an access to ITIS sectors report now. Thanks!

No Sync option in this broken sector

mdoering · 2022-02-16T17:02:06Z

Fails to rematch, but the name stays and it simply remains as broken.
I guess I hand over to you to manually rematch it, delete it or do what you think should be done with that sector.
There should be 116 names in the project for that sector still.

yroskov · 2022-02-16T17:02:59Z

OK, I'll do re-assembly of Archostemata from new ITIS.

mdoering · 2022-02-16T17:03:10Z

No Sync option in this broken sector

Obviously not if its broken. You can manually edit the sector if you know where in ITIS this now belongs

mdoering · 2022-02-16T17:03:58Z

Let me try to fix this, I am surprised the rematch did not work cause it still exists:
https://data.catalogueoflife.org/dataset/2144/taxon/109217

mdoering · 2022-02-16T17:05:46Z

Worked like a charm to update the subject by searching for Archostemata in the edit menu!
https://data.catalogueoflife.org/catalogue/3/sector?limit=100&name=Archostemata&offset=0&subjectDatasetKey=2144

yroskov · 2022-02-16T17:05:55Z

woops: I already kicked off deletion of Archostemata as a sector in Col

mdoering · 2022-02-16T17:06:37Z

Also fine, but to preserve history it is always preferrable to rematch and keep the sector

mdoering · 2022-02-16T17:09:16Z

I verified all sectors in the database and this was the only 1 record missing a subject name. Must have been some weird relict so there is no need to protect the UI against this unexpected response.

yroskov · 2022-02-16T17:09:50Z

Good. many thanks!

yroskov mentioned this issue Feb 16, 2022

CLB: blank sector in the report on ITIS sectors - it's not flagged as a broken CatalogueOfLife/backend#1105

Closed

mdoering closed this as completed Feb 16, 2022

yroskov mentioned this issue Feb 16, 2022

ITIS (id 2144): test report CatalogueOfLife/testing#8

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLB: failure of Project - Sectors report #1007

CLB: failure of Project - Sectors report #1007

yroskov commented Feb 16, 2022 •

edited

Loading

yroskov commented Feb 16, 2022

timrobertson100 commented Feb 16, 2022 •

edited

Loading

timrobertson100 commented Feb 16, 2022 •

edited

Loading

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022 •

edited

Loading

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022

yroskov commented Feb 16, 2022

yroskov commented Feb 16, 2022 •

edited

Loading

yroskov commented Feb 16, 2022

yroskov commented Feb 16, 2022

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022 •

edited

Loading

yroskov commented Feb 16, 2022

mdoering commented Feb 16, 2022

yroskov commented Feb 16, 2022

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022

yroskov commented Feb 16, 2022

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022

yroskov commented Feb 16, 2022

CLB: failure of Project - Sectors report #1007

CLB: failure of Project - Sectors report #1007

Comments

yroskov commented Feb 16, 2022 • edited Loading

yroskov commented Feb 16, 2022

timrobertson100 commented Feb 16, 2022 • edited Loading

timrobertson100 commented Feb 16, 2022 • edited Loading

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022 • edited Loading

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022

yroskov commented Feb 16, 2022

yroskov commented Feb 16, 2022 • edited Loading

yroskov commented Feb 16, 2022

yroskov commented Feb 16, 2022

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022 • edited Loading

yroskov commented Feb 16, 2022

mdoering commented Feb 16, 2022

yroskov commented Feb 16, 2022

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022

yroskov commented Feb 16, 2022

mdoering commented Feb 16, 2022

mdoering commented Feb 16, 2022

yroskov commented Feb 16, 2022

yroskov commented Feb 16, 2022 •

edited

Loading

timrobertson100 commented Feb 16, 2022 •

edited

Loading

timrobertson100 commented Feb 16, 2022 •

edited

Loading

mdoering commented Feb 16, 2022 •

edited

Loading

yroskov commented Feb 16, 2022 •

edited

Loading

mdoering commented Feb 16, 2022 •

edited

Loading