Multiple portal metric bugs are persisting

[mburrus]

Describe the bug
Some portals experience no metric issues and all graphs/tables/wheels function as expected. Other portal metric pages have one or more of the following issues. These issues appear to be unique to portal metrics only.

I couldn't identify why only some portals have these issues or what is causing just one issue to appear versus all three issues.

1. The download/view/citation tables incorrectly report that the count is 0, rather than stating that the tables are failing to render (metrics-service#85, ess-dive-catalog#455)
2. Some metric graphs don't render (it's inconsistent which graphs don't work) and there's no spiny wheel or any messaging, there's just empty space
3. The summary wheel, file formats wheel, and time period graph have a never-ending spiny wheel

Other Considerations

• Is the following issue related to this? Or is this issue completely resolved? I think it might be unrelated/resolved. metacatui#1927: Portal metrics are missing from recent months
• It seems like either (a) any portal with datasets that were created recently will have at least one issue OR (b) any portal that was submitted recently will have at least one issue.

To Reproduce
Check out these portals.

ESS-DIVE

• Portals with all three bugs: watershed sfa, ATWE
• Portals with no issues: WHONDRS, SPRUCE, CDIAC

Arctic Data Center

• Portals with all three bugs: AIS, SocialScience
• Portals with at least one bug: DBO, ICECAPS, GreenTracs
• Experiences no issues: didn't find one, I opened ~10 portals

DataOne

• I did not find any portals with issues

[rushirajnenuji]

Thanks for reporting this @mburrus! I believe the issue (#1927) was resolved. Let me check what's causing this issue and I'll update the ticket when I have narrowed down the bug.

[vchendrix]

A quick look at the portals shows a 500 error on the metrics service for https://data.ess-dive.lbl.gov/portals/watershed-function-sfa/Metrics

Metrics Service

Payload

metricsRequest: {"metricsPage":{"total":0,"start":0,"count":0},"metrics":["citations","downloads","views"],"filterBy":[{"filterType":"portal","values":["watershed-function-sfa"],"interpretAs":"list"},{"filterType":"month","values":["01/01/2012","08/31/2022"],"interpretAs":"range"},{"filterType":"query","values":["(((projectText:\"Watershed Function SFA\" OR title:\"Water Quality Parameters at the Elkhorn Slough National Estuarine Research Reserve \(ESNERR\)\")) OR (isPartOf:\"urn\:uuid\:eff58bde\-8bc9\-4947\-bcb2\-556a4549a81e\" OR seriesId:\"doi\:10.15485\/1734791\")) AND (-obsoletedBy:* AND formatType:METADATA)"],"interpretAs":"list"}],"groupBy":["month"]}

Response

{"title": "500 Internal Server Error"}

Solr Service

Form Data

q: (((projectText:"Watershed%20Function%20SFA"%20OR%20title:"Water%20Quality%20Parameters%20at%20the%20Elkhorn%20Slough%20National%20Estuarine%20Research%20Reserve%20%5C(ESNERR%5C)"))%20OR%20(isPartOf:"urn%5C%3Auuid%5C%3Aeff58bde%5C-8bc9%5C-4947%5C-bcb2%5C-556a4549a81e"%20OR%20seriesId:"doi%5C%3A10.15485%5C%2F1734791"))%20AND%20(-obsoletedBy:*%20AND%20formatType:METADATA)
fq: -formatId:*dataone.org/collections* AND -formatId:*dataone.org/portals* AND formatType:METADATA AND -obsoletedBy:*
stats: true
stats.field: size
facet: true
facet.field: formatId
facet.field: beginDate
facet.field: endDate
f.formatId.facet.mincount: 1
f.formatId.facet.missing: false
f.beginDate.facet.mincount: 1
f.endDate.facet.mincount: 1
f.beginDate.facet.missing: false
f.endDate.facet.missing: false
facet.limit: -1
facet.range: dateUploaded
facet.range.start: 1900-01-01T00:00:00.000Z
facet.range.end: 2022-08-31T15:42:02.602Z
facet.range.gap: +1MONTH
f.dateUploaded.facet.missing: true
rows: 0
wt: json
facet.query: {!key=<1950}(beginDate:[* TO 1949-12-31T23:59:59.999Z/YEAR])
facet.query: {!key=1950-1959}(beginDate:[* TO 1959-12-31T23:59:59.999Z/YEAR] AND endDate:[1950-01-01T00:00:00.000Z/YEAR TO *])
facet.query: {!key=1960-1969}(beginDate:[* TO 1969-12-31T23:59:59.999Z/YEAR] AND endDate:[1960-01-01T00:00:00.000Z/YEAR TO *])
facet.query: {!key=1970-1979}(beginDate:[* TO 1979-12-31T23:59:59.999Z/YEAR] AND endDate:[1970-01-01T00:00:00.000Z/YEAR TO *])
facet.query: {!key=1980-1989}(beginDate:[* TO 1989-12-31T23:59:59.999Z/YEAR] AND endDate:[1980-01-01T00:00:00.000Z/YEAR TO *])
facet.query: {!key=1990-1999}(beginDate:[* TO 1999-12-31T23:59:59.999Z/YEAR] AND endDate:[1990-01-01T00:00:00.000Z/YEAR TO *])
facet.query: {!key=2000-2009}(beginDate:[* TO 2009-12-31T23:59:59.999Z/YEAR] AND endDate:[2000-01-01T00:00:00.000Z/YEAR TO *])
facet.query: {!key=2010-2019}(beginDate:[* TO 2019-12-31T23:59:59.999Z/YEAR] AND endDate:[2010-01-01T00:00:00.000Z/YEAR TO *])
facet.query: {!key=2020-2022}(beginDate:[* TO 2022-12-31T23:59:59.999Z/YEAR] AND endDate:[2020-01-01T00:00:00.000Z/YEAR TO *])

Response

<?xml version="1.0" encoding="UTF-8"?><error detailCode="2821" errorCode="500" name="ServiceFailure">
    <description>Solr server error: Error from server at http://db-solr:8983/solr/dataone: undefined field %20OR%20title</description>
</error>

[vchendrix]

The errors above are the same for https://data.ess-dive.lbl.gov/portals/ATWE/Metrics

[rushirajnenuji]

Update: the usage and citation metrics issues have been sorted out.

• re: infinite spinners, at an initial glance both title and isPartOf seem valid SOLR fields, so I'll investigate further and report back on why SOLR is returning 500 Internal Server Error

[mbjones]

@rushirajnenuji could you update the status on these metrics service issues please?

[rushirajnenuji]

I've added a fix for the infinite spinners caused by 500 Internal Server Error from SOLR. The issue was MetacatUI URL encoding POST q parameter sent as part of the request body, which was causing errors with SOLR. The fix is currently in bugfix-2041-StatsModel-POST-queries branch and is ready for testing.

[robyngit]

bugfix-2041-StatsModel-POST-queries is merged into develop now for the next release. What remains to do for this issue?

[rushirajnenuji]

This was the last piece that needed fixing, the usage and citation issues were resolved a couple of weeks ago. We can close this issue when we get this to production.