Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output fieldName of missing_recommended_field notice even if there is no column #1425

Closed
takohei opened this issue May 1, 2023 · 4 comments · Fixed by #1511
Closed

Output fieldName of missing_recommended_field notice even if there is no column #1425

takohei opened this issue May 1, 2023 · 4 comments · Fixed by #1511
Assignees
Labels
bug Something isn't working (crash, a rule has a problem) good first issue Good task for newcomers to work on.

Comments

@takohei
Copy link

takohei commented May 1, 2023

Describe the bug

It is specified in RULES.md that the notice of missing_recommended_field should have fieldNames.

However, if there is no column in the header, the fieldName is not output.
So it is difficult for us to identify which field is unset.

Steps/Code to Reproduce

Validate attached feed.
It has no column for feed_contact_email and no value for feed_contact_url in the feed_info.txt.

Expected Results

In report.json, both feed_contact_email and feed_contact_url notices have fieldNames.

    {
      "code": "missing_recommended_field",
      "severity": "WARNING",
      "totalNotices": 2,
      "sampleNotices": [
        {
          "filename": "feed_info.txt",
          "csvRowNumber": 2.0
          "fieldName": "feed_contact_email "
        },
        {
          "filename": "feed_info.txt",
          "csvRowNumber": 2.0,
          "fieldName": "feed_contact_url"
        }
      ]
    }

Actual Results

In report.json, the feed_contact_url notice has fieldName, but the feed_contact_email notice has no fieldName.

    {
      "code": "missing_recommended_field",
      "severity": "WARNING",
      "totalNotices": 2,
      "sampleNotices": [
        {
          "filename": "feed_info.txt",
          "csvRowNumber": 2.0
        },
        {
          "filename": "feed_info.txt",
          "csvRowNumber": 2.0,
          "fieldName": "feed_contact_url"
        }
      ]
    }

Screenshots

The detail table of report.json has value but no header of fieldName.
image

Files used

no_mail_column_no_url_value_akocity.zip
report.json.txt

Validator version

4.1.0

Operating system

Windows

Java version

18

Additional notes

This is just a guess as I do not have a development environment, but the following code may not get the column name if there is no column header.

} else if (level == FieldLevelEnum.RECOMMENDED && s == null) {
noticeContainer.addValidationNotice(
new MissingRecommendedFieldNotice(
fileName, getRowNumber(), header.getColumnName(columnIndex)));
}

@takohei takohei added bug Something isn't working (crash, a rule has a problem) status: Needs triage Applied to all new issues labels May 1, 2023
@takohei
Copy link
Author

takohei commented May 2, 2023

I checked the variables using the debugger and my guess was correct.

If a recommended column is not in the header, columnIndex will be -1 and getColumnName() will not return the column name, so the feedName will not be output.
I am not sure how to fix it properly.

@isabelle-dr
Copy link
Contributor

Thank you for opening this issue @takohei!
Our team will have a look at it shortly.

@isabelle-dr isabelle-dr added status: Ready An issue that is ready to be worked on. and removed status: Needs triage Applied to all new issues labels May 2, 2023
@takohei
Copy link
Author

takohei commented May 2, 2023

@isabelle-dr
Thanks for your reply and triage.

As a test, I changed the argument of axX()@RowParser.java from FieldLevelEnum to GtfsColumnDescriptor and it worked well.
As follows.

  public String asString(int columnIndex, GtfsColumnDescriptor columnDescriptor) {
    String s = row.asString(columnIndex);
    if (columnDescriptor.fieldLevel() == FieldLevelEnum.REQUIRED && s == null) {
      noticeContainer.addValidationNotice(
          new MissingRequiredFieldNotice(
              fileName, getRowNumber(), columnDescriptor.columnName()));
    } else if (columnDescriptor.fieldLevel() == FieldLevelEnum.RECOMMENDED && s == null) {
      noticeContainer.addValidationNotice(
          new MissingRecommendedFieldNotice(
              fileName, getRowNumber(), columnDescriptor.columnName()));
    }

@isabelle-dr isabelle-dr added this to the Now milestone Jun 12, 2023
@isabelle-dr isabelle-dr added the good first issue Good task for newcomers to work on. label Jun 12, 2023
@qcdyx qcdyx self-assigned this Jun 12, 2023
@qcdyx qcdyx added status: Work in progress A PR that would close this issue has been opened. and removed status: Ready An issue that is ready to be worked on. labels Jun 12, 2023
@qcdyx
Copy link
Contributor

qcdyx commented Jun 20, 2023

Hello @takohei, you are right. Using the GtfsColumnDescriptor is the way to go. I fix this issue by using this approach. Thanks for digging deep into the code.

@qcdyx qcdyx removed the status: Work in progress A PR that would close this issue has been opened. label Jun 21, 2023
qcdyx added a commit that referenced this issue Jun 21, 2023
…if there is no column (#1511)

* replace FieldLevelEnum with GtfsColumnDescriptor

* use columnDescriptor instead of columnDescriptor.fieldLevel() in TableDescriptorGenerator

* fix broken tests

* fixe asString_recommended_missing test
format code

* format code

* remove the old asString method

---------
Close #1425 
Co-authored-by: David Gamez <1192523+davidgamez@users.noreply.github.com>
@emmambd emmambd removed this from the Now milestone Jan 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working (crash, a rule has a problem) good first issue Good task for newcomers to work on.
Projects
4 participants