One unit test is failing on OpenBSD and FreeBSD

[rwestphal]

One test fails when running make test on OpenBSD or FreeBSD:

99% tests passed, 1 tests failed out of 104

Total Test time (real) =  16.40 sec

The following tests FAILED:
         40 - test_conformance (Failed)
Errors while running CTest
*** Error code 8

Here's the full output when I run the associated test program manually:

$ ./tests/test_conformance
[==========] Running 8 test(s).
[ RUN      ] test_implemented1_yin
libyang[0]: Module "b@2015-04-04" in another revision "2015-01-01" already implemented.
libyang[0]: Module "b" parsing failed.
[       OK ] test_implemented1_yin
[ RUN      ] test_implemented1_yang
libyang[0]: Module "b@2015-04-04" in another revision "2015-01-01" already implemented.
libyang[0]: Module "b" parsing failed.
[       OK ] test_implemented1_yang
[ RUN      ] test_implemented2_yin
libyang[0]: Module "b" in another revision already implemented.
libyang[0]: Making the augment target module "b" implemented failed.
libyang[0]: Module "a" parsing failed.
[       OK ] test_implemented2_yin
[ RUN      ] test_implemented2_yang
libyang[0]: Module "b" in another revision already implemented.
libyang[0]: Making the augment target module "b" implemented failed.
libyang[0]: Module "a" parsing failed.
[       OK ] test_implemented2_yang
[ RUN      ] test_implemented_info_yin
[  ERROR   ] --- "<yang-library xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
  <module-set>
    <name>complete</name>
    <checksum>10</checksum>
    <import-only-module>
      <name>ietf-yang-metadata</name>
      <revision>2016-08-05</revision>
      <namespace>urn:ietf:params:xml:ns:yang:ietf-yang-metadata</namespace>
    </import-only-module>
    <module>
      <name>yang</name>
      <revision>2017-02-20</revision>
      <namespace>urn:ietf:params:xml:ns:yang:1</namespace>
    </module>
    <import-only-module>
      <name>ietf-inet-types</name>
      <revision>2013-07-15</revision>
      <namespace>urn:ietf:params:xml:ns:yang:ietf-inet-types</namespace>
    </import-only-module>
    <import-only-module>
      <name>ietf-yang-types</name>
      <revision>2013-07-15</revision>
      <namespace>urn:ietf:params:xml:ns:yang:ietf-yang-types</namespace>
    </import-only-module>
    <import-only-module>
      <name>ietf-datastores</name>
      <revision>2017-08-17</revision>
      <namespace>urn:iet[  FAILED  ] test_implemented_info_yin
[ RUN      ] test_implemented_info_yang
[  ERROR   ] --- "<yang-library xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
  <module-set>
    <name>complete</name>
    <checksum>10</checksum>
    <import-only-module>
      <name>ietf-yang-metadata</name>
      <revision>2016-08-05</revision>
      <namespace>urn:ietf:params:xml:ns:yang:ietf-yang-metadata</namespace>
    </import-only-module>
    <module>
      <name>yang</name>
      <revision>2017-02-20</revision>
      <namespace>urn:ietf:params:xml:ns:yang:1</namespace>
    </module>
    <import-only-module>
      <name>ietf-inet-types</name>
      <revision>2013-07-15</revision>
      <namespace>urn:ietf:params:xml:ns:yang:ietf-inet-types</namespace>
    </import-only-module>
    <import-only-module>
      <name>ietf-yang-types</name>
      <revision>2013-07-15</revision>
      <namespace>urn:ietf:params:xml:ns:yang:ietf-yang-types</namespace>
    </import-only-module>
    <import-only-module>
      <name>ietf-datastores</name>
      <revision>2017-08-17</revision>
      <namespace>urn:iet[  FAILED  ] test_implemented_info_yang
[ RUN      ] test_revision_date_yin
libyang[0]: Invalid date "2018-02-29", valid date in format "YYYY-MM-DD" expected.
libyang[0]: Module "x" parsing failed.
libyang[0]: Invalid date "18-02-28", valid date in format "YYYY-MM-DD" expected.
libyang[0]: Module "x" parsing failed.
libyang[0]: Invalid date "today", valid date in format "YYYY-MM-DD" expected.
libyang[0]: Module "x" parsing failed.
[       OK ] test_revision_date_yin
[ RUN      ] test_revision_date_yang
libyang[0]: Invalid date "2018-02-29", valid date in format "YYYY-MM-DD" expected.
libyang[0]: Module "x" parsing failed.
libyang[0]: Invalid date "18-02-28", valid date in format "YYYY-MM-DD" expected.
libyang[0]: Module "x" parsing failed.
libyang[0]: Invalid date "today", valid date in format "YYYY-MM-DD" expected.
libyang[0]: Module "x" parsing failed.
[       OK ] test_revision_date_yang
[==========] 8 test(s) run.
[  PASSED  ] 6 test(s).
[  FAILED  ] 2 test(s), listed below:
[  FAILED  ] test_implemented_info_yin
[  FAILED  ] test_implemented_info_yang

 2 FAILED TEST(S)

Opening the issue just to keep track of the problem. Any help would be appreciated.

[michalvasko]

Hi,
firstly, I don't know how old this output is, but just to be safe, please try it again with the current libyang devel. If it is still failing then simply add some print of the compared strings (cmocka fails to properly print this information). Something like instead of

437 assert_string_equal(data, template);

use

435 fputs(data, stdout);
436 fputs(template, stdout);
437 assert_string_equal(data, template);

and then post both the strings so that we can clearly see the difference.

Regards,
Michal

[rwestphal]

Hi Michal,

I pulled the latest changes and introduced the two fputs() you suggested in the source of the unit test. Here's the full output of test_conformance now: https://hastebin.com/akexiwenop.txt

Here's the diff between data and template: https://www.diffchecker.com/yfzZRjuv

Not sure if that helps finding out what the problem is...

Unrelated to this problem, I had to do this small change to fix the build on OpenBSD:

--- a/tests/data/test_state_lists.c
+++ b/tests/data/test_state_lists.c
@@ -15,6 +15,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <setjmp.h>
+#include <stdarg.h>
 #include <cmocka.h>
 
 #include "libyang.h

test_state_lists.c was probably created before my PR so it wasn't including <stdarg.h> like the other tests.

[michalvasko]

Hi,
the diff helped and I would try to look at tree_schema.c:1137 function lys_parse_set_filename. There are likely some differences on OpenBSD (no /proc?) so make changes accordingly and the tests should pass. Also, I will add the missing include.

Regards,
Michal

[rwestphal]

Thanks Michal, I see where is the problem now. The BSDs indeed don't have a /proc virtual file system, and unfortunately they don't have F_GETPATH as well nor anything equivalent. I'll leave this issue open until the day I find a reliable solution to this problem (if at all possible). From a quick inspection, it seems only users of the lys_parse_fd() function on *BSD will be affected by this issue, so I guess it's not a critical problem for the time being.

[michalvasko]

Hi Renato,
it is unfortunate that BSDs does not support this. You could, perhaps, add #ifdefs into this test and condition the missing lines only for platforms different from all BSDs. But it is up to you.

Regards,
Michal

[rwestphal]

Ok, I'll think about it or try to implement a workaround for the BSDs.

[snar]

Shall be fixed in devel version since 0.16.27