Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Two questions about open/R #37

Closed
silveric10 opened this issue Nov 6, 2018 · 3 comments
Closed

Two questions about open/R #37

silveric10 opened this issue Nov 6, 2018 · 3 comments

Comments

@silveric10
Copy link

Please use this template for reporting suspected bugs or requests for help.

Issue Description

first question, the NetlinkSocketTestis failed.
second question, there is a file named "platform_linux" will be compiled in /usr/local/sbin, I think it works as the platform agent for linux to program the routes, but when I run it after I run "run_openr", some core dump issues come up.

Environment

ubuntu-16.04

Minimal test code / Steps to reproduce the issue

first issue:
Start 13: NetlinkTypesTest
13/17 Test #13: NetlinkTypesTest ................. Passed 0.09 sec
Start 14: NetlinkSocketTest
14/17 Test #14: NetlinkSocketTest ................***Failed 28.45 sec
Start 15: PrefixManagerTest
15/17 Test #15: PrefixManagerTest ................ Passed 0.35 sec
Start 16: SparkTest
16/17 Test #16: SparkTest ........................ Passed 27.93 sec
Start 17: MockIoProviderTest
17/17 Test #17: MockIoProviderTest ............... Passed 0.28 sec

94% tests passed, 1 tests failed out of 17

Total Test time (real) = 424.98 sec

The following tests FAILED:
14 - NetlinkSocketTest (Failed)
Errors while running CTest
Makefile:117: recipe for target 'test' failed
make: *** [test] Error 8

second issue:
E1106 08:28:46.837707 4258 ThriftServer.cpp:397] Got an exception while setting up the server: 98failed to bind to async server socket: [::]:60099: Address already in use
E1106 08:28:46.844789 4259 ThriftServer.cpp:397] Got an exception while setting up the server: 98failed to bind to async server socket: [::]:60100: Address already in use
terminate called after throwing an instance of 'std::system_error'
what(): 98failed to bind to async server socket: [::]:60100: Address already in use
*** Aborted at 1541521726 (unix time) try "date -d @1541521726" if you are using GNU date ***
PC: @ 0x7fca01a47428 gsignal
terminate called recursively
*** SIGABRT (@0x3e80000109e) received by PID 4254 (TID 0x7fc9f9bbc700) from PID 4254; stack trace: ***
@ 0x7fca0300b390 (unknown)
@ 0x7fca01a47428 gsignal
@ 0x7fca01a4902a abort
@ 0x7fca0239984d __gnu_cxx::__verbose_terminate_handler()
@ 0x7fca023976b6 (unknown)
@ 0x7fca02397701 std::terminate()
@ 0x7fca023c2d38 (unknown)
@ 0x7fca030016ba start_thread
@ 0x7fca01b1941d clone
Aborted (core dumped)

What's the actual result?

What's the expected result?

I want to know if the "NetlinkSocketTest failed" will lead the openr doesn't work properly. And How can I enable the linux platform agent "platform_linux" after I run "run_openr"? Or the "run_openr" already contain the linux fib agent and I do not need to run another?

Thankyou

@saifhhasan
Copy link
Contributor

Q1 - Can you show the output of netlink socket test failure ? Did you run with "sudo" .. sudo ./netlink_socket_test

Q2 - Crash happens because ports Open/R trying to bind on is already in use by some other process on your system. Do sudo netstat -natp | grep <port> and kill that process. Then try to run openr

@silveric10
Copy link
Author

Hi there:
The output of netlink socket test failure is attached below.
[==========] Running 24 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 24 tests from NetlinkSocketFixture
[ RUN ] NetlinkSocketFixture.EmptyRouteTest
[ OK ] NetlinkSocketFixture.EmptyRouteTest (127 ms)
[ RUN ] NetlinkSocketFixture.SingleRouteTest
[ OK ] NetlinkSocketFixture.SingleRouteTest (117 ms)
[ RUN ] NetlinkSocketFixture.SingleRouteTestV4
[ OK ] NetlinkSocketFixture.SingleRouteTestV4 (99 ms)
[ RUN ] NetlinkSocketFixture.NullRouteTest
[ OK ] NetlinkSocketFixture.NullRouteTest (123 ms)
[ RUN ] NetlinkSocketFixture.NullRouteV4Test
[ OK ] NetlinkSocketFixture.NullRouteV4Test (126 ms)
[ RUN ] NetlinkSocketFixture.UpdateRouteTest
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:353: Failure
Expected: 2
To be equal to: rt2.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:366: Failure
Expected: 1
To be equal to: count
Which is: 0
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:391: Failure
Expected: 2
To be equal to: rt3.getNextHops().size()
Which is: 1
[ FAILED ] NetlinkSocketFixture.UpdateRouteTest (208 ms)
[ RUN ] NetlinkSocketFixture.UpdateRouteTestV4
[ OK ] NetlinkSocketFixture.UpdateRouteTestV4 (116 ms)
[ RUN ] NetlinkSocketFixture.UpdateMultiRouteTest
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:469: Failure
Expected: 3
To be equal to: rt.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:470: Failure
Value of: CompareNextHops(nexthops, rt)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:481: Failure
Expected: 1
To be equal to: count
Which is: 0
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:500: Failure
Expected: 2
To be equal to: rt1.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:501: Failure
Value of: CompareNextHops(nexthops, rt1)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:512: Failure
Expected: 1
To be equal to: count
Which is: 0
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:527: Failure
Expected: 3
To be equal to: rt2.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:528: Failure
Value of: CompareNextHops(nexthops, rt2)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:539: Failure
Expected: 1
To be equal to: count
Which is: 0
[ FAILED ] NetlinkSocketFixture.UpdateMultiRouteTest (144 ms)
[ RUN ] NetlinkSocketFixture.UpdateMultiRouteTestV4
[ OK ] NetlinkSocketFixture.UpdateMultiRouteTestV4 (107 ms)
[ RUN ] NetlinkSocketFixture.SyncRouteTest
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:612: Failure
Expected: 2
To be equal to: count
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:618: Failure
Expected: 2
To be equal to: rt1.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:620: Failure
Value of: CompareNextHops(nexthops1, rt1)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:653: Failure
Expected: 2
To be equal to: count
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:660: Failure
Expected: 2
To be equal to: rt4.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:662: Failure
Value of: CompareNextHops(nexthops2, rt4)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:685: Failure
Expected: 1
To be equal to: count
Which is: 0
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:689: Failure
Expected: 2
To be equal to: rt5.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:690: Failure
Value of: CompareNextHops(nexthops2, rt5)
Actual: false
Expected: true
[ FAILED ] NetlinkSocketFixture.SyncRouteTest (147 ms)
[ RUN ] NetlinkSocketFixture.SyncRouteTestV4
[ OK ] NetlinkSocketFixture.SyncRouteTestV4 (108 ms)
[ RUN ] NetlinkSocketFixture.ModifyMulticastRouteTest
[ OK ] NetlinkSocketFixture.ModifyMulticastRouteTest (96 ms)
[ RUN ] NetlinkSocketFixture.MultiPathTest
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1050: Failure
Expected: 2
To be equal to: rt.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1051: Failure
Value of: CompareNextHops(nexthops1, rt)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1062: Failure
Expected: 2
To be equal to: rt2.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1063: Failure
Expected: 2
To be equal to: rt3.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1064: Failure
Value of: CompareNextHops(nexthops1, rt2)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1065: Failure
Value of: CompareNextHops(nexthops2, rt3)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1089: Failure
Expected: 2
To be equal to: count
Which is: 0
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1097: Failure
Expected: 2
To be equal to: rt4.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1098: Failure
Value of: CompareNextHops(nexthops2, rt4)
Actual: false
Expected: true
[ FAILED ] NetlinkSocketFixture.MultiPathTest (130 ms)
[ RUN ] NetlinkSocketFixture.DeleteNonExistingRouteTest
E1107 02:19:24.090428 3069 NetlinkSocket.cpp:492] Trying to delete non-existing prefix fc00:cafe:3::4/128
[ OK ] NetlinkSocketFixture.DeleteNonExistingRouteTest (107 ms)
[ RUN ] NetlinkSocketFixture.MultiProtocolUnicastTest
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1320: Failure
Expected: 2
To be equal to: rt7.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1321: Failure
Value of: CompareNextHops(nextHopsV6, rt7)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1358: Failure
Expected: 4
To be equal to: count
Which is: 3
[ FAILED ] NetlinkSocketFixture.MultiProtocolUnicastTest (212 ms)
[ RUN ] NetlinkSocketFixture.MutiProtocolMulticastRouteTest
[ OK ] NetlinkSocketFixture.MutiProtocolMulticastRouteTest (76 ms)
[ RUN ] NetlinkSocketFixture.MultiProtocolSyncUnicastRouteTest
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1577: Failure
Expected: 2
To be equal to: rt1.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1579: Failure
Value of: CompareNextHops(nextHops1V6, rt1)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1610: Failure
Expected: 4
To be equal to: count
Which is: 5
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1631: Failure
Expected: 2
To be equal to: rt6.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1633: Failure
Value of: CompareNextHops(nextHops2V6, rt6)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1670: Failure
Expected: 2
To be equal to: rt9.getNextHops().size()
Which is: 1
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1671: Failure
Value of: CompareNextHops(nextHops2V6, rt9)
Actual: false
Expected: true
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1683: Failure
Expected: 1
To be equal to: count
Which is: 2
E1107 02:19:24.633577 3083 NetlinkSocket.cpp:492] Trying to delete non-existing prefix fc00:cafe:3::3/128
/home/fanya/openr/openr/nl/tests/NetlinkSocketTest.cpp:1721: Failure
Expected: 0
To be equal to: count
Which is: 1
[ FAILED ] NetlinkSocketFixture.MultiProtocolSyncUnicastRouteTest (284 ms)
[ RUN ] NetlinkSocketFixture.MultiProtocolSyncLinkRouteTest
[ OK ] NetlinkSocketFixture.MultiProtocolSyncLinkRouteTest (132 ms)
[ RUN ] NetlinkSocketFixture.IfNametoIfIndexTest
[ OK ] NetlinkSocketFixture.IfNametoIfIndexTest (95 ms)
[ RUN ] NetlinkSocketFixture.AddDelIfAddressBaseTest
[ OK ] NetlinkSocketFixture.AddDelIfAddressBaseTest (112 ms)
[ RUN ] NetlinkSocketFixture.AddDelDuplicatedIfAddressTest
[ OK ] NetlinkSocketFixture.AddDelDuplicatedIfAddressTest (84 ms)
[ RUN ] NetlinkSocketFixture.AddressSyncTest
[ OK ] NetlinkSocketFixture.AddressSyncTest (216 ms)
[ RUN ] NetlinkSocketFixture.AddressFlushTest
[ OK ] NetlinkSocketFixture.AddressFlushTest (105 ms)
[ RUN ] NetlinkSocketFixture.GetAddrsTest
[ OK ] NetlinkSocketFixture.GetAddrsTest (164 ms)
[----------] 24 tests from NetlinkSocketFixture (3245 ms total)

[----------] Global test environment tear-down
[==========] 24 tests from 1 test case ran. (3245 ms total)
[ PASSED ] 18 tests.
[ FAILED ] 6 tests, listed below:
[ FAILED ] NetlinkSocketFixture.UpdateRouteTest
[ FAILED ] NetlinkSocketFixture.UpdateMultiRouteTest
[ FAILED ] NetlinkSocketFixture.SyncRouteTest
[ FAILED ] NetlinkSocketFixture.MultiPathTest
[ FAILED ] NetlinkSocketFixture.MultiProtocolUnicastTest
[ FAILED ] NetlinkSocketFixture.MultiProtocolSyncUnicastRouteTest

6 FAILED TESTS

About the question 2, the ports 60099 and 60100 are alreay bind by Open/R,
tcp6 0 0 :::60099 :::* LISTEN 4574/openr
tcp6 0 0 ::1:48066 ::1:60099 ESTABLISHED 4574/openr
tcp6 0 0 ::1:60099 ::1:48066 ESTABLISHED 4574/openr
tcp6 0 0 :::60100 :::* LISTEN 4574/openr
tcp6 0 0 ::1:49384 ::1:60100 ESTABLISHED 4574/openr
tcp6 0 0 ::1:60100 ::1:49384 ESTABLISHED 4574/openr

What I mean excatly is that first I run openr with "sudo ./run_openr.sh", then I want to enable linux platform agent so I tap "sudo ./platform_linux", then the core dumped occures. Do I need to active "platform_linux" or not? The tech-support log is attached below.

breeze tech-support

-------- openr config file --------

Missing Config File

-------- openr runtime params --------

4574 openr --alloc_prefix_len=128 --assume_drained=false --config_store_filepath=/tmp/aq_persistent_config_store.bin --decision_debounce_max_ms=250 --decision_debounce_min_ms=10 --domain=openr --dryrun=false --enable_subnet_validation=true --enable_fib_sync=true --enable_health_checker=false --enable_legacy_flooding=true --enable_lfa=false --enable_netlink_fib_handler=true --enable_netlink_system_handler=true --enable_old_decision_module=false --enable_perf_measurement=true --enable_prefix_alloc=false --enable_rtt_metric=true --enable_segment_routing=false --enable_spark=true --enable_v4=true --enable_watchdog=true --fib_handler_port=60100 --health_checker_ping_interval_s=3 --ifname_prefix=terra,nic1,nic2,ens33,ens38 --iface_regex_exclude= --iface_regex_include= --ip_tos=192 --key_prefix_filters= --kvstore_flood_msg_per_sec=0 --kvstore_flood_msg_burst_size=0 --kvstore_zmq_hwm=65536 --link_flap_initial_backoff_ms=1000 --link_flap_max_backoff_ms=60000 --loopback_iface=lo --memory_limit_mb=300 --minloglevel=0 --node_name=yf --override_loopback_addr=false --prefixes= --redistribute_ifaces=lo1 --seed_prefix= --set_loopback_address=false --spark_fastinit_keepalive_time_ms=100 --spark_hold_time_s=30 --spark_keepalive_time_s=3 --set_leaf_node=false --static_prefix_alloc=false --logbufsecs=0 --logtostderr --max_log_size=1 --v=1

-------- openr version --------

Current Version : 20180307
Lowest Supported Version : 20180307

-------- openr build information --------

Build Information
Built by:
Built on:
Built at:
Build path:
Package Name:
Package Version:
Package Release:
Build Revision:
Build Upstream Revision:
Build Platform:
Build Rule: (, , )

-------- openr config --------

Missing Prefix Allocator config
Missing Link Monitor config

Prefix Type


-------- breeze prefixmgr view --------

Type Prefix


-------- breeze lm links --------

== Node Overload: NO ==

Interface Status Metric Override Addresses


ens33 Up 192.168.75.128
fe80::c9ae:9c2a:1bea:6865
ens38 Up 192.168.199.128
fe80::50a5:60f4:54ea:bc27

-------- breeze kvstore peers --------

== yf's peers ==

-------- breeze kvstore nodes --------

Node V6-Loopback V4-Loopback


  • yf N/A N/A

-------- breeze kvstore adj --------

-------- breeze kvstore prefixes --------

yf
Prefix Type


-------- breeze kvstore keys --ttl --------

== Available keys in KvStore ==

Key Originator Ver Hash TTL - Ver


prefix:yf yf 1 +1103d60e82d7d1a3 0:04:11.406000 - 9

-------- breeze decision validate --------

Decision is in sync with KvStore if nothing shows up

-------- breeze decision routes --------

== Routes for yf ==

No routes found.

-------- breeze fib validate --------

PASS
Decision and Openr-Fib routing table match
PASS
Openr-Fib and FibAgent routing table match
PASS
Route validation successful

-------- breeze fib routes-computed --------

== Routes for yf ==

No routes found.

-------- breeze fib routes-installed --------

== yf's FIB routes by client 786 ==

-------- breeze perf fib --------

-------- breeze monitor counters --------

== yf's counters ==

decision.prefix_db_update.count.0 : 1
decision.prefix_db_update.count.3600 : 1
decision.prefix_db_update.count.60 : 0
decision.prefix_db_update.count.600 : 0
fib.num_routes : 0
fib.process_interface_db.count.0 : 1
fib.process_interface_db.count.3600 : 1
fib.process_interface_db.count.60 : 0
fib.process_interface_db.count.600 : 0
fib.require_routedb_sync : 0
fib.sync_fib_calls.count.0 : 1
fib.sync_fib_calls.count.3600 : 1
fib.sync_fib_calls.count.60 : 0
fib.sync_fib_calls.count.600 : 0
kvstore.cmd_key_dump_.count.0 : 1
kvstore.cmd_key_dump_.count.3600 : 1
kvstore.cmd_key_dump_.count.60 : 0
kvstore.cmd_key_dump_.count.600 : 0
kvstore.cmd_key_get.count.0 : 11
kvstore.cmd_key_get.count.3600 : 11
kvstore.cmd_key_get.count.60 : 1
kvstore.cmd_key_get.count.600 : 9
kvstore.cmd_key_set.count.0 : 10
kvstore.cmd_key_set.count.3600 : 10
kvstore.cmd_key_set.count.60 : 1
kvstore.cmd_key_set.count.600 : 8
kvstore.num_keys : 1
kvstore.num_peers : 0
kvstore.pending_full_sync : 0
kvstore.received_key_vals.sum.0 : 10
kvstore.received_key_vals.sum.3600 : 10
kvstore.received_key_vals.sum.60 : 1
kvstore.received_key_vals.sum.600 : 8
kvstore.received_publications.count.0 : 10
kvstore.received_publications.count.3600 : 10
kvstore.received_publications.count.60 : 1
kvstore.received_publications.count.600 : 8
kvstore.updated_key_vals.sum.0 : 10
kvstore.updated_key_vals.sum.3600 : 10
kvstore.updated_key_vals.sum.60 : 1
kvstore.updated_key_vals.sum.600 : 8
link_monitor.adjacencies : 0
link_monitor.advertise_links.sum.0 : 1
link_monitor.advertise_links.sum.3600 : 1
link_monitor.advertise_links.sum.60 : 0
link_monitor.advertise_links.sum.600 : 0
process.cpu.pct : 0.009998
process.memory.rss : 7.24132e+07
spark.hello_packet_sent.sum.0 : 488
spark.hello_packet_sent.sum.3600 : 488
spark.hello_packet_sent.sum.60 : 38
spark.hello_packet_sent.sum.600 : 390
spark.my_seq_num : 489
spark.num_adjacent_neighbors : 0
spark.num_tracked_interfaces : 2
spark.num_tracked_neighbors : 0
spark.pending_timers : 3

-------- breeze monitor logs --------

time 02:22:36 2018-11-07
domain openr
node_name yf
event IFACE_UPDATE
entity Fib

@saifhhasan
Copy link
Contributor

Pardon me for delayed response. Not sure why netlink tests are failing on your system. I would encourage you to poke around test code. We use netlink library 3.4.0+ with patch https://github.com/facebook/openr/blob/master/build/fix-route-obj-attr-list.patch

Open/R by default have platform_linux server built into it and enabled by default. You can run in either mode as follows

openr --enable_netlink_fib_handler=true ....<other-args>

or

openr --enable_netlink_fib_handler=false .... <other-args>
./platform-linux

facebook-github-bot pushed a commit that referenced this issue Jul 24, 2019
Summary:
This is just facebookarchive/bistro#37, backported to `fbcode_builder`.

snarkmaster
Pull Request resolved: facebookarchive/bistro#37

Test Plan: Push and watch Travis

Reviewed By: simpkins

Differential Revision: D16453080

Pulled By: snarkmaster

fbshipit-source-id: a15eaead931f046c41e50f8e3b412ef68b172d65
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants