New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci-kubernetes-e2e-gce-examples: broken test run #39379

Closed
k8s-merge-robot opened this Issue Jan 3, 2017 · 393 comments

Comments

Projects
None yet
3 participants
@k8s-merge-robot
Contributor

k8s-merge-robot commented Jan 3, 2017

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1900/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc42071c220>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.1.11\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.1.11\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.1.11\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.1.11\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  18:35:27 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  18:35:28 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.1.11; broadcast_rpc_address=10.180.1.11; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.1.11; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  18:35:28 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  18:35:28 Global memtable on-heap threshold is enabled at 128MB\nINFO  18:35:28 Global memtable off-heap threshold is enabled at 128MB\nINFO  18:35:28 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\nINFO  18:35:28 Overriding RING_DELAY to 30000ms\nWARN  18:35:28 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  18:35:28 Hostname: cassandra-bh42h\nINFO  18:35:28 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  18:35:28 Heap size: 512.000MiB/512.000MiB\nINFO  18:35:28 Code Cache Non-heap memory: init = 2555904(2496K) used = 4245248(4145K) committed = 4259840(4160K) max = 251658240(245760K)\nINFO  18:35:28 Metaspace Non-heap memory: init = 0(0K) used = 17316664(16910K) committed = 17956864(17536K) max = -1(-1K)\nINFO  18:35:28 Compressed Class Space Non-heap memory: init = 0(0K) used = 2075400(2026K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  18:35:28 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)\nINFO  18:35:28 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  18:35:28 G1 Old Gen Heap memory: init = 508559360(496640K) used = 996864(973K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  18:35:28 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  18:35:28 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.1.11, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  18:35:29 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  18:35:29 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  18:35:29 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  18:35:29 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  18:35:29 Initializing SIGAR library\nWARN  18:35:29 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  18:35:29 Directory /var/lib/cassandra/hints doesn't exist\nINFO  18:35:29 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  18:35:30 Initializing system.IndexInfo\nINFO  18:35:31 Initializing system.batches\nINFO  18:35:31 Initializing system.paxos\nINFO  18:35:31 Initializing system.local\nINFO  18:35:31 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  18:35:32 Initializing key cache with capacity of 25 MBs.\nINFO  18:35:32 Initializing row cache with capacity of 0 MBs\nINFO  18:35:32 Initializing counter cache with capacity of 12 MBs\nINFO  18:35:32 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  18:35:32 Initializing system.peers\nINFO  18:35:32 Initializing system.peer_events\nINFO  18:35:32 Initializing system.range_xfers\nINFO  18:35:32 Initializing system.compaction_history\nINFO  18:35:32 Initializing system.sstable_activity\nINFO  18:35:32 Initializing system.size_estimates\nINFO  18:35:32 Initializing system.available_ranges\nINFO  18:35:32 Initializing system.views_builds_in_progress\nINFO  18:35:32 Initializing system.built_views\nINFO  18:35:32 Initializing system.hints\nINFO  18:35:32 Initializing system.batchlog\nINFO  18:35:32 Initializing system.schema_keyspaces\nINFO  18:35:32 Initializing system.schema_columnfamilies\nINFO  18:35:32 Initializing system.schema_columns\nINFO  18:35:32 Initializing system.schema_triggers\nINFO  18:35:32 Initializing system.schema_usertypes\nINFO  18:35:32 Initializing system.schema_functions\nINFO  18:35:32 Initializing system.schema_aggregates\nINFO  18:35:32 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  18:35:32 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  18:35:34 Populating token metadata from system tables\nINFO  18:35:34 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\nWARN  18:35:34 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  18:35:34 Token metadata: \nINFO  18:35:34 Initializing system_schema.keyspaces\nINFO  18:35:34 Initializing system_schema.tables\nINFO  18:35:34 Initializing system_schema.columns\nINFO  18:35:34 Initializing system_schema.triggers\nINFO  18:35:34 Initializing system_schema.dropped_columns\nINFO  18:35:34 Initializing system_schema.views\nINFO  18:35:34 Initializing system_schema.types\nINFO  18:35:34 Initializing system_schema.functions\nINFO  18:35:34 Initializing system_schema.aggregates\nINFO  18:35:34 Initializing system_schema.indexes\nINFO  18:35:34 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  18:35:34 Completed loading (4 ms; 2 keys) KeyCache cache\nINFO  18:35:34 Replaying /cassandra_data/commitlog/CommitLog-6-1483468406810.log, /cassandra_data/commitlog/CommitLog-6-1483468406811.log\nINFO  18:35:36 Log replay complete, 92 replayed mutations\nINFO  18:35:36 Populating token metadata from system tables\nINFO  18:35:36 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\nWARN  18:35:36 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  18:35:36 Token metadata: \nINFO  18:35:36 Cassandra version: 3.9\nINFO  18:35:36 Thrift API version: 20.1.0\nINFO  18:35:36 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  18:35:36 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  18:35:36 Starting Messaging Service on /10.180.1.11:7000 (eth0)\nINFO  18:35:36 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\nWARN  18:35:36 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  18:36:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\nWARN  18:36:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 18:36:08 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  18:36:08 No local state or state is in silent shutdown, not announcing shutdown\nINFO  18:36:08 Waiting for messaging service to quiesce\nINFO  18:36:08 MessagingService has terminated the accept() thread\nINFO  18:36:08 Paused hints dispatch\nINFO  18:36:08 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.1.11
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.1.11
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.1.11
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.1.11
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  18:35:27 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  18:35:28 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.1.11; broadcast_rpc_address=10.180.1.11; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.1.11; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  18:35:28 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  18:35:28 Global memtable on-heap threshold is enabled at 128MB
    INFO  18:35:28 Global memtable off-heap threshold is enabled at 128MB
    INFO  18:35:28 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    INFO  18:35:28 Overriding RING_DELAY to 30000ms
    WARN  18:35:28 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  18:35:28 Hostname: cassandra-bh42h
    INFO  18:35:28 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  18:35:28 Heap size: 512.000MiB/512.000MiB
    INFO  18:35:28 Code Cache Non-heap memory: init = 2555904(2496K) used = 4245248(4145K) committed = 4259840(4160K) max = 251658240(245760K)
    INFO  18:35:28 Metaspace Non-heap memory: init = 0(0K) used = 17316664(16910K) committed = 17956864(17536K) max = -1(-1K)
    INFO  18:35:28 Compressed Class Space Non-heap memory: init = 0(0K) used = 2075400(2026K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  18:35:28 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)
    INFO  18:35:28 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  18:35:28 G1 Old Gen Heap memory: init = 508559360(496640K) used = 996864(973K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  18:35:28 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  18:35:28 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.1.11, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  18:35:29 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  18:35:29 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  18:35:29 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  18:35:29 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  18:35:29 Initializing SIGAR library
    WARN  18:35:29 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  18:35:29 Directory /var/lib/cassandra/hints doesn't exist
    INFO  18:35:29 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  18:35:30 Initializing system.IndexInfo
    INFO  18:35:31 Initializing system.batches
    INFO  18:35:31 Initializing system.paxos
    INFO  18:35:31 Initializing system.local
    INFO  18:35:31 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  18:35:32 Initializing key cache with capacity of 25 MBs.
    INFO  18:35:32 Initializing row cache with capacity of 0 MBs
    INFO  18:35:32 Initializing counter cache with capacity of 12 MBs
    INFO  18:35:32 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  18:35:32 Initializing system.peers
    INFO  18:35:32 Initializing system.peer_events
    INFO  18:35:32 Initializing system.range_xfers
    INFO  18:35:32 Initializing system.compaction_history
    INFO  18:35:32 Initializing system.sstable_activity
    INFO  18:35:32 Initializing system.size_estimates
    INFO  18:35:32 Initializing system.available_ranges
    INFO  18:35:32 Initializing system.views_builds_in_progress
    INFO  18:35:32 Initializing system.built_views
    INFO  18:35:32 Initializing system.hints
    INFO  18:35:32 Initializing system.batchlog
    INFO  18:35:32 Initializing system.schema_keyspaces
    INFO  18:35:32 Initializing system.schema_columnfamilies
    INFO  18:35:32 Initializing system.schema_columns
    INFO  18:35:32 Initializing system.schema_triggers
    INFO  18:35:32 Initializing system.schema_usertypes
    INFO  18:35:32 Initializing system.schema_functions
    INFO  18:35:32 Initializing system.schema_aggregates
    INFO  18:35:32 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  18:35:32 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  18:35:34 Populating token metadata from system tables
    INFO  18:35:34 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    WARN  18:35:34 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  18:35:34 Token metadata: 
    INFO  18:35:34 Initializing system_schema.keyspaces
    INFO  18:35:34 Initializing system_schema.tables
    INFO  18:35:34 Initializing system_schema.columns
    INFO  18:35:34 Initializing system_schema.triggers
    INFO  18:35:34 Initializing system_schema.dropped_columns
    INFO  18:35:34 Initializing system_schema.views
    INFO  18:35:34 Initializing system_schema.types
    INFO  18:35:34 Initializing system_schema.functions
    INFO  18:35:34 Initializing system_schema.aggregates
    INFO  18:35:34 Initializing system_schema.indexes
    INFO  18:35:34 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  18:35:34 Completed loading (4 ms; 2 keys) KeyCache cache
    INFO  18:35:34 Replaying /cassandra_data/commitlog/CommitLog-6-1483468406810.log, /cassandra_data/commitlog/CommitLog-6-1483468406811.log
    INFO  18:35:36 Log replay complete, 92 replayed mutations
    INFO  18:35:36 Populating token metadata from system tables
    INFO  18:35:36 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    WARN  18:35:36 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  18:35:36 Token metadata: 
    INFO  18:35:36 Cassandra version: 3.9
    INFO  18:35:36 Thrift API version: 20.1.0
    INFO  18:35:36 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  18:35:36 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  18:35:36 Starting Messaging Service on /10.180.1.11:7000 (eth0)
    INFO  18:35:36 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    WARN  18:35:36 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  18:36:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    WARN  18:36:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-tfrtt/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    Exception (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
    ERROR 18:36:08 Exception encountered during startup
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    WARN  18:36:08 No local state or state is in silent shutdown, not announcing shutdown
    INFO  18:36:08 Waiting for messaging service to quiesce
    INFO  18:36:08 MessagingService has terminated the accept() thread
    INFO  18:36:08 Paused hints dispatch
    INFO  18:36:08 Paused hints dispatch
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:236

Issues about this test specifically: #27978 #28817

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc420a54030>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 18:37:44.917  INFO 7 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-9kqw2 with PID 7 (/bootstrapper.jar started by root in /)\n2017-01-03 18:37:44.942  INFO 7 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 18:37:45.080  INFO 7 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 18:37:45 GMT 2017]; root of context hierarchy\n2017-01-03 18:37:46.906  INFO 7 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 18:37:46.931  INFO 7 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 18:37:47.411  WARN 7 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-88649/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 18:37:47.411  INFO 7 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 18:37:47.730  INFO 7 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 18:37:47.732  INFO 7 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 18:37:47.732  WARN 7 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 18:37:47.746  INFO 7 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.1.12]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 18:37:47.782  INFO 7 --- [           main] com.hazelcast.system                     : [10.180.1.12]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.1.12]:5701\n2017-01-03 18:37:47.783  INFO 7 --- [           main] com.hazelcast.system                     : [10.180.1.12]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 18:37:47.783  INFO 7 --- [           main] com.hazelcast.system                     : [10.180.1.12]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 18:37:48.211  INFO 7 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.1.12]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 18:37:49.429  INFO 7 --- [           main] com.hazelcast.instance.Node              : [10.180.1.12]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 18:37:49.444  INFO 7 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.1.12]:5701 [someGroup] [3.7] [10.180.1.12]:5701 is STARTING\n2017-01-03 18:37:49.727  INFO 7 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.1.12]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 18:37:49.734  INFO 7 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.1.12]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 18:37:49.754  INFO 7 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.1.12]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-03 18:37:49.781  INFO 7 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.1.12]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.1.12]:5701 - 6f1f7159-dc2a-4481-bc7f-3bc133937aad this\n}\n\n2017-01-03 18:37:49.860  INFO 7 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.1.12]:5701 [someGroup] [3.7] [10.180.1.12]:5701 is STARTED\n2017-01-03 18:37:49.866  INFO 7 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 6.036 seconds (JVM running for 7.055)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-03 18:37:44.917  INFO 7 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-9kqw2 with PID 7 (/bootstrapper.jar started by root in /)
    2017-01-03 18:37:44.942  INFO 7 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default
    2017-01-03 18:37:45.080  INFO 7 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 18:37:45 GMT 2017]; root of context hierarchy
    2017-01-03 18:37:46.906  INFO 7 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
    2017-01-03 18:37:46.931  INFO 7 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
    2017-01-03 18:37:47.411  WARN 7 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed
    
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-88649/endpoints/hazelcast
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]
    	at com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]
    	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]
    	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]
    
    2017-01-03 18:37:47.411  INFO 7 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.
    2017-01-03 18:37:47.730  INFO 7 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []
    2017-01-03 18:37:47.732  INFO 7 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.
    2017-01-03 18:37:47.732  WARN 7 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.
    2017-01-03 18:37:47.746  INFO 7 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.1.12]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
    2017-01-03 18:37:47.782  INFO 7 --- [           main] com.hazelcast.system                     : [10.180.1.12]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.1.12]:5701
    2017-01-03 18:37:47.783  INFO 7 --- [           main] com.hazelcast.system                     : [10.180.1.12]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
    2017-01-03 18:37:47.783  INFO 7 --- [           main] com.hazelcast.system                     : [10.180.1.12]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1
    2017-01-03 18:37:48.211  INFO 7 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.1.12]:5701 [someGroup] [3.7] Backpressure is disabled
    2017-01-03 18:37:49.429  INFO 7 --- [           main] com.hazelcast.instance.Node              : [10.180.1.12]:5701 [someGroup] [3.7] Creating TcpIpJoiner
    2017-01-03 18:37:49.444  INFO 7 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.1.12]:5701 [someGroup] [3.7] [10.180.1.12]:5701 is STARTING
    2017-01-03 18:37:49.727  INFO 7 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.1.12]:5701 [someGroup] [3.7] Starting 2 partition threads
    2017-01-03 18:37:49.734  INFO 7 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.1.12]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)
    2017-01-03 18:37:49.754  INFO 7 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.1.12]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
    2017-01-03 18:37:49.781  INFO 7 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.1.12]:5701 [someGroup] [3.7] 
    
    
    Members [1] {
    	Member [10.180.1.12]:5701 - 6f1f7159-dc2a-4481-bc7f-3bc133937aad this
    }
    
    2017-01-03 18:37:49.860  INFO 7 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.1.12]:5701 [someGroup] [3.7] [10.180.1.12]:5701 is STARTED
    2017-01-03 18:37:49.866  INFO 7 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 6.036 seconds (JVM running for 7.055)
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:556

Issues about this test specifically: #27850 #30672 #33271

Failed: [k8s.io] [Feature:Example] [k8s.io] RethinkDB should create and stop rethinkdb servers {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:524
Expected error:
    <*errors.errorString | 0xc420e76430>: {
        s: "Failed to find \"Server ready\", last result: \"Checking for other nodes\nMy host: 10.180.1.16\nEndpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-2dvfg/endpoints/rethinkdb-driver\nLooking for IPs...\nparse error: Invalid numeric literal at line 1, column 5\n\"",
    }
    Failed to find "Server ready", last result: "Checking for other nodes
    My host: 10.180.1.16
    Endpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-2dvfg/endpoints/rethinkdb-driver
    Looking for IPs...
    parse error: Invalid numeric literal at line 1, column 5
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:502

Issues about this test specifically: #30581 #32755

Failed: Test {e2e.go}

exit status 1

Issues about this test specifically: #33361 #38663

Previous issues for this suite: #36930 #37437 #38393 #38578

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Jan 3, 2017

Contributor

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1901/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc4218f3500>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.2.18\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.2.18\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.2.18\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.2.18\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  19:52:07 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  19:52:08 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.2.18; broadcast_rpc_address=10.180.2.18; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.2.18; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  19:52:08 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  19:52:08 Global memtable on-heap threshold is enabled at 128MB\nINFO  19:52:08 Global memtable off-heap threshold is enabled at 128MB\nINFO  19:52:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nINFO  19:52:08 Overriding RING_DELAY to 30000ms\nWARN  19:52:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  19:52:08 Hostname: cassandra-675w2\nINFO  19:52:08 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  19:52:08 Heap size: 512.000MiB/512.000MiB\nINFO  19:52:08 Code Cache Non-heap memory: init = 2555904(2496K) used = 4355072(4253K) committed = 4390912(4288K) max = 251658240(245760K)\nINFO  19:52:08 Metaspace Non-heap memory: init = 0(0K) used = 17387488(16979K) committed = 17694720(17280K) max = -1(-1K)\nINFO  19:52:08 Compressed Class Space Non-heap memory: init = 0(0K) used = 2076176(2027K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  19:52:08 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)\nINFO  19:52:08 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  19:52:08 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1027584(1003K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  19:52:08 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  19:52:08 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.2.18, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  19:52:08 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  19:52:08 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  19:52:08 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  19:52:08 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  19:52:08 Initializing SIGAR library\nWARN  19:52:08 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  19:52:08 Directory /var/lib/cassandra/hints doesn't exist\nINFO  19:52:09 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  19:52:09 Initializing system.IndexInfo\nINFO  19:52:10 Initializing system.batches\nINFO  19:52:11 Initializing system.paxos\nINFO  19:52:11 Initializing system.local\nINFO  19:52:11 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  19:52:11 Initializing key cache with capacity of 25 MBs.\nINFO  19:52:11 Initializing row cache with capacity of 0 MBs\nINFO  19:52:11 Initializing counter cache with capacity of 12 MBs\nINFO  19:52:11 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  19:52:11 Initializing system.peers\nINFO  19:52:11 Initializing system.peer_events\nINFO  19:52:11 Initializing system.range_xfers\nINFO  19:52:11 Initializing system.compaction_history\nINFO  19:52:11 Initializing system.sstable_activity\nINFO  19:52:11 Initializing system.size_estimates\nINFO  19:52:11 Initializing system.available_ranges\nINFO  19:52:11 Initializing system.views_builds_in_progress\nINFO  19:52:11 Initializing system.built_views\nINFO  19:52:11 Initializing system.hints\nINFO  19:52:11 Initializing system.batchlog\nINFO  19:52:11 Initializing system.schema_keyspaces\nINFO  19:52:11 Initializing system.schema_columnfamilies\nINFO  19:52:11 Initializing system.schema_columns\nINFO  19:52:11 Initializing system.schema_triggers\nINFO  19:52:11 Initializing system.schema_usertypes\nINFO  19:52:11 Initializing system.schema_functions\nINFO  19:52:11 Initializing system.schema_aggregates\nINFO  19:52:11 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  19:52:12 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  19:52:13 Populating token metadata from system tables\nINFO  19:52:13 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nWARN  19:52:13 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  19:52:13 Token metadata: \nINFO  19:52:13 Initializing system_schema.keyspaces\nINFO  19:52:13 Initializing system_schema.tables\nINFO  19:52:13 Initializing system_schema.columns\nINFO  19:52:13 Initializing system_schema.triggers\nINFO  19:52:13 Initializing system_schema.dropped_columns\nINFO  19:52:13 Initializing system_schema.views\nINFO  19:52:13 Initializing system_schema.types\nINFO  19:52:13 Initializing system_schema.functions\nINFO  19:52:13 Initializing system_schema.aggregates\nINFO  19:52:13 Initializing system_schema.indexes\nINFO  19:52:13 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  19:52:13 Completed loading (10 ms; 2 keys) KeyCache cache\nINFO  19:52:13 Replaying /cassandra_data/commitlog/CommitLog-6-1483473007983.log, /cassandra_data/commitlog/CommitLog-6-1483473007984.log\nINFO  19:52:15 Log replay complete, 92 replayed mutations\nINFO  19:52:15 Populating token metadata from system tables\nINFO  19:52:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nWARN  19:52:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  19:52:15 Token metadata: \nINFO  19:52:15 Cassandra version: 3.9\nINFO  19:52:15 Thrift API version: 20.1.0\nINFO  19:52:15 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  19:52:15 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  19:52:15 Starting Messaging Service on /10.180.2.18:7000 (eth0)\nINFO  19:52:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nWARN  19:52:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  19:52:46 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nWARN  19:52:46 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 19:52:46 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  19:52:46 No local state or state is in silent shutdown, not announcing shutdown\nINFO  19:52:46 Waiting for messaging service to quiesce\nINFO  19:52:46 MessagingService has terminated the accept() thread\nINFO  19:52:46 Paused hints dispatch\nINFO  19:52:46 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.2.18
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.2.18
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.2.18
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.2.18
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  19:52:07 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  19:52:08 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.2.18; broadcast_rpc_address=10.180.2.18; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.2.18; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  19:52:08 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  19:52:08 Global memtable on-heap threshold is enabled at 128MB
    INFO  19:52:08 Global memtable off-heap threshold is enabled at 128MB
    INFO  19:52:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    INFO  19:52:08 Overriding RING_DELAY to 30000ms
    WARN  19:52:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  19:52:08 Hostname: cassandra-675w2
    INFO  19:52:08 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  19:52:08 Heap size: 512.000MiB/512.000MiB
    INFO  19:52:08 Code Cache Non-heap memory: init = 2555904(2496K) used = 4355072(4253K) committed = 4390912(4288K) max = 251658240(245760K)
    INFO  19:52:08 Metaspace Non-heap memory: init = 0(0K) used = 17387488(16979K) committed = 17694720(17280K) max = -1(-1K)
    INFO  19:52:08 Compressed Class Space Non-heap memory: init = 0(0K) used = 2076176(2027K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  19:52:08 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)
    INFO  19:52:08 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  19:52:08 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1027584(1003K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  19:52:08 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  19:52:08 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.2.18, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  19:52:08 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  19:52:08 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  19:52:08 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  19:52:08 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  19:52:08 Initializing SIGAR library
    WARN  19:52:08 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  19:52:08 Directory /var/lib/cassandra/hints doesn't exist
    INFO  19:52:09 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  19:52:09 Initializing system.IndexInfo
    INFO  19:52:10 Initializing system.batches
    INFO  19:52:11 Initializing system.paxos
    INFO  19:52:11 Initializing system.local
    INFO  19:52:11 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  19:52:11 Initializing key cache with capacity of 25 MBs.
    INFO  19:52:11 Initializing row cache with capacity of 0 MBs
    INFO  19:52:11 Initializing counter cache with capacity of 12 MBs
    INFO  19:52:11 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  19:52:11 Initializing system.peers
    INFO  19:52:11 Initializing system.peer_events
    INFO  19:52:11 Initializing system.range_xfers
    INFO  19:52:11 Initializing system.compaction_history
    INFO  19:52:11 Initializing system.sstable_activity
    INFO  19:52:11 Initializing system.size_estimates
    INFO  19:52:11 Initializing system.available_ranges
    INFO  19:52:11 Initializing system.views_builds_in_progress
    INFO  19:52:11 Initializing system.built_views
    INFO  19:52:11 Initializing system.hints
    INFO  19:52:11 Initializing system.batchlog
    INFO  19:52:11 Initializing system.schema_keyspaces
    INFO  19:52:11 Initializing system.schema_columnfamilies
    INFO  19:52:11 Initializing system.schema_columns
    INFO  19:52:11 Initializing system.schema_triggers
    INFO  19:52:11 Initializing system.schema_usertypes
    INFO  19:52:11 Initializing system.schema_functions
    INFO  19:52:11 Initializing system.schema_aggregates
    INFO  19:52:11 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  19:52:12 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  19:52:13 Populating token metadata from system tables
    INFO  19:52:13 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    WARN  19:52:13 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  19:52:13 Token metadata: 
    INFO  19:52:13 Initializing system_schema.keyspaces
    INFO  19:52:13 Initializing system_schema.tables
    INFO  19:52:13 Initializing system_schema.columns
    INFO  19:52:13 Initializing system_schema.triggers
    INFO  19:52:13 Initializing system_schema.dropped_columns
    INFO  19:52:13 Initializing system_schema.views
    INFO  19:52:13 Initializing system_schema.types
    INFO  19:52:13 Initializing system_schema.functions
    INFO  19:52:13 Initializing system_schema.aggregates
    INFO  19:52:13 Initializing system_schema.indexes
    INFO  19:52:13 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  19:52:13 Completed loading (10 ms; 2 keys) KeyCache cache
    INFO  19:52:13 Replaying /cassandra_data/commitlog/CommitLog-6-1483473007983.log, /cassandra_data/commitlog/CommitLog-6-1483473007984.log
    INFO  19:52:15 Log replay complete, 92 replayed mutations
    INFO  19:52:15 Populating token metadata from system tables
    INFO  19:52:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    WARN  19:52:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  19:52:15 Token metadata: 
    INFO  19:52:15 Cassandra version: 3.9
    INFO  19:52:15 Thrift API version: 20.1.0
    INFO  19:52:15 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  19:52:15 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  19:52:15 Starting Messaging Service on /10.180.2.18:7000 (eth0)
    INFO  19:52:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    WARN  19:52:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  19:52:46 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    WARN  19:52:46 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    Exception (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
    ERROR 19:52:46 Exception encountered during startup
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    WARN  19:52:46 No local state or state is in silent shutdown, not announcing shutdown
    INFO  19:52:46 Waiting for messaging service to quiesce
    INFO  19:52:46 MessagingService has terminated the accept() thread
    INFO  19:52:46 Paused hints dispatch
    INFO  19:52:46 Paused hints dispatch
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:236

Issues about this test specifically: #27978 #28817

Failed: [k8s.io] [Feature:Example] [k8s.io] RethinkDB should create and stop rethinkdb servers {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:524
Expected error:
    <*errors.errorString | 0xc420e73720>: {
        s: "Failed to find \"Server ready\", last result: \"Checking for other nodes\nMy host: 10.180.2.13\nEndpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-sq4t2/endpoints/rethinkdb-driver\nLooking for IPs...\nparse error: Invalid numeric literal at line 1, column 5\n\"",
    }
    Failed to find "Server ready", last result: "Checking for other nodes
    My host: 10.180.2.13
    Endpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-sq4t2/endpoints/rethinkdb-driver
    Looking for IPs...
    parse error: Invalid numeric literal at line 1, column 5
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:502

Issues about this test specifically: #30581 #32755

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc4218e7230>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 19:36:49.350  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-81hgf with PID 5 (/bootstrapper.jar started by root in /)\n2017-01-03 19:36:49.365  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 19:36:49.496  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 19:36:49 GMT 2017]; root of context hierarchy\n2017-01-03 19:36:51.054  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 19:36:51.071  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 19:36:51.474  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-wqprf/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 19:36:51.474  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 19:36:51.718  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 19:36:51.720  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 19:36:51.720  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 19:36:51.734  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.17]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 19:36:51.771  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.17]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.17]:5701\n2017-01-03 19:36:51.772  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.17]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 19:36:51.772  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.17]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 19:36:52.178  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.17]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 19:36:53.303  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.2.17]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 19:36:53.334  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.17]:5701 [someGroup] [3.7] [10.180.2.17]:5701 is STARTING\n2017-01-03 19:36:53.565  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.17]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 19:36:53.572  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.17]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 19:36:53.588 
Contributor

k8s-merge-robot commented Jan 3, 2017

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1901/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc4218f3500>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.2.18\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.2.18\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.2.18\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.2.18\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  19:52:07 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  19:52:08 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.2.18; broadcast_rpc_address=10.180.2.18; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.2.18; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  19:52:08 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  19:52:08 Global memtable on-heap threshold is enabled at 128MB\nINFO  19:52:08 Global memtable off-heap threshold is enabled at 128MB\nINFO  19:52:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nINFO  19:52:08 Overriding RING_DELAY to 30000ms\nWARN  19:52:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  19:52:08 Hostname: cassandra-675w2\nINFO  19:52:08 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  19:52:08 Heap size: 512.000MiB/512.000MiB\nINFO  19:52:08 Code Cache Non-heap memory: init = 2555904(2496K) used = 4355072(4253K) committed = 4390912(4288K) max = 251658240(245760K)\nINFO  19:52:08 Metaspace Non-heap memory: init = 0(0K) used = 17387488(16979K) committed = 17694720(17280K) max = -1(-1K)\nINFO  19:52:08 Compressed Class Space Non-heap memory: init = 0(0K) used = 2076176(2027K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  19:52:08 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)\nINFO  19:52:08 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  19:52:08 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1027584(1003K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  19:52:08 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  19:52:08 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.2.18, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  19:52:08 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  19:52:08 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  19:52:08 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  19:52:08 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  19:52:08 Initializing SIGAR library\nWARN  19:52:08 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  19:52:08 Directory /var/lib/cassandra/hints doesn't exist\nINFO  19:52:09 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  19:52:09 Initializing system.IndexInfo\nINFO  19:52:10 Initializing system.batches\nINFO  19:52:11 Initializing system.paxos\nINFO  19:52:11 Initializing system.local\nINFO  19:52:11 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  19:52:11 Initializing key cache with capacity of 25 MBs.\nINFO  19:52:11 Initializing row cache with capacity of 0 MBs\nINFO  19:52:11 Initializing counter cache with capacity of 12 MBs\nINFO  19:52:11 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  19:52:11 Initializing system.peers\nINFO  19:52:11 Initializing system.peer_events\nINFO  19:52:11 Initializing system.range_xfers\nINFO  19:52:11 Initializing system.compaction_history\nINFO  19:52:11 Initializing system.sstable_activity\nINFO  19:52:11 Initializing system.size_estimates\nINFO  19:52:11 Initializing system.available_ranges\nINFO  19:52:11 Initializing system.views_builds_in_progress\nINFO  19:52:11 Initializing system.built_views\nINFO  19:52:11 Initializing system.hints\nINFO  19:52:11 Initializing system.batchlog\nINFO  19:52:11 Initializing system.schema_keyspaces\nINFO  19:52:11 Initializing system.schema_columnfamilies\nINFO  19:52:11 Initializing system.schema_columns\nINFO  19:52:11 Initializing system.schema_triggers\nINFO  19:52:11 Initializing system.schema_usertypes\nINFO  19:52:11 Initializing system.schema_functions\nINFO  19:52:11 Initializing system.schema_aggregates\nINFO  19:52:11 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  19:52:12 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  19:52:13 Populating token metadata from system tables\nINFO  19:52:13 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nWARN  19:52:13 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  19:52:13 Token metadata: \nINFO  19:52:13 Initializing system_schema.keyspaces\nINFO  19:52:13 Initializing system_schema.tables\nINFO  19:52:13 Initializing system_schema.columns\nINFO  19:52:13 Initializing system_schema.triggers\nINFO  19:52:13 Initializing system_schema.dropped_columns\nINFO  19:52:13 Initializing system_schema.views\nINFO  19:52:13 Initializing system_schema.types\nINFO  19:52:13 Initializing system_schema.functions\nINFO  19:52:13 Initializing system_schema.aggregates\nINFO  19:52:13 Initializing system_schema.indexes\nINFO  19:52:13 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  19:52:13 Completed loading (10 ms; 2 keys) KeyCache cache\nINFO  19:52:13 Replaying /cassandra_data/commitlog/CommitLog-6-1483473007983.log, /cassandra_data/commitlog/CommitLog-6-1483473007984.log\nINFO  19:52:15 Log replay complete, 92 replayed mutations\nINFO  19:52:15 Populating token metadata from system tables\nINFO  19:52:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nWARN  19:52:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  19:52:15 Token metadata: \nINFO  19:52:15 Cassandra version: 3.9\nINFO  19:52:15 Thrift API version: 20.1.0\nINFO  19:52:15 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  19:52:15 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  19:52:15 Starting Messaging Service on /10.180.2.18:7000 (eth0)\nINFO  19:52:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nWARN  19:52:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  19:52:46 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\nWARN  19:52:46 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 19:52:46 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  19:52:46 No local state or state is in silent shutdown, not announcing shutdown\nINFO  19:52:46 Waiting for messaging service to quiesce\nINFO  19:52:46 MessagingService has terminated the accept() thread\nINFO  19:52:46 Paused hints dispatch\nINFO  19:52:46 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.2.18
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.2.18
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.2.18
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.2.18
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  19:52:07 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  19:52:08 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.2.18; broadcast_rpc_address=10.180.2.18; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.2.18; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  19:52:08 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  19:52:08 Global memtable on-heap threshold is enabled at 128MB
    INFO  19:52:08 Global memtable off-heap threshold is enabled at 128MB
    INFO  19:52:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    INFO  19:52:08 Overriding RING_DELAY to 30000ms
    WARN  19:52:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  19:52:08 Hostname: cassandra-675w2
    INFO  19:52:08 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  19:52:08 Heap size: 512.000MiB/512.000MiB
    INFO  19:52:08 Code Cache Non-heap memory: init = 2555904(2496K) used = 4355072(4253K) committed = 4390912(4288K) max = 251658240(245760K)
    INFO  19:52:08 Metaspace Non-heap memory: init = 0(0K) used = 17387488(16979K) committed = 17694720(17280K) max = -1(-1K)
    INFO  19:52:08 Compressed Class Space Non-heap memory: init = 0(0K) used = 2076176(2027K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  19:52:08 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)
    INFO  19:52:08 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  19:52:08 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1027584(1003K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  19:52:08 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  19:52:08 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.2.18, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  19:52:08 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  19:52:08 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  19:52:08 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  19:52:08 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  19:52:08 Initializing SIGAR library
    WARN  19:52:08 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  19:52:08 Directory /var/lib/cassandra/hints doesn't exist
    INFO  19:52:09 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  19:52:09 Initializing system.IndexInfo
    INFO  19:52:10 Initializing system.batches
    INFO  19:52:11 Initializing system.paxos
    INFO  19:52:11 Initializing system.local
    INFO  19:52:11 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  19:52:11 Initializing key cache with capacity of 25 MBs.
    INFO  19:52:11 Initializing row cache with capacity of 0 MBs
    INFO  19:52:11 Initializing counter cache with capacity of 12 MBs
    INFO  19:52:11 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  19:52:11 Initializing system.peers
    INFO  19:52:11 Initializing system.peer_events
    INFO  19:52:11 Initializing system.range_xfers
    INFO  19:52:11 Initializing system.compaction_history
    INFO  19:52:11 Initializing system.sstable_activity
    INFO  19:52:11 Initializing system.size_estimates
    INFO  19:52:11 Initializing system.available_ranges
    INFO  19:52:11 Initializing system.views_builds_in_progress
    INFO  19:52:11 Initializing system.built_views
    INFO  19:52:11 Initializing system.hints
    INFO  19:52:11 Initializing system.batchlog
    INFO  19:52:11 Initializing system.schema_keyspaces
    INFO  19:52:11 Initializing system.schema_columnfamilies
    INFO  19:52:11 Initializing system.schema_columns
    INFO  19:52:11 Initializing system.schema_triggers
    INFO  19:52:11 Initializing system.schema_usertypes
    INFO  19:52:11 Initializing system.schema_functions
    INFO  19:52:11 Initializing system.schema_aggregates
    INFO  19:52:11 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  19:52:12 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  19:52:13 Populating token metadata from system tables
    INFO  19:52:13 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    WARN  19:52:13 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  19:52:13 Token metadata: 
    INFO  19:52:13 Initializing system_schema.keyspaces
    INFO  19:52:13 Initializing system_schema.tables
    INFO  19:52:13 Initializing system_schema.columns
    INFO  19:52:13 Initializing system_schema.triggers
    INFO  19:52:13 Initializing system_schema.dropped_columns
    INFO  19:52:13 Initializing system_schema.views
    INFO  19:52:13 Initializing system_schema.types
    INFO  19:52:13 Initializing system_schema.functions
    INFO  19:52:13 Initializing system_schema.aggregates
    INFO  19:52:13 Initializing system_schema.indexes
    INFO  19:52:13 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  19:52:13 Completed loading (10 ms; 2 keys) KeyCache cache
    INFO  19:52:13 Replaying /cassandra_data/commitlog/CommitLog-6-1483473007983.log, /cassandra_data/commitlog/CommitLog-6-1483473007984.log
    INFO  19:52:15 Log replay complete, 92 replayed mutations
    INFO  19:52:15 Populating token metadata from system tables
    INFO  19:52:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    WARN  19:52:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  19:52:15 Token metadata: 
    INFO  19:52:15 Cassandra version: 3.9
    INFO  19:52:15 Thrift API version: 20.1.0
    INFO  19:52:15 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  19:52:15 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  19:52:15 Starting Messaging Service on /10.180.2.18:7000 (eth0)
    INFO  19:52:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    WARN  19:52:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  19:52:46 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    WARN  19:52:46 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-m0ltc/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    Exception (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
    ERROR 19:52:46 Exception encountered during startup
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    WARN  19:52:46 No local state or state is in silent shutdown, not announcing shutdown
    INFO  19:52:46 Waiting for messaging service to quiesce
    INFO  19:52:46 MessagingService has terminated the accept() thread
    INFO  19:52:46 Paused hints dispatch
    INFO  19:52:46 Paused hints dispatch
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:236

Issues about this test specifically: #27978 #28817

Failed: [k8s.io] [Feature:Example] [k8s.io] RethinkDB should create and stop rethinkdb servers {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:524
Expected error:
    <*errors.errorString | 0xc420e73720>: {
        s: "Failed to find \"Server ready\", last result: \"Checking for other nodes\nMy host: 10.180.2.13\nEndpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-sq4t2/endpoints/rethinkdb-driver\nLooking for IPs...\nparse error: Invalid numeric literal at line 1, column 5\n\"",
    }
    Failed to find "Server ready", last result: "Checking for other nodes
    My host: 10.180.2.13
    Endpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-sq4t2/endpoints/rethinkdb-driver
    Looking for IPs...
    parse error: Invalid numeric literal at line 1, column 5
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:502

Issues about this test specifically: #30581 #32755

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc4218e7230>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 19:36:49.350  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-81hgf with PID 5 (/bootstrapper.jar started by root in /)\n2017-01-03 19:36:49.365  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 19:36:49.496  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 19:36:49 GMT 2017]; root of context hierarchy\n2017-01-03 19:36:51.054  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 19:36:51.071  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 19:36:51.474  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-wqprf/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 19:36:51.474  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 19:36:51.718  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 19:36:51.720  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 19:36:51.720  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 19:36:51.734  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.17]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 19:36:51.771  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.17]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.17]:5701\n2017-01-03 19:36:51.772  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.17]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 19:36:51.772  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.17]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 19:36:52.178  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.17]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 19:36:53.303  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.2.17]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 19:36:53.334  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.17]:5701 [someGroup] [3.7] [10.180.2.17]:5701 is STARTING\n2017-01-03 19:36:53.565  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.17]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 19:36:53.572  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.17]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 19:36:53.588 
@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Jan 3, 2017

Contributor

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1902/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc420c2f010>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 20:19:48.817  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-dn58m with PID 6 (/bootstrapper.jar started by root in /)\n2017-01-03 20:19:48.824  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 20:19:48.912  INFO 6 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 20:19:48 GMT 2017]; root of context hierarchy\n2017-01-03 20:19:50.555  INFO 6 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 20:19:50.577  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 20:19:51.026  WARN 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-q2d79/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 20:19:51.026  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 20:19:51.291  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 20:19:51.294  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 20:19:51.294  WARN 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 20:19:51.307  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.7]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 20:19:51.339  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.7]:5701\n2017-01-03 20:19:51.340  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 20:19:51.340  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 20:19:51.706  INFO 6 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.7]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 20:19:52.753  INFO 6 --- [           main] com.hazelcast.instance.Node              : [10.180.2.7]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 20:19:52.758  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.7]:5701 [someGroup] [3.7] [10.180.2.7]:5701 is STARTING\n2017-01-03 20:19:52.983  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.7]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 20:19:52.987  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.7]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 20:19:52.999  INFO 6 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.7]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-03 20:19:53.025  INFO 6 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.7]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.2.7]:5701 - ee40f65e-9689-4e63-aabe-3ae056b5397f this\n}\n\n2017-01-03 20:19:53.096  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.7]:5701 [someGroup] [3.7] [10.180.2.7]:5701 is STARTED\n2017-01-03 20:19:53.108  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.139 seconds (JVM running for 5.807)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-03 20:19:48.817  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-dn58m with PID 6 (/bootstrapper.jar started by root in /)
    2017-01-03 20:19:48.824  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default
    2017-01-03 20:19:48.912  INFO 6 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 20:19:48 GMT 2017]; root of context hierarchy
    2017-01-03 20:19:50.555  INFO 6 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
    2017-01-03 20:19:50.577  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
    2017-01-03 20:19:51.026  WARN 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed
    
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-q2d79/endpoints/hazelcast
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]
    	at com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]
    	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]
    	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]
    
    2017-01-03 20:19:51.026  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.
    2017-01-03 20:19:51.291  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []
    2017-01-03 20:19:51.294  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.
    2017-01-03 20:19:51.294  WARN 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.
    2017-01-03 20:19:51.307  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.7]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
    2017-01-03 20:19:51.339  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.7]:5701
    2017-01-03 20:19:51.340  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
    2017-01-03 20:19:51.340  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1
    2017-01-03 20:19:51.706  INFO 6 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.7]:5701 [someGroup] [3.7] Backpressure is disabled
    2017-01-03 20:19:52.753  INFO 6 --- [           main] com.hazelcast.instance.Node              : [10.180.2.7]:5701 [someGroup] [3.7] Creating TcpIpJoiner
    2017-01-03 20:19:52.758  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.7]:5701 [someGroup] [3.7] [10.180.2.7]:5701 is STARTING
    2017-01-03 20:19:52.983  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.7]:5701 [someGroup] [3.7] Starting 2 partition threads
    2017-01-03 20:19:52.987  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.7]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)
    2017-01-03 20:19:52.999  INFO 6 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.7]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
    2017-01-03 20:19:53.025  INFO 6 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.7]:5701 [someGroup] [3.7] 
    
    
    Members [1] {
    	Member [10.180.2.7]:5701 - ee40f65e-9689-4e63-aabe-3ae056b5397f this
    }
    
    2017-01-03 20:19:53.096  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.7]:5701 [someGroup] [3.7] [10.180.2.7]:5701 is STARTED
    2017-01-03 20:19:53.108  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.139 seconds (JVM running for 5.807)
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:556

Issues about this test specifically: #27850 #30672 #33271

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc4217289f0>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.0.13\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.0.13\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.13\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.0.13\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  20:45:52 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  20:45:52 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.13; broadcast_rpc_address=10.180.0.13; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.13; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  20:45:52 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  20:45:52 Global memtable on-heap threshold is enabled at 128MB\nINFO  20:45:52 Global memtable off-heap threshold is enabled at 128MB\nINFO  20:45:52 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nINFO  20:45:52 Overriding RING_DELAY to 30000ms\nWARN  20:45:52 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  20:45:52 Hostname: cassandra-8z2xb\nINFO  20:45:52 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  20:45:52 Heap size: 512.000MiB/512.000MiB\nINFO  20:45:52 Code Cache Non-heap memory: init = 2555904(2496K) used = 4170432(4072K) committed = 4194304(4096K) max = 251658240(245760K)\nINFO  20:45:52 Metaspace Non-heap memory: init = 0(0K) used = 17241752(16837K) committed = 17694720(17280K) max = -1(-1K)\nINFO  20:45:52 Compressed Class Space Non-heap memory: init = 0(0K) used = 2069920(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  20:45:52 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)\nINFO  20:45:52 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  20:45:52 G1 Old Gen Heap memory: init = 508559360(496640K) used = 990208(967K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  20:45:52 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  20:45:52 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.13, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  20:45:52 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  20:45:52 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  20:45:52 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  20:45:52 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  20:45:52 Initializing SIGAR library\nWARN  20:45:52 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  20:45:53 Directory /var/lib/cassandra/hints doesn't exist\nINFO  20:45:53 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  20:45:54 Initializing system.IndexInfo\nINFO  20:45:55 Initializing system.batches\nINFO  20:45:55 Initializing system.paxos\nINFO  20:45:55 Initializing system.local\nINFO  20:45:55 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  20:45:55 Initializing key cache with capacity of 25 MBs.\nINFO  20:45:55 Initializing row cache with capacity of 0 MBs\nINFO  20:45:55 Initializing counter cache with capacity of 12 MBs\nINFO  20:45:55 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  20:45:55 Initializing system.peers\nINFO  20:45:55 Initializing system.peer_events\nINFO  20:45:55 Initializing system.range_xfers\nINFO  20:45:56 Initializing system.compaction_history\nINFO  20:45:56 Initializing system.sstable_activity\nINFO  20:45:56 Initializing system.size_estimates\nINFO  20:45:56 Initializing system.available_ranges\nINFO  20:45:56 Initializing system.views_builds_in_progress\nINFO  20:45:56 Initializing system.built_views\nINFO  20:45:56 Initializing system.hints\nINFO  20:45:56 Initializing system.batchlog\nINFO  20:45:56 Initializing system.schema_keyspaces\nINFO  20:45:56 Initializing system.schema_columnfamilies\nINFO  20:45:56 Initializing system.schema_columns\nINFO  20:45:56 Initializing system.schema_triggers\nINFO  20:45:56 Initializing system.schema_usertypes\nINFO  20:45:56 Initializing system.schema_functions\nINFO  20:45:56 Initializing system.schema_aggregates\nINFO  20:45:56 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  20:45:56 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  20:45:57 Populating token metadata from system tables\nINFO  20:45:57 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nWARN  20:45:57 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  20:45:57 Token metadata: \nINFO  20:45:57 Initializing system_schema.keyspaces\nINFO  20:45:57 Initializing system_schema.tables\nINFO  20:45:58 Initializing system_schema.columns\nINFO  20:45:58 Initializing system_schema.triggers\nINFO  20:45:58 Initializing system_schema.dropped_columns\nINFO  20:45:58 Initializing system_schema.views\nINFO  20:45:58 Initializing system_schema.types\nINFO  20:45:58 Initializing system_schema.functions\nINFO  20:45:58 Initializing system_schema.aggregates\nINFO  20:45:58 Initializing system_schema.indexes\nINFO  20:45:58 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  20:45:58 Completed loading (5 ms; 2 keys) KeyCache cache\nINFO  20:45:58 Replaying /cassandra_data/commitlog/CommitLog-6-1483476227115.log, /cassandra_data/commitlog/CommitLog-6-1483476227116.log\nINFO  20:46:00 Log replay complete, 92 replayed mutations\nINFO  20:46:00 Populating token metadata from system tables\nINFO  20:46:00 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nWARN  20:46:00 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  20:46:00 Token metadata: \nINFO  20:46:00 Cassandra version: 3.9\nINFO  20:46:00 Thrift API version: 20.1.0\nINFO  20:46:00 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  20:46:00 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  20:46:00 Starting Messaging Service on /10.180.0.13:7000 (eth0)\nINFO  20:46:00 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nWARN  20:46:00 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  20:46:31 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nWARN  20:46:31 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 20:46:31 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  20:46:31 No local state or state is in silent shutdown, not announcing shutdown\nINFO  20:46:31 Waiting for messaging service to quiesce\nINFO  20:46:31 MessagingService has terminated the accept() thread\nINFO  20:46:31 Paused hints dispatch\nINFO  20:46:31 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.0.13
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.0.13
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.13
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.0.13
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  20:45:52 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  20:45:52 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.13; broadcast_rpc_address=10.180.0.13; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.13; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  20:45:52 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  20:45:52 Global memtable on-heap threshold is enabled at 128MB
    INFO  20:45:52 Global memtable off-heap threshold is enabled at 128MB
    INFO  20:45:52 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    INFO  20:45:52 Overriding RING_DELAY to 30000ms
    WARN  20:45:52 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  20:45:52 Hostname: cassandra-8z2xb
    INFO  20:45:52 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  20:45:52 Heap size: 512.000MiB/512.000MiB
    INFO  20:45:52 Code Cache Non-heap memory: init = 2555904(2496K) used = 4170432(4072K) committed = 4194304(4096K) max = 251658240(245760K)
    INFO  20:45:52 Metaspace Non-heap memory: init = 0(0K) used = 17241752(16837K) committed = 17694720(17280K) max = -1(-1K)
    INFO  20:45:52 Compressed Class Space Non-heap memory: init = 0(0K) used = 2069920(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  20:45:52 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)
    INFO  20:45:52 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  20:45:52 G1 Old Gen Heap memory: init = 508559360(496640K) used = 990208(967K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  20:45:52 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  20:45:52 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.13, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  20:45:52 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  20:45:52 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  20:45:52 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  20:45:52 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  20:45:52 Initializing SIGAR library
    WARN  20:45:52 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  20:45:53 Directory /var/lib/cassandra/hints doesn't exist
    INFO  20:45:53 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  20:45:54 Initializing system.IndexInfo
    INFO  20:45:55 Initializing system.batches
    INFO  20:45:55 Initializing system.paxos
    INFO  20:45:55 Initializing system.local
    INFO  20:45:55 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  20:45:55 Initializing key cache with capacity of 25 MBs.
    INFO  20:45:55 Initializing row cache with capacity of 0 MBs
    INFO  20:45:55 Initializing counter cache with capacity of 12 MBs
    INFO  20:45:55 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  20:45:55 Initializing system.peers
    INFO  20:45:55 Initializing system.peer_events
    INFO  20:45:55 Initializing system.range_xfers
    INFO  20:45:56 Initializing system.compaction_history
    INFO  20:45:56 Initializing system.sstable_activity
    INFO  20:45:56 Initializing system.size_estimates
    INFO  20:45:56 Initializing system.available_ranges
    INFO  20:45:56 Initializing system.views_builds_in_progress
    INFO  20:45:56 Initializing system.built_views
    INFO  20:45:56 Initializing system.hints
    INFO  20:45:56 Initializing system.batchlog
    INFO  20:45:56 Initializing system.schema_keyspaces
    INFO  20:45:56 Initializing system.schema_columnfamilies
    INFO  20:45:56 Initializing system.schema_columns
    INFO  20:45:56 Initializing system.schema_triggers
    INFO  20:45:56 Initializing system.schema_usertypes
    INFO  20:45:56 Initializing system.schema_functions
    INFO  20:45:56 Initializing system.schema_aggregates
    INFO  20:45:56 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  20:45:56 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  20:45:57 Populating token metadata from system tables
    INFO  20:45:57 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    WARN  20:45:57 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  20:45:57 Token metadata: 
    INFO  20:45:57 Initializing system_schema.keyspaces
    INFO  20:45:57 Initializing system_schema.tables
    INFO  20:45:58 Initializing system_schema.columns
    INFO  20:45:58 Initializing system_schema.triggers
    INFO  20:45:58 Initializing system_schema.dropped_columns
    INFO  20:45:58 Initializing system_schema.views
    INFO  20:45:58 Initializing system_schema.types
    INFO  20:45:58 Initializing system_schema.functions
    INFO  20:45:58 Initializing system_schema.aggregates
    INFO  20:45:58 Initializing system_schema.indexes
    INFO  20:45:58 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  20:45:58 Completed loading (5 ms; 2 keys) KeyCache cache
    INFO  20:45:58 Replaying /cassandra_data/commitlog/CommitLog-6-1483476227115.log, /cassandra_data/commitlog/CommitLog-6-1483476227116.log
    INFO  20:46:00 Log replay complete, 92 replayed mutations
    INFO  20:46:00 Populating token metadata from system tables
    INFO  20:46:00 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    WARN  20:46:00 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  20:46:00 Token metadata: 
    INFO  20:46:00 Cassandra version: 3.9
    INFO  20:46:00 Thrift API version: 20.1.0
    INFO  20:46:00 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  20:46:00 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  20:46:00 Starting Messaging Service on /10.180.0.13:7000 (eth0)
    INFO  20:46:00 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    WARN  20:46:00 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[
Contributor

k8s-merge-robot commented Jan 3, 2017

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1902/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc420c2f010>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 20:19:48.817  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-dn58m with PID 6 (/bootstrapper.jar started by root in /)\n2017-01-03 20:19:48.824  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 20:19:48.912  INFO 6 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 20:19:48 GMT 2017]; root of context hierarchy\n2017-01-03 20:19:50.555  INFO 6 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 20:19:50.577  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 20:19:51.026  WARN 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-q2d79/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 20:19:51.026  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 20:19:51.291  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 20:19:51.294  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 20:19:51.294  WARN 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 20:19:51.307  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.7]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 20:19:51.339  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.7]:5701\n2017-01-03 20:19:51.340  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 20:19:51.340  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 20:19:51.706  INFO 6 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.7]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 20:19:52.753  INFO 6 --- [           main] com.hazelcast.instance.Node              : [10.180.2.7]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 20:19:52.758  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.7]:5701 [someGroup] [3.7] [10.180.2.7]:5701 is STARTING\n2017-01-03 20:19:52.983  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.7]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 20:19:52.987  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.7]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 20:19:52.999  INFO 6 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.7]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-03 20:19:53.025  INFO 6 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.7]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.2.7]:5701 - ee40f65e-9689-4e63-aabe-3ae056b5397f this\n}\n\n2017-01-03 20:19:53.096  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.7]:5701 [someGroup] [3.7] [10.180.2.7]:5701 is STARTED\n2017-01-03 20:19:53.108  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.139 seconds (JVM running for 5.807)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-03 20:19:48.817  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-dn58m with PID 6 (/bootstrapper.jar started by root in /)
    2017-01-03 20:19:48.824  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default
    2017-01-03 20:19:48.912  INFO 6 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 20:19:48 GMT 2017]; root of context hierarchy
    2017-01-03 20:19:50.555  INFO 6 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
    2017-01-03 20:19:50.577  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
    2017-01-03 20:19:51.026  WARN 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed
    
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-q2d79/endpoints/hazelcast
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]
    	at com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]
    	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]
    	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]
    
    2017-01-03 20:19:51.026  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.
    2017-01-03 20:19:51.291  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []
    2017-01-03 20:19:51.294  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.
    2017-01-03 20:19:51.294  WARN 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.
    2017-01-03 20:19:51.307  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.7]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
    2017-01-03 20:19:51.339  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.7]:5701
    2017-01-03 20:19:51.340  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
    2017-01-03 20:19:51.340  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.7]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1
    2017-01-03 20:19:51.706  INFO 6 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.7]:5701 [someGroup] [3.7] Backpressure is disabled
    2017-01-03 20:19:52.753  INFO 6 --- [           main] com.hazelcast.instance.Node              : [10.180.2.7]:5701 [someGroup] [3.7] Creating TcpIpJoiner
    2017-01-03 20:19:52.758  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.7]:5701 [someGroup] [3.7] [10.180.2.7]:5701 is STARTING
    2017-01-03 20:19:52.983  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.7]:5701 [someGroup] [3.7] Starting 2 partition threads
    2017-01-03 20:19:52.987  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.7]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)
    2017-01-03 20:19:52.999  INFO 6 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.7]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
    2017-01-03 20:19:53.025  INFO 6 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.7]:5701 [someGroup] [3.7] 
    
    
    Members [1] {
    	Member [10.180.2.7]:5701 - ee40f65e-9689-4e63-aabe-3ae056b5397f this
    }
    
    2017-01-03 20:19:53.096  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.7]:5701 [someGroup] [3.7] [10.180.2.7]:5701 is STARTED
    2017-01-03 20:19:53.108  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.139 seconds (JVM running for 5.807)
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:556

Issues about this test specifically: #27850 #30672 #33271

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc4217289f0>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.0.13\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.0.13\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.13\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.0.13\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  20:45:52 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  20:45:52 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.13; broadcast_rpc_address=10.180.0.13; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.13; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  20:45:52 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  20:45:52 Global memtable on-heap threshold is enabled at 128MB\nINFO  20:45:52 Global memtable off-heap threshold is enabled at 128MB\nINFO  20:45:52 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nINFO  20:45:52 Overriding RING_DELAY to 30000ms\nWARN  20:45:52 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  20:45:52 Hostname: cassandra-8z2xb\nINFO  20:45:52 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  20:45:52 Heap size: 512.000MiB/512.000MiB\nINFO  20:45:52 Code Cache Non-heap memory: init = 2555904(2496K) used = 4170432(4072K) committed = 4194304(4096K) max = 251658240(245760K)\nINFO  20:45:52 Metaspace Non-heap memory: init = 0(0K) used = 17241752(16837K) committed = 17694720(17280K) max = -1(-1K)\nINFO  20:45:52 Compressed Class Space Non-heap memory: init = 0(0K) used = 2069920(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  20:45:52 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)\nINFO  20:45:52 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  20:45:52 G1 Old Gen Heap memory: init = 508559360(496640K) used = 990208(967K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  20:45:52 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  20:45:52 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.13, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  20:45:52 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  20:45:52 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  20:45:52 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  20:45:52 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  20:45:52 Initializing SIGAR library\nWARN  20:45:52 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  20:45:53 Directory /var/lib/cassandra/hints doesn't exist\nINFO  20:45:53 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  20:45:54 Initializing system.IndexInfo\nINFO  20:45:55 Initializing system.batches\nINFO  20:45:55 Initializing system.paxos\nINFO  20:45:55 Initializing system.local\nINFO  20:45:55 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  20:45:55 Initializing key cache with capacity of 25 MBs.\nINFO  20:45:55 Initializing row cache with capacity of 0 MBs\nINFO  20:45:55 Initializing counter cache with capacity of 12 MBs\nINFO  20:45:55 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  20:45:55 Initializing system.peers\nINFO  20:45:55 Initializing system.peer_events\nINFO  20:45:55 Initializing system.range_xfers\nINFO  20:45:56 Initializing system.compaction_history\nINFO  20:45:56 Initializing system.sstable_activity\nINFO  20:45:56 Initializing system.size_estimates\nINFO  20:45:56 Initializing system.available_ranges\nINFO  20:45:56 Initializing system.views_builds_in_progress\nINFO  20:45:56 Initializing system.built_views\nINFO  20:45:56 Initializing system.hints\nINFO  20:45:56 Initializing system.batchlog\nINFO  20:45:56 Initializing system.schema_keyspaces\nINFO  20:45:56 Initializing system.schema_columnfamilies\nINFO  20:45:56 Initializing system.schema_columns\nINFO  20:45:56 Initializing system.schema_triggers\nINFO  20:45:56 Initializing system.schema_usertypes\nINFO  20:45:56 Initializing system.schema_functions\nINFO  20:45:56 Initializing system.schema_aggregates\nINFO  20:45:56 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  20:45:56 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  20:45:57 Populating token metadata from system tables\nINFO  20:45:57 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nWARN  20:45:57 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  20:45:57 Token metadata: \nINFO  20:45:57 Initializing system_schema.keyspaces\nINFO  20:45:57 Initializing system_schema.tables\nINFO  20:45:58 Initializing system_schema.columns\nINFO  20:45:58 Initializing system_schema.triggers\nINFO  20:45:58 Initializing system_schema.dropped_columns\nINFO  20:45:58 Initializing system_schema.views\nINFO  20:45:58 Initializing system_schema.types\nINFO  20:45:58 Initializing system_schema.functions\nINFO  20:45:58 Initializing system_schema.aggregates\nINFO  20:45:58 Initializing system_schema.indexes\nINFO  20:45:58 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  20:45:58 Completed loading (5 ms; 2 keys) KeyCache cache\nINFO  20:45:58 Replaying /cassandra_data/commitlog/CommitLog-6-1483476227115.log, /cassandra_data/commitlog/CommitLog-6-1483476227116.log\nINFO  20:46:00 Log replay complete, 92 replayed mutations\nINFO  20:46:00 Populating token metadata from system tables\nINFO  20:46:00 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nWARN  20:46:00 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  20:46:00 Token metadata: \nINFO  20:46:00 Cassandra version: 3.9\nINFO  20:46:00 Thrift API version: 20.1.0\nINFO  20:46:00 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  20:46:00 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  20:46:00 Starting Messaging Service on /10.180.0.13:7000 (eth0)\nINFO  20:46:00 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nWARN  20:46:00 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  20:46:31 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\nWARN  20:46:31 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 20:46:31 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  20:46:31 No local state or state is in silent shutdown, not announcing shutdown\nINFO  20:46:31 Waiting for messaging service to quiesce\nINFO  20:46:31 MessagingService has terminated the accept() thread\nINFO  20:46:31 Paused hints dispatch\nINFO  20:46:31 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.0.13
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.0.13
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.13
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.0.13
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  20:45:52 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  20:45:52 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.13; broadcast_rpc_address=10.180.0.13; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.13; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  20:45:52 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  20:45:52 Global memtable on-heap threshold is enabled at 128MB
    INFO  20:45:52 Global memtable off-heap threshold is enabled at 128MB
    INFO  20:45:52 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    INFO  20:45:52 Overriding RING_DELAY to 30000ms
    WARN  20:45:52 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  20:45:52 Hostname: cassandra-8z2xb
    INFO  20:45:52 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  20:45:52 Heap size: 512.000MiB/512.000MiB
    INFO  20:45:52 Code Cache Non-heap memory: init = 2555904(2496K) used = 4170432(4072K) committed = 4194304(4096K) max = 251658240(245760K)
    INFO  20:45:52 Metaspace Non-heap memory: init = 0(0K) used = 17241752(16837K) committed = 17694720(17280K) max = -1(-1K)
    INFO  20:45:52 Compressed Class Space Non-heap memory: init = 0(0K) used = 2069920(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  20:45:52 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)
    INFO  20:45:52 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  20:45:52 G1 Old Gen Heap memory: init = 508559360(496640K) used = 990208(967K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  20:45:52 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  20:45:52 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.13, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  20:45:52 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  20:45:52 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  20:45:52 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  20:45:52 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  20:45:52 Initializing SIGAR library
    WARN  20:45:52 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  20:45:53 Directory /var/lib/cassandra/hints doesn't exist
    INFO  20:45:53 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  20:45:54 Initializing system.IndexInfo
    INFO  20:45:55 Initializing system.batches
    INFO  20:45:55 Initializing system.paxos
    INFO  20:45:55 Initializing system.local
    INFO  20:45:55 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  20:45:55 Initializing key cache with capacity of 25 MBs.
    INFO  20:45:55 Initializing row cache with capacity of 0 MBs
    INFO  20:45:55 Initializing counter cache with capacity of 12 MBs
    INFO  20:45:55 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  20:45:55 Initializing system.peers
    INFO  20:45:55 Initializing system.peer_events
    INFO  20:45:55 Initializing system.range_xfers
    INFO  20:45:56 Initializing system.compaction_history
    INFO  20:45:56 Initializing system.sstable_activity
    INFO  20:45:56 Initializing system.size_estimates
    INFO  20:45:56 Initializing system.available_ranges
    INFO  20:45:56 Initializing system.views_builds_in_progress
    INFO  20:45:56 Initializing system.built_views
    INFO  20:45:56 Initializing system.hints
    INFO  20:45:56 Initializing system.batchlog
    INFO  20:45:56 Initializing system.schema_keyspaces
    INFO  20:45:56 Initializing system.schema_columnfamilies
    INFO  20:45:56 Initializing system.schema_columns
    INFO  20:45:56 Initializing system.schema_triggers
    INFO  20:45:56 Initializing system.schema_usertypes
    INFO  20:45:56 Initializing system.schema_functions
    INFO  20:45:56 Initializing system.schema_aggregates
    INFO  20:45:56 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  20:45:56 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  20:45:57 Populating token metadata from system tables
    INFO  20:45:57 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    WARN  20:45:57 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  20:45:57 Token metadata: 
    INFO  20:45:57 Initializing system_schema.keyspaces
    INFO  20:45:57 Initializing system_schema.tables
    INFO  20:45:58 Initializing system_schema.columns
    INFO  20:45:58 Initializing system_schema.triggers
    INFO  20:45:58 Initializing system_schema.dropped_columns
    INFO  20:45:58 Initializing system_schema.views
    INFO  20:45:58 Initializing system_schema.types
    INFO  20:45:58 Initializing system_schema.functions
    INFO  20:45:58 Initializing system_schema.aggregates
    INFO  20:45:58 Initializing system_schema.indexes
    INFO  20:45:58 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  20:45:58 Completed loading (5 ms; 2 keys) KeyCache cache
    INFO  20:45:58 Replaying /cassandra_data/commitlog/CommitLog-6-1483476227115.log, /cassandra_data/commitlog/CommitLog-6-1483476227116.log
    INFO  20:46:00 Log replay complete, 92 replayed mutations
    INFO  20:46:00 Populating token metadata from system tables
    INFO  20:46:00 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    WARN  20:46:00 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  20:46:00 Token metadata: 
    INFO  20:46:00 Cassandra version: 3.9
    INFO  20:46:00 Thrift API version: 20.1.0
    INFO  20:46:00 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  20:46:00 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  20:46:00 Starting Messaging Service on /10.180.0.13:7000 (eth0)
    INFO  20:46:00 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    WARN  20:46:00 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-x8t07/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[
@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Jan 3, 2017

Contributor

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1903/
Multiple broken tests:

Failed: Test {e2e.go}

exit status 1

Issues about this test specifically: #33361 #38663

Failed: [k8s.io] [Feature:Example] [k8s.io] RethinkDB should create and stop rethinkdb servers {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:524
Expected error:
    <*errors.errorString | 0xc420a3c2c0>: {
        s: "Failed to find \"Server ready\", last result: \"Checking for other nodes\nMy host: 10.180.2.6\nEndpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-5xj7b/endpoints/rethinkdb-driver\nLooking for IPs...\nparse error: Invalid numeric literal at line 1, column 5\n\"",
    }
    Failed to find "Server ready", last result: "Checking for other nodes
    My host: 10.180.2.6
    Endpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-5xj7b/endpoints/rethinkdb-driver
    Looking for IPs...
    parse error: Invalid numeric literal at line 1, column 5
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:502

Issues about this test specifically: #30581 #32755

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc42184de80>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.3.7\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.3.7\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.3.7\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.3.7\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  21:41:30 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  21:41:30 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.3.7; broadcast_rpc_address=10.180.3.7; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.3.7; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  21:41:30 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  21:41:30 Global memtable on-heap threshold is enabled at 128MB\nINFO  21:41:30 Global memtable off-heap threshold is enabled at 128MB\nINFO  21:41:30 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nINFO  21:41:31 Overriding RING_DELAY to 30000ms\nWARN  21:41:31 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  21:41:31 Hostname: cassandra-4skv3\nINFO  21:41:31 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  21:41:31 Heap size: 512.000MiB/512.000MiB\nINFO  21:41:31 Code Cache Non-heap memory: init = 2555904(2496K) used = 4387008(4284K) committed = 4456448(4352K) max = 251658240(245760K)\nINFO  21:41:31 Metaspace Non-heap memory: init = 0(0K) used = 17447152(17038K) committed = 17956864(17536K) max = -1(-1K)\nINFO  21:41:31 Compressed Class Space Non-heap memory: init = 0(0K) used = 2083232(2034K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  21:41:31 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)\nINFO  21:41:31 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  21:41:31 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1134080(1107K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  21:41:31 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  21:41:31 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.3.7, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  21:41:31 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  21:41:31 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  21:41:31 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  21:41:31 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  21:41:31 Initializing SIGAR library\nWARN  21:41:31 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  21:41:31 Directory /var/lib/cassandra/hints doesn't exist\nINFO  21:41:31 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  21:41:32 Initializing system.IndexInfo\nINFO  21:41:33 Initializing system.batches\nINFO  21:41:33 Initializing system.paxos\nINFO  21:41:33 Initializing system.local\nINFO  21:41:33 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  21:41:33 Initializing key cache with capacity of 25 MBs.\nINFO  21:41:34 Initializing row cache with capacity of 0 MBs\nINFO  21:41:34 Initializing counter cache with capacity of 12 MBs\nINFO  21:41:34 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  21:41:34 Initializing system.peers\nINFO  21:41:34 Initializing system.peer_events\nINFO  21:41:34 Initializing system.range_xfers\nINFO  21:41:34 Initializing system.compaction_history\nINFO  21:41:34 Initializing system.sstable_activity\nINFO  21:41:34 Initializing system.size_estimates\nINFO  21:41:34 Initializing system.available_ranges\nINFO  21:41:34 Initializing system.views_builds_in_progress\nINFO  21:41:34 Initializing system.built_views\nINFO  21:41:34 Initializing system.hints\nINFO  21:41:34 Initializing system.batchlog\nINFO  21:41:34 Initializing system.schema_keyspaces\nINFO  21:41:34 Initializing system.schema_columnfamilies\nINFO  21:41:34 Initializing system.schema_columns\nINFO  21:41:34 Initializing system.schema_triggers\nINFO  21:41:34 Initializing system.schema_usertypes\nINFO  21:41:34 Initializing system.schema_functions\nINFO  21:41:34 Initializing system.schema_aggregates\nINFO  21:41:34 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  21:41:34 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  21:41:35 Populating token metadata from system tables\nINFO  21:41:35 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nWARN  21:41:35 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  21:41:35 Token metadata: \nINFO  21:41:36 Initializing system_schema.keyspaces\nINFO  21:41:36 Initializing system_schema.tables\nINFO  21:41:36 Initializing system_schema.columns\nINFO  21:41:36 Initializing system_schema.triggers\nINFO  21:41:36 Initializing system_schema.dropped_columns\nINFO  21:41:36 Initializing system_schema.views\nINFO  21:41:36 Initializing system_schema.types\nINFO  21:41:36 Initializing system_schema.functions\nINFO  21:41:36 Initializing system_schema.aggregates\nINFO  21:41:36 Initializing system_schema.indexes\nINFO  21:41:36 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  21:41:36 Completed loading (2 ms; 2 keys) KeyCache cache\nINFO  21:41:36 Replaying /cassandra_data/commitlog/CommitLog-6-1483479570799.log, /cassandra_data/commitlog/CommitLog-6-1483479570800.log\nINFO  21:41:38 Log replay complete, 92 replayed mutations\nINFO  21:41:38 Populating token metadata from system tables\nINFO  21:41:38 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nWARN  21:41:38 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  21:41:38 Token metadata: \nINFO  21:41:38 Cassandra version: 3.9\nINFO  21:41:38 Thrift API version: 20.1.0\nINFO  21:41:38 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  21:41:38 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  21:41:38 Starting Messaging Service on /10.180.3.7:7000 (eth0)\nINFO  21:41:38 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nWARN  21:41:38 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  21:42:09 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nWARN  21:42:09 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 21:42:09 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  21:42:09 No local state or state is in silent shutdown, not announcing shutdown\nINFO  21:42:09 Waiting for messaging service to quiesce\nINFO  21:42:09 MessagingService has terminated the accept() thread\nINFO  21:42:09 Paused hints dispatch\nINFO  21:42:09 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.3.7
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.3.7
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.3.7
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.3.7
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  21:41:30 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  21:41:30 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.3.7; broadcast_rpc_address=10.180.3.7; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.3.7; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  21:41:30 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  21:41:30 Global memtable on-heap threshold is enabled at 128MB
    INFO  21:41:30 Global memtable off-heap threshold is enabled at 128MB
    INFO  21:41:30 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    INFO  21:41:31 Overriding RING_DELAY to 30000ms
    WARN  21:41:31 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  21:41:31 Hostname: cassandra-4skv3
    INFO  21:41:31 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  21:41:31 Heap size: 512.000MiB/512.000MiB
    INFO  21:41:31 Code Cache Non-heap memory: init = 2555904(2496K) used = 4387008(4284K) committed = 4456448(4352K) max = 251658240(245760K)
    INFO  21:41:31 Metaspace Non-heap memory: init = 0(0K) used = 17447152(17038K) committed = 17956864(17536K) max = -1(-1K)
    INFO  21:41:31 Compressed Class Space Non-heap memory: init = 0(0K) used = 2083232(2034K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  21:41:31 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)
    INFO  21:41:31 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  21:41:31 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1134080(1107K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  21:41:31 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  21:41:31 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.3.7, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  21:41:31 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  21:41:31 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  21:41:31 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  21:41:31 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  21:41:31 Initializing SIGAR library
    WARN  21:41:31 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  21:41:31 Directory /var/lib/cassandra/hints doesn't exist
    INFO  21:41:31 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  21:41:32 Initializing system.IndexInfo
    INFO  21:41:33 Initializing system.batches
    INFO  21:41:33 Initializing system.paxos
    INFO  21:41:33 Initializing system.local
    INFO  21:41:33 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  21:41:33 Initializing key cache with capacity of 25 MBs.
    INFO  21:41:34 Initializing row cache with capacity of 0 MBs
    INFO  21:41:34 Initializing counter cache with capacity of 12 MBs
    INFO  21:41:34 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  21:41:34 Initializing system.peers
    INFO  21:41:34 Initializing system.peer_events
    INFO  21:41:34 Initializing system.range_xfers
    INFO  21:41:34 Initializing system.compaction_history
    INFO  21:41:34 Initializing system.sstable_activity
    INFO  21:41:34 Initializing system.size_estimates
    INFO  21:41:34 Initializing system.available_ranges
    INFO  21:41:34 Initializing system.views_builds_in_progress
    INFO  21:41:34 Initializing system.built_views
    INFO  21:41:34 Initializing system.hints
    INFO  21:41:34 Initializing system.batchlog
    INFO  21:41:34 Initializing system.schema_keyspaces
    INFO  21:41:34 Initializing system.schema_columnfamilies
    INFO  21:41:34 Initializing system.schema_columns
    INFO  21:41:34 Initializing system.schema_triggers
    INFO  21:41:34 Initializing system.schema_usertypes
    INFO  21:41:34 Initializing system.schema_functions
    INFO  21:41:34 Initializing system.schema_aggregates
    INFO  21:41:34 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  21:41:34 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  21:41:35 Populating token metadata from system tables
    INFO  21:41:35 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    WARN  21:41:35 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  21:41:35 Token metadata: 
    INFO  21:41:36 Initializing system_schema.keyspaces
    INFO  21:41:36 Initializing system_schema.tables
    INFO  21:41:36 Initializing system_schema.columns
    INFO  21:41:36 Initializing system_schema.triggers
    INFO  21:41:36 Initializing system_schema.dropped_columns
    INFO  21:41:36 Initializing system_schema.views
    INFO  21:41:36 Initializing system_schema.types
    INFO  21:41:36 Initializing system_schema.functions
    INFO  21:41:36 Initializing system_schema.aggregates
    INFO  21:41:36 Initializing system_schema.indexes
    INFO  21:41:36 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  21:41:36 Completed loading (2 ms; 2 keys) KeyCache cache
    INFO  21:41:36 Replaying /cassandra_data/commitlog/CommitLog-6-1483479570799.log, /cassandra_data/commitlog/CommitLog-6-1483479570800.log
    INFO  21:41:38 Log replay complete, 92 replayed mutations
    INFO  21:41:38 Populating token metadata from system tables
    INFO  21:41:38 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    WARN  21:41:38 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  21:41:38 Token metadata: 
    INFO  21:41:38 Cassandra version: 3.9
    INFO  21:41:38 Thrift API version: 20.1.0
    INFO  21:41:38 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  21:41:38 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  21:41:38 Starting Messaging Service on /10.180.3.7:7000 (eth0)
    INFO  21:41:38 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    WARN  21:41:38 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  21:42:09 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    WARN  21:42:09 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    Exception (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
    ERROR 21:42:09 Exception encountered during startup
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    WARN  21:42:09 No local state or state is in silent shutdown, not announcing shutdown
    INFO  21:42:09 Waiting for messaging service to quiesce
    INFO  21:42:09 MessagingService has terminated the accept() thread
    INFO  21:42:09 Paused hints dispatch
    INFO  21:42:09 Paused hints dispatch
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:236

Issues about this test specifically: #27978 #28817

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc420b1c400>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 21:49:20.597  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-54660 with PID 6 (/bootstrapper.jar started by root in /)\n2017-01-03 21:49:20.605  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 21:49:20.727  INFO 6 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 21:49:20 GMT 2017]; root of context hierarchy\n2017-01-03 21:49:22.738  INFO 6 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 21:49:22.761  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 21:49:23.228  WARN 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-s9zcw/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 21:49:23.230  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 21:49:23.466  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 21:49:23.468  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 21:49:23.468  WARN 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 21:49:23.483  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.18]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 21:49:23.519  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.18]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.18]:5701\n2017-01-03 21:49:23.520  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.18]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 21:49:23.520  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.18]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 21:49:23.895  INFO 6 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.18]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 21:49:24.916  INFO 6 --- [           main] com.hazelcast.instance.Node              : [10.180.2.18]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 21:49:24.921  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.18]:5701 [someGroup] [3.7] [10.180.2.18]:5701 is STARTING\n2017-01-03 21:49:25.224  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.18]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 21:49:25.227  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.18]:5701 [someGroup] [3.7] Starting 
Contributor

k8s-merge-robot commented Jan 3, 2017

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1903/
Multiple broken tests:

Failed: Test {e2e.go}

exit status 1

Issues about this test specifically: #33361 #38663

Failed: [k8s.io] [Feature:Example] [k8s.io] RethinkDB should create and stop rethinkdb servers {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:524
Expected error:
    <*errors.errorString | 0xc420a3c2c0>: {
        s: "Failed to find \"Server ready\", last result: \"Checking for other nodes\nMy host: 10.180.2.6\nEndpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-5xj7b/endpoints/rethinkdb-driver\nLooking for IPs...\nparse error: Invalid numeric literal at line 1, column 5\n\"",
    }
    Failed to find "Server ready", last result: "Checking for other nodes
    My host: 10.180.2.6
    Endpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-5xj7b/endpoints/rethinkdb-driver
    Looking for IPs...
    parse error: Invalid numeric literal at line 1, column 5
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:502

Issues about this test specifically: #30581 #32755

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc42184de80>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.3.7\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.3.7\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.3.7\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.3.7\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  21:41:30 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  21:41:30 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.3.7; broadcast_rpc_address=10.180.3.7; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.3.7; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  21:41:30 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  21:41:30 Global memtable on-heap threshold is enabled at 128MB\nINFO  21:41:30 Global memtable off-heap threshold is enabled at 128MB\nINFO  21:41:30 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nINFO  21:41:31 Overriding RING_DELAY to 30000ms\nWARN  21:41:31 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  21:41:31 Hostname: cassandra-4skv3\nINFO  21:41:31 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  21:41:31 Heap size: 512.000MiB/512.000MiB\nINFO  21:41:31 Code Cache Non-heap memory: init = 2555904(2496K) used = 4387008(4284K) committed = 4456448(4352K) max = 251658240(245760K)\nINFO  21:41:31 Metaspace Non-heap memory: init = 0(0K) used = 17447152(17038K) committed = 17956864(17536K) max = -1(-1K)\nINFO  21:41:31 Compressed Class Space Non-heap memory: init = 0(0K) used = 2083232(2034K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  21:41:31 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)\nINFO  21:41:31 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  21:41:31 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1134080(1107K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  21:41:31 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  21:41:31 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.3.7, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  21:41:31 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  21:41:31 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  21:41:31 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  21:41:31 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  21:41:31 Initializing SIGAR library\nWARN  21:41:31 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  21:41:31 Directory /var/lib/cassandra/hints doesn't exist\nINFO  21:41:31 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  21:41:32 Initializing system.IndexInfo\nINFO  21:41:33 Initializing system.batches\nINFO  21:41:33 Initializing system.paxos\nINFO  21:41:33 Initializing system.local\nINFO  21:41:33 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  21:41:33 Initializing key cache with capacity of 25 MBs.\nINFO  21:41:34 Initializing row cache with capacity of 0 MBs\nINFO  21:41:34 Initializing counter cache with capacity of 12 MBs\nINFO  21:41:34 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  21:41:34 Initializing system.peers\nINFO  21:41:34 Initializing system.peer_events\nINFO  21:41:34 Initializing system.range_xfers\nINFO  21:41:34 Initializing system.compaction_history\nINFO  21:41:34 Initializing system.sstable_activity\nINFO  21:41:34 Initializing system.size_estimates\nINFO  21:41:34 Initializing system.available_ranges\nINFO  21:41:34 Initializing system.views_builds_in_progress\nINFO  21:41:34 Initializing system.built_views\nINFO  21:41:34 Initializing system.hints\nINFO  21:41:34 Initializing system.batchlog\nINFO  21:41:34 Initializing system.schema_keyspaces\nINFO  21:41:34 Initializing system.schema_columnfamilies\nINFO  21:41:34 Initializing system.schema_columns\nINFO  21:41:34 Initializing system.schema_triggers\nINFO  21:41:34 Initializing system.schema_usertypes\nINFO  21:41:34 Initializing system.schema_functions\nINFO  21:41:34 Initializing system.schema_aggregates\nINFO  21:41:34 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  21:41:34 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  21:41:35 Populating token metadata from system tables\nINFO  21:41:35 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nWARN  21:41:35 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  21:41:35 Token metadata: \nINFO  21:41:36 Initializing system_schema.keyspaces\nINFO  21:41:36 Initializing system_schema.tables\nINFO  21:41:36 Initializing system_schema.columns\nINFO  21:41:36 Initializing system_schema.triggers\nINFO  21:41:36 Initializing system_schema.dropped_columns\nINFO  21:41:36 Initializing system_schema.views\nINFO  21:41:36 Initializing system_schema.types\nINFO  21:41:36 Initializing system_schema.functions\nINFO  21:41:36 Initializing system_schema.aggregates\nINFO  21:41:36 Initializing system_schema.indexes\nINFO  21:41:36 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  21:41:36 Completed loading (2 ms; 2 keys) KeyCache cache\nINFO  21:41:36 Replaying /cassandra_data/commitlog/CommitLog-6-1483479570799.log, /cassandra_data/commitlog/CommitLog-6-1483479570800.log\nINFO  21:41:38 Log replay complete, 92 replayed mutations\nINFO  21:41:38 Populating token metadata from system tables\nINFO  21:41:38 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nWARN  21:41:38 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  21:41:38 Token metadata: \nINFO  21:41:38 Cassandra version: 3.9\nINFO  21:41:38 Thrift API version: 20.1.0\nINFO  21:41:38 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  21:41:38 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  21:41:38 Starting Messaging Service on /10.180.3.7:7000 (eth0)\nINFO  21:41:38 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nWARN  21:41:38 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  21:42:09 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\nWARN  21:42:09 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 21:42:09 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  21:42:09 No local state or state is in silent shutdown, not announcing shutdown\nINFO  21:42:09 Waiting for messaging service to quiesce\nINFO  21:42:09 MessagingService has terminated the accept() thread\nINFO  21:42:09 Paused hints dispatch\nINFO  21:42:09 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.3.7
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.3.7
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.3.7
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.3.7
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  21:41:30 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  21:41:30 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.3.7; broadcast_rpc_address=10.180.3.7; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.3.7; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  21:41:30 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  21:41:30 Global memtable on-heap threshold is enabled at 128MB
    INFO  21:41:30 Global memtable off-heap threshold is enabled at 128MB
    INFO  21:41:30 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    INFO  21:41:31 Overriding RING_DELAY to 30000ms
    WARN  21:41:31 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  21:41:31 Hostname: cassandra-4skv3
    INFO  21:41:31 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  21:41:31 Heap size: 512.000MiB/512.000MiB
    INFO  21:41:31 Code Cache Non-heap memory: init = 2555904(2496K) used = 4387008(4284K) committed = 4456448(4352K) max = 251658240(245760K)
    INFO  21:41:31 Metaspace Non-heap memory: init = 0(0K) used = 17447152(17038K) committed = 17956864(17536K) max = -1(-1K)
    INFO  21:41:31 Compressed Class Space Non-heap memory: init = 0(0K) used = 2083232(2034K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  21:41:31 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)
    INFO  21:41:31 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  21:41:31 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1134080(1107K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  21:41:31 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  21:41:31 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.3.7, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  21:41:31 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  21:41:31 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  21:41:31 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  21:41:31 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  21:41:31 Initializing SIGAR library
    WARN  21:41:31 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  21:41:31 Directory /var/lib/cassandra/hints doesn't exist
    INFO  21:41:31 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  21:41:32 Initializing system.IndexInfo
    INFO  21:41:33 Initializing system.batches
    INFO  21:41:33 Initializing system.paxos
    INFO  21:41:33 Initializing system.local
    INFO  21:41:33 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  21:41:33 Initializing key cache with capacity of 25 MBs.
    INFO  21:41:34 Initializing row cache with capacity of 0 MBs
    INFO  21:41:34 Initializing counter cache with capacity of 12 MBs
    INFO  21:41:34 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  21:41:34 Initializing system.peers
    INFO  21:41:34 Initializing system.peer_events
    INFO  21:41:34 Initializing system.range_xfers
    INFO  21:41:34 Initializing system.compaction_history
    INFO  21:41:34 Initializing system.sstable_activity
    INFO  21:41:34 Initializing system.size_estimates
    INFO  21:41:34 Initializing system.available_ranges
    INFO  21:41:34 Initializing system.views_builds_in_progress
    INFO  21:41:34 Initializing system.built_views
    INFO  21:41:34 Initializing system.hints
    INFO  21:41:34 Initializing system.batchlog
    INFO  21:41:34 Initializing system.schema_keyspaces
    INFO  21:41:34 Initializing system.schema_columnfamilies
    INFO  21:41:34 Initializing system.schema_columns
    INFO  21:41:34 Initializing system.schema_triggers
    INFO  21:41:34 Initializing system.schema_usertypes
    INFO  21:41:34 Initializing system.schema_functions
    INFO  21:41:34 Initializing system.schema_aggregates
    INFO  21:41:34 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  21:41:34 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  21:41:35 Populating token metadata from system tables
    INFO  21:41:35 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    WARN  21:41:35 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  21:41:35 Token metadata: 
    INFO  21:41:36 Initializing system_schema.keyspaces
    INFO  21:41:36 Initializing system_schema.tables
    INFO  21:41:36 Initializing system_schema.columns
    INFO  21:41:36 Initializing system_schema.triggers
    INFO  21:41:36 Initializing system_schema.dropped_columns
    INFO  21:41:36 Initializing system_schema.views
    INFO  21:41:36 Initializing system_schema.types
    INFO  21:41:36 Initializing system_schema.functions
    INFO  21:41:36 Initializing system_schema.aggregates
    INFO  21:41:36 Initializing system_schema.indexes
    INFO  21:41:36 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  21:41:36 Completed loading (2 ms; 2 keys) KeyCache cache
    INFO  21:41:36 Replaying /cassandra_data/commitlog/CommitLog-6-1483479570799.log, /cassandra_data/commitlog/CommitLog-6-1483479570800.log
    INFO  21:41:38 Log replay complete, 92 replayed mutations
    INFO  21:41:38 Populating token metadata from system tables
    INFO  21:41:38 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    WARN  21:41:38 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  21:41:38 Token metadata: 
    INFO  21:41:38 Cassandra version: 3.9
    INFO  21:41:38 Thrift API version: 20.1.0
    INFO  21:41:38 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  21:41:38 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  21:41:38 Starting Messaging Service on /10.180.3.7:7000 (eth0)
    INFO  21:41:38 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    WARN  21:41:38 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  21:42:09 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    WARN  21:42:09 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-j0fk3/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    Exception (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
    ERROR 21:42:09 Exception encountered during startup
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    WARN  21:42:09 No local state or state is in silent shutdown, not announcing shutdown
    INFO  21:42:09 Waiting for messaging service to quiesce
    INFO  21:42:09 MessagingService has terminated the accept() thread
    INFO  21:42:09 Paused hints dispatch
    INFO  21:42:09 Paused hints dispatch
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:236

Issues about this test specifically: #27978 #28817

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc420b1c400>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 21:49:20.597  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-54660 with PID 6 (/bootstrapper.jar started by root in /)\n2017-01-03 21:49:20.605  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 21:49:20.727  INFO 6 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 21:49:20 GMT 2017]; root of context hierarchy\n2017-01-03 21:49:22.738  INFO 6 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 21:49:22.761  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 21:49:23.228  WARN 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-s9zcw/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 21:49:23.230  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 21:49:23.466  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 21:49:23.468  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 21:49:23.468  WARN 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 21:49:23.483  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.18]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 21:49:23.519  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.18]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.18]:5701\n2017-01-03 21:49:23.520  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.18]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 21:49:23.520  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.18]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 21:49:23.895  INFO 6 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.18]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 21:49:24.916  INFO 6 --- [           main] com.hazelcast.instance.Node              : [10.180.2.18]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 21:49:24.921  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.18]:5701 [someGroup] [3.7] [10.180.2.18]:5701 is STARTING\n2017-01-03 21:49:25.224  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.18]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 21:49:25.227  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.18]:5701 [someGroup] [3.7] Starting 
@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Jan 3, 2017

Contributor

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1904/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] RethinkDB should create and stop rethinkdb servers {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:524
Expected error:
    <*errors.errorString | 0xc4205ec120>: {
        s: "Failed to find \"Server ready\", last result: \"Checking for other nodes\nMy host: 10.180.0.5\nEndpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l79gx/endpoints/rethinkdb-driver\nLooking for IPs...\nparse error: Invalid numeric literal at line 1, column 5\n\"",
    }
    Failed to find "Server ready", last result: "Checking for other nodes
    My host: 10.180.0.5
    Endpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l79gx/endpoints/rethinkdb-driver
    Looking for IPs...
    parse error: Invalid numeric literal at line 1, column 5
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:502

Issues about this test specifically: #30581 #32755

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc4206b11a0>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 22:39:47.599  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-5cvfb with PID 5 (/bootstrapper.jar started by root in /)\n2017-01-03 22:39:47.612  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 22:39:47.767  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 22:39:47 GMT 2017]; root of context hierarchy\n2017-01-03 22:39:49.633  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 22:39:49.652  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 22:39:50.023  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-cnjs5/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 22:39:50.023  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 22:39:50.265  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 22:39:50.267  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 22:39:50.267  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 22:39:50.282  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.0.14]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.0.14]:5701\n2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 22:39:50.700  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.0.14]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 22:39:51.548  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.0.14]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 22:39:51.559  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.0.14]:5701 [someGroup] [3.7] [10.180.0.14]:5701 is STARTING\n2017-01-03 22:39:51.825  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.0.14]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 22:39:51.827  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.0.14]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 22:39:51.845  INFO 5 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.0.14]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-03 22:39:51.868  INFO 5 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.0.14]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.0.14]:5701 - c7b06b22-bb44-42b1-b16c-4c2609fefaa4 this\n}\n\n2017-01-03 22:39:51.931  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.0.14]:5701 [someGroup] [3.7] [10.180.0.14]:5701 is STARTED\n2017-01-03 22:39:51.935  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.321 seconds (JVM running for 6.039)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-03 22:39:47.599  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-5cvfb with PID 5 (/bootstrapper.jar started by root in /)
    2017-01-03 22:39:47.612  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default
    2017-01-03 22:39:47.767  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 22:39:47 GMT 2017]; root of context hierarchy
    2017-01-03 22:39:49.633  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
    2017-01-03 22:39:49.652  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
    2017-01-03 22:39:50.023  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed
    
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-cnjs5/endpoints/hazelcast
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]
    	at com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]
    	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]
    	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]
    
    2017-01-03 22:39:50.023  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.
    2017-01-03 22:39:50.265  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []
    2017-01-03 22:39:50.267  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.
    2017-01-03 22:39:50.267  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.
    2017-01-03 22:39:50.282  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.0.14]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
    2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.0.14]:5701
    2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
    2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1
    2017-01-03 22:39:50.700  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.0.14]:5701 [someGroup] [3.7] Backpressure is disabled
    2017-01-03 22:39:51.548  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.0.14]:5701 [someGroup] [3.7] Creating TcpIpJoiner
    2017-01-03 22:39:51.559  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.0.14]:5701 [someGroup] [3.7] [10.180.0.14]:5701 is STARTING
    2017-01-03 22:39:51.825  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.0.14]:5701 [someGroup] [3.7] Starting 2 partition threads
    2017-01-03 22:39:51.827  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.0.14]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)
    2017-01-03 22:39:51.845  INFO 5 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.0.14]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
    2017-01-03 22:39:51.868  INFO 5 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.0.14]:5701 [someGroup] [3.7] 
    
    
    Members [1] {
    	Member [10.180.0.14]:5701 - c7b06b22-bb44-42b1-b16c-4c2609fefaa4 this
    }
    
    2017-01-03 22:39:51.931  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.0.14]:5701 [someGroup] [3.7] [10.180.0.14]:5701 is STARTED
    2017-01-03 22:39:51.935  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.321 seconds (JVM running for 6.039)
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:556

Issues about this test specifically: #27850 #30672 #33271

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc420bfaed0>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.0.18\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.0.18\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.18\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.0.18\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  22:57:42 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  22:57:42 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.18; broadcast_rpc_address=10.180.0.18; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.18; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  22:57:42 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  22:57:42 Global memtable on-heap threshold is enabled at 128MB\nINFO  22:57:42 Global memtable off-heap threshold is enabled at 128MB\nINFO  22:57:42 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nINFO  22:57:42 Overriding RING_DELAY to 30000ms\nWARN  22:57:43 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  22:57:43 Hostname: cassandra-p0hg7\nINFO  22:57:43 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  22:57:43 Heap size: 512.000MiB/512.000MiB\nINFO  22:57:43 Code Cache Non-heap memory: init = 2555904(2496K) used = 4396416(4293K) committed = 4456448(4352K) max = 251658240(245760K)\nINFO  22:57:43 Metaspace Non-heap memory: init = 0(0K) used = 17257592(16853K) committed = 17694720(17280K) max = -1(-1K)\nINFO  22:57:43 Compressed Class Space Non-heap memory: init = 0(0K) used = 2070480(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  22:57:43 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)\nINFO  22:57:43 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  22:57:43 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1083392(1058K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  22:57:43 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  22:57:43 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.18, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  22:57:43 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  22:57:43 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  22:57:43 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  22:57:43 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  22:57:43 Initializing SIGAR library\nWARN  22:57:43 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  22:57:43 Directory /var/lib/cassandra/hints doesn't exist\nINFO  22:57:43 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  22:57:43 Initializing system.IndexInfo\nINFO  22:57:45 Initializing system.batches\nINFO  22:57:45 Initializing system.paxos\nINFO  22:57:45 Initializing system.local\nINFO  22:57:45 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  22:57:45 Initializing key cache with capacity of 25 MBs.\nINFO  22:57:45 Initializing row cache with capacity of 0 MBs\nINFO  22:57:45 Initializing counter cache with capacity of 12 MBs\nINFO  22:57:45 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  22:57:45 Initializing system.peers\nINFO  22:57:45 Initializing system.peer_events\nINFO  22:57:45 Initializing system.range_xfers\nINFO  22:57:45 Initializing system.compaction_history\nINFO  22:57:45 Initializing system.sstable_activity\nINFO  22:57:45 Initializing system.size_estimates\nINFO  22:57:45 Initializing system.available_ranges\nINFO  22:57:45 Initializing system.views_builds_in_progress\nINFO  22:57:45 Initializing system.built_views\nINFO  22:57:45 Initializing system.hints\nINFO  22:57:45 Initializing system.batchlog\nINFO  22:57:45 Initializing system.schema_keyspaces\nINFO  22:57:45 Initializing system.schema_columnfamilies\nINFO  22:57:45 Initializing system.schema_columns\nINFO  22:57:45 Initializing system.schema_triggers\nINFO  22:57:45 Initializing system.schema_usertypes\nINFO  22:57:45 Initializing system.schema_functions\nINFO  22:57:45 Initializing system.schema_aggregates\nINFO  22:57:45 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  22:57:46 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  22:57:47 Populating token metadata from system tables\nINFO  22:57:47 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nWARN  22:57:47 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  22:57:47 Token metadata: \nINFO  22:57:47 Initializing system_schema.keyspaces\nINFO  22:57:47 Initializing system_schema.tables\nINFO  22:57:47 Initializing system_schema.columns\nINFO  22:57:47 Initializing system_schema.triggers\nINFO  22:57:47 Initializing system_schema.dropped_columns\nINFO  22:57:47 Initializing system_schema.views\nINFO  22:57:47 Initializing system_schema.types\nINFO  22:57:47 Initializing system_schema.functions\nINFO  22:57:47 Initializing system_schema.aggregates\nINFO  22:57:47 Initializing system_schema.indexes\nINFO  22:57:47 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  22:57:47 Completed loading (2 ms; 2 keys) KeyCache cache\nINFO  22:57:47 Replaying /cassandra_data/commitlog/CommitLog-6-1483484140294.log, /cassandra_data/commitlog/CommitLog-6-1483484140295.log\nINFO  22:57:49 Log replay complete, 92 replayed mutations\nINFO  22:57:49 Populating token metadata from system tables\nINFO  22:57:49 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nWARN  22:57:49 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  22:57:49 Token metadata: \nINFO  22:57:50 Cassandra version: 3.9\nINFO  22:57:50 Thrift API version: 20.1.0\nINFO  22:57:50 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  22:57:50 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  22:57:50 Starting Messaging Service on /10.180.0.18:7000 (eth0)\nINFO  22:57:50 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nWARN  22:57:50 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  22:58:21 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nWARN  22:58:21 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 22:58:21 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  22:58:21 No local state or state is in silent shutdown, not announcing shutdown\nINFO  22:58:21 Waiting for messaging service to quiesce\nINFO  22:58:21 MessagingService has terminated the accept() thread\nINFO  22:58:21 Paused hints dispatch\nINFO  22:58:21 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.0.18
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.0.18
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.18
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.0.18
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  22:57:42 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  22:57:42 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.18; broadcast_rpc_address=10.180.0.18; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.18; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  22:57:42 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  22:57:42 Global memtable on-heap threshold is enabled at 128MB
    INFO  22:57:42 Global memtable off-heap threshold is enabled at 128MB
    INFO  22:57:42 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    INFO  22:57:42 Overriding RING_DELAY to 30000ms
    WARN  22:57:43 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  22:57:43 Hostname: cassandra-p0hg7
    INFO  22:57:43 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  22:57:43 Heap size: 512.000MiB/512.000MiB
    INFO  22:57:43 Code Cache Non-heap memory: init = 2555904(2496K) used = 4396416(4293K) committed = 4456448(4352K) max = 251658240(245760K)
    INFO  22:57:43 Metaspace Non-heap memory: init = 0(0K) used = 17257592(16853K) committed = 17694720(17280K) max = -1(-1K)
    INFO  22:57:43 Compressed Class Space Non-heap memory: init = 0(0K) used = 2070480(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  22:57:43 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)
    INFO  22:57:43 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  22:57:43 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1083392(1058K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  22:57:43 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  22:57:43 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.18, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  22:57:43 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  22:57:43 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  22:57:43 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  22:57:43 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  22:57:43 Initializing SIGAR library
    WARN  22:57:43 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  22:57:43 Directory /var/lib/cassandra/hints doesn't exist
    INFO  22:57:43 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  22:57:43 Initializing system.IndexInfo
    INFO  22:57:45 Initializing system.batches
    INFO  22:57:45 Initializing system.paxos
    INFO  22:57:45 Initializing system.local
    INFO  22:57:45 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  22:57:45 Initializing key cache with capacity of 25 MBs.
    INFO  22:57:45 Initializing row cache with capacity of 0 MBs
    INFO  22:57:45 Initializing counter cache with capacity of 12 MBs
    INFO  22:57:45 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  22:57:45 Initializing system.peers
    INFO  22:57:45 Initializing system.peer_events
    INFO  22:57:45 Initializing system.range_xfers
    INFO  22:57:45 Initializing system.compaction_history
    INFO  22:57:45 Initializing system.sstable_activity
    INFO  22:57:45 Initializing system.size_estimates
    INFO  22:57:45 Initializing system.available_ranges
    INFO  22:57:45 Initializing system.views_builds_in_progress
    INFO  22:57:45 Initializing system.built_views
    INFO  22:57:45 Initializing system.hints
    INFO  22:57:45 Initializing system.batchlog
    INFO  22:57:45 Initializing system.schema_keyspaces
    INFO  22:57:45 Initializing system.schema_columnfamilies
    INFO  22:57:45 Initializing system.schema_columns
    INFO  22:57:45 Initializing system.schema_triggers
    INFO  22:57:45 Initializing system.schema_usertypes
    INFO  22:57:45 Initializing system.schema_functions
    INFO  22:57:45 Initializing system.schema_aggregates
    INFO  22:57:45 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  22:57:46 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  22:57:47 Populating token metadata from system tables
    INFO  22:57:47 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    WARN  22:57:47 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  22:57:47 Token metadata: 
    INFO  22:57:47 Initializing system_schema.keyspaces
    INFO  22:57:47 Initializing system_schema.tables
    INFO  22:57:47 Initializing system_schema.columns
    INFO  22:57:47 Initializing system_schema.triggers
    INFO  22:57:47 Initializing system_schema.dropped_columns
    INFO  22:57:47 Initializing system_schema.views
    INFO  22:57:47 Initializing system_schema.types
    INFO  22:57:47 Initializing system_schema.functions
    INFO  22:57:47 Initializing system_schema.aggregates
    INFO  22:57:47 Initializing system_schema.indexes
    INFO  22:57:47 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  22:57:47 Completed loading (2 ms; 2 keys) KeyCache cache
    INFO  22:57:47 Replaying /cassandra_data/commitlog/CommitLog-6-1483484140294.log, /cassandra_data/commitlog/CommitLog-6-1483484140295.log
    INFO  22:57:49 Log replay complete, 92 replayed mutations
    INFO  22:57:49 Populating token metadata from system tables
    INFO  22:57:49 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    WARN  22:57:49 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  22:57:49 Token metadata: 
    INFO  22:57:50 Cassandra version: 3.9
    INFO  22:57:50 Thrift API version: 20.1.0
    INFO  22:57:50 CQL supported versions: 3.4.2 (defa
Contributor

k8s-merge-robot commented Jan 3, 2017

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1904/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] RethinkDB should create and stop rethinkdb servers {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:524
Expected error:
    <*errors.errorString | 0xc4205ec120>: {
        s: "Failed to find \"Server ready\", last result: \"Checking for other nodes\nMy host: 10.180.0.5\nEndpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l79gx/endpoints/rethinkdb-driver\nLooking for IPs...\nparse error: Invalid numeric literal at line 1, column 5\n\"",
    }
    Failed to find "Server ready", last result: "Checking for other nodes
    My host: 10.180.0.5
    Endpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l79gx/endpoints/rethinkdb-driver
    Looking for IPs...
    parse error: Invalid numeric literal at line 1, column 5
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:502

Issues about this test specifically: #30581 #32755

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc4206b11a0>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 22:39:47.599  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-5cvfb with PID 5 (/bootstrapper.jar started by root in /)\n2017-01-03 22:39:47.612  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 22:39:47.767  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 22:39:47 GMT 2017]; root of context hierarchy\n2017-01-03 22:39:49.633  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 22:39:49.652  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 22:39:50.023  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-cnjs5/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 22:39:50.023  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 22:39:50.265  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 22:39:50.267  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 22:39:50.267  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 22:39:50.282  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.0.14]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.0.14]:5701\n2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 22:39:50.700  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.0.14]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 22:39:51.548  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.0.14]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 22:39:51.559  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.0.14]:5701 [someGroup] [3.7] [10.180.0.14]:5701 is STARTING\n2017-01-03 22:39:51.825  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.0.14]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 22:39:51.827  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.0.14]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 22:39:51.845  INFO 5 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.0.14]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-03 22:39:51.868  INFO 5 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.0.14]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.0.14]:5701 - c7b06b22-bb44-42b1-b16c-4c2609fefaa4 this\n}\n\n2017-01-03 22:39:51.931  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.0.14]:5701 [someGroup] [3.7] [10.180.0.14]:5701 is STARTED\n2017-01-03 22:39:51.935  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.321 seconds (JVM running for 6.039)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-03 22:39:47.599  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-5cvfb with PID 5 (/bootstrapper.jar started by root in /)
    2017-01-03 22:39:47.612  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default
    2017-01-03 22:39:47.767  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 22:39:47 GMT 2017]; root of context hierarchy
    2017-01-03 22:39:49.633  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
    2017-01-03 22:39:49.652  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
    2017-01-03 22:39:50.023  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed
    
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-cnjs5/endpoints/hazelcast
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]
    	at com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]
    	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]
    	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]
    
    2017-01-03 22:39:50.023  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.
    2017-01-03 22:39:50.265  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []
    2017-01-03 22:39:50.267  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.
    2017-01-03 22:39:50.267  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.
    2017-01-03 22:39:50.282  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.0.14]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
    2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.0.14]:5701
    2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
    2017-01-03 22:39:50.317  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.0.14]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1
    2017-01-03 22:39:50.700  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.0.14]:5701 [someGroup] [3.7] Backpressure is disabled
    2017-01-03 22:39:51.548  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.0.14]:5701 [someGroup] [3.7] Creating TcpIpJoiner
    2017-01-03 22:39:51.559  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.0.14]:5701 [someGroup] [3.7] [10.180.0.14]:5701 is STARTING
    2017-01-03 22:39:51.825  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.0.14]:5701 [someGroup] [3.7] Starting 2 partition threads
    2017-01-03 22:39:51.827  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.0.14]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)
    2017-01-03 22:39:51.845  INFO 5 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.0.14]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
    2017-01-03 22:39:51.868  INFO 5 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.0.14]:5701 [someGroup] [3.7] 
    
    
    Members [1] {
    	Member [10.180.0.14]:5701 - c7b06b22-bb44-42b1-b16c-4c2609fefaa4 this
    }
    
    2017-01-03 22:39:51.931  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.0.14]:5701 [someGroup] [3.7] [10.180.0.14]:5701 is STARTED
    2017-01-03 22:39:51.935  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.321 seconds (JVM running for 6.039)
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:556

Issues about this test specifically: #27850 #30672 #33271

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc420bfaed0>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.0.18\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.0.18\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.18\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.0.18\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  22:57:42 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  22:57:42 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.18; broadcast_rpc_address=10.180.0.18; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.18; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  22:57:42 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  22:57:42 Global memtable on-heap threshold is enabled at 128MB\nINFO  22:57:42 Global memtable off-heap threshold is enabled at 128MB\nINFO  22:57:42 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nINFO  22:57:42 Overriding RING_DELAY to 30000ms\nWARN  22:57:43 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  22:57:43 Hostname: cassandra-p0hg7\nINFO  22:57:43 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  22:57:43 Heap size: 512.000MiB/512.000MiB\nINFO  22:57:43 Code Cache Non-heap memory: init = 2555904(2496K) used = 4396416(4293K) committed = 4456448(4352K) max = 251658240(245760K)\nINFO  22:57:43 Metaspace Non-heap memory: init = 0(0K) used = 17257592(16853K) committed = 17694720(17280K) max = -1(-1K)\nINFO  22:57:43 Compressed Class Space Non-heap memory: init = 0(0K) used = 2070480(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  22:57:43 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)\nINFO  22:57:43 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  22:57:43 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1083392(1058K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  22:57:43 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  22:57:43 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.18, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  22:57:43 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  22:57:43 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  22:57:43 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  22:57:43 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  22:57:43 Initializing SIGAR library\nWARN  22:57:43 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  22:57:43 Directory /var/lib/cassandra/hints doesn't exist\nINFO  22:57:43 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  22:57:43 Initializing system.IndexInfo\nINFO  22:57:45 Initializing system.batches\nINFO  22:57:45 Initializing system.paxos\nINFO  22:57:45 Initializing system.local\nINFO  22:57:45 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  22:57:45 Initializing key cache with capacity of 25 MBs.\nINFO  22:57:45 Initializing row cache with capacity of 0 MBs\nINFO  22:57:45 Initializing counter cache with capacity of 12 MBs\nINFO  22:57:45 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  22:57:45 Initializing system.peers\nINFO  22:57:45 Initializing system.peer_events\nINFO  22:57:45 Initializing system.range_xfers\nINFO  22:57:45 Initializing system.compaction_history\nINFO  22:57:45 Initializing system.sstable_activity\nINFO  22:57:45 Initializing system.size_estimates\nINFO  22:57:45 Initializing system.available_ranges\nINFO  22:57:45 Initializing system.views_builds_in_progress\nINFO  22:57:45 Initializing system.built_views\nINFO  22:57:45 Initializing system.hints\nINFO  22:57:45 Initializing system.batchlog\nINFO  22:57:45 Initializing system.schema_keyspaces\nINFO  22:57:45 Initializing system.schema_columnfamilies\nINFO  22:57:45 Initializing system.schema_columns\nINFO  22:57:45 Initializing system.schema_triggers\nINFO  22:57:45 Initializing system.schema_usertypes\nINFO  22:57:45 Initializing system.schema_functions\nINFO  22:57:45 Initializing system.schema_aggregates\nINFO  22:57:45 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  22:57:46 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  22:57:47 Populating token metadata from system tables\nINFO  22:57:47 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nWARN  22:57:47 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  22:57:47 Token metadata: \nINFO  22:57:47 Initializing system_schema.keyspaces\nINFO  22:57:47 Initializing system_schema.tables\nINFO  22:57:47 Initializing system_schema.columns\nINFO  22:57:47 Initializing system_schema.triggers\nINFO  22:57:47 Initializing system_schema.dropped_columns\nINFO  22:57:47 Initializing system_schema.views\nINFO  22:57:47 Initializing system_schema.types\nINFO  22:57:47 Initializing system_schema.functions\nINFO  22:57:47 Initializing system_schema.aggregates\nINFO  22:57:47 Initializing system_schema.indexes\nINFO  22:57:47 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  22:57:47 Completed loading (2 ms; 2 keys) KeyCache cache\nINFO  22:57:47 Replaying /cassandra_data/commitlog/CommitLog-6-1483484140294.log, /cassandra_data/commitlog/CommitLog-6-1483484140295.log\nINFO  22:57:49 Log replay complete, 92 replayed mutations\nINFO  22:57:49 Populating token metadata from system tables\nINFO  22:57:49 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nWARN  22:57:49 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  22:57:49 Token metadata: \nINFO  22:57:50 Cassandra version: 3.9\nINFO  22:57:50 Thrift API version: 20.1.0\nINFO  22:57:50 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  22:57:50 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  22:57:50 Starting Messaging Service on /10.180.0.18:7000 (eth0)\nINFO  22:57:50 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nWARN  22:57:50 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  22:58:21 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\nWARN  22:58:21 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 22:58:21 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  22:58:21 No local state or state is in silent shutdown, not announcing shutdown\nINFO  22:58:21 Waiting for messaging service to quiesce\nINFO  22:58:21 MessagingService has terminated the accept() thread\nINFO  22:58:21 Paused hints dispatch\nINFO  22:58:21 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.0.18
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.0.18
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.18
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.0.18
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  22:57:42 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  22:57:42 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.18; broadcast_rpc_address=10.180.0.18; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.18; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  22:57:42 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  22:57:42 Global memtable on-heap threshold is enabled at 128MB
    INFO  22:57:42 Global memtable off-heap threshold is enabled at 128MB
    INFO  22:57:42 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    INFO  22:57:42 Overriding RING_DELAY to 30000ms
    WARN  22:57:43 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  22:57:43 Hostname: cassandra-p0hg7
    INFO  22:57:43 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  22:57:43 Heap size: 512.000MiB/512.000MiB
    INFO  22:57:43 Code Cache Non-heap memory: init = 2555904(2496K) used = 4396416(4293K) committed = 4456448(4352K) max = 251658240(245760K)
    INFO  22:57:43 Metaspace Non-heap memory: init = 0(0K) used = 17257592(16853K) committed = 17694720(17280K) max = -1(-1K)
    INFO  22:57:43 Compressed Class Space Non-heap memory: init = 0(0K) used = 2070480(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  22:57:43 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)
    INFO  22:57:43 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  22:57:43 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1083392(1058K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  22:57:43 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  22:57:43 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.18, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  22:57:43 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  22:57:43 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  22:57:43 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  22:57:43 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  22:57:43 Initializing SIGAR library
    WARN  22:57:43 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  22:57:43 Directory /var/lib/cassandra/hints doesn't exist
    INFO  22:57:43 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  22:57:43 Initializing system.IndexInfo
    INFO  22:57:45 Initializing system.batches
    INFO  22:57:45 Initializing system.paxos
    INFO  22:57:45 Initializing system.local
    INFO  22:57:45 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  22:57:45 Initializing key cache with capacity of 25 MBs.
    INFO  22:57:45 Initializing row cache with capacity of 0 MBs
    INFO  22:57:45 Initializing counter cache with capacity of 12 MBs
    INFO  22:57:45 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  22:57:45 Initializing system.peers
    INFO  22:57:45 Initializing system.peer_events
    INFO  22:57:45 Initializing system.range_xfers
    INFO  22:57:45 Initializing system.compaction_history
    INFO  22:57:45 Initializing system.sstable_activity
    INFO  22:57:45 Initializing system.size_estimates
    INFO  22:57:45 Initializing system.available_ranges
    INFO  22:57:45 Initializing system.views_builds_in_progress
    INFO  22:57:45 Initializing system.built_views
    INFO  22:57:45 Initializing system.hints
    INFO  22:57:45 Initializing system.batchlog
    INFO  22:57:45 Initializing system.schema_keyspaces
    INFO  22:57:45 Initializing system.schema_columnfamilies
    INFO  22:57:45 Initializing system.schema_columns
    INFO  22:57:45 Initializing system.schema_triggers
    INFO  22:57:45 Initializing system.schema_usertypes
    INFO  22:57:45 Initializing system.schema_functions
    INFO  22:57:45 Initializing system.schema_aggregates
    INFO  22:57:45 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  22:57:46 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  22:57:47 Populating token metadata from system tables
    INFO  22:57:47 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    WARN  22:57:47 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  22:57:47 Token metadata: 
    INFO  22:57:47 Initializing system_schema.keyspaces
    INFO  22:57:47 Initializing system_schema.tables
    INFO  22:57:47 Initializing system_schema.columns
    INFO  22:57:47 Initializing system_schema.triggers
    INFO  22:57:47 Initializing system_schema.dropped_columns
    INFO  22:57:47 Initializing system_schema.views
    INFO  22:57:47 Initializing system_schema.types
    INFO  22:57:47 Initializing system_schema.functions
    INFO  22:57:47 Initializing system_schema.aggregates
    INFO  22:57:47 Initializing system_schema.indexes
    INFO  22:57:47 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  22:57:47 Completed loading (2 ms; 2 keys) KeyCache cache
    INFO  22:57:47 Replaying /cassandra_data/commitlog/CommitLog-6-1483484140294.log, /cassandra_data/commitlog/CommitLog-6-1483484140295.log
    INFO  22:57:49 Log replay complete, 92 replayed mutations
    INFO  22:57:49 Populating token metadata from system tables
    INFO  22:57:49 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    WARN  22:57:49 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-k3s0s/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  22:57:49 Token metadata: 
    INFO  22:57:50 Cassandra version: 3.9
    INFO  22:57:50 Thrift API version: 20.1.0
    INFO  22:57:50 CQL supported versions: 3.4.2 (defa
@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Jan 4, 2017

Contributor

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1905/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc4206b2440>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.2.8\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.2.8\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.2.8\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.2.8\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  23:25:16 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  23:25:17 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.2.8; broadcast_rpc_address=10.180.2.8; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.2.8; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  23:25:17 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  23:25:17 Global memtable on-heap threshold is enabled at 128MB\nINFO  23:25:17 Global memtable off-heap threshold is enabled at 128MB\nINFO  23:25:17 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nINFO  23:25:17 Overriding RING_DELAY to 30000ms\nWARN  23:25:17 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  23:25:17 Hostname: cassandra-967wt\nINFO  23:25:17 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  23:25:17 Heap size: 512.000MiB/512.000MiB\nINFO  23:25:17 Code Cache Non-heap memory: init = 2555904(2496K) used = 4343680(4241K) committed = 4390912(4288K) max = 251658240(245760K)\nINFO  23:25:17 Metaspace Non-heap memory: init = 0(0K) used = 17260264(16855K) committed = 17694720(17280K) max = -1(-1K)\nINFO  23:25:17 Compressed Class Space Non-heap memory: init = 0(0K) used = 2070480(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  23:25:17 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)\nINFO  23:25:17 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  23:25:17 G1 Old Gen Heap memory: init = 508559360(496640K) used = 984064(961K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  23:25:17 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  23:25:17 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.2.8, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  23:25:17 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  23:25:17 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  23:25:17 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  23:25:17 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  23:25:17 Initializing SIGAR library\nWARN  23:25:17 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  23:25:17 Directory /var/lib/cassandra/hints doesn't exist\nINFO  23:25:17 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  23:25:18 Initializing system.IndexInfo\nINFO  23:25:19 Initializing system.batches\nINFO  23:25:19 Initializing system.paxos\nINFO  23:25:19 Initializing system.local\nINFO  23:25:19 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  23:25:20 Initializing key cache with capacity of 25 MBs.\nINFO  23:25:20 Initializing row cache with capacity of 0 MBs\nINFO  23:25:20 Initializing counter cache with capacity of 12 MBs\nINFO  23:25:20 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  23:25:20 Initializing system.peers\nINFO  23:25:20 Initializing system.peer_events\nINFO  23:25:20 Initializing system.range_xfers\nINFO  23:25:20 Initializing system.compaction_history\nINFO  23:25:20 Initializing system.sstable_activity\nINFO  23:25:20 Initializing system.size_estimates\nINFO  23:25:20 Initializing system.available_ranges\nINFO  23:25:20 Initializing system.views_builds_in_progress\nINFO  23:25:20 Initializing system.built_views\nINFO  23:25:20 Initializing system.hints\nINFO  23:25:20 Initializing system.batchlog\nINFO  23:25:20 Initializing system.schema_keyspaces\nINFO  23:25:20 Initializing system.schema_columnfamilies\nINFO  23:25:20 Initializing system.schema_columns\nINFO  23:25:20 Initializing system.schema_triggers\nINFO  23:25:20 Initializing system.schema_usertypes\nINFO  23:25:20 Initializing system.schema_functions\nINFO  23:25:20 Initializing system.schema_aggregates\nINFO  23:25:20 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  23:25:20 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  23:25:22 Populating token metadata from system tables\nINFO  23:25:22 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nWARN  23:25:22 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  23:25:22 Token metadata: \nINFO  23:25:22 Initializing system_schema.keyspaces\nINFO  23:25:22 Initializing system_schema.tables\nINFO  23:25:22 Initializing system_schema.columns\nINFO  23:25:22 Initializing system_schema.triggers\nINFO  23:25:22 Initializing system_schema.dropped_columns\nINFO  23:25:22 Initializing system_schema.views\nINFO  23:25:22 Initializing system_schema.types\nINFO  23:25:22 Initializing system_schema.functions\nINFO  23:25:22 Initializing system_schema.aggregates\nINFO  23:25:22 Initializing system_schema.indexes\nINFO  23:25:22 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  23:25:22 Completed loading (2 ms; 2 keys) KeyCache cache\nINFO  23:25:22 Replaying /cassandra_data/commitlog/CommitLog-6-1483485794838.log, /cassandra_data/commitlog/CommitLog-6-1483485794839.log\nINFO  23:25:24 Log replay complete, 92 replayed mutations\nINFO  23:25:24 Populating token metadata from system tables\nINFO  23:25:24 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nWARN  23:25:24 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  23:25:24 Token metadata: \nINFO  23:25:24 Cassandra version: 3.9\nINFO  23:25:24 Thrift API version: 20.1.0\nINFO  23:25:24 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  23:25:24 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  23:25:24 Starting Messaging Service on /10.180.2.8:7000 (eth0)\nINFO  23:25:24 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nWARN  23:25:24 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  23:25:55 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nWARN  23:25:55 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 23:25:55 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  23:25:55 No local state or state is in silent shutdown, not announcing shutdown\nINFO  23:25:55 Waiting for messaging service to quiesce\nINFO  23:25:55 MessagingService has terminated the accept() thread\nINFO  23:25:55 Paused hints dispatch\nINFO  23:25:55 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.2.8
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.2.8
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.2.8
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.2.8
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  23:25:16 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  23:25:17 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.2.8; broadcast_rpc_address=10.180.2.8; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.2.8; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  23:25:17 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  23:25:17 Global memtable on-heap threshold is enabled at 128MB
    INFO  23:25:17 Global memtable off-heap threshold is enabled at 128MB
    INFO  23:25:17 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    INFO  23:25:17 Overriding RING_DELAY to 30000ms
    WARN  23:25:17 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  23:25:17 Hostname: cassandra-967wt
    INFO  23:25:17 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  23:25:17 Heap size: 512.000MiB/512.000MiB
    INFO  23:25:17 Code Cache Non-heap memory: init = 2555904(2496K) used = 4343680(4241K) committed = 4390912(4288K) max = 251658240(245760K)
    INFO  23:25:17 Metaspace Non-heap memory: init = 0(0K) used = 17260264(16855K) committed = 17694720(17280K) max = -1(-1K)
    INFO  23:25:17 Compressed Class Space Non-heap memory: init = 0(0K) used = 2070480(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  23:25:17 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)
    INFO  23:25:17 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  23:25:17 G1 Old Gen Heap memory: init = 508559360(496640K) used = 984064(961K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  23:25:17 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  23:25:17 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.2.8, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  23:25:17 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  23:25:17 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  23:25:17 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  23:25:17 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  23:25:17 Initializing SIGAR library
    WARN  23:25:17 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  23:25:17 Directory /var/lib/cassandra/hints doesn't exist
    INFO  23:25:17 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  23:25:18 Initializing system.IndexInfo
    INFO  23:25:19 Initializing system.batches
    INFO  23:25:19 Initializing system.paxos
    INFO  23:25:19 Initializing system.local
    INFO  23:25:19 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  23:25:20 Initializing key cache with capacity of 25 MBs.
    INFO  23:25:20 Initializing row cache with capacity of 0 MBs
    INFO  23:25:20 Initializing counter cache with capacity of 12 MBs
    INFO  23:25:20 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  23:25:20 Initializing system.peers
    INFO  23:25:20 Initializing system.peer_events
    INFO  23:25:20 Initializing system.range_xfers
    INFO  23:25:20 Initializing system.compaction_history
    INFO  23:25:20 Initializing system.sstable_activity
    INFO  23:25:20 Initializing system.size_estimates
    INFO  23:25:20 Initializing system.available_ranges
    INFO  23:25:20 Initializing system.views_builds_in_progress
    INFO  23:25:20 Initializing system.built_views
    INFO  23:25:20 Initializing system.hints
    INFO  23:25:20 Initializing system.batchlog
    INFO  23:25:20 Initializing system.schema_keyspaces
    INFO  23:25:20 Initializing system.schema_columnfamilies
    INFO  23:25:20 Initializing system.schema_columns
    INFO  23:25:20 Initializing system.schema_triggers
    INFO  23:25:20 Initializing system.schema_usertypes
    INFO  23:25:20 Initializing system.schema_functions
    INFO  23:25:20 Initializing system.schema_aggregates
    INFO  23:25:20 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  23:25:20 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  23:25:22 Populating token metadata from system tables
    INFO  23:25:22 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    WARN  23:25:22 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  23:25:22 Token metadata: 
    INFO  23:25:22 Initializing system_schema.keyspaces
    INFO  23:25:22 Initializing system_schema.tables
    INFO  23:25:22 Initializing system_schema.columns
    INFO  23:25:22 Initializing system_schema.triggers
    INFO  23:25:22 Initializing system_schema.dropped_columns
    INFO  23:25:22 Initializing system_schema.views
    INFO  23:25:22 Initializing system_schema.types
    INFO  23:25:22 Initializing system_schema.functions
    INFO  23:25:22 Initializing system_schema.aggregates
    INFO  23:25:22 Initializing system_schema.indexes
    INFO  23:25:22 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  23:25:22 Completed loading (2 ms; 2 keys) KeyCache cache
    INFO  23:25:22 Replaying /cassandra_data/commitlog/CommitLog-6-1483485794838.log, /cassandra_data/commitlog/CommitLog-6-1483485794839.log
    INFO  23:25:24 Log replay complete, 92 replayed mutations
    INFO  23:25:24 Populating token metadata from system tables
    INFO  23:25:24 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    WARN  23:25:24 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  23:25:24 Token metadata: 
    INFO  23:25:24 Cassandra version: 3.9
    INFO  23:25:24 Thrift API version: 20.1.0
    INFO  23:25:24 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  23:25:24 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  23:25:24 Starting Messaging Service on /10.180.2.8:7000 (eth0)
    INFO  23:25:24 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    WARN  23:25:24 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  23:25:55 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    WARN  23:25:55 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    Exception (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
    ERROR 23:25:55 Exception encountered during startup
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    WARN  23:25:55 No local state or state is in silent shutdown, not announcing shutdown
    INFO  23:25:55 Waiting for messaging service to quiesce
    INFO  23:25:55 MessagingService has terminated the accept() thread
    INFO  23:25:55 Paused hints dispatch
    INFO  23:25:55 Paused hints dispatch
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:236

Issues about this test specifically: #27978 #28817

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc421756a00>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 23:27:52.080  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-jbxw0 with PID 6 (/bootstrapper.jar started by root in /)\n2017-01-03 23:27:52.092  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 23:27:52.266  INFO 6 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 23:27:52 GMT 2017]; root of context hierarchy\n2017-01-03 23:27:54.255  INFO 6 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 23:27:54.275  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 23:27:54.698  WARN 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-hvxb1/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 23:27:54.706  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 23:27:55.000  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 23:27:55.002  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 23:27:55.003  WARN 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 23:27:55.022  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.9]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 23:27:55.059  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.9]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.9]:5701\n2017-01-03 23:27:55.059  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.9]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 23:27:55.059  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.9]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 23:27:55.497  INFO 6 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.9]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 23:27:56.449  INFO 6 --- [           main] com.hazelcast.instance.Node              : [10.180.2.9]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 23:27:56.455  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.9]:5701 [someGroup] [3.7] [10.180.2.9]:5701 is STARTING\n2017-01-03 23:27:56.809  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.9]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 23:27:56.813  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.9]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 23:27:56.826  INFO 6 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.9]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-03 23:27:56.850  INFO 6 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.9]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.2.9]:5701 - f3884a5e-4142-4da3-8a24-3eb202fa5480 this\n}\n\n2017-01-03 23:27:56.934  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.9]:5701 [someGroup] [3.7] [10.180.2.9]:5701 is STARTED\n2017-01-03 23:27:56.944  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.805 seconds (JVM running for 6.559)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-03 23:27:52.080  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-jbxw0 with PID 6 (/bootstrapper.jar started by root in /)
    2017-01-03 23:27:52.092  INFO 6 --- [           main] com.git
Contributor

k8s-merge-robot commented Jan 4, 2017

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1905/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc4206b2440>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.2.8\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.2.8\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.2.8\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.2.8\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  23:25:16 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  23:25:17 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.2.8; broadcast_rpc_address=10.180.2.8; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.2.8; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  23:25:17 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  23:25:17 Global memtable on-heap threshold is enabled at 128MB\nINFO  23:25:17 Global memtable off-heap threshold is enabled at 128MB\nINFO  23:25:17 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nINFO  23:25:17 Overriding RING_DELAY to 30000ms\nWARN  23:25:17 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  23:25:17 Hostname: cassandra-967wt\nINFO  23:25:17 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  23:25:17 Heap size: 512.000MiB/512.000MiB\nINFO  23:25:17 Code Cache Non-heap memory: init = 2555904(2496K) used = 4343680(4241K) committed = 4390912(4288K) max = 251658240(245760K)\nINFO  23:25:17 Metaspace Non-heap memory: init = 0(0K) used = 17260264(16855K) committed = 17694720(17280K) max = -1(-1K)\nINFO  23:25:17 Compressed Class Space Non-heap memory: init = 0(0K) used = 2070480(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  23:25:17 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)\nINFO  23:25:17 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  23:25:17 G1 Old Gen Heap memory: init = 508559360(496640K) used = 984064(961K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  23:25:17 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  23:25:17 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.2.8, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  23:25:17 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  23:25:17 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  23:25:17 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  23:25:17 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  23:25:17 Initializing SIGAR library\nWARN  23:25:17 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  23:25:17 Directory /var/lib/cassandra/hints doesn't exist\nINFO  23:25:17 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  23:25:18 Initializing system.IndexInfo\nINFO  23:25:19 Initializing system.batches\nINFO  23:25:19 Initializing system.paxos\nINFO  23:25:19 Initializing system.local\nINFO  23:25:19 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  23:25:20 Initializing key cache with capacity of 25 MBs.\nINFO  23:25:20 Initializing row cache with capacity of 0 MBs\nINFO  23:25:20 Initializing counter cache with capacity of 12 MBs\nINFO  23:25:20 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  23:25:20 Initializing system.peers\nINFO  23:25:20 Initializing system.peer_events\nINFO  23:25:20 Initializing system.range_xfers\nINFO  23:25:20 Initializing system.compaction_history\nINFO  23:25:20 Initializing system.sstable_activity\nINFO  23:25:20 Initializing system.size_estimates\nINFO  23:25:20 Initializing system.available_ranges\nINFO  23:25:20 Initializing system.views_builds_in_progress\nINFO  23:25:20 Initializing system.built_views\nINFO  23:25:20 Initializing system.hints\nINFO  23:25:20 Initializing system.batchlog\nINFO  23:25:20 Initializing system.schema_keyspaces\nINFO  23:25:20 Initializing system.schema_columnfamilies\nINFO  23:25:20 Initializing system.schema_columns\nINFO  23:25:20 Initializing system.schema_triggers\nINFO  23:25:20 Initializing system.schema_usertypes\nINFO  23:25:20 Initializing system.schema_functions\nINFO  23:25:20 Initializing system.schema_aggregates\nINFO  23:25:20 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  23:25:20 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  23:25:22 Populating token metadata from system tables\nINFO  23:25:22 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nWARN  23:25:22 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  23:25:22 Token metadata: \nINFO  23:25:22 Initializing system_schema.keyspaces\nINFO  23:25:22 Initializing system_schema.tables\nINFO  23:25:22 Initializing system_schema.columns\nINFO  23:25:22 Initializing system_schema.triggers\nINFO  23:25:22 Initializing system_schema.dropped_columns\nINFO  23:25:22 Initializing system_schema.views\nINFO  23:25:22 Initializing system_schema.types\nINFO  23:25:22 Initializing system_schema.functions\nINFO  23:25:22 Initializing system_schema.aggregates\nINFO  23:25:22 Initializing system_schema.indexes\nINFO  23:25:22 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  23:25:22 Completed loading (2 ms; 2 keys) KeyCache cache\nINFO  23:25:22 Replaying /cassandra_data/commitlog/CommitLog-6-1483485794838.log, /cassandra_data/commitlog/CommitLog-6-1483485794839.log\nINFO  23:25:24 Log replay complete, 92 replayed mutations\nINFO  23:25:24 Populating token metadata from system tables\nINFO  23:25:24 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nWARN  23:25:24 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  23:25:24 Token metadata: \nINFO  23:25:24 Cassandra version: 3.9\nINFO  23:25:24 Thrift API version: 20.1.0\nINFO  23:25:24 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  23:25:24 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  23:25:24 Starting Messaging Service on /10.180.2.8:7000 (eth0)\nINFO  23:25:24 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nWARN  23:25:24 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  23:25:55 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\nWARN  23:25:55 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 23:25:55 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  23:25:55 No local state or state is in silent shutdown, not announcing shutdown\nINFO  23:25:55 Waiting for messaging service to quiesce\nINFO  23:25:55 MessagingService has terminated the accept() thread\nINFO  23:25:55 Paused hints dispatch\nINFO  23:25:55 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.2.8
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.2.8
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.2.8
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.2.8
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  23:25:16 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  23:25:17 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.2.8; broadcast_rpc_address=10.180.2.8; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.2.8; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  23:25:17 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  23:25:17 Global memtable on-heap threshold is enabled at 128MB
    INFO  23:25:17 Global memtable off-heap threshold is enabled at 128MB
    INFO  23:25:17 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    INFO  23:25:17 Overriding RING_DELAY to 30000ms
    WARN  23:25:17 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  23:25:17 Hostname: cassandra-967wt
    INFO  23:25:17 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  23:25:17 Heap size: 512.000MiB/512.000MiB
    INFO  23:25:17 Code Cache Non-heap memory: init = 2555904(2496K) used = 4343680(4241K) committed = 4390912(4288K) max = 251658240(245760K)
    INFO  23:25:17 Metaspace Non-heap memory: init = 0(0K) used = 17260264(16855K) committed = 17694720(17280K) max = -1(-1K)
    INFO  23:25:17 Compressed Class Space Non-heap memory: init = 0(0K) used = 2070480(2021K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  23:25:17 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)
    INFO  23:25:17 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  23:25:17 G1 Old Gen Heap memory: init = 508559360(496640K) used = 984064(961K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  23:25:17 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  23:25:17 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.2.8, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  23:25:17 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  23:25:17 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  23:25:17 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  23:25:17 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  23:25:17 Initializing SIGAR library
    WARN  23:25:17 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  23:25:17 Directory /var/lib/cassandra/hints doesn't exist
    INFO  23:25:17 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  23:25:18 Initializing system.IndexInfo
    INFO  23:25:19 Initializing system.batches
    INFO  23:25:19 Initializing system.paxos
    INFO  23:25:19 Initializing system.local
    INFO  23:25:19 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  23:25:20 Initializing key cache with capacity of 25 MBs.
    INFO  23:25:20 Initializing row cache with capacity of 0 MBs
    INFO  23:25:20 Initializing counter cache with capacity of 12 MBs
    INFO  23:25:20 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  23:25:20 Initializing system.peers
    INFO  23:25:20 Initializing system.peer_events
    INFO  23:25:20 Initializing system.range_xfers
    INFO  23:25:20 Initializing system.compaction_history
    INFO  23:25:20 Initializing system.sstable_activity
    INFO  23:25:20 Initializing system.size_estimates
    INFO  23:25:20 Initializing system.available_ranges
    INFO  23:25:20 Initializing system.views_builds_in_progress
    INFO  23:25:20 Initializing system.built_views
    INFO  23:25:20 Initializing system.hints
    INFO  23:25:20 Initializing system.batchlog
    INFO  23:25:20 Initializing system.schema_keyspaces
    INFO  23:25:20 Initializing system.schema_columnfamilies
    INFO  23:25:20 Initializing system.schema_columns
    INFO  23:25:20 Initializing system.schema_triggers
    INFO  23:25:20 Initializing system.schema_usertypes
    INFO  23:25:20 Initializing system.schema_functions
    INFO  23:25:20 Initializing system.schema_aggregates
    INFO  23:25:20 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  23:25:20 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  23:25:22 Populating token metadata from system tables
    INFO  23:25:22 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    WARN  23:25:22 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  23:25:22 Token metadata: 
    INFO  23:25:22 Initializing system_schema.keyspaces
    INFO  23:25:22 Initializing system_schema.tables
    INFO  23:25:22 Initializing system_schema.columns
    INFO  23:25:22 Initializing system_schema.triggers
    INFO  23:25:22 Initializing system_schema.dropped_columns
    INFO  23:25:22 Initializing system_schema.views
    INFO  23:25:22 Initializing system_schema.types
    INFO  23:25:22 Initializing system_schema.functions
    INFO  23:25:22 Initializing system_schema.aggregates
    INFO  23:25:22 Initializing system_schema.indexes
    INFO  23:25:22 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  23:25:22 Completed loading (2 ms; 2 keys) KeyCache cache
    INFO  23:25:22 Replaying /cassandra_data/commitlog/CommitLog-6-1483485794838.log, /cassandra_data/commitlog/CommitLog-6-1483485794839.log
    INFO  23:25:24 Log replay complete, 92 replayed mutations
    INFO  23:25:24 Populating token metadata from system tables
    INFO  23:25:24 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    WARN  23:25:24 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  23:25:24 Token metadata: 
    INFO  23:25:24 Cassandra version: 3.9
    INFO  23:25:24 Thrift API version: 20.1.0
    INFO  23:25:24 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  23:25:24 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  23:25:24 Starting Messaging Service on /10.180.2.8:7000 (eth0)
    INFO  23:25:24 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    WARN  23:25:24 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  23:25:55 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    WARN  23:25:55 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-gg647/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    Exception (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
    ERROR 23:25:55 Exception encountered during startup
    java.lang.RuntimeException: Unable to gossip with any seeds
    	at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    WARN  23:25:55 No local state or state is in silent shutdown, not announcing shutdown
    INFO  23:25:55 Waiting for messaging service to quiesce
    INFO  23:25:55 MessagingService has terminated the accept() thread
    INFO  23:25:55 Paused hints dispatch
    INFO  23:25:55 Paused hints dispatch
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:236

Issues about this test specifically: #27978 #28817

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc421756a00>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-03 23:27:52.080  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-jbxw0 with PID 6 (/bootstrapper.jar started by root in /)\n2017-01-03 23:27:52.092  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-03 23:27:52.266  INFO 6 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Tue Jan 03 23:27:52 GMT 2017]; root of context hierarchy\n2017-01-03 23:27:54.255  INFO 6 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-03 23:27:54.275  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-03 23:27:54.698  WARN 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-hvxb1/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-03 23:27:54.706  INFO 6 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-03 23:27:55.000  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-03 23:27:55.002  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-03 23:27:55.003  WARN 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-03 23:27:55.022  INFO 6 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.9]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-03 23:27:55.059  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.9]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.9]:5701\n2017-01-03 23:27:55.059  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.9]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-03 23:27:55.059  INFO 6 --- [           main] com.hazelcast.system                     : [10.180.2.9]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-03 23:27:55.497  INFO 6 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.9]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-03 23:27:56.449  INFO 6 --- [           main] com.hazelcast.instance.Node              : [10.180.2.9]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-03 23:27:56.455  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.9]:5701 [someGroup] [3.7] [10.180.2.9]:5701 is STARTING\n2017-01-03 23:27:56.809  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.9]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-03 23:27:56.813  INFO 6 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.9]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-03 23:27:56.826  INFO 6 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.9]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-03 23:27:56.850  INFO 6 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.9]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.2.9]:5701 - f3884a5e-4142-4da3-8a24-3eb202fa5480 this\n}\n\n2017-01-03 23:27:56.934  INFO 6 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.9]:5701 [someGroup] [3.7] [10.180.2.9]:5701 is STARTED\n2017-01-03 23:27:56.944  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.805 seconds (JVM running for 6.559)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-03 23:27:52.080  INFO 6 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-jbxw0 with PID 6 (/bootstrapper.jar started by root in /)
    2017-01-03 23:27:52.092  INFO 6 --- [           main] com.git
@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Jan 4, 2017

Contributor

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1906/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc420d19320>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-04 00:46:50.088  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-08bsz with PID 5 (/bootstrapper.jar started by root in /)\n2017-01-04 00:46:50.112  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-04 00:46:50.230  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Wed Jan 04 00:46:50 GMT 2017]; root of context hierarchy\n2017-01-04 00:46:51.859  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-04 00:46:51.879  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-04 00:46:52.322  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-kkt6g/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-04 00:46:52.323  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-04 00:46:52.557  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-04 00:46:52.558  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-04 00:46:52.559  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-04 00:46:52.570  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.13]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.13]:5701\n2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-04 00:46:52.948  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.13]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-04 00:46:53.998  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.2.13]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-04 00:46:54.000  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.13]:5701 [someGroup] [3.7] [10.180.2.13]:5701 is STARTING\n2017-01-04 00:46:54.289  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.13]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-04 00:46:54.292  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.13]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-04 00:46:54.334  INFO 5 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.13]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-04 00:46:54.393  INFO 5 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.13]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.2.13]:5701 - 64015084-a86f-446a-b8ba-1cea6544b787 this\n}\n\n2017-01-04 00:46:54.502  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.13]:5701 [someGroup] [3.7] [10.180.2.13]:5701 is STARTED\n2017-01-04 00:46:54.504  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.465 seconds (JVM running for 6.281)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-04 00:46:50.088  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-08bsz with PID 5 (/bootstrapper.jar started by root in /)
    2017-01-04 00:46:50.112  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default
    2017-01-04 00:46:50.230  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Wed Jan 04 00:46:50 GMT 2017]; root of context hierarchy
    2017-01-04 00:46:51.859  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
    2017-01-04 00:46:51.879  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
    2017-01-04 00:46:52.322  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed
    
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-kkt6g/endpoints/hazelcast
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]
    	at com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]
    	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]
    	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]
    
    2017-01-04 00:46:52.323  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.
    2017-01-04 00:46:52.557  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []
    2017-01-04 00:46:52.558  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.
    2017-01-04 00:46:52.559  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.
    2017-01-04 00:46:52.570  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.13]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
    2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.13]:5701
    2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
    2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1
    2017-01-04 00:46:52.948  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.13]:5701 [someGroup] [3.7] Backpressure is disabled
    2017-01-04 00:46:53.998  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.2.13]:5701 [someGroup] [3.7] Creating TcpIpJoiner
    2017-01-04 00:46:54.000  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.13]:5701 [someGroup] [3.7] [10.180.2.13]:5701 is STARTING
    2017-01-04 00:46:54.289  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.13]:5701 [someGroup] [3.7] Starting 2 partition threads
    2017-01-04 00:46:54.292  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.13]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)
    2017-01-04 00:46:54.334  INFO 5 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.13]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
    2017-01-04 00:46:54.393  INFO 5 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.13]:5701 [someGroup] [3.7] 
    
    
    Members [1] {
    	Member [10.180.2.13]:5701 - 64015084-a86f-446a-b8ba-1cea6544b787 this
    }
    
    2017-01-04 00:46:54.502  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.13]:5701 [someGroup] [3.7] [10.180.2.13]:5701 is STARTED
    2017-01-04 00:46:54.504  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.465 seconds (JVM running for 6.281)
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:556

Issues about this test specifically: #27850 #30672 #33271

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc420920140>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.0.21\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.0.21\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.21\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.0.21\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  01:02:00 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  01:02:01 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.21; broadcast_rpc_address=10.180.0.21; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.21; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  01:02:01 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  01:02:01 Global memtable on-heap threshold is enabled at 128MB\nINFO  01:02:01 Global memtable off-heap threshold is enabled at 128MB\nINFO  01:02:01 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nINFO  01:02:01 Overriding RING_DELAY to 30000ms\nWARN  01:02:01 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:02:01 Hostname: cassandra-8v0mt\nINFO  01:02:01 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  01:02:01 Heap size: 512.000MiB/512.000MiB\nINFO  01:02:01 Code Cache Non-heap memory: init = 2555904(2496K) used = 4263232(4163K) committed = 4325376(4224K) max = 251658240(245760K)\nINFO  01:02:01 Metaspace Non-heap memory: init = 0(0K) used = 17426976(17018K) committed = 17956864(17536K) max = -1(-1K)\nINFO  01:02:01 Compressed Class Space Non-heap memory: init = 0(0K) used = 2080368(2031K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  01:02:01 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)\nINFO  01:02:01 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  01:02:01 G1 Old Gen Heap memory: init = 508559360(496640K) used = 964608(942K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  01:02:01 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  01:02:01 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.21, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  01:02:01 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  01:02:01 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  01:02:01 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  01:02:01 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  01:02:01 Initializing SIGAR library\nWARN  01:02:01 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  01:02:01 Directory /var/lib/cassandra/hints doesn't exist\nINFO  01:02:01 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  01:02:02 Initializing system.IndexInfo\nINFO  01:02:04 Initializing system.batches\nINFO  01:02:04 Initializing system.paxos\nINFO  01:02:04 Initializing system.local\nINFO  01:02:04 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  01:02:04 Initializing key cache with capacity of 25 MBs.\nINFO  01:02:04 Initializing row cache with capacity of 0 MBs\nINFO  01:02:04 Initializing counter cache with capacity of 12 MBs\nINFO  01:02:04 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  01:02:04 Initializing system.peers\nINFO  01:02:04 Initializing system.peer_events\nINFO  01:02:04 Initializing system.range_xfers\nINFO  01:02:04 Initializing system.compaction_history\nINFO  01:02:04 Initializing system.sstable_activity\nINFO  01:02:04 Initializing system.size_estimates\nINFO  01:02:04 Initializing system.available_ranges\nINFO  01:02:04 Initializing system.views_builds_in_progress\nINFO  01:02:04 Initializing system.built_views\nINFO  01:02:04 Initializing system.hints\nINFO  01:02:04 Initializing system.batchlog\nINFO  01:02:04 Initializing system.schema_keyspaces\nINFO  01:02:04 Initializing system.schema_columnfamilies\nINFO  01:02:04 Initializing system.schema_columns\nINFO  01:02:04 Initializing system.schema_triggers\nINFO  01:02:04 Initializing system.schema_usertypes\nINFO  01:02:04 Initializing system.schema_functions\nINFO  01:02:04 Initializing system.schema_aggregates\nINFO  01:02:04 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  01:02:05 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  01:02:06 Populating token metadata from system tables\nINFO  01:02:06 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nWARN  01:02:06 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:02:06 Token metadata: \nINFO  01:02:06 Initializing system_schema.keyspaces\nINFO  01:02:06 Initializing system_schema.tables\nINFO  01:02:06 Initializing system_schema.columns\nINFO  01:02:06 Initializing system_schema.triggers\nINFO  01:02:06 Initializing system_schema.dropped_columns\nINFO  01:02:06 Initializing system_schema.views\nINFO  01:02:06 Initializing system_schema.types\nINFO  01:02:06 Initializing system_schema.functions\nINFO  01:02:06 Initializing system_schema.aggregates\nINFO  01:02:06 Initializing system_schema.indexes\nINFO  01:02:06 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  01:02:06 Completed loading (2 ms; 2 keys) KeyCache cache\nINFO  01:02:07 Replaying /cassandra_data/commitlog/CommitLog-6-1483491598766.log, /cassandra_data/commitlog/CommitLog-6-1483491598767.log\nINFO  01:02:08 Log replay complete, 92 replayed mutations\nINFO  01:02:08 Populating token metadata from system tables\nINFO  01:02:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nWARN  01:02:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:02:08 Token metadata: \nINFO  01:02:09 Cassandra version: 3.9\nINFO  01:02:09 Thrift API version: 20.1.0\nINFO  01:02:09 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  01:02:09 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  01:02:09 Starting Messaging Service on /10.180.0.21:7000 (eth0)\nINFO  01:02:09 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nWARN  01:02:09 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:02:40 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nWARN  01:02:40 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 01:02:40 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  01:02:40 No local state or state is in silent shutdown, not announcing shutdown\nINFO  01:02:40 Waiting for messaging service to quiesce\nINFO  01:02:40 MessagingService has terminated the accept() thread\nINFO  01:02:40 Paused hints dispatch\nINFO  01:02:40 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.0.21
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.0.21
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.21
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.0.21
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  01:02:00 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  01:02:01 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.21; broadcast_rpc_address=10.180.0.21; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.21; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  01:02:01 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  01:02:01 Global memtable on-heap threshold is enabled at 128MB
    INFO  01:02:01 Global memtable off-heap threshold is enabled at 128MB
    INFO  01:02:01 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    INFO  01:02:01 Overriding RING_DELAY to 30000ms
    WARN  01:02:01 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:02:01 Hostname: cassandra-8v0mt
    INFO  01:02:01 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  01:02:01 Heap size: 512.000MiB/512.000MiB
    INFO  01:02:01 Code Cache Non-heap memory: init = 2555904(2496K) used = 4263232(4163K) committed = 4325376(4224K) max = 251658240(245760K)
    INFO  01:02:01 Metaspace Non-heap memory: init = 0(0K) used = 17426976(17018K) committed = 17956864(17536K) max = -1(-1K)
    INFO  01:02:01 Compressed Class Space Non-heap memory: init = 0(0K) used = 2080368(2031K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  01:02:01 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)
    INFO  01:02:01 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  01:02:01 G1 Old Gen Heap memory: init = 508559360(496640K) used = 964608(942K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  01:02:01 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  01:02:01 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.21, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  01:02:01 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  01:02:01 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  01:02:01 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  01:02:01 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  01:02:01 Initializing SIGAR library
    WARN  01:02:01 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  01:02:01 Directory /var/lib/cassandra/hints doesn't exist
    INFO  01:02:01 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  01:02:02 Initializing system.IndexInfo
    INFO  01:02:04 Initializing system.batches
    INFO  01:02:04 Initializing system.paxos
    INFO  01:02:04 Initializing system.local
    INFO  01:02:04 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  01:02:04 Initializing key cache with capacity of 25 MBs.
    INFO  01:02:04 Initializing row cache with capacity of 0 MBs
    INFO  01:02:04 Initializing counter cache with capacity of 12 MBs
    INFO  01:02:04 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  01:02:04 Initializing system.peers
    INFO  01:02:04 Initializing system.peer_events
    INFO  01:02:04 Initializing system.range_xfers
    INFO  01:02:04 Initializing system.compaction_history
    INFO  01:02:04 Initializing system.sstable_activity
    INFO  01:02:04 Initializing system.size_estimates
    INFO  01:02:04 Initializing system.available_ranges
    INFO  01:02:04 Initializing system.views_builds_in_progress
    INFO  01:02:04 Initializing system.built_views
    INFO  01:02:04 Initializing system.hints
    INFO  01:02:04 Initializing system.batchlog
    INFO  01:02:04 Initializing system.schema_keyspaces
    INFO  01:02:04 Initializing system.schema_columnfamilies
    INFO  01:02:04 Initializing system.schema_columns
    INFO  01:02:04 Initializing system.schema_triggers
    INFO  01:02:04 Initializing system.schema_usertypes
    INFO  01:02:04 Initializing system.schema_functions
    INFO  01:02:04 Initializing system.schema_aggregates
    INFO  01:02:04 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  01:02:05 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  01:02:06 Populating token metadata from system tables
    INFO  01:02:06 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    WARN  01:02:06 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:02:06 Token metadata: 
    INFO  01:02:06 Initializing system_schema.keyspaces
    INFO  01:02:06 Initializing system_schema.tables
    INFO  01:02:06 Initializing system_schema.columns
    INFO  01:02:06 Initializing system_schema.triggers
    INFO  01:02:06 Initializing system_schema.dropped_columns
    INFO  01:02:06 Initializing system_schema.views
    INFO  01:02:06 Initializing system_schema.types
    INFO  01:02:06 Initializing system_schema.functions
    INFO  01:02:06 Initializing system_schema.aggregates
    INFO  01:02:06 Initializing system_schema.indexes
    INFO  01:02:06 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  01:02:06 Completed loading (2 ms; 2 keys) KeyCache cache
    INFO  01:02:07 Replaying /cassandra_data/commitlog/CommitLog-6-1483491598766.log, /cassandra_data/commitlog/CommitLog-6-1483491598767.log
    INFO  01:02:08 Log replay complete, 92 replayed mutations
    INFO  01:02:08 Populating token metadata from system tables
    INFO  01:02:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    WARN  01:02:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:02:08 Token metadata: 
    INFO  01:02:09 Cassandra version: 3.9
    INFO  01:02:09 Thrift API version: 20.1.0
    INFO  01:02:09 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  01:02:09 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  01:02:09 Starting Messaging Service on /10.180.0.21:7000 (eth0)
    INFO  01:02:09 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    WARN  01:02:09 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessControlle
Contributor

k8s-merge-robot commented Jan 4, 2017

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1906/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] Hazelcast should create and scale hazelcast {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:558
Expected error:
    <*errors.errorString | 0xc420d19320>: {
        s: "Failed to find \"Members [2]\", last result: \"2017-01-04 00:46:50.088  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-08bsz with PID 5 (/bootstrapper.jar started by root in /)\n2017-01-04 00:46:50.112  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default\n2017-01-04 00:46:50.230  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Wed Jan 04 00:46:50 GMT 2017]; root of context hierarchy\n2017-01-04 00:46:51.859  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup\n2017-01-04 00:46:51.879  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..\n2017-01-04 00:46:52.322  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed\n\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-kkt6g/endpoints/hazelcast\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]\n\tat com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]\n\tat org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]\n\tat com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]\n\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]\n\tat org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]\n\tat org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]\n\n2017-01-04 00:46:52.323  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.\n2017-01-04 00:46:52.557  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []\n2017-01-04 00:46:52.558  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.\n2017-01-04 00:46:52.559  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.\n2017-01-04 00:46:52.570  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.13]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true\n2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.13]:5701\n2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.\n2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1\n2017-01-04 00:46:52.948  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.13]:5701 [someGroup] [3.7] Backpressure is disabled\n2017-01-04 00:46:53.998  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.2.13]:5701 [someGroup] [3.7] Creating TcpIpJoiner\n2017-01-04 00:46:54.000  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.13]:5701 [someGroup] [3.7] [10.180.2.13]:5701 is STARTING\n2017-01-04 00:46:54.289  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.13]:5701 [someGroup] [3.7] Starting 2 partition threads\n2017-01-04 00:46:54.292  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.13]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)\n2017-01-04 00:46:54.334  INFO 5 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.13]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads\n2017-01-04 00:46:54.393  INFO 5 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.13]:5701 [someGroup] [3.7] \n\n\nMembers [1] {\n\tMember [10.180.2.13]:5701 - 64015084-a86f-446a-b8ba-1cea6544b787 this\n}\n\n2017-01-04 00:46:54.502  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.13]:5701 [someGroup] [3.7] [10.180.2.13]:5701 is STARTED\n2017-01-04 00:46:54.504  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.465 seconds (JVM running for 6.281)\n\"",
    }
    Failed to find "Members [2]", last result: "2017-01-04 00:46:50.088  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Starting Application on hazelcast-08bsz with PID 5 (/bootstrapper.jar started by root in /)
    2017-01-04 00:46:50.112  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : No active profile set, falling back to default profiles: default
    2017-01-04 00:46:50.230  INFO 5 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@46fbb2c1: startup date [Wed Jan 04 00:46:50 GMT 2017]; root of context hierarchy
    2017-01-04 00:46:51.859  INFO 5 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
    2017-01-04 00:46:51.879  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
    2017-01-04 00:46:52.322  WARN 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Request to Kubernetes API failed
    
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local/api/v1/namespaces/e2e-tests-examples-kkt6g/endpoints/hazelcast
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92-internal]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_92-internal]
    	at com.github.pires.hazelcast.HazelcastDiscoveryController.run(HazelcastDiscoveryController.java:118) ~[classes!/:na]
    	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:782) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:769) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:134) [spring-boot-1.4.0.RELEASE.jar!/:1.4.0.RELEASE]
    	at com.github.pires.hazelcast.Application.main(Application.java:28) [classes!/:na]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_92-internal]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_92-internal]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_92-internal]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_92-internal]
    	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [bootstrapper.jar:na]
    	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58) [bootstrapper.jar:na]
    
    2017-01-04 00:46:52.323  INFO 5 --- [           main] c.g.p.h.HazelcastDiscoveryController     : Found 0 pods running Hazelcast.
    2017-01-04 00:46:52.557  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: []
    2017-01-04 00:46:52.558  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Prefer IPv4 stack is true.
    2017-01-04 00:46:52.559  WARN 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Could not find a matching address to start with! Picking one of non-loopback addresses.
    2017-01-04 00:46:52.570  INFO 5 --- [           main] c.h.instance.DefaultAddressPicker        : [LOCAL] [someGroup] [3.7] Picked [10.180.2.13]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
    2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Hazelcast 3.7 (20160817 - 1302600) starting at [10.180.2.13]:5701
    2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
    2017-01-04 00:46:52.611  INFO 5 --- [           main] com.hazelcast.system                     : [10.180.2.13]:5701 [someGroup] [3.7] Configured Hazelcast Serialization version : 1
    2017-01-04 00:46:52.948  INFO 5 --- [           main] c.h.s.i.o.impl.BackpressureRegulator     : [10.180.2.13]:5701 [someGroup] [3.7] Backpressure is disabled
    2017-01-04 00:46:53.998  INFO 5 --- [           main] com.hazelcast.instance.Node              : [10.180.2.13]:5701 [someGroup] [3.7] Creating TcpIpJoiner
    2017-01-04 00:46:54.000  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.13]:5701 [someGroup] [3.7] [10.180.2.13]:5701 is STARTING
    2017-01-04 00:46:54.289  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.13]:5701 [someGroup] [3.7] Starting 2 partition threads
    2017-01-04 00:46:54.292  INFO 5 --- [           main] c.h.s.i.o.impl.OperationExecutorImpl     : [10.180.2.13]:5701 [someGroup] [3.7] Starting 3 generic threads (1 dedicated for priority tasks)
    2017-01-04 00:46:54.334  INFO 5 --- [           main] c.h.n.t.n.NonBlockingIOThreadingModel    : [10.180.2.13]:5701 [someGroup] [3.7] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
    2017-01-04 00:46:54.393  INFO 5 --- [           main] com.hazelcast.cluster.impl.TcpIpJoiner   : [10.180.2.13]:5701 [someGroup] [3.7] 
    
    
    Members [1] {
    	Member [10.180.2.13]:5701 - 64015084-a86f-446a-b8ba-1cea6544b787 this
    }
    
    2017-01-04 00:46:54.502  INFO 5 --- [           main] com.hazelcast.core.LifecycleService      : [10.180.2.13]:5701 [someGroup] [3.7] [10.180.2.13]:5701 is STARTED
    2017-01-04 00:46:54.504  INFO 5 --- [           main] com.github.pires.hazelcast.Application   : Started Application in 5.465 seconds (JVM running for 6.281)
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:556

Issues about this test specifically: #27850 #30672 #33271

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc420920140>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.0.21\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.0.21\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.21\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.0.21\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  01:02:00 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  01:02:01 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.21; broadcast_rpc_address=10.180.0.21; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.21; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  01:02:01 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  01:02:01 Global memtable on-heap threshold is enabled at 128MB\nINFO  01:02:01 Global memtable off-heap threshold is enabled at 128MB\nINFO  01:02:01 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nINFO  01:02:01 Overriding RING_DELAY to 30000ms\nWARN  01:02:01 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:02:01 Hostname: cassandra-8v0mt\nINFO  01:02:01 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  01:02:01 Heap size: 512.000MiB/512.000MiB\nINFO  01:02:01 Code Cache Non-heap memory: init = 2555904(2496K) used = 4263232(4163K) committed = 4325376(4224K) max = 251658240(245760K)\nINFO  01:02:01 Metaspace Non-heap memory: init = 0(0K) used = 17426976(17018K) committed = 17956864(17536K) max = -1(-1K)\nINFO  01:02:01 Compressed Class Space Non-heap memory: init = 0(0K) used = 2080368(2031K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  01:02:01 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)\nINFO  01:02:01 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  01:02:01 G1 Old Gen Heap memory: init = 508559360(496640K) used = 964608(942K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  01:02:01 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  01:02:01 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.21, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  01:02:01 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  01:02:01 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  01:02:01 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  01:02:01 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  01:02:01 Initializing SIGAR library\nWARN  01:02:01 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  01:02:01 Directory /var/lib/cassandra/hints doesn't exist\nINFO  01:02:01 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  01:02:02 Initializing system.IndexInfo\nINFO  01:02:04 Initializing system.batches\nINFO  01:02:04 Initializing system.paxos\nINFO  01:02:04 Initializing system.local\nINFO  01:02:04 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  01:02:04 Initializing key cache with capacity of 25 MBs.\nINFO  01:02:04 Initializing row cache with capacity of 0 MBs\nINFO  01:02:04 Initializing counter cache with capacity of 12 MBs\nINFO  01:02:04 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  01:02:04 Initializing system.peers\nINFO  01:02:04 Initializing system.peer_events\nINFO  01:02:04 Initializing system.range_xfers\nINFO  01:02:04 Initializing system.compaction_history\nINFO  01:02:04 Initializing system.sstable_activity\nINFO  01:02:04 Initializing system.size_estimates\nINFO  01:02:04 Initializing system.available_ranges\nINFO  01:02:04 Initializing system.views_builds_in_progress\nINFO  01:02:04 Initializing system.built_views\nINFO  01:02:04 Initializing system.hints\nINFO  01:02:04 Initializing system.batchlog\nINFO  01:02:04 Initializing system.schema_keyspaces\nINFO  01:02:04 Initializing system.schema_columnfamilies\nINFO  01:02:04 Initializing system.schema_columns\nINFO  01:02:04 Initializing system.schema_triggers\nINFO  01:02:04 Initializing system.schema_usertypes\nINFO  01:02:04 Initializing system.schema_functions\nINFO  01:02:04 Initializing system.schema_aggregates\nINFO  01:02:04 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  01:02:05 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  01:02:06 Populating token metadata from system tables\nINFO  01:02:06 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nWARN  01:02:06 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:02:06 Token metadata: \nINFO  01:02:06 Initializing system_schema.keyspaces\nINFO  01:02:06 Initializing system_schema.tables\nINFO  01:02:06 Initializing system_schema.columns\nINFO  01:02:06 Initializing system_schema.triggers\nINFO  01:02:06 Initializing system_schema.dropped_columns\nINFO  01:02:06 Initializing system_schema.views\nINFO  01:02:06 Initializing system_schema.types\nINFO  01:02:06 Initializing system_schema.functions\nINFO  01:02:06 Initializing system_schema.aggregates\nINFO  01:02:06 Initializing system_schema.indexes\nINFO  01:02:06 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  01:02:06 Completed loading (2 ms; 2 keys) KeyCache cache\nINFO  01:02:07 Replaying /cassandra_data/commitlog/CommitLog-6-1483491598766.log, /cassandra_data/commitlog/CommitLog-6-1483491598767.log\nINFO  01:02:08 Log replay complete, 92 replayed mutations\nINFO  01:02:08 Populating token metadata from system tables\nINFO  01:02:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nWARN  01:02:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:02:08 Token metadata: \nINFO  01:02:09 Cassandra version: 3.9\nINFO  01:02:09 Thrift API version: 20.1.0\nINFO  01:02:09 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  01:02:09 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  01:02:09 Starting Messaging Service on /10.180.0.21:7000 (eth0)\nINFO  01:02:09 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nWARN  01:02:09 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:02:40 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\nWARN  01:02:40 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 01:02:40 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  01:02:40 No local state or state is in silent shutdown, not announcing shutdown\nINFO  01:02:40 Waiting for messaging service to quiesce\nINFO  01:02:40 MessagingService has terminated the accept() thread\nINFO  01:02:40 Paused hints dispatch\nINFO  01:02:40 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.0.21
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.0.21
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.0.21
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.0.21
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  01:02:00 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  01:02:01 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.0.21; broadcast_rpc_address=10.180.0.21; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.0.21; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  01:02:01 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  01:02:01 Global memtable on-heap threshold is enabled at 128MB
    INFO  01:02:01 Global memtable off-heap threshold is enabled at 128MB
    INFO  01:02:01 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    INFO  01:02:01 Overriding RING_DELAY to 30000ms
    WARN  01:02:01 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:02:01 Hostname: cassandra-8v0mt
    INFO  01:02:01 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  01:02:01 Heap size: 512.000MiB/512.000MiB
    INFO  01:02:01 Code Cache Non-heap memory: init = 2555904(2496K) used = 4263232(4163K) committed = 4325376(4224K) max = 251658240(245760K)
    INFO  01:02:01 Metaspace Non-heap memory: init = 0(0K) used = 17426976(17018K) committed = 17956864(17536K) max = -1(-1K)
    INFO  01:02:01 Compressed Class Space Non-heap memory: init = 0(0K) used = 2080368(2031K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  01:02:01 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 32505856(31744K) max = -1(-1K)
    INFO  01:02:01 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  01:02:01 G1 Old Gen Heap memory: init = 508559360(496640K) used = 964608(942K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  01:02:01 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  01:02:01 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.0.21, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  01:02:01 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  01:02:01 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  01:02:01 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  01:02:01 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  01:02:01 Initializing SIGAR library
    WARN  01:02:01 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  01:02:01 Directory /var/lib/cassandra/hints doesn't exist
    INFO  01:02:01 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  01:02:02 Initializing system.IndexInfo
    INFO  01:02:04 Initializing system.batches
    INFO  01:02:04 Initializing system.paxos
    INFO  01:02:04 Initializing system.local
    INFO  01:02:04 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  01:02:04 Initializing key cache with capacity of 25 MBs.
    INFO  01:02:04 Initializing row cache with capacity of 0 MBs
    INFO  01:02:04 Initializing counter cache with capacity of 12 MBs
    INFO  01:02:04 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  01:02:04 Initializing system.peers
    INFO  01:02:04 Initializing system.peer_events
    INFO  01:02:04 Initializing system.range_xfers
    INFO  01:02:04 Initializing system.compaction_history
    INFO  01:02:04 Initializing system.sstable_activity
    INFO  01:02:04 Initializing system.size_estimates
    INFO  01:02:04 Initializing system.available_ranges
    INFO  01:02:04 Initializing system.views_builds_in_progress
    INFO  01:02:04 Initializing system.built_views
    INFO  01:02:04 Initializing system.hints
    INFO  01:02:04 Initializing system.batchlog
    INFO  01:02:04 Initializing system.schema_keyspaces
    INFO  01:02:04 Initializing system.schema_columnfamilies
    INFO  01:02:04 Initializing system.schema_columns
    INFO  01:02:04 Initializing system.schema_triggers
    INFO  01:02:04 Initializing system.schema_usertypes
    INFO  01:02:04 Initializing system.schema_functions
    INFO  01:02:04 Initializing system.schema_aggregates
    INFO  01:02:04 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  01:02:05 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  01:02:06 Populating token metadata from system tables
    INFO  01:02:06 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    WARN  01:02:06 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:02:06 Token metadata: 
    INFO  01:02:06 Initializing system_schema.keyspaces
    INFO  01:02:06 Initializing system_schema.tables
    INFO  01:02:06 Initializing system_schema.columns
    INFO  01:02:06 Initializing system_schema.triggers
    INFO  01:02:06 Initializing system_schema.dropped_columns
    INFO  01:02:06 Initializing system_schema.views
    INFO  01:02:06 Initializing system_schema.types
    INFO  01:02:06 Initializing system_schema.functions
    INFO  01:02:06 Initializing system_schema.aggregates
    INFO  01:02:06 Initializing system_schema.indexes
    INFO  01:02:06 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  01:02:06 Completed loading (2 ms; 2 keys) KeyCache cache
    INFO  01:02:07 Replaying /cassandra_data/commitlog/CommitLog-6-1483491598766.log, /cassandra_data/commitlog/CommitLog-6-1483491598767.log
    INFO  01:02:08 Log replay complete, 92 replayed mutations
    INFO  01:02:08 Populating token metadata from system tables
    INFO  01:02:08 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    WARN  01:02:08 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:02:08 Token metadata: 
    INFO  01:02:09 Cassandra version: 3.9
    INFO  01:02:09 Thrift API version: 20.1.0
    INFO  01:02:09 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  01:02:09 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  01:02:09 Starting Messaging Service on /10.180.0.21:7000 (eth0)
    INFO  01:02:09 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    WARN  01:02:09 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-fm3tp/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessControlle
@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot Jan 4, 2017

Contributor

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gce-examples/1907/
Multiple broken tests:

Failed: [k8s.io] [Feature:Example] [k8s.io] RethinkDB should create and stop rethinkdb servers {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:524
Expected error:
    <*errors.errorString | 0xc420d9bb20>: {
        s: "Failed to find \"Server ready\", last result: \"Checking for other nodes\nMy host: 10.180.3.10\nEndpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-njdw1/endpoints/rethinkdb-driver\nLooking for IPs...\nparse error: Invalid numeric literal at line 1, column 5\n\"",
    }
    Failed to find "Server ready", last result: "Checking for other nodes
    My host: 10.180.3.10
    Endpont url: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-njdw1/endpoints/rethinkdb-driver
    Looking for IPs...
    parse error: Invalid numeric literal at line 1, column 5
    "
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:502

Issues about this test specifically: #30581 #32755

Failed: [k8s.io] [Feature:Example] [k8s.io] Cassandra should create and scale cassandra {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/examples.go:247
Expected error:
    <*errors.errorString | 0xc4207309a0>: {
        s: "Failed to find \"Starting listening for CQL clients\", last result: \"Starting Cassandra on 10.180.1.12\nCASSANDRA_CONF_DIR /etc/cassandra\nCASSANDRA_CFG /etc/cassandra/cassandra.yaml\nCASSANDRA_AUTO_BOOTSTRAP true\nCASSANDRA_BROADCAST_ADDRESS 10.180.1.12\nCASSANDRA_BROADCAST_RPC_ADDRESS 10.180.1.12\nCASSANDRA_CLUSTER_NAME 'Test Cluster'\nCASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC\nCASSANDRA_CONCURRENT_COMPACTORS\nCASSANDRA_CONCURRENT_READS\nCASSANDRA_CONCURRENT_WRITES\nCASSANDRA_COUNTER_CACHE_SIZE_IN_MB\nCASSANDRA_DC\nCASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd\nCASSANDRA_ENDPOINT_SNITCH SimpleSnitch\nCASSANDRA_GC_WARN_THRESHOLD_IN_MS\nCASSANDRA_INTERNODE_COMPRESSION\nCASSANDRA_KEY_CACHE_SIZE_IN_MB\nCASSANDRA_LISTEN_ADDRESS 10.180.1.12\nCASSANDRA_LISTEN_INTERFACE\nCASSANDRA_MEMTABLE_ALLOCATION_TYPE\nCASSANDRA_MEMTABLE_CLEANUP_THRESHOLD\nCASSANDRA_MEMTABLE_FLUSH_WRITERS\nCASSANDRA_MIGRATION_WAIT 1\nCASSANDRA_NUM_TOKENS 32\nCASSANDRA_RACK\nCASSANDRA_RING_DELAY 30000\nCASSANDRA_RPC_ADDRESS 0.0.0.0\nCASSANDRA_RPC_INTERFACE\nCASSANDRA_SEEDS\nCASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider\nOpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory\n\nINFO  01:48:15 Configuration location: file:/etc/cassandra/cassandra.yaml\nINFO  01:48:15 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.1.12; broadcast_rpc_address=10.180.1.12; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.1.12; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\nINFO  01:48:15 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\nINFO  01:48:15 Global memtable on-heap threshold is enabled at 128MB\nINFO  01:48:15 Global memtable off-heap threshold is enabled at 128MB\nINFO  01:48:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\nINFO  01:48:15 Overriding RING_DELAY to 30000ms\nWARN  01:48:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:48:15 Hostname: cassandra-8dm7l\nINFO  01:48:15 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111\nINFO  01:48:15 Heap size: 512.000MiB/512.000MiB\nINFO  01:48:15 Code Cache Non-heap memory: init = 2555904(2496K) used = 4233984(4134K) committed = 4259840(4160K) max = 251658240(245760K)\nINFO  01:48:15 Metaspace Non-heap memory: init = 0(0K) used = 17218136(16814K) committed = 17694720(17280K) max = -1(-1K)\nINFO  01:48:15 Compressed Class Space Non-heap memory: init = 0(0K) used = 2065280(2016K) committed = 2228224(2176K) max = 1073741824(1048576K)\nINFO  01:48:15 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)\nINFO  01:48:15 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)\nINFO  01:48:15 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1444864(1411K) committed = 508559360(496640K) max = 536870912(524288K)\nINFO  01:48:15 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\nINFO  01:48:15 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.1.12, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\nWARN  01:48:15 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\nINFO  01:48:15 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\nWARN  01:48:15 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\nWARN  01:48:15 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release\nINFO  01:48:15 Initializing SIGAR library\nWARN  01:48:15 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false \nWARN  01:48:16 Directory /var/lib/cassandra/hints doesn't exist\nINFO  01:48:16 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\nINFO  01:48:16 Initializing system.IndexInfo\nINFO  01:48:18 Initializing system.batches\nINFO  01:48:18 Initializing system.paxos\nINFO  01:48:18 Initializing system.local\nINFO  01:48:18 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap\nINFO  01:48:18 Initializing key cache with capacity of 25 MBs.\nINFO  01:48:18 Initializing row cache with capacity of 0 MBs\nINFO  01:48:18 Initializing counter cache with capacity of 12 MBs\nINFO  01:48:18 Scheduling counter cache save to every 7200 seconds (going to save all keys).\nINFO  01:48:18 Initializing system.peers\nINFO  01:48:18 Initializing system.peer_events\nINFO  01:48:18 Initializing system.range_xfers\nINFO  01:48:18 Initializing system.compaction_history\nINFO  01:48:18 Initializing system.sstable_activity\nINFO  01:48:18 Initializing system.size_estimates\nINFO  01:48:18 Initializing system.available_ranges\nINFO  01:48:18 Initializing system.views_builds_in_progress\nINFO  01:48:18 Initializing system.built_views\nINFO  01:48:18 Initializing system.hints\nINFO  01:48:18 Initializing system.batchlog\nINFO  01:48:18 Initializing system.schema_keyspaces\nINFO  01:48:18 Initializing system.schema_columnfamilies\nINFO  01:48:18 Initializing system.schema_columns\nINFO  01:48:18 Initializing system.schema_triggers\nINFO  01:48:18 Initializing system.schema_usertypes\nINFO  01:48:18 Initializing system.schema_functions\nINFO  01:48:18 Initializing system.schema_aggregates\nINFO  01:48:18 Not submitting build tasks for views in keyspace system as storage service is not initialized\nINFO  01:48:19 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\nINFO  01:48:20 Populating token metadata from system tables\nINFO  01:48:20 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\nWARN  01:48:20 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:48:20 Token metadata: \nINFO  01:48:20 Initializing system_schema.keyspaces\nINFO  01:48:20 Initializing system_schema.tables\nINFO  01:48:20 Initializing system_schema.columns\nINFO  01:48:20 Initializing system_schema.triggers\nINFO  01:48:20 Initializing system_schema.dropped_columns\nINFO  01:48:20 Initializing system_schema.views\nINFO  01:48:20 Initializing system_schema.types\nINFO  01:48:20 Initializing system_schema.functions\nINFO  01:48:20 Initializing system_schema.aggregates\nINFO  01:48:20 Initializing system_schema.indexes\nINFO  01:48:20 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\nINFO  01:48:20 Completed loading (19 ms; 2 keys) KeyCache cache\nINFO  01:48:20 Replaying /cassandra_data/commitlog/CommitLog-6-1483494363831.log, /cassandra_data/commitlog/CommitLog-6-1483494363832.log\nINFO  01:48:22 Log replay complete, 92 replayed mutations\nINFO  01:48:22 Populating token metadata from system tables\nINFO  01:48:22 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\nWARN  01:48:22 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:48:22 Token metadata: \nINFO  01:48:22 Cassandra version: 3.9\nINFO  01:48:22 Thrift API version: 20.1.0\nINFO  01:48:22 CQL supported versions: 3.4.2 (default: 3.4.2)\nINFO  01:48:22 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes\nINFO  01:48:22 Starting Messaging Service on /10.180.1.12:7000 (eth0)\nINFO  01:48:22 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\nWARN  01:48:22 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.buildSeedsList(Gossiper.java:1402) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1347) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nINFO  01:48:53 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\nWARN  01:48:53 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml\njava.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]\n\tat java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]\n\tat sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]\n\tat io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]\n\tat org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nException (java.lang.RuntimeException) encountered during startup: Unable to gossip with any seeds\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386)\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484)\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648)\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548)\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385)\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)\nERROR 01:48:53 Exception encountered during startup\njava.lang.RuntimeException: Unable to gossip with any seeds\n\tat org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1386) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:484) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:778) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:648) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.StorageService.initServer(StorageService.java:548) ~[apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:385) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]\n\tat org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]\nWARN  01:48:53 No local state or state is in silent shutdown, not announcing shutdown\nINFO  01:48:53 Waiting for messaging service to quiesce\nINFO  01:48:53 MessagingService has terminated the accept() thread\nINFO  01:48:53 Paused hints dispatch\nINFO  01:48:53 Paused hints dispatch\n\"",
    }
    Failed to find "Starting listening for CQL clients", last result: "Starting Cassandra on 10.180.1.12
    CASSANDRA_CONF_DIR /etc/cassandra
    CASSANDRA_CFG /etc/cassandra/cassandra.yaml
    CASSANDRA_AUTO_BOOTSTRAP true
    CASSANDRA_BROADCAST_ADDRESS 10.180.1.12
    CASSANDRA_BROADCAST_RPC_ADDRESS 10.180.1.12
    CASSANDRA_CLUSTER_NAME 'Test Cluster'
    CASSANDRA_COMPACTION_THROUGHPUT_MB_PER_SEC
    CASSANDRA_CONCURRENT_COMPACTORS
    CASSANDRA_CONCURRENT_READS
    CASSANDRA_CONCURRENT_WRITES
    CASSANDRA_COUNTER_CACHE_SIZE_IN_MB
    CASSANDRA_DC
    CASSANDRA_DISK_OPTIMIZATION_STRATEGY ssd
    CASSANDRA_ENDPOINT_SNITCH SimpleSnitch
    CASSANDRA_GC_WARN_THRESHOLD_IN_MS
    CASSANDRA_INTERNODE_COMPRESSION
    CASSANDRA_KEY_CACHE_SIZE_IN_MB
    CASSANDRA_LISTEN_ADDRESS 10.180.1.12
    CASSANDRA_LISTEN_INTERFACE
    CASSANDRA_MEMTABLE_ALLOCATION_TYPE
    CASSANDRA_MEMTABLE_CLEANUP_THRESHOLD
    CASSANDRA_MEMTABLE_FLUSH_WRITERS
    CASSANDRA_MIGRATION_WAIT 1
    CASSANDRA_NUM_TOKENS 32
    CASSANDRA_RACK
    CASSANDRA_RING_DELAY 30000
    CASSANDRA_RPC_ADDRESS 0.0.0.0
    CASSANDRA_RPC_INTERFACE
    CASSANDRA_SEEDS
    CASSANDRA_SEED_PROVIDER io.k8s.cassandra.KubernetesSeedProvider
    OpenJDK 64-Bit Server VM warning: Cannot open file /var/log/cassandra/gc.log due to No such file or directory
    
    INFO  01:48:15 Configuration location: file:/etc/cassandra/cassandra.yaml
    INFO  01:48:15 Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.180.1.12; broadcast_rpc_address=10.180.1.12; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=null; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/cassandra_data/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=null; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@43ee72e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=all; internode_recv_buff_size_in_bytes=null; internode_send_buff_size_in_bytes=null; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.180.1.12; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=1; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=32; otc_coalescing_strategy=TIMEHORIZON; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/cassandra_data/saved_caches; seed_provider=io.k8s.cassandra.KubernetesSeedProvider{seeds=}; server_encryption_options=<REDACTED>; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@23529fee; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
    INFO  01:48:15 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
    INFO  01:48:15 Global memtable on-heap threshold is enabled at 128MB
    INFO  01:48:15 Global memtable off-heap threshold is enabled at 128MB
    INFO  01:48:15 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra
    INFO  01:48:15 Overriding RING_DELAY to 30000ms
    WARN  01:48:15 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:784) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:125) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:576) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:48:15 Hostname: cassandra-8dm7l
    INFO  01:48:15 JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_111
    INFO  01:48:15 Heap size: 512.000MiB/512.000MiB
    INFO  01:48:15 Code Cache Non-heap memory: init = 2555904(2496K) used = 4233984(4134K) committed = 4259840(4160K) max = 251658240(245760K)
    INFO  01:48:15 Metaspace Non-heap memory: init = 0(0K) used = 17218136(16814K) committed = 17694720(17280K) max = -1(-1K)
    INFO  01:48:15 Compressed Class Space Non-heap memory: init = 0(0K) used = 2065280(2016K) committed = 2228224(2176K) max = 1073741824(1048576K)
    INFO  01:48:15 G1 Eden Space Heap memory: init = 28311552(27648K) used = 26214400(25600K) committed = 33554432(32768K) max = -1(-1K)
    INFO  01:48:15 G1 Survivor Space Heap memory: init = 0(0K) used = 4194304(4096K) committed = 4194304(4096K) max = -1(-1K)
    INFO  01:48:15 G1 Old Gen Heap memory: init = 508559360(496640K) used = 1444864(1411K) committed = 508559360(496640K) max = 536870912(524288K)
    INFO  01:48:15 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.39.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.3.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.3.jar:/usr/share/cassandra/lib/primitive-1.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.9.jar:/usr/share/cassandra/apache-cassandra-thrift-3.9.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
    INFO  01:48:15 JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseG1GC, -XX:G1RSetUpdatingPauseTimePercent=5, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Dcassandra.migration_task_wait_in_seconds=1, -Dcassandra.ring_delay_ms=30000, -Xms512M, -Xmx512M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Djava.rmi.server.hostname=10.180.1.12, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
    WARN  01:48:15 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
    INFO  01:48:15 jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
    WARN  01:48:15 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
    WARN  01:48:15 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
    INFO  01:48:15 Initializing SIGAR library
    WARN  01:48:15 Cassandra server running in degraded mode. Is swap disabled? : true,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : false 
    WARN  01:48:16 Directory /var/lib/cassandra/hints doesn't exist
    INFO  01:48:16 Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
    INFO  01:48:16 Initializing system.IndexInfo
    INFO  01:48:18 Initializing system.batches
    INFO  01:48:18 Initializing system.paxos
    INFO  01:48:18 Initializing system.local
    INFO  01:48:18 Global buffer pool is enabled, when pool is exhausted (max is 128.000MiB) it will allocate on heap
    INFO  01:48:18 Initializing key cache with capacity of 25 MBs.
    INFO  01:48:18 Initializing row cache with capacity of 0 MBs
    INFO  01:48:18 Initializing counter cache with capacity of 12 MBs
    INFO  01:48:18 Scheduling counter cache save to every 7200 seconds (going to save all keys).
    INFO  01:48:18 Initializing system.peers
    INFO  01:48:18 Initializing system.peer_events
    INFO  01:48:18 Initializing system.range_xfers
    INFO  01:48:18 Initializing system.compaction_history
    INFO  01:48:18 Initializing system.sstable_activity
    INFO  01:48:18 Initializing system.size_estimates
    INFO  01:48:18 Initializing system.available_ranges
    INFO  01:48:18 Initializing system.views_builds_in_progress
    INFO  01:48:18 Initializing system.built_views
    INFO  01:48:18 Initializing system.hints
    INFO  01:48:18 Initializing system.batchlog
    INFO  01:48:18 Initializing system.schema_keyspaces
    INFO  01:48:18 Initializing system.schema_columnfamilies
    INFO  01:48:18 Initializing system.schema_columns
    INFO  01:48:18 Initializing system.schema_triggers
    INFO  01:48:18 Initializing system.schema_usertypes
    INFO  01:48:18 Initializing system.schema_functions
    INFO  01:48:18 Initializing system.schema_aggregates
    INFO  01:48:18 Not submitting build tasks for views in keyspace system as storage service is not initialized
    INFO  01:48:19 Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
    INFO  01:48:20 Populating token metadata from system tables
    INFO  01:48:20 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra
    WARN  01:48:20 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:257) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:48:20 Token metadata: 
    INFO  01:48:20 Initializing system_schema.keyspaces
    INFO  01:48:20 Initializing system_schema.tables
    INFO  01:48:20 Initializing system_schema.columns
    INFO  01:48:20 Initializing system_schema.triggers
    INFO  01:48:20 Initializing system_schema.dropped_columns
    INFO  01:48:20 Initializing system_schema.views
    INFO  01:48:20 Initializing system_schema.types
    INFO  01:48:20 Initializing system_schema.functions
    INFO  01:48:20 Initializing system_schema.aggregates
    INFO  01:48:20 Initializing system_schema.indexes
    INFO  01:48:20 Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
    INFO  01:48:20 Completed loading (19 ms; 2 keys) KeyCache cache
    INFO  01:48:20 Replaying /cassandra_data/commitlog/CommitLog-6-1483494363831.log, /cassandra_data/commitlog/CommitLog-6-1483494363832.log
    INFO  01:48:22 Log replay complete, 92 replayed mutations
    INFO  01:48:22 Populating token metadata from system tables
    INFO  01:48:22 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra
    WARN  01:48:22 Request to kubernetes apiserver failed, using default seeds in cassandra.yaml
    java.io.IOException: Server returned HTTP response code: 403 for URL: https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:91) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1466) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1464) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_111]
    	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782) ~[na:1.8.0_111]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1463) ~[na:1.8.0_111]
    	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_111]
    	at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:132) ~[kubernetes-cassandra.jar:na]
    	at org.apache.cassandra.config.DatabaseDescriptor.getSeeds(DatabaseDescriptor.java:1475) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.shouldBootstrap(StorageService.java:721) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.StorageService.populateTokenMetadata(StorageService.java:537) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:331) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.jar:3.9]
    	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.jar:3.9]
    INFO  01:48:22 Token metadata: 
    INFO  01:48:22 Cassandra version: 3.9
    INFO  01:48:22 Thrift API version: 20.1.0
    INFO  01:48:22 CQL supported versions: 3.4.2 (default: 3.4.2)
    INFO  01:48:22 Initializing index summary manager with a memory pool size of 25 MB and a resize interval of 60 minutes
    INFO  01:48:22 Starting Messaging Service on /10.180.1.12:7000 (eth0)
    INFO  01:48:22 Getting endpoints from https://10.0.0.1:443/api/v1/namespaces/e2e-tests-examples-l9jp4/endpoints/cassandra
    WARN  01:48:22 Request to kubernetes apiser