New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open a Bolt connector to your embedded instance to get GUI administration and other benefits. #7608

Open
remilmi opened this Issue Jul 19, 2016 · 15 comments

Comments

Projects
None yet
8 participants
@remilmi

remilmi commented Jul 19, 2016

The documentation, specifically section 2.14 of http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded says that enabling a bolt connector on an embedded instance allows you to get GUI administration and other benefits. However the documentation doesn't specify to to access the embedded instance using the Neo4j Browser.

  • Neo4j version: 3.0.3
  • Operating system: Windows 7 Enterprise
  • API/Driver: Java API/Bolt
  • Steps to reproduce
    1. Create an embedded database with a bolt connector enabled according to: http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded section 2.14
    2. Not sure, the documentation doesn't say how the browser is used once the bolt connector is enabled.
  • Expected behavior: Can access the embedded DB with the Neo4j browser as the documentation claims -> "Doing so allows you to connect the services Neo4j Browser to your embedded instance."
  • Actual behavior: Can't access the embedded instance using the Neo4j Browser.
@spacecowboy

This comment has been minimized.

Contributor

spacecowboy commented Jul 21, 2016

Once you have enabled bolt, open the browser and configure the appropriate ip, port, username, password, in the sidebar as shown here:

selection_054

@remilmi

This comment has been minimized.

remilmi commented Jul 21, 2016

I'm assuming I need to start the neo4j server first so that it's hosted by something. I also disabled bolt on the neo4j server so there wouldn't be a port conflict.

I have the following running in JBoss:

graphDb = new GraphDatabaseFactory() .newEmbeddedDatabaseBuilder(new File(imseaConfig.getNeo4jLocation())) .setConfig(bolt.enabled, "true") .setConfig(bolt.type, "BOLT") .setConfig(bolt.encryption_level, GraphDatabaseSettings.BoltConnector.EncryptionLevel.DISABLED.toString()) .setConfig(bolt.address, "0.0.0.0:7687") .newGraphDatabase();

Once JBoss is started 7687 is open.

I checked the "Use bolt protocol when available" box and entered localhost:7687.
When I try to connect I'm prompted with a Username/Password form. I enter the username and password, click connect and nothing. Eventually a red banner will appear saying I'm not connected and also eventually and only some times will I get the following error in the browser:

WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using the wrong connection URL or temporary network problems. If you have enabled encryption, ensure your browser is configured to trust the certificate Neo4j is configured to use. WebSocket readyState is: 3

And the only thing I see in the developer console is:
Firefox can't establish a connection to the server at wss://localhost:7687/.

Do you see anything obvious I'm not doing correctly?

@spacecowboy

This comment has been minimized.

Contributor

spacecowboy commented Jul 26, 2016

No, can't see anything wrong with that. I know that a while back we had a bug which prevented the browser from working over bolt, I am unsure if that has been fixed in 3.0.3..

@spacecowboy

This comment has been minimized.

Contributor

spacecowboy commented Jul 26, 2016

@remilmi the log shows that you are trying to connect to wss://localhost:7687/ which is a secure connection, but you have disabled encryption.

Make sure you open the browser at http://localhost:7474 rather than https://localhost:7473 to have encryption disabled in the browser as well. Or, enable encryption.

@remilmi

This comment has been minimized.

remilmi commented Jul 27, 2016

I tried with encryption set to both optional and required going to 7474 and 7473 for each but with the same result. I couldn't find the bug report you mentioned earlier, can you send me the link for it please?

@spacecowboy

This comment has been minimized.

Contributor

spacecowboy commented Jul 28, 2016

There is no bug report. It is fixed anyway.

Upgrading this to a bug.

@spacecowboy spacecowboy added bug browser and removed question labels Jul 28, 2016

@anneiam

This comment has been minimized.

anneiam commented Dec 16, 2016

Trying to connect the Neo4j browser to embedded neo instance (with Bolt connector enabled), it says "connected" and I initially see my nodes/labels/etc in the "Database View" sidebar, but I'm unable to run any queries/commands. After some time, it disconnects and reports "web socket connection failure".

Immediately upon connecting, I got the following exception (repeatedly) in the neo4j log:

2016-12-16 20:53:36.086+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [UnknownError]: Exception invoking method resources, reference 8d09ef86-da3e-49c7-a850-afdede0ddef2. Exception invoking method resources
javax.management.RuntimeOperationsException: Exception invoking method resources
	at org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute(BaseModelMBean.java:196)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
	at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:131)
	at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:119)
	at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.lambda$apply$0(JmxQueryProcedure.java:91)
	at org.neo4j.kernel.builtinprocs.JmxQueryProcedure$$Lambda$346/713579480.get(Unknown Source)
	at org.neo4j.collection.RawIterator$2.fetchNextOrNull(RawIterator.java:84)
	at org.neo4j.collection.PrefetchingRawIterator.peek(PrefetchingRawIterator.java:50)
	at org.neo4j.collection.PrefetchingRawIterator.hasNext(PrefetchingRawIterator.java:36)
	at org.neo4j.kernel.impl.api.OperationsFacade$1.hasNext(OperationsFacade.java:1583)
	at org.neo4j.cypher.internal.spi.v3_1.TransactionBoundQueryContext$$anon$2.hasNext(TransactionBoundQueryContext.scala:633)
	at org.neo4j.cypher.internal.spi.v3_1.ExceptionTranslationSupport$$anon$1$$anonfun$hasNext$1.apply$mcZ$sp(ExceptionTranslationSupport.scala:47)
	at org.neo4j.cypher.internal.spi.v3_1.ExceptionTranslationSupport$$anon$1$$anonfun$hasNext$1.apply(ExceptionTranslationSupport.scala:47)
	at org.neo4j.cypher.internal.spi.v3_1.ExceptionTranslationSupport$$anon$1$$anonfun$hasNext$1.apply(ExceptionTranslationSupport.scala:47)
	at org.neo4j.cypher.internal.spi.v3_1.ExceptionTranslationSupport$class.translateException(ExceptionTranslationSupport.scala:32)
	at org.neo4j.cypher.internal.compatibility.ExceptionTranslatingQueryContextFor3_1.translateException(ExceptionTranslatingQueryContextFor3_1.scala:34)
	at org.neo4j.cypher.internal.spi.v3_1.ExceptionTranslationSupport$$anon$1.hasNext(ExceptionTranslationSupport.scala:47)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at org.neo4j.cypher.internal.spi.v3_1.ExceptionTranslationSupport$$anon$1.foreach(ExceptionTranslationSupport.scala:46)
	at org.neo4j.cypher.internal.compiler.v3_1.executionplan.procs.ProcedureExecutionResult.accept(ProcedureExecutionResult.scala:69)
	at org.neo4j.cypher.internal.compatibility.ExecutionResultWrapperFor3_1.accept(CompatibilityFor3_1.scala:402)
	at org.neo4j.cypher.internal.compatibility.ClosingExecutionResult$$anonfun$accept$1.apply$mcV$sp(ClosingExecutionResult.scala:169)
	at org.neo4j.cypher.internal.compatibility.ClosingExecutionResult$$anonfun$accept$1.apply(ClosingExecutionResult.scala:168)
	at org.neo4j.cypher.internal.compatibility.ClosingExecutionResult$$anonfun$accept$1.apply(ClosingExecutionResult.scala:168)
	at org.neo4j.cypher.internal.compatibility.exceptionHandlerFor3_1$runSafely$.apply(CompatibilityFor3_1.scala:190)
	at org.neo4j.cypher.internal.compatibility.ClosingExecutionResult.accept(ClosingExecutionResult.scala:168)
	at org.neo4j.cypher.internal.javacompat.ExecutionResult.accept(ExecutionResult.java:317)
	at org.neo4j.bolt.v1.runtime.cypher.CypherAdapterStream.accept(CypherAdapterStream.java:69)
	at org.neo4j.bolt.v1.messaging.BoltMessageRouter$ResultHandler.onRecords(BoltMessageRouter.java:131)
	at org.neo4j.bolt.v1.runtime.BoltStateMachine$State$3.lambda$pullAll$0(BoltStateMachine.java:451)
	at org.neo4j.bolt.v1.runtime.BoltStateMachine$State$3$$Lambda$337/802474997.accept(Unknown Source)
	at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State$1.streamResult(TransactionStateMachine.java:211)
	at org.neo4j.bolt.v1.runtime.TransactionStateMachine.streamResult(TransactionStateMachine.java:93)
	at org.neo4j.bolt.v1.runtime.BoltStateMachine$State$3.pullAll(BoltStateMachine.java:450)
	at org.neo4j.bolt.v1.runtime.BoltStateMachine.pullAll(BoltStateMachine.java:233)
	at org.neo4j.bolt.v1.messaging.BoltMessageRouter.lambda$onPullAll$6(BoltMessageRouter.java:99)
	at org.neo4j.bolt.v1.messaging.BoltMessageRouter$$Lambda$327/2408107.perform(Unknown Source)
	at org.neo4j.bolt.v1.runtime.concurrent.RunnableBoltWorker.execute(RunnableBoltWorker.java:136)
	at org.neo4j.bolt.v1.runtime.concurrent.RunnableBoltWorker.executeBatch(RunnableBoltWorker.java:123)
	at org.neo4j.bolt.v1.runtime.concurrent.RunnableBoltWorker.run(RunnableBoltWorker.java:94)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
	at org.apache.catalina.mbeans.NamingResourcesMBean.getResources(NamingResourcesMBean.java:117)
	at sun.reflect.GeneratedMethodAccessor946.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute(BaseModelMBean.java:187)
	... 41 more
@ghost

This comment has been minimized.

ghost commented Dec 17, 2016

We are running Neo4j 3.1.0 as an embedded graph.db and also would like to enable the 7474 neo4j HTTP browser as we did in 2.3.7.
Conf parameters are setting 7474 to HTTP from BOLT in 3.1.0, in 3.0.7 the port behaves like a bolt websocket port

Bottom line is:

3.0.7 brings up 7474 as a bolt port
3.1.0 does not bring up 7474 at all

In 3.1 the gui is different for the 3.0 procedure where a standalone browser is used to adjust the embedded graph.db - the bolt-http connector is still not up though.

screenshot 2016-12-17 23 43 56

Can you advise if the bolt+routing protocol workaround for 3.1 is valid - when it was put in to 3.1 the issue states "when in a causal cluster"

cl: Add support for the bolt+routing protocol when in a Causal Cluster
https://github.com/neo4j/neo4j-browser/pull/322

To check - I switched back to 3.0.7 (deleted graph.db) - get different behaviour
I get the same malformed websocket message I see in 7474 that I get if I access bolt 7687 directly in 3.1.0

likely because 7474 ends up configured with BOLT instead of HTTP

2016-12-18 15:03:26.189+0000 INFO  [o.n.b.v.r.Sessions] Bolt Server extension loaded.
2016-12-18 15:03:26.190+0000 INFO  [o.n.b.v.r.Sessions] Bolt enabled on 0.0.0.0:7687.
2016-12-18 15:03:26.190+0000 INFO  [o.n.b.v.r.Sessions] Bolt enabled on 0.0.0.0:7474.

This is with or without the bolt settings applied on the factory and running only with

        <entry key="dbms.connector.http.address" value="0.0.0.0:7474" />
        <entry key="dbms.connector.http.enabled" value="true" />
        <entry key="dbms.connector.http.type" value="HTTP" />

and with/without bolt settings applied

public class ExtendedHighlyAvailableGraphDatabaseFactory extends HighlyAvailableGraphDatabaseFactory {
    private HaMonitor haMonitor;
    private GraphDatabaseSettings.BoltConnector bolt = GraphDatabaseSettings.boltConnector( "0" );
    @Override
     protected GraphDatabaseBuilder.DatabaseCreator createDatabaseCreator(
                final File storeDir, final GraphDatabaseFactoryState state)  {
            return new GraphDatabaseBuilder.DatabaseCreator() {
                @Override
                public GraphDatabaseService newDatabase( final Map<String, String> config ) {
                    // section 2.14 of http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded
//                    config.put( bolt.type.name(), "BOLT" );
//                    config.put( bolt.enabled.name(), "true" );
//                    config.put( bolt.address.name(), "0.0.0.0:7687" );

running

           <neo4j.version>3.0.7</neo4j.version>
            <!--neo4j.version>3.1.0</neo4j.version-->
   </properties>
  <dependencies>
      <dependency>
        <groupId>org.neo4j</groupId>
        <artifactId>neo4j-bolt</artifactId>
        <version>${neo4j.version}</version>
    </dependency> 

"not a WebSocket handshake request: missing upgrade"

A 400 this time
Request URL:http://localhost:7474/
Request Method:GET
Status Code:400 Bad Request
Remote Address:127.0.0.1:7474

3.0.7
2016-12-18 15:03:26.189+0000 INFO  [o.n.b.v.r.Sessions] Bolt Server extension loaded.
2016-12-18 15:03:26.190+0000 INFO  [o.n.b.v.r.Sessions] Bolt enabled on 0.0.0.0:7687.
2016-12-18 15:03:26.190+0000 INFO  [o.n.b.v.r.Sessions] Bolt enabled on 0.0.0.0:7474.
2016-12-18 15:24:46.851+0000 INFO  [o.n.k.i.DiagnosticsManager] Kernel version: 3.0.7,844933cd439bde936b04e66f1c0ca8d617794a0a

obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7474
tcp46      0      0  *.7474                 *.*                    LISTEN      131072 131072  35888      0
tcp6       0      0  ::1.7474               ::1.49967              TIME_WAIT   407223 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49976              TIME_WAIT   407444 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49978              TIME_WAIT   407223 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49979              TIME_WAIT   407223 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49980              TIME_WAIT   407223 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49982              TIME_WAIT   407223 146808  35888      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080
tcp4       0      0  127.0.0.1.8080         127.0.0.1.49948        FIN_WAIT_2  408104 146988  35888      0
tcp46      0      0  *.8080                 *.*                    LISTEN      131072 131072  35888      0
tcp6       0      0  ::1.8080               ::1.49961              TIME_WAIT   407402 146808  35888      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7687
tcp46      0      0  *.7687                 *.*                    LISTEN      131072 131072  35888      0
3.1.0
2016-12-18 15:27:02.128+0000 INFO  [o.n.b.v.r.WorkerFactory] Bolt Server extension loaded.
2016-12-18 15:27:02.128+0000 INFO  [o.n.b.v.r.WorkerFactory] Bolt enabled on 0.0.0.0:7687.
2016-12-18 15:27:05.457+0000 INFO  [o.n.k.i.DiagnosticsManager] Kernel version: 3.1.0,16a782b42d76ca37db72958eb2565cf6aa671a29

obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7474
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080
tcp6       0      0  ::1.8080               ::1.51243              ESTABLISHED 407402 146808  36204      0
tcp6       0      0  ::1.51243              ::1.8080               ESTABLISHED 407624 146808  24134      0
tcp4       0      0  127.0.0.1.8080         127.0.0.1.51236        ESTABLISHED 408104 146988  36204      0
tcp4     994      0  127.0.0.1.51236        127.0.0.1.8080         ESTABLISHED 408128 146988  13649      0
tcp46      0      0  *.8080                 *.*                    LISTEN      131072 131072  36204      0
tcp4       0      0  127.0.0.1.8080         127.0.0.1.51231        TIME_WAIT   408300 146988  36188      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7687
tcp46      0      0  *.7687                 *.*                    LISTEN      131072 131072  36204      0

//I'll try 3.2.0-alpha02 when it comes up on maven central as well.

The KernelExtensionFactory impls contain only one Bolt subclass that could setup the 7687 and 7474 connectors. If I force the http connector as type BOLT - I register both of them (7474 is then a websocket port) - but this is not a solution.


Debugging the GraphDatabase Initialization I see () implementations of KernelExtensionFactory
    @Override
    public Lifecycle newInstance( KernelContext context, Dependencies dependencies ) throws Throwable

each KernelExtensionFactory implements newInstance() 
BoltKernelExtension
Lucene* (3 for indexing)
Metrics*
OnlineBackup*
QueryLogger*
ShellServer*
Udc*

connectors	ArrayList<E>  (id=197)	
	elementData	Object[10]  (id=208)	
		[0]	SocketTransport  (id=211)	
			address	ListenSocketAddress  (id=218)	
				hostname	"localhost" (id=278)	
				port	7687	
		[1]	SocketTransport  (id=212)	
			address	ListenSocketAddress  (id=235)	
				hostname	"localhost" (id=275)	
				port	7474	
public class BoltKernelExtension extends KernelExtensionFactory<BoltKernelExtension.Dependencies>

        if ( connectors.size() > 0 && !config.get( GraphDatabaseSettings.disconnected ) )
        {
            life.add( new NettyServer( scheduler.threadFactory( boltNetworkIO ), connectors ) );
            log.info( "Bolt Server extension loaded." );
            for ( ProtocolInitializer connector : connectors )
            {
                logService.getUserLog( WorkerFactory.class ).info( "Bolt enabled on %s.", connector.address() );

It looks like we need to be running the connector setup that occurs in NeoServer and its subclasses down to EnterpriseNeoServer where we setup the following - however this code only runs for non-embedded servers

            new Neo4jBrowserModule( webServer ),
      HttpConnector httpConnector = ClientConnectorSettings.httpConnector( config, ClientConnectorSettings.HttpConnector.Encryption.NONE )
              .orElseThrow( () ->
                      new IllegalArgumentException( "An HTTP connector must be configured to run the server" ) );
      httpListenAddress = config.get( httpConnector.listen_address );
      httpAdvertisedAddress = config.get( httpConnector.advertised_address );

I did run into an issue with SSL creation during OPTIONAL (the default) only when a certain logging parameter was set to DEBUG - I'll raise a separate issue on this.
The fix was to set tls_level on the indexed 0 bolt property not directly on bolt.tls_level

      <entry key="dbms.connector.bolt.tls_level" value="DISABLED" /> <!--  no effect - missing index 0 -->
        <entry key="dbms.logs.debug.level" value="DEBUG"/
causes

java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty-tcnative-osx-x86_64, netty-tcnative]
	at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:173)
	at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:380)
	at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:99)
	at io.netty.handler.ssl.SslContext.defaultProvider(SslContext.java:117)
	at io.netty.handler.ssl.SslContext.defaultServerProvider(SslContext.java:104)
	at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:404)
	at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:402)
	at org.neo4j.bolt.BoltKernelExtension.createSslContext(BoltKernelExtension.java:224)
	at org.neo4j.bolt.BoltKernelExtension.lambda$newInstance$0(BoltKernelExtension.java:182)

fixed with
                    config.put( bolt.encryption_level.name(), "DISABLED" );
which renders in jconsole as 
dbms.connector.0.bolt.tls_level

Just to verify our custom 4 level constructor override of GraphDatabaseService to load our custom HA listener is causing us connector deviation I ran a very basic embedded setup - same issue

        GraphDatabaseSettings.BoltConnector bolt = GraphDatabaseSettings.boltConnector( "0" );
    	GraphDatabaseService graphDb = new GraphDatabaseFactory()
        .newEmbeddedDatabaseBuilder( new File("/ec2-user/graph2.db" ))
        .setConfig(bolt.type, "BOLT" )
    	.setConfig( bolt.enabled, "true" )
    	.setConfig( bolt.address, "0.0.0.0:7687" )
    	.setConfig( bolt.encryption_level, "DISABLED" ) //dbms.connector.0.tls_level
    	.setConfig("dbms.connector.http.address","0.0.0.0:7474")
    	.setConfig("dbms.connector.http.enabled","true" )
    	.setConfig("dbms.connector.http.type", "HTTP" )
    	.setConfig("dbms.connector.http.tls_level", "DISABLED")
        .newGraphDatabase();

graphDb GraphDatabaseFacade (id=198)
community single [/ec2-user/graph2.db]

TODO:
I still have a couple experiments and debugging I can do to try to force 7474 port creation on the embedded instance. Sad because 7474 works so well on a standalone server via the sh command

[root@obrien bin]# ./neo4j console
Starting Neo4j.
2016-12-18 23:25:26.073+0000 INFO  No SSL certificate found, generating a self-signed certificate..
2016-12-18 23:25:26.499+0000 INFO  Starting...
2016-12-18 23:25:27.227+0000 INFO  Bolt enabled on localhost:7687.
2016-12-18 23:25:27.241+0000 INFO  Initiating metrics...
2016-12-18 23:25:30.054+0000 INFO  Started.
2016-12-18 23:25:30.239+0000 INFO  Mounted REST API at: /db/manage
2016-12-18 23:25:31.331+0000 INFO  Remote interface available at http://localhost:7474/

linked
https://github.com/neo4j/neo4j-browser/issues/342

Attaching small maven war eclipse project used to test
biometric.nbi.web-0.0.11-SNAPSHOT_eclipse_maven_src.zip

@ghost

This comment has been minimized.

ghost commented Dec 19, 2016

Anne's/J.F.'s workaround using 2 neo4j servers is good - I verified it on my setup - I get the same NPE as well.

Use the browser from a standalone to run queries against the embedded servers's bolt 7688 address without having to shutdown either graph.db

  1. graph1.db managed by embedded neo4j (nsp live db)
  2. graph2.db managed by standalone neo4j (app db - stub in the context of nsp)
  3. launch embedded neo4j on bolt 7688
    dbms.connector.bolt.address=127.0.0.1:7688
  4. launch standalone neo4j on bolt 7687 (default)
  5. in http://localhost:7474 point the bolt address to 7688 (didn't realize we could edit this greyed box)
  6. now queries from 7474 will run against the embedded graph1.db instead of the standalone graph2.db
    Going forward:
    Currently a pure embedded server cannot instantiate an http port (the code in in NeoServer) - we would need to revert back to the Wrapping class removed in 3.x
    No effect

{noformat}



{noformat}

Ideally a procedure to wrap the embedded server' graphDatabaseFacade so that 7474 can be exposed via NeoServer code would be good in the future

The jmx exception does not block querying the db but should be looked at


match(n) return(n);

2016-12-19 16:46:27.595+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [UnknownError]: Exception invoking method writeBufSize. See debug.log for more details, reference dcc80076-665c-4cde-be3b-041c9253647c.
2016-12-19 16:46:27.596+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [UnknownError]: Exception invoking method writeBufSize, reference dcc80076-665c-4cde-be3b-041c9253647c. Exception invoking method writeBufSize
javax.management.RuntimeOperationsException: Exception invoking method writeBufSize
	at org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute(BaseModelMBean.java:196)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
	at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:131)
	at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:119)

override_bolt_7687_to7688_run_browser_across_2_neo4j_servers_screenshot 2016-12-19 11 47 28

@ghost

This comment has been minimized.

ghost commented Dec 20, 2016

It turns out with a bit of help from the following post we can still bring up a wrapping NeoServer around our embedded mode Neo4j 3.1.0 server. This code however is not integrated yet with our HA listener overrides on the constructor tree we bootstrap in Spring - so is a POC.

With this NeoServer wrapper code - we don’t need the secondary workaround where we bring up a 2nd Neo4j server mode instance, bring up its’ 7474 browser and modify the bolt address (default 7687) in the config to point to the exposed (7688) embedded mode graph.db - without bringing either Neo4j server down as long as they point to different graph.db instances - in effect sharing the 7474 browser as a bolt client.

Disclaimer: I believe a proper architecture involves a clustered set of server mode bolt enabled Neo4j servers.
This undocumented/internal API is used as a temporary workaround to get the neo4j browser up on an embedded Neo4j server in 3.1.0 as it used to work under the WrappingNeoServerBootstrapper in 2.3 so we can continue to use in-jvm Dijkstra algos until we are able to rework/benchmark the same or greater traversal speed via a bolt enabled cluster.

FrontController/spring service bean starts graph

2016-12-20 01:28:04.186+0000 INFO  Starting...
2016-12-20 01:28:04.533+0000 INFO  Bolt enabled on localhost:7687.
2016-12-20 01:28:04.534+0000 INFO  Initiating metrics...
2016-12-20 01:28:06.091+0000 INFO  Started.
2016-12-20 01:28:07.009+0000 INFO  Remote interface available at http://localhost:7575/
<state>community single [/ec2-user/graph2.db] : community single [/ec2-user/graph2.db/data/databases/graph.db]</state>

after opening browser

    	// http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper
    	// Warning: internal/unsupported API - NeoServer wrappers are usually only run in  server mode - not embedded
        ServerBootstrapper serverBootstrapper = new EnterpriseBootstrapper();//CommunityBootstrapper();
        NeoServer directNeoServer = serverBootstrapper.getServer();
        int i = serverBootstrapper.start(new File("/ec2-user"), // will resolve to /ec2-user/data/databases/graph.db 
        		Optional.of(new File("/Users/michaelobrien/Documents/Neo4j/.neo4j.conf")), //  stub only
        				    	Pair.of("dbms.connector.http.address","0.0.0.0:7575"),
        				    	Pair.of("dbms.connector.http.enabled","true" ),
        				    	Pair.of("dbms.connector.http.type", "HTTP" ),
        				    	Pair.of("dbms.connector.http.tls_level", "DISABLED")
        				    	// BOLT not advised if we run an embedded Neo4j DB inside a NeoServer wrapper
        				    	// BOLT should be enabled if you run a standalone server's browser against this embedded bolt port
        				    	// WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using the wrong connection URL or temporary network problems. If you have enabled encryption, ensure your browser is configured to trust the certificate Neo4j is configured to use. WebSocket `readyState` is: 3
         				    	,Pair.of("dbms.connector.bolt.address","0.0.0.0:7688"),
        				    	Pair.of("dbms.connector.bolt.enabled","true" ),
        				    	Pair.of("dbms.connector.bolt.type", "HTTP" ),
        				    	Pair.of("dbms.connector.bolt.tls_level", "DISABLED")       				    	
        				);
        NeoServer neoServer = serverBootstrapper.getServer();
        GraphDatabaseService graph = neoServer.getDatabase().getGraph();  

Debugging we see that we get the proper Jetty server available usually only to server mode Neo4j not embedded mode.

AbstractNeoServer.<clinit>() line: 114	
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45	
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90	
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161	

CommunityNeoServer.<clinit>() line: 54	
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45	
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90	
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161	

Config.get(Setting<T>) line: 137	
EnterpriseNeoServer(AbstractNeoServer).<init>(Config, Database$Factory, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 159	
EnterpriseNeoServer(CommunityNeoServer).<init>(Config, Database$Factory, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 69	
EnterpriseNeoServer.<init>(Config, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 105	
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45	
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90	
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161	

Jetty9WebServer.<init>(LogProvider, Config) line: 135	
EnterpriseNeoServer(CommunityNeoServer).createWebServer() line: 89	
EnterpriseNeoServer.createWebServer() line: 131	
EnterpriseNeoServer(AbstractNeoServer).init() line: 181	
EnterpriseNeoServer(AbstractNeoServer).start() line: 196	
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91	
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161	

GraphDatabaseFacadeFactory.initFacade(File, Map<String,String>, Dependencies, GraphDatabaseFacade) line: 141	
EnterpriseGraphDatabase.<init>(File, Map<String,String>, Dependencies) line: 36	
EnterpriseNeoServer.lambda$static$1(Config, GraphDatabaseFacadeFactory$Dependencies) line: 90	
1274309106.newGraphDatabase(Config, GraphDatabaseFacadeFactory$Dependencies) line: not available	
LifecycleManagingDatabase.start() line: 89	
LifeSupport$LifecycleInstance.start() line: 433	
LifeSupport.start() line: 107	
EnterpriseNeoServer(AbstractNeoServer).start() line: 199	
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91	
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161	

Jetty9WebServer.loadJAXRSResource(SessionManager, String, JaxRsServletHolderFactory) line: 536	
Jetty9WebServer.loadJAXRSClasses(SessionManager, String) line: 522	
Jetty9WebServer.loadAllMounts() line: 409	
Jetty9WebServer.start() line: 171	
EnterpriseNeoServer(AbstractNeoServer).startWebServer() line: 326	
EnterpriseNeoServer(AbstractNeoServer).start() line: 218	
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91	
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161	

serverBootstrapper	EnterpriseBootstrapper  (id=299)	
	dependencies	GraphDatabaseDependencies  (id=347)	
	log	FormattedLog  (id=573)	
	server	EnterpriseNeoServer  (id=357)	
	serverAddress	"0.0.0.0:7575" (id=575)	
	shutdownHook	ServerBootstrapper$1  (id=577)	

neoServer	EnterpriseNeoServer  (id=357)	
….	
	webServer	Jetty9WebServer  (id=406)	

from NeoServer
graph	EnterpriseGraphDatabase  (id=417)	
	contextFactory	Neo4jTransactionalContextFactory  (id=1119)	
	defaultTransactionTimeout	0	
	indexManager	Suppliers$1  (id=1121)	
	nodeActions	StandardNodeActions  (id=1123)	
	relActions	StandardRelationshipActions  (id=1128)	
	schema	SchemaImpl  (id=1132)	
	spi	ClassicCoreSPI  (id=425)	

Without NeoServer
graphDb	GraphDatabaseFacade  (id=150)	
	contextFactory	Neo4jTransactionalContextFactory  (id=1182)	
	defaultTransactionTimeout	0	
	indexManager	Suppliers$1  (id=1183)	
	nodeActions	StandardNodeActions  (id=1184)	
	relActions	StandardRelationshipActions  (id=1208)	
	schema	SchemaImpl  (id=1219)	
	spi	ClassicCoreSPI  (id=160)	
2016-12-20 02:15:02.825+0000 INFO  Starting...
2016-12-20 02:15:03.304+0000 INFO  Bolt enabled on 0.0.0.0:7688.
2016-12-20 02:15:03.316+0000 INFO  Initiating metrics...
2016-12-20 02:15:06.232+0000 INFO  Started.
2016-12-20 02:15:06.412+0000 INFO  Mounted REST API at: /db/manage
2016-12-20 02:15:07.103+0000 INFO  Remote interface available at http://localhost:7575/
<state> : enterprise single [/ec2-user/data/databases/graph.db]</state>

settings and logs

cat /ec2-user/logs/debug.log
2016-12-20 02:15:03.225+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.http.enabled=true
2016-12-20 02:15:03.225+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.security.auth_enabled=true
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.type=HTTP
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] unsupported.dbms.directories.neo4j_home=/ec2-user
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.directories.import=import
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.tls_level=DISABLED
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.http.type=HTTP
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.logs.http.enabled=false
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] unsupported.dbms.edition=enterprise
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.address=0.0.0.0:7688
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.http.address=0.0.0.0:7575
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.enabled=true
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.http.tls_level=DISABLED
2016-12-20 02:15:03.304+0000 INFO  [o.n.b.v.r.WorkerFactory] Bolt Server extension loaded.
2016-12-20 02:15:03.304+0000 INFO  [o.n.b.v.r.WorkerFactory] Bolt enabled on 0.0.0.0:7688.
2016-12-20 02:15:04.687+0000 INFO  [o.n.k.i.DiagnosticsManager] Kernel version: 3.1.0,16a782b42d76ca37db72958eb2565cf6aa671a29

These errors are non-fatal related to JMX and can be fixed

2016-12-20 02:17:30.759+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [UnknownError]: Exception invoking method writeBufSize, reference 59659a03-461e-4532-9b98-7afe3a2e5704. Exception invoking method writeBufSize
javax.management.RuntimeOperationsException: Exception invoking method writeBufSize
	at org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute(BaseModelMBean.java:196)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
	at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:131)
Caused by: java.lang.NullPointerException
	at org.apache.tomcat.util.net.SocketProperties.getTxBufSize(SocketProperties.java:300)

7575 http and 7688 bolt ports

obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7575
tcp46      0      0  *.7575                 *.*                    LISTEN      131072 131072  49013      0
tcp4       0      0  127.0.0.1.7575         127.0.0.1.60685        TIME_WAIT   407296 146988  49013      0
tcp6       0      0  ::1.7575               ::1.60699              TIME_WAIT   407284 146808  49013      0
tcp6       0      0  ::1.7575               ::1.60700              TIME_WAIT   407284 146808  49013      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7688
tcp6       0      0  ::1.7688               ::1.60704              ESTABLISHED 406582 146808  49013      0
tcp6       0      0  ::1.60704              ::1.7688               ESTABLISHED 398196 146808  48165      0
tcp6       0      0  ::1.7688               ::1.60702              ESTABLISHED 406570 146808  49013      0
tcp6       0      0  ::1.60702              ::1.7688               ESTABLISHED 398185 146808  48165      0
tcp6       0      0  ::1.7688               ::1.60701              ESTABLISHED 407255 146808  49013      0
tcp6       0      0  ::1.60701              ::1.7688               ESTABLISHED 407628 146808  48165      0
tcp46      0      0  *.7688                 *.*                    LISTEN      131072 131072  49013      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080
tcp4       0      0  127.0.0.1.8080         127.0.0.1.60584        FIN_WAIT_2  408104 146988  49013      0
tcp4     994      0  127.0.0.1.60584        127.0.0.1.8080         CLOSE_WAIT  408128 146988  42992      0
tcp46      0      0  *.8080                 *.*                    LISTEN      131072 131072  49013      0
udp4       0      0  *.*                    *.*                                196724   9216  38080      0

This code however is not integrated yet with our HA listener overrides on the constructor tree we bootstrap in Spring - the following scrubbed/distilled mirror POC of our corporate production code will be adjusted in the next post.

public class ExtendedHighlyAvailableGraphDatabaseFactory extends HighlyAvailableGraphDatabaseFactory {
    private HaMonitor haMonitor;
    private GraphDatabaseSettings.BoltConnector bolt = GraphDatabaseSettings.boltConnector( "0" );
    @Override
     protected GraphDatabaseBuilder.DatabaseCreator createDatabaseCreator(
                final File storeDir, final GraphDatabaseFactoryState state)  {
            return new GraphDatabaseBuilder.DatabaseCreator() {
                @Override
                public GraphDatabaseService newDatabase( final Map<String, String> config ) {
                    HighlyAvailableGraphDatabase haDB = 
                    		new HighlyAvailableGraphDatabase( storeDir, config, state.databaseDependencies());
                    haMonitor.setDb(haDB);
                    HighAvailabilityMemberStateMachine memberStateMachine = 
                    		haDB.getDependencyResolver().resolveDependency(HighAvailabilityMemberStateMachine.class);
                    if ( memberStateMachine != null ) {
                        memberStateMachine.addHighAvailabilityMemberListener(haMonitor);
                    }
                    return haDB;
                } };}
@ghost

This comment has been minimized.

ghost commented Dec 21, 2016

We can get an HA embedded mode server up wrapped by a NeoServer - working out passing parameters with TypeSafe again still.

Dec 20, 2016 11:23:51 PM org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor <init>
2016-12-21 04:24:34.172+0000 INFO  Starting...
2016-12-21 04:25:02.323+0000 INFO  Write transactions to database disabled
2016-12-21 04:25:05.196+0000 INFO  Bolt enabled on 0.0.0.0:7688.
2016-12-21 04:25:05.314+0000 INFO  Initiating metrics...
2016-12-21 04:25:14.874+0000 INFO  Attempting to join cluster of [127.0.0.1:5001]
2016-12-21 04:25:16.883+0000 INFO  Could not join cluster of [127.0.0.1:5001]
2016-12-21 04:25:16.884+0000 INFO  Creating new cluster with name [neo4j.ha]...
2016-12-21 04:25:16.923+0000 INFO  Instance 1 (this server)  entered the cluster
2016-12-21 04:25:16.966+0000 INFO  Instance 1 (this server)  was elected as coordinator
2016-12-21 04:25:17.154+0000 INFO  I am 1, moving to master
2016-12-21 04:25:17.281+0000 INFO  Instance 1 (this server)  was elected as coordinator
2016-12-21 04:25:17.452+0000 INFO  I am 1, successfully moved to master
2016-12-21 04:25:17.506+0000 INFO  Instance 1 (this server)  is available as master at ha://127.0.0.1:6001?serverId=1 with StoreId{creationTime=1482199697648, randomId=7800059877674392627, storeVersion=15531981201765894, upgradeTime=1482199697648, upgradeId=1}
2016-12-21 04:25:25.306+0000 INFO  Database available for write transactions
2016-12-21 04:25:39.442+0000 INFO  Started.
2016-12-21 04:25:42.429+0000 INFO  Mounted REST API at: /db/manage
2016-12-21 04:26:02.890+0000 INFO  Remote interface available at http://localhost:7575/

haDB HighlyAvailableGraphDatabase (id=178)

@obriensystems

This comment has been minimized.

obriensystems commented Dec 26, 2016

example tomcat 8 war maven project for above POC
https://github.com/obrienlabs/nbi-neo4j-embedded-aws-war
in obrienlabs/nbi-neo4j-embedded-aws-war#1

Endpoints

Add a node to the embedded graph db
http://neo4j.ca-central-1.elasticbeanstalk.com/FrontController?action=graph
(bolt enabled) http browser - user:neo4j pass:password
http://neo4j.ca-central-1.elasticbeanstalk.com:7575/browser/

@obriensystems

This comment has been minimized.

obriensystems commented Dec 26, 2016

linking related (for the NPE)
#8548

@vvavepacket

This comment has been minimized.

vvavepacket commented May 17, 2017

are you guys able to access custom user procedures using BOLT? I have an embedded database remotely, and i can view the contents (view nodes) via bolt using neo4j browser (running on different server),, but it seems it cant find the procedures?

@developerworks

This comment has been minimized.

developerworks commented Feb 3, 2018

In latest version of neo4j embedded mode, The inner class GraphDatabaseSettings.BoltConnector was deprecated, you should use BoltConnector like this:

import lombok.extern.slf4j.Slf4j;
import org.neo4j.graphdb.*;
import org.neo4j.graphdb.factory.GraphDatabaseFactory;
import org.neo4j.graphdb.factory.GraphDatabaseSettings;
import org.neo4j.graphdb.index.Index;
import org.neo4j.kernel.api.exceptions.KernelException;
import org.neo4j.kernel.configuration.BoltConnector;
import org.neo4j.kernel.configuration.Settings;
import org.neo4j.kernel.impl.proc.Procedures;
import org.neo4j.kernel.internal.GraphDatabaseAPI;
import org.neo4j.shell.ShellLobby;
import org.neo4j.shell.ShellServer;
import org.neo4j.shell.ShellSettings;
import org.neo4j.shell.kernel.GraphDatabaseShellServer;

import java.io.BufferedReader;
import java.io.File;
import java.io.InputStreamReader;
import java.nio.charset.StandardCharsets;
import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.*;

...
...
...
BoltConnector bolt = new BoltConnector("0");
Path path = Paths.get("conf/neo4j.conf");

graphDb = new GraphDatabaseFactory()
    .newEmbeddedDatabaseBuilder(DATABASE_DIRECTORY)
    .setConfig(GraphDatabaseSettings.pagecache_memory, "512M")
    .setConfig(GraphDatabaseSettings.string_block_size, "60")
    .setConfig(GraphDatabaseSettings.array_block_size, "300")
    .setConfig(GraphDatabaseSettings.store_internal_log_level, "DEBUG")
    .loadPropertiesFromFile(path.toAbsolutePath().toString())
    .setConfig(bolt.enabled, "true")
    .setConfig(bolt.type, "BOLT")
    .setConfig(bolt.listen_address, "0.0.0.0:7687")
    .setConfig(bolt.encryption_level, BoltConnector.EncryptionLevel.OPTIONAL.toString())
    .setConfig(ShellSettings.remote_shell_enabled, Settings.TRUE)
    .newGraphDatabase();
...
...
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment