Support mapping of named paths. [DATAGRAPH-1437]

[spring-projects-issues]

Sledge Hammer opened DATAGRAPH-1437 and commented

Using the Movies data set, in previous versions, I had a Movie repo as:

@Query("MATCH path=shortestPath((p1:Person)-[*0..15]-(p2:Person)) WHERE toLower(p1.name) = toLower($from) AND toLower(p2.name) = toLower($to) " +
        "RETURN relationships(path) AS relationships, nodes(path) AS nodes")
public List<Map<String, Object>> findShortestPath(String from, String to);

In the current version, this throws an exception in SingleValueMappingFunction:

Records with more than one value cannot be converted without a mapper.

No documentation on how to implement this in 6.0. I tried the following:

1. consume a bean of Driver and do a query. This resulted in an exception:

		Result result = driver.session().run("MATCH path=shortestPath((p1:Person)-[*0..15]-(p2:Person)) WHERE toLower(p1.name) = toLower('Al Pacino') AND toLower(p2.name) = toLower('Kevin Bacon') " +
				   "RETURN relationships(path) AS relationships, nodes(path) AS nodes");

		List<Record> rows = result.list();

org.neo4j.driver.exceptions.ClientException: Database access is not allowed for user 'movies' with roles [movies_reader].
	at org.neo4j.driver.internal.util.Futures.blockingGet(Futures.java:143)
	at org.neo4j.driver.internal.InternalResult.blockingGet(InternalResult.java:128)
	at org.neo4j.driver.internal.InternalResult.list(InternalResult.java:105)

2. tried with a bean of Session, no bean found
3. tried with a bean of Neo4jTemplate, same exception as Spaces after header names in template.mf #1
4. tried with a bean of Neo4jClient, same exception as Spaces after header names in template.mf #1
5. I was able to finally get it, at least executing as:

		QueryRunner queryRunner = Neo4jTransactionManager.retrieveTransaction(driver, "movies");
		
		if (queryRunner == null) {
			queryRunner = driver.session(Neo4jTransactionUtils.defaultSessionConfig("movies"));
		}

		Result result = queryRunner.run("MATCH path=shortestPath((p1:Person)-[*0..15]-(p2:Person)) WHERE toLower(p1.name) = toLower('kevin bacon') AND toLower(p2.name) = toLower('al pacino') " +
				   "RETURN relationships(path) AS relationships, nodes(path) AS nodes");

So issue #1, it seems like methods #1, 3 & 4 are not carrying the credentials properly?

Issue #2 is this result is very low level, is there a higher level abstraction for path queries as there was in previous versions?

Re the permissions, my config class is as followings and entity queries through a Neo4jRepository with custom queries are not having the permissions issue:

@Configuration
@EnableNeo4jRepositories(basePackages="org.xxx.yyy.movies.repositories",
						 transactionManagerRef="moviesTransactionManager")
public class MoviesDataSourceConfig {

	private final Environment environment;

	@Autowired
	public MoviesDataSourceConfig(Environment environment) {
		this.environment = environment;
	}

	@Bean
	public Driver moviesDriver() {
		return GraphDatabase.driver(Preconditions.checkNotNull(this.environment.getProperty("org.neo4j.driver.movies.uri")),
									AuthTokens.basic(Preconditions.checkNotNull(this.environment.getProperty("org.neo4j.driver.movies.authentication.username")),
													 Preconditions.checkNotNull(this.environment.getProperty("org.neo4j.driver.movies.authentication.password"))));
	}

	@Bean
	protected DatabaseSelectionProvider moviesDatabaseSelectionProvider() {
		return DatabaseSelectionProvider.createStaticDatabaseSelectionProvider("movies");
	}

	@Bean
	public Neo4jTransactionManager moviesTransactionManager(@Qualifier("moviesDriver") Driver driver,
															@Qualifier("moviesDatabaseSelectionProvider") DatabaseSelectionProvider databaseSelectionProvider)
	{
		return new Neo4jTransactionManager(driver, databaseSelectionProvider);
	}
}

---

Affects: 6.0.1 (2020.0.1)

Issue Links:

• DATAGRAPH-1441 Register necessary callbacks internally inside the mapping context

Backported to: 6.0.2 (2020.0.2)

[spring-projects-issues]

Michael Simons commented

1. SDN 6 does currently not support path mapping.
2. There is no session bean. The OGM session is just not there, the driver session is a short lived object that is not supposed to be kept around.
3. Regarding your points 3 and 4 of missing beans: It is related to your configuration. Why are you running this your own? Spring Boot provides dedicated support for those beans (literally all of which you mention). 

In any way: The repositories use the underlying template anyway. So the behavior must be the same. The correct way to jump into this is at the level of the Neo4jClient (integrated with Spring transactions) or the pure driver. 

The reason for the "missing" credentials: I cannot tell from your config which driver you get hold of the first place, it might be the one you configured, it might be Springs. 

In any case: You requesting a database session from it that points to the default database to which apparently your user doesn't have access to.

If you run it through the transaction manager and specify the name of the database, you get a query runner tied to the database.

 

You can remove all of your custom config and replace it with this:

 

spring.neo4j.uri=bolt://localhost:7687
spring.neo4j.authentication.username=neo4j
spring.neo4j.authentication.password=secret
spring.data.neo4j.database=movies

This will at least fix your bean issues.

Regarding the path mapping: What would you expect us to map? A list of entities that are on the path? That's absolutely something to consider and I would appreciate your feedback here.

[spring-projects-issues]

Sledge Hammer commented

Hi Michael,

My config class is posted up above (MoviesDataSourceConfig). The reason I am doing it that way is actually another issue I'm having after the migration :). Consider that I have two Neo databases on the same server: Movies and Fitness. In a typical enterprise environment, you'll have different sets of credentials for each database, so I have a "movies" user/pw and a "fitness" user/pw. The Neo4jRepository's DO work. Although I haven't gotten multiple databases working yet.

How would you suggest setting up multiple DBs in the same app with different creds?

Re: path mapping... using the Movies database and the query I posted as an example, I think I would expect a list of "relationship objects" that contains a reference to the Person and Movie for that leg or connection or whatever you want to call it.

Not so clear cut in 6 since you redid the annotations and now a relationship object can only contain one entity node.

[spring-projects-issues]

Michael Simons commented

Thanks for getting back!

Are we speaking 

Multiple databases as in "Multiple Neo4j instances"? Or

Multiple databases as in "One neo4j instance with multiple database"? If it's the later, you might do have different credentials for different users, but maybe not.

If it is the later, you might want to have a look here https://medium.com/neo4j/reactive-multi-tenancy-with-neo4j-4-0-and-sdn-rx-d8ae0754c35 or respectively here https://docs.spring.io/spring-data/neo4j/docs/current/reference/html/#faq.multidatabase

The former is a user case idea, back then based on SDN/RX (the prototype).

Having a closer look at your config, I guess you are speaking of multiple Neo4js. That's fine too and from that perspective, your config makes sense.

Regarding the missing credentials I just can assume that you might have grabbed the wrong driver bean then… 

If you can share the whole setup that would be very helpful

[spring-projects-issues]

Sledge Hammer commented

Hi Michael,

I have a single instance of Neo4j with multiple databases:

1. Movies - user: movies, pass:xxx
2. Fitness - user: fitness, pass:yyy

So 2 databases pointing to localhost:7687. Each database has a different username / password.

In my code, (at least in the old version), I have 2 config classes:

MoviesDataSourceConfig -> basePackages="org.xxx.yyy.movies.repositories"
FitnessDataSourceConfig ->
basePackages="org.xxx.yyy.fitness.repositories"

If I try that in 6 as I have the config classes above (picture an exact clone of that one for the FitnessDataSourceConfig except with the base package changed), I get an exception on startup saying that there were multiple driver beans found.

Note, the app doesn't run under the credentials, that's just for the Neo user, so my properties is something like:

1. Movies configuration settings

org.neo4j.driver.movies.uri=bolt://localhost:7687
org.neo4j.driver.movies.authentication.username=movies
org.neo4j.driver.movies.authentication.password=xxx

1. Fitness configuration settings

org.neo4j.driver.fitness.uri=bolt://localhost:7687
org.neo4j.driver.fitness.authentication.username=fitness
org.neo4j.driver.fitness.authentication.password=yyy

And my 2 config classes read those properties when they new up the driver as seen above.

The Medium sample you linked, that one uses the Spring credentials to hit the DB where that isn't the case in my app. The Spring user is completely unrelated to the Neo4j users. The Neo4j users are internal to the app to connect to Neo

[spring-projects-issues]

Michael Simons commented

Have a look here, please: https://github.com/michael-simons/neo4j-examples-and-tips/tree/master/examples/sdn6-multidb-multi-connections

Especially this: https://github.com/michael-simons/neo4j-examples-and-tips/blob/master/examples/sdn6-multidb-multi-connections/src/main/java/org/neo4j/tips/sdn/sdn6multidbmulticonnections/Neo4jPropertiesConfig.java
This https://github.com/michael-simons/neo4j-examples-and-tips/blob/master/examples/sdn6-multidb-multi-connections/src/main/java/org/neo4j/tips/sdn/sdn6multidbmulticonnections/movies/MoviesConfig.java
(and it's pendant for fitness)

As well as the usage:

https://github.com/michael-simons/neo4j-examples-and-tips/blob/master/examples/sdn6-multidb-multi-connections/src/main/java/org/neo4j/tips/sdn/sdn6multidbmulticonnections/Example.java

You could spare some effort if you have only one connection (driver instance) with one user that has rights in both databases…

[spring-projects-issues]

Sledge Hammer commented

Yeah, enterprise level security though :). Minimum rights principal. Samples look promising. I'll try it out! Thanks!

[spring-projects-issues]

Michael Simons commented

Hey :) I updated my example and removed some noise. I also use the data properties now for the database name.
I discovered one issue with SDN 6 which I am gonna fix next.
After that I'll jump into the path mapping

[spring-projects-issues]

Michael Simons commented

We will be able to map the paths in the next release.

Here's how it's going to look like: https://github.com/spring-projects/spring-data-neo4j/blob/issue/pathsupport/src/test/java/org/springframework/data/neo4j/integration/movies/AdvancedMappingIT.java

There will be some prose coming up.
Basically you will be able to map all entities of a given type a long the path, plus filling relationships that are on the path and if you need more, you can aggregate it to it

[spring-projects-issues]

Michael Simons commented

This is now ready for the next release: 9ac3b99