Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ActiveRel: a relationship wrapper #393

Closed
subvertallchris opened this issue Jul 28, 2014 · 45 comments
Closed

ActiveRel: a relationship wrapper #393

subvertallchris opened this issue Jul 28, 2014 · 45 comments
Labels
Milestone

Comments

@subvertallchris
Copy link
Contributor

Since the great debate over the has_many API has winded down, I think the time is right for another divisive discussion: relationship wrapping.

Check out the 'activerel' branch. 53688e8 It is built upon the most recent commits to the query_experiment (prior to tonight's updates) and uuid branches and its gemfile is pointed to my fork of neo4j-core to get some changes that make it possible. It's less finished than I'd like but since I'm leaving in the morning to go away until Friday, I wanted to share what I have to get discussion started.

ActiveRel turns relationships into first-class citizens. This is important because Neo4j makes it clear that relationships are not just the by-products of an association, they are objects and they deserve the same love and respect as nodes. Being able to create/destroy/validate from a dedicated relationship classes instead of objects on either end of the relationship makes sense.

It makes sense from an OOP perspective, too. It answers the question, "If you have a relationship between Show and Band, which model should be responsible for the properties and validations of the relationship?" Rails would say either Show or Band, but that doesn't seem true to me with Neo4j, so ActiveRel answers with a third class that can be responsible for the relationship itself. (Obviously, simple associations can and should be handled by the node models, but this adds an option for heavy lifting with highly relational data.)

So that's what I've attempted to do. With ActiveRel, you can do something like this:

class Band
    include Neo4j::ActiveNode  
    property :name
    property :description
    property :formed, type: Integer

    has_many :outbound, :shows, type: 'playing_shows', rel_class: ShowBand
end

class Show
    include Neo4j::ActiveNode
  property :title,      type: String
  property :date,       type: DateTime
  property :cost
  property :created_at  
  property :updated_at

    has_many :inbound, :bands, type: 'playing_shows', rel_class: ShowBand
end

class ShowBand
  include Neo4j::ActiveRel
  before_save :do_this_thing
  validates_presence_of :slot

  outbound_class Band
  inbound_class  Show

  rel_type 'playing_shows'

  property :slot
  property :custom_url
  property :custom_description

  def do_this_thing
    #some special behavior
  end
end

show = Show.create(title: 'Emperor in NYC')
band = Band.create(name: 'Emperor')

playing_show = ShowBand.new
playing_show.inbound = show
playing_show.outbound = band
playing_show.custom_description = "Classic Norwegian Black Metal"
playing_show.custom_url = "emperorhorde.com"
playing_show.save
# false

playing_show.errors.full_messages
#  => ["Slot can't be blank"] 

Because relationships can't use labels, it sticks a _classname property on relationships when defined and the wrapping process looks for this.

You can use the class method find, access it from a node using rels, use Neo4j::Relationship.load with an ID if you have one, or anything else that calls Neo4j::Relationship.load at some point.

Neo4j::Relationship.load(5361)
#  => #<ShowBand custom_description: "Classic Norwegian Black Metal", custom_url: "emperorhorde.com", slot: 1> 
ShowBand.find(5361)
# => #<ShowBand custom_description: "Classic Norwegian Black Metal", custom_url: "emperorhorde.com", slot: 1> 
show.rels
 #=> [#<ShowBand custom_description: "Classic Norwegian Black Metal", custom_url: "emperorhorde.com", slot: 1>]

The most divisive part of what I've done might be the reorganization of ActiveNode files. When I started this, I found that ActiveNode was really built with only nodes in mind. The first draft of ActiveRel just included ActiveNode and defined its own methods when those included contained node-specific behavior, but this led to three problems:

  • Mess. active_rel.rb was a grab-bag of methods and it seemed guaranteed to eventually be a huge, sprawling mess that would eventually require being split up into separate modules.
  • Unexpected dependencies. When I see ActiveNode, I think that the implication is really that it deals with nodes, not nodes and relationships. By making ActiveRel dependent on ActiveNode, code that was written for nodes and intended for use by nodes was suddenly responsible for relationships as well, and I felt that lack of an explicit declaration of a method as shared behavior did a disservice to anyone trying to maintain the code in the future. This makes it clear when a method is shared and when it's specific to one type of object.
  • Access to inappropriate methods. For all the shared behavior, there are a lot of methods ActiveRel objects should not have, either because they require heavy reworking for ActiveRel or they're totally inappropriate. Rather than working backwards, overriding those methods, it just seemed better to add in methods, either by generalizing shared behavior or building relationship-specific methods, as we go.

The result of the reorganization is Neo4j::Library and neo4j/library/, which contains modules of methods extracted from Neo4j::ActiveNode modules that apply to both relationships and nodes. This is a work in progress, just like the rest of this.

Right now, what I've done is both very unfinished but very usable, depending on what you want from it. It has no specs at the moment, I ran out of time. The only failing specs other than those related to the need for an updated has_one method is before_validation, which just does not want to work and I am way too tired to figure out what the story is with that now. (I did fix a bunch of failing specs caused by the new has_many method along the way!)

There's a lot to do other than specs, so think of this as a working model. In my opinion, the highest priority issues after specs:

  • If it is to be implemented, I think that has_many/has_one should delegate to the ActiveRel class when a rel_class in specified, but I haven't thought through exactly how this would work. (Would you still need to declare direction, type, classes, etc,... in the has_many call? Things like that.) For the moment, all including the rel_type option in has_many does is tag it during creation so it will be wrapped if you return it in the future.
  • It requires relationships to be between the declared inbound and outbound classes and does not yet support :all.
  • Persistence needs to be modified so you can't change the inbound or outbound nodes once a relationship is created.

Things that do work:

  • wrapping of relationships with _classname property matching existing class constant
  • class methods create and find
  • instance methods id, neo_id, inbound, outbound, destroy, save
  • callbacks and validations other than before_validation

Again, sorry to be sharing something with no specs, but I wanted to show where I am and get the ball rolling since I expect to be iPhone-only until Friday and the more time that passes, the harder it will be to merge the changes into master.

@andreasronge
Copy link
Member

Great. Will have a look at this later this week. I would like to explore alternatives as well, for example: maybe without using classname property and instead use labels of incoming and outgoing nodes for a relationship to decide which relationship ruby class it should be.

@andreasronge
Copy link
Member

What do you think about renaming Neo4j::Library to Neo4j::ActiveEntity. I think that in the neo4j source code they name things that are either a node or relationship an entity. Library feels a bit too general.

@subvertallchris
Copy link
Contributor Author

That sounds good! I was just looking for something generic to get the ball
rolling.

On Wednesday, July 30, 2014, Andreas Ronge notifications@github.com wrote:

What do you think about renaming Neo4j::Library to Neo4j::ActiveEntity. I
think that in the neo4j source code they name things that are either a node
or relationship an entity. Library feels a bit too general.


Reply to this email directly or view it on GitHub
#393 (comment).

@subvertallchris
Copy link
Contributor Author

Also, I'm not really married to much of what was done in there , I just
wanted to have something shareable to use as a starting point before
leaving on my trip.

If we can find a good way to use the related nodes to determine class, that
could be cool. Maybe some kind of naming scheme similar to how Rails
expects models to match table and controller names, only we use the
relationship names? We should think carefully about it, there's a lot to be
said for the directness and simplicity of "_classname", even if it isn't
very sophisticated.

On Wednesday, July 30, 2014, Chris Grigg chris@subvertallmedia.com wrote:

That sounds good! I was just looking for something generic to get the ball
rolling.

On Wednesday, July 30, 2014, Andreas Ronge <notifications@github.com
javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

What do you think about renaming Neo4j::Library to Neo4j::ActiveEntity.
I think that in the neo4j source code they name things that are either a
node or relationship an entity. Library feels a bit too general.


Reply to this email directly or view it on GitHub
#393 (comment).

@subvertallchris
Copy link
Contributor Author

I think that this should wait until the query_experiment branch is merged in and we can tackle it as a team, since there's so much to cover between specs, moving code around, and figuring out exactly how we want it to behave and interact with what's already there. What do you think? @cheerfulstoic, what about you?

@andreasronge
Copy link
Member

I agree, we should wait until query_experiment is in master.

@subvertallchris
Copy link
Contributor Author

I just pushed a hideous where method to this branch for demonstration purposes. Since you can't query for relationships directly, you need to match based on the declared inbound and outbound node labels and relationship type. Core Query needs to be modified to use Neo4j::Relationship.load so it wraps correctly, but I'm feeding the results into Neo4j::Relationships.load in the meantime so it will work.

24bc3a2#diff-149d2ed2cc215f0946334b0d9ea975c5R8

I don't think it will be necessary to worry about finders on ActiveRel too much but having some methods to generate cypher matches will be nice. The preferred method of finding a relationship is to query for the node, then work with its rels. Otherwise, the big benefit is being able to create from ActiveRel classes and take advantage of validations and callbacks.

@buildc0de
Copy link

@subvertallchris To answer your question in #401, I sure understand re: specs and work 😈 and I have, indeed, played with the existing branch. I am trying it out in my current project where relationships are used to record financial transactions...

@subvertallchris
Copy link
Contributor Author

I just pushed a ton of changes to this branch. They include a ton of specs, support for all the basic persistence features you'd expect, fixed the broken before_validation callback. There are also updates on neo4j-core so Gemfile is now pointed at my dev branch. Still a few specs that should be added, specifically testing for failures, but I'm way too tired to keep going tonight.

Also haven't checked this at all in JRuby, so expect nothing to work correctly in Embedded mode yet. Failing specs are result of missing changes from query_experiment, which I'm not going to bother merging until it's all done.

Haven't changed Library to ActiveEntity or whatever it should be, will worry about that later.

@subvertallchris
Copy link
Contributor Author

Among the nicer changes, it lazily loads related nodes if you pull up a rel using something like bands.rels.first. Use inbound and outbound instance methods to access those nodes, but they won't be called until you need them. This prevents extra queries if you just want to load the relationship to work with its properties. You can also initialize new instances of an ActiveRel class and pass inbound and outbound along with nodes:

class MyRel
  include Neo4j::ActiveRel
  outbound_class Class2
  inbound_class Class1

  property :this_thing
end

MyRel.create(outbound: node1, inbound: node2, this_thing: 'words here')
# or
r = MyRel.new(outbound: node1, inbound: node2, this_thing: 'words_here')
r.save

Class method where should be working but it's still pretty crude and not really the focus of this. It'll need some more attention.

All callbacks and validations are working on these classes.

@andreasronge
Copy link
Member

This is really nice. Just one minor thing: I would prefer using start_node and end_node instead of outbound_class and inbound_class. This is also what the neo4j docs uses (
http://api.neo4j.org/1.5/org/neo4j/graphdb/Relationship.html) and is also used in neo4j-core.

@cheerfulstoic
Copy link
Contributor

Just got a chance to read through the thread. Seems really good overall. Haven't looked at the code yet.

I was also going to mention the start / end thing. Maybe just start and end because the argument isn't a node but rather a class / model? Will users be able to specify that any class / label is allowed as the start / end?

Also, maybe use type instead of rel_type for consistency with new has_many syntax.

@subvertallchris
Copy link
Contributor Author

Would you guys be ok with just from_class/to_class and from/to? It reads like English, feels like the Ruby way to do things.

class MyRel
  include Neo4j::ActiveNode
  from_class Show
  to_class Band
end

r = MyRel.new
r.from = show
r.to = band
r.save

Good idea on type, I'll definitely tweak that. rel_type is redundant anyway, we already know we're working with rels.

@subvertallchris
Copy link
Contributor Author

Changed rel_type to type, didn't do anything with inbound/outbound/etc,... yet.

@cheerfulstoic, when things settle down a little for you, I'd love your help making query a bit more usable. I threw something together that works unless the outbound class in the model is set to :any but I'm sure there's a better way to handle it.

@andreasronge
Copy link
Member

Nice that you changed to type and from and to, I like it. However, I'm still a bit worried that the API is not consistent since I think there are users that will use both the neo4j-core api and neo4j.

Btw the new and create method can't accept just one hash argument, e.g. MyRel.new(from: node1, to: node2, this_thing: 'words_here') since there will be a clash if there is a property called from or to.

@subvertallchris
Copy link
Contributor Author

Didn't change from and to yet cause I knew there would be more discussion. ;-)

How about this: class methods are from_class/to_class, instance methods are from_node/to_node, BUT they are all aliased so someone can also use start_[class|node]/end_[class|node] if they prefer. I see what you're saying about keeping the API consistent with core but I think it's a bit less intuitive for Rails users, so this will give everyone exactly what they want.

class MyRel
  include Neo4j::ActiveRel

  from_class   MyStartClass
  to_class     MyEndClass
  #or!
  start_class  MyStartClass
  end_class    MyEndClass
end

MyRel.new(from_node: node1, to_node: node2)
#or
MyRel.new(start_node: node1, end_node: node2)

This will let everyone use the methods that fit their workflow.

@subvertallchris
Copy link
Contributor Author

You can see it at 855a2db

@cheerfulstoic
Copy link
Contributor

@subvertallchris We've just gotten to the cottage where we'll be spending the next ~4 weeks (aside from the few days I'll be in Sweden, of course ;) The wi-fi's not great, so lay down your questions/ideas and I'll see what I can do to help (either here or in personal E-Mail)

@subvertallchris
Copy link
Contributor Author

The big thing would really be to just take a quick look at https://github.com/andreasronge/neo4j/blob/2e04822f277e48974e9deaa48b9c924778dafcdd/lib/neo4j/active_rel/query.rb. I'm looking at the outbound class and using query_as relative to that, or I'm doing Neo4j::Session.query if outbound class is :all. It all works, I just don't love it.

@subvertallchris
Copy link
Contributor Author

Beyond that, do you think we're ready to merge query_experiment in now that callbacks are implemented?

@subvertallchris subvertallchris added this to the 3.0.0 milestone Aug 13, 2014
@cheerfulstoic
Copy link
Contributor

I'm looking at the code now (might have to go if Theo wakes), but I'm def down with merging query_experiment

@subvertallchris
Copy link
Contributor Author

@andreasronge
Copy link
Member

It's very much appreciated. Will have a look at it tonight.

@andreasronge
Copy link
Member

I run codeclimate on the branch (https://codeclimate.com/github/andreasronge/neo4j/compare/activerel)
I found it interesting. Not too many code smells.

@subvertallchris
Copy link
Contributor Author

I pride myself on having minimal smell.

On Friday, August 15, 2014, Andreas Ronge notifications@github.com wrote:

I run codeclimate on the branch (
https://codeclimate.com/github/andreasronge/neo4j/compare/activerel)
I found it interesting. Not too many code smells.


Reply to this email directly or view it on GitHub
#393 (comment).

@cheerfulstoic
Copy link
Contributor

That's good since we're going to be working together in person...

Brian
;p

On Friday, August 15, 2014, Chris Grigg notifications@github.com wrote:

I pride myself on having minimal smell.

On Friday, August 15, 2014, Andreas Ronge <notifications@github.com
javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

I run codeclimate on the branch (
https://codeclimate.com/github/andreasronge/neo4j/compare/activerel)
I found it interesting. Not too many code smells.


Reply to this email directly or view it on GitHub
#393 (comment).


Reply to this email directly or view it on GitHub
#393 (comment).

@subvertallchris
Copy link
Contributor Author

3.8 from Codeclimate!? Unacceptable! I'll fix that. ;-)

This does remind me of something, though: we might want to use this as more motivation to remove some e2e tests and create unit tests for Neo4j::Shared's methods. :cache_key, for example, doesn't have a unit test and only appears in ActiveNode's e2e, even though it is a shared method. I don't think we should delay merging into master to clear it up since coverage level is still high, but it's one more reason to move to unit.

@andreasronge
Copy link
Member

+1 for merging earlier to master as long it works: specs and a simple real rails app ( exempels/blog for example)

@subvertallchris
Copy link
Contributor Author

I'll look at your example app, should be able to modify that. There are already specs for all the new code, 100% coverage (EDIT: except active_rel.rb itself) cause I have no life.

@andreasronge
Copy link
Member

Great work !
Btw, Neo4j::ActiveRel::IdProperty does not make sense for ActiveRel.

@subvertallchris
Copy link
Contributor Author

Yeah, it'll come out. It's an empty module, just held over from when I had something or other in there.

@subvertallchris
Copy link
Contributor Author

Example app updated for ActiveRel: 39d476c
Just storing a timestamp in the rel plus notes.

Removed ActiveRel::IdProperty: 1551b1c

@andreasronge
Copy link
Member

Some comments/questions from the wiki page

property types are not required unless you want something other than String.

Is this really true ? I would think it gets whatever you assign the property to, but maybe active_attr converts every property value to string by default.

Pass the :rel_type option in a declared association with the constant of an ActiveRel model. When that relationship is created, it will add a hidden _classname property with that model's name.

Why does it add a _classname property if there a declared association with a rel_type which match an ActiveRel model ?
3.

What happens if two ActiveRel models have the same rel_type ?
4.

I still don't like ActiveRel#create using one parameter hash, e.g. EnrolledIn.create(from_node: student, to_node: lesson, since: 2014) since we mix properties with to_node and from_node. Should either be two hash properties or three arguments (node,node, hash).
5.

What happens if the rel_type does not match. Example:

class Student
  include Neo4j::ActiveNode
  has_many :out, :lessons, type: 'something_else_not_match_EnrolledIn_rel_type',  rel_class: EnrolledIn
end

@subvertallchris
Copy link
Contributor Author

  1. I always assumed that everything lacking a type was converted to string, but it looks like I was wrong, it handles fixnum just fine. Can edit that out, I was just making the point that properties in ActiveRel are the same as in ActiveNode.
  2. _classname lets the relationship classes become independent of the related objects. Relying on nodes to figure out the class requires an additional query to at least one of the nodes.
  3. I guess there's really no reason not to do (node, node, hash). I'll make that change.

As for the last question, until a moment ago, it would create the relationship and use whatever type was set in the model, so you were expected to list it twice, once in at least one ActiveNode model and again in ActiveRel, or you'd get mismatched types. Student.lessons << node1 would have type 'something_else_not_match_EnrolledIn_rel_type', EnrolledIn.create would have type 'enrolled_in'. This is dumb.

To fix it, I'm just finishing up some tests for a change that is going to make the model definitions even easier. If you specify :rel_class, you do not need to include :origin or :type. It will use the type provided by the class. This means that you can have shorter associations when using an ActiveRel class:

class EnrolledIn
  include Neo4j::ActiveRel
  from_class Student
  to_class Lesson
  type 'enrolled_in'
end

class Student
  include Neo4j::ActiveNode
  # has_many :out, :lessons, type: 'enrolled_in', rel_class: EnrolledIn
  has_many :out, :lessons, rel_class: EnrolledIn
end

class Lesson
  include Neo4j::ActiveNode
  # has_many :in, :students, origin: :lessons, rel_class: EnrolledIn
  has_many :in, :students, rel_class: EnrolledIn
end

And you only have to define the type once in the ActiveRel model! Callbacks and Validations still won't run if you do student.lessons << node, you'll need to do EnrolledIn.create(student, lesson) for that to happen.

I'll have this change pushed in a few minutes.

@subvertallchris
Copy link
Contributor Author

I actually don't think I can do (node, node, props) without making significant changes to other code. What about a constant of illegal property names to prevent conflicts? There should probably be one of those anyway.

@andreasronge
Copy link
Member

Ok, keep it as it is now, but add some validation, e.g. declaring a property with name from_node should raise an exception.

I really like your example where you don't have to specify origin,type since that is already given in the ActiveRel class. Nice !

I guess you need to raise an exception if the relationship type does not match, the declared type on ActiveNode class and the declared type on ActiveRel class.

Btw, what happens if I add a node of wrong class to an association ? An exception is raised ?

@subvertallchris
Copy link
Contributor Author

Cool, it'll be done in a few minutes. If anyone else has a property they want to prevent, they can just add it into the ILLEGAL_PROPS array in Shared::Property. Super easy.

Yeah, I like how much cleaner things look when you move the type to the ActiveRel class! It raises an exception if you even pass the type or origin option along with rel_class.

If you add a node of the wrong class, an exception is raised, but this is still controlled by the ActiveNode association if you're creating from an ActiveNode model. This might seem strange but bear with me here.

(This is really long, sorry.)

# here's where it causes a problem
class Student
  include Neo4j::ActiveNode
  has_many :out, :lessons, rel_class: EnrolledIn
end

class Lesson
  include Neo4j::ActiveNode
  has_many :in, :students, rel_class: EnrolledIn
end

class EnrolledIn
  include Neo4j::ActiveRel
  from_class Student
  to_class Band #obviously not what we want
end

student = Student.first
lesson = Lesson.first
band = Band.first

student.lessons << band
# raises exception because Student is excepting object of class Lesson, it does not look at EnrolledIn for this

EnrolledIn.create(from_node: student, to_node: band)
# does NOT raise an exception, this matches the model's definition

This is an example of when it goes bad but the benefits really outweigh the negatives. You can reuse an ActiveRel model for multiple ActiveNode models by using :any.

class LikesRel
  include Neo4j::ActiveRel
  from_class User
  to_class :any

  type 'likes_thing'

  property :my_prop
  property :another_prop

  validate :likeable_node

  def likeable_node
    errors.add(:to_node, 'destination object cannot be liked') unless to_node.respond_to?(:liked_by)
  end
end

class User
  include Neo4j::ActiveNode
  has_many :out, :favorite_bands, model_class: Band, rel_class: LikesRel
  has_many :out, :favorite_foods, model_class: Food, rel_class: LikesRel
  has_many :out, :liked_things, model_class: :any, rel_class: LikesRel
end

LikesRel.new(from_node: user, to_node: Nickelback.create).valid?
# => false
# cause nobody likes Nickelback... get it? ;-)

Associations build cypher queries with labels, so you can be sure that user.favorite_bands and user.favorite_foods will give you different results, but you can centralize type and control behavior by using ActiveRel.

@subvertallchris
Copy link
Contributor Author

I think we're good to merge this into master if you're into it.

@subvertallchris
Copy link
Contributor Author

Awesome! Want to do a new release?

@andreasronge
Copy link
Member

@subvertallchris Yes, why not. Do you have an account on rubygems.org ?
Maybe you can do it yourself after I give you access to releasing the gem ?
Just edit the CHANGELOG, update version, test with a rails app and then release, rake release

@subvertallchris
Copy link
Contributor Author

Sounds good! I just registered on rubygems as subvertallchris.

@andreasronge
Copy link
Member

Great, I will give you access. Btw I've already released v3.0.0.alpha.10

@subvertallchris
Copy link
Contributor Author

Thank you! Could you give me access to neo4j-core on github?

@andreasronge
Copy link
Member

Sorry, I thought you already had access to neo4j-core.
You now have access to released both gems: https://rubygems.org/profiles/subvertallchris

@subvertallchris
Copy link
Contributor Author

Perfect, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants