Deprecate :include-refsets? in favor of :include-reflikes?

[clyfe]

Naming of this config option is a bit misleading, it's called :include-refsets? but what it actually does is :include-reflikes?, and it's misleading particularly post #68

[clyfe]

Also, find-keys should default to :include-refsets? true because a bunch of keys are left out on exec. Sample:

(def config
  {:some/handler 1
   :some/other 2
   :jetty/server (ig/refset :handler)})

(def system (ig/init [:jetty/server]))

;; => init ... build ... dependent-keys ... find-keys 
;; => just :jetty/server ends up in the map

[weavejester]

Naming of this config option is a bit misleading, it's called :include-refsets? but what it actually does is :include-reflikes?

Yes, that looks like a mistake. Let me think of the best way to handle this... perhaps via a :ref-filter option instead.

[weavejester]

Also, find-keys should default to :include-refsets? true because a bunch of keys are left out on exec.

No, that part is deliberate. If you have a configuration:

{::a (ig/ref ::b), ::b 1}

Then if you want to initiate ::a, you have to also initiate ::b, because a ref must connect to exactly 1 key. However:

{::a (ig/refset ::b), ::b 1}

Because a refset can be empty, it's perfectly valid to initiate ::a and not ::b, therefore if you want ::b it needs to be explicit.

From a practical perspective, this allows you to initiate an explicit subset of a refset.

[weavejester]

This is a larger problem than it seems at first consideration. How can we tell whether a reflike behaves like a ref or a refset, and would adding more to the RefLike protocol add more complexity than the functionality justifies? I think this needs to be given some further thought.

[clyfe]

I guess there are 2 possible behaviors: lazy/minimal or greedy/maximal.

Note, the (current) lazy/minimal one makes duct core's default (exec-config config profiles [:duct/daemon]) initializer not-very-useful, in a {:duct/daemon (ig/refset :my/handlers)} scenario - no handlers get loaded and the program doesn't do anything useful.

[weavejester]

The minimal strategy allows you to exclude keys from a refset, whereas a maximal strategy does not. With a maximal strategy you'd need to either lose functionality, or change how the keys are specified.

Now it's true that in that particular case, the Duct default isn't a good one. But that's a very specific use case, and I think if you're going to do something unusual, overriding the default to something like [:duct/daemon :duct/handler] isn't that burdensome a task.

Plus the way Duct is currently written, profiles and modules rely on the minimal strategy in order to define ordering. With a refset you can effectively say "if this key exists, ensure you come after it".

[weavejester]

I think we need a little time to ensure that reflikes work consistently, so what I'll do is to create a new reflike branch and revert #68 on master.

However, I will rename the :include-refsets? option to something like :optional-deps? in order to not tie it to refsets in particular. As I doubt many (if any) people are using this and Integrant is yet to hit 1.0, I'm okay with a breaking change in this case.

I'm thinking that we may want to have two additional protocol functions on reflike: ref-deps and ref-mandatory-deps, which take a collection of keys from the configuration, and return only the keys that the ref depends on.

[weavejester]

Whether we call the option :eager or :optional-deps?, we still need some way of determining which dependencies are optional for a reflike, and which dependencies are mandatory. In other words, we need two functions:

(ref-optional-deps reflike keys)
(ref-mandatory-deps reflike keys)

Or possibly:

(ref-all-deps reflike keys)
(ref-mandatory-deps reflike keys)

[clyfe]

Sorry, I deleted my prior comment because it was bonkers, before you replied.
Yes, we need to change RefLike as you mentioned.

The case I'm making now is for :include-refsets?/:include-reflikes?/:optional-deps? to be toggle-able from ig/init - down the call chain - up to dependency-graph; while at the same time keeping the functions signatures reasonable. At the moment this is my main concern.

[clyfe]

My thoughts come and go; to go back at my earlier (deleted) point (not all bonkers), better stated:

In the great scheme of things a reflike can:

• get precisely one thing (ref) off the map
• get zero or more - anything else (reflike, refmap, refwhatever, anything else but the above).

In light of this, optional/mandatory is a complication; whether this is a distinction that's needed, we have yet to have a concrete case.

So rename to :include-reflikes? and keep RefLike signature seems sufficient to me. (Plus toggle-able from ig/init as stated above.)

[clyfe]

Let me know on which variant you're sold and I'll sketch a PR.

[weavejester]

The case I'm making now is for :include-refsets?/:include-reflikes?/:optional-deps? to be toggle-able from ig/init - down the call chain - up to dependency-graph; while at the same time keeping the functions signatures reasonable. At the moment this is my main concern.

I don't think we need that. It's more straightforward to just be explicit about the keys you want. Automatically including dependencies was only ever intended to pull in mandatory dependencies; i.e. it automatically includes the minimal set of keys that conform to what the user requires.

So rename to :include-reflikes? and keep RefLike signature seems sufficient to me. (Plus toggle-able from ig/init as stated above.)

I don't think that's a good idea. Remember, anything we implement is going to be set in stone. We cannot remove options or behavior after Integrant hits 1.0, which ideally I'd like to do sometime this year.

This also sets up special, hardcoded behavior for refs over other RefLikes. Again, this doesn't strike me as good design; ideally I'd like RefLikes to be consistent in how they behave.

[clyfe]

I don't think we need that.

I arrived here wanting a way for (ig/init [:my/server]) to pull in all my :handlers.

It's more straightforward to just be explicit about the keys you want.

Say, handlers are in the hundreds or more, and added somewhat dynamically by a mechanism. Not quite straightforward, nor do I like it.

It's ergonomic in EDN to declare deps as {:my/server (ig/refset :my/handler)} rather than be explicit about each handler; also the latter not as feasible in my "mechanism" case above.

I'll leave this alone for now, but likely come back to it later. This is the main reason for this ticket.

I don't think that's a good idea.

Ok, continuing with the "optional/mandatory" variant, I'll try to have a PR soon; sketch:

1. RefLikes have it in them to decide what's mandatory or optional; protocol:

(defprotocol RefLike
  (ref-key [r] "Return the key of the reference.")
  (ref-mandatory-keys r config "Returns the mandatory keys from `config`.")
  (ref-all-keys r config "Returns the mandatory and optional keys from `config`.")
  (ref-resolve [r config resolvef] "Return the resolved value."))

2. find-derived-refs uses ref-mandatory-keys or ref-all-keys depending on :optional-deps? flag.

[weavejester]

Say, handlers are in the hundreds or more, and added somewhat dynamically by a mechanism. Not quite straightforward, nor do I like it.

It doesn't matter how many handlers there are, as long as their keys share an ancestor. So for instance you might write:

(ig/init config [:duct/daemon :my/handler)])

If you're using refset to pull in all your handlers, then you must have already created a common ancestor. By specifying the :my/handler key to initialize, you also initialize all derived keys.

Alternatively, you could create a custom RefLike where all of the keys are mandatory.

RefLikes have it in them to decide what's mandatory or optional; protocol

Unless I've missed something, I don't think the ref-mandatory-keys and ref-all-keys need access to the configuration. They just need a collection of keys to choose from.

[weavejester]

I've added a reflike branch for pursuing this feature. I'll revert the change in master when I have time to resolve all the conflicts, with the intent of introducing the reflike feature into a 1.1.0-alpha.

[clyfe]

Let's release reflike.