basilisp.core/load fn caching

[ikappaki]

Hi,

as per its docstring, basilisp.core/loaded files are not cached and will be reread every-time they are loaded:

  Note that unlike ``require``\\, files loaded by ``load`` will not be cached and will
  thus incur compilation time on subsequent loads.

I believe this can cause degraded performance, since every time a library is loaded that uses load to separate files under one namespace it has to read, analyze and compile those files.

Is there a chance this limitation been lifted? I think this will get it closer to how Clojure works.

Thanks

[chrisrink10]

Namespaces are cached by the Basilisp import mechanism (which is supported and facilitated by Python's own importlib machinery). Figuring out how to cache arbitrarily loaded code in a way that is compatible with the existing caching mechanism seems extremely challenging and complex.

But setting aside the technical aspects, I simply don't agree with this style of code organization and do not wish to encourage it. The same effect could be achieved by splitting a namespace into a sub-namespaces with implementation details and then requiring those into the "main"/public API namespace. This style of organization would be eligible for namespace caching, while the load supported style is not.

I've not done a comprehensive survey of Clojure libraries, but from my recollection this is not a common idiom outside of Clojure (clojure/clojure) itself.

[ikappaki]

Namespaces are cached by the Basilisp import mechanism (which is supported and facilitated by Python's own importlib machinery). Figuring out how to cache arbitrarily loaded code in a way that is compatible with the existing caching mechanism seems extremely challenging and complex.

Right.

But setting aside the technical aspects, I simply don't agree with this style of code organization and do not wish to encourage it. The same effect could be achieved by splitting a namespace into a sub-namespaces with implementation details and then requiring those into the "main"/public API namespace. This style of organization would be eligible for namespace caching, while the load supported style is not.

In the case of clojure.pprint, the namespace is split into five different loaded files, with some of them being relatively chunky.

We can discuss the best course of action once I commit the initial draft, another option would be to concatenate all files into one, but that defeats the purpose of modularity.

I've not done a comprehensive survey of Clojure libraries, but from my recollection this is not a common idiom outside of Clojure (clojure/clojure) itself.

I haven't seen any other such case either.

Thanks