Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage from Shadow-CLJS? #64

Closed
teodorlu opened this issue May 2, 2019 · 19 comments

Comments

Projects
None yet
3 participants
@teodorlu
Copy link

commented May 2, 2019

Hello!

I'm hitting some problems trying to use Oz in a Shadow-CLJS project, and I've got a question or two.

Questions

  • Are anyone using Oz with Shadow-CLJS?
  • Should this be solved in user projects, or is it feasible and reasonable to do something in Oz to make using Oz from Shadow-CLJS a smooth experience?

Problem description

When I added (:require [oz.core :as oz]) to my ns form in a .cljs file, I got this error message:

The required namespace "cljsjs.vega" is not available, it was required by "oz/core.cljs".
The namespace was provided via :foreign-libs which is not supported.
Please refer to https://shadow-cljs.github.io/docs/UsersGuide.html#cljsjs for more information.

From what I can tell from the linked section from the Shadow-CLJS user guide,

  • Shadow-CLJS doesn't support CLJSJS
  • Oz uses Vega and Vega-lite from CLJSJS
  • Oz needs to keep working without NPM to give that smooth Clojure experience being able to spin up a visualization from the REPL.

I don't understand all the implications of the section in the guide. I guess it might be possible to install Vega and Vega-lite directly with NPM in my project and point Oz to those.

Teodor

@teodorlu teodorlu changed the title Usage with Shadow-CLJS? Usage from Shadow-CLJS? May 2, 2019

@metasoarous

This comment has been minimized.

Copy link
Owner

commented May 2, 2019

Hi @teodorlu. Thanks for raising this issue.

I would love to be able to be able to support Shadown-CLJS if possible. However, I'm a bit surprised and dismayed that it does not support CLJSJS. I know very little about Shadow-CLJS, aside from the basics, so I'm not really the person to work on this. But if someone comes up with a good way of supporting it, I'd be all ears.

Does anyone know why CLJSJS isn't supported in Shadow-CLJS?

Thanks again for posting

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 2, 2019

Glad to hear you'd like to support Shadow-CLJS! I think his point is that Shadow can use NPM directly, thus doesn't need to go through Clojars. There's some discussion on this in the guide. For the record, this prompted me to swap out Shadow-CLJS for Figwheel, which I got working.

From what I can understand in the Shadow CLJS guide, the user might be able to fix this by providing namespaces for cljsjs.vega (and the rest), that map these over to NPM packages, which can be installed from NPM with Shadow. Thomas Heller seems to have done this for quite a few libraries already.

A possible path to a fix could be to create the required mappings, try to get them merged into the shadow-cljsjs repo, and add a section on usage from Shadow-CLJS in the README.

@metasoarous

This comment has been minimized.

Copy link
Owner

commented May 2, 2019

Great! Glad that you have things working with Figwheel.

Looking over some of those library mappings, they seem rather straightforward. Your suggestion seems like a great path forward towards providing Shadow-CLJS support.

Thanks again!

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 3, 2019

No problem! Glad to be able to contribute.

And thanks a lot for Oz -- I think you're really getting how Vega interaction from Clojure(Script) should be.

  • Being able to spin up a visualization from a Clojure (JVM) REPL is a sweet experience
  • ... and then being able to re-use that visualization code if I'm going to create a some sort of dashboard is nice.

Keep up the good work!

Teodor

@ivanminutillo

This comment has been minimized.

Copy link

commented May 8, 2019

Hi :)
I've just started to play a bit with oz, after I've watched this awesome video
I created this repo to share with you how I managed to have a proper setup with oz + reagent + shadow-cljs, as I found myself in the same situation of @teodorlu...
Right now all works smooth, the only warning remained is the following

 TypeError: "vegaTooltip.vegaLite is not a function" render_vega_lite core.cljs:35

but I can live with that for the stage of my current experiments :)
Hope it can be usefull, and keep up the good work 👍

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 8, 2019

@ivanminutillo That sounds fantastic! I'll check it out when I get the chance.

Do you have an opinion on how this should be presented to new users?

Edit: downloaded and took it for a test run. Looks good to me, thanks for the effort!

  • Confirmed working with globally installed shadow-cljs from NPM
  • Confirmed working with editor-integrated CIDER REPL.

No issues! I think it sounds reasonable to create a PR to thheller/shadow-cljsjs and create a "Usage from Shadow-CLJS README section.

@ivanminutillo

This comment has been minimized.

Copy link

commented May 8, 2019

No issues! I think it sounds reasonable to create a PR to thheller/shadow-cljsjs and create a "Usage from Shadow-CLJS README section.

I think it is a good way to go 👍

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 9, 2019

Another question, if you don't mind.

In your shadow-cljs.edn, you include your dependencies without :exclusions for the cljsjs libs. Does that mean they still get loaded?

;; from shadow-cljs.edn
 :dependencies [[binaryage/devtools "0.9.10"]
                [proto-repl "0.3.1"]
                [metasoarous/oz "1.6.0-alpha2"]
                [reagent "0.8.1"]]

From the reagent readme, they show how to provide "your own react version":

;; ...
                [reagent "0.x.x" :exclusions [cljsjs/react cljsjs/react-dom]]
;; ...

I'm asking because I really don't know. Might be able to fix that if I throw some time on it.

@ivanminutillo

This comment has been minimized.

Copy link

commented May 11, 2019

Hey, sorry for late answer... I used this template to get started and as you can see there is no trace of :exclusions in the shadow-cljs.edn file.
I am interested in what you come up with this, if you have time to investigate (i am still a newbe in the clojurescript realm :))

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 14, 2019

No problem, open source takes time. I think my hypothesis above is right, checking now.

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 14, 2019

@metasoarous I'm starting to feel that Oz in reality covers two use cases:

  1. Plain Clojure data -> Vega visualizations
  2. A system for making this actually feasible when working on a Clojure (JVM).

I'm going to assume that (1.) mostly is a pure data transformation. Please correct me if that isn't the case.

Splitting (1.) and (2.) into separate packages would allow using (1.) without pulling in dependencies from (2.) Do you think there's any feasibility to this? Vega and React would still have to be provided. To be clear, I'm not proposing we (?) start work on this, but I'd like to know your opinion.


Use case I'm thinking from: As a user, I'd like to use Oz from a normal Clojure REPL interactively, and re-use those visualizations on the web.

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 14, 2019

@ivanminutillo, I've got some results, and a question.

CLJSJS dependency exclusion

  1. I've verified your example project pulls down CLJSJS deps,
  2. With some trial and error, I managed to exclude them all,
  3. And the project still works!

See my changes for details.

shadow-cljsjs mappings

I'll create a PR to shadow-cljsjs. @ivanminutillo, this is your work, so if you'd rather have your own name in the shadow-cljsjs repo, I can delete the PR.

Edit: Created the pull request.

@ivanminutillo

This comment has been minimized.

Copy link

commented May 15, 2019

great work @teodorlu ! it works like a charm with :exclusions!
no problem at all with names on pull-request 👍
I'll start to test some more data-viz with reagent and oz during the next weeks, if it interests you, we could share some experiments on a shared repository to build some materials for future examples/guides ?

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 15, 2019

Definitely interested!

I can't promise that I'll a lot of time to spare, but I think ClojureScript + Vega really makes sense for data visualization. I'm experimenting with the role of the database as well -- what does it feel like to use a data model in datascript connected to visualizations in Oz? I imagine that workflow could be a nice place to work. Feel free to tag me, and we can take it from there!

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 15, 2019

As for this thread, this is my plan:

  1. Wait for the changes to shadow-cljsjs to get released
  2. Remove the cljsjs.{vega,vega-lite,...} namespaces from the example and validate things working
  3. Create a docs/usage-from-shadow-cljs.md or something else aligned with what makes sense for @metasoarous.
@teodorlu

This comment has been minimized.

Copy link
Author

commented May 20, 2019

Update: as of Shadow-CLJS 2.8.37 (pushed a few days ago), CLJSJS mappings for Vega and Vega-lite are provided upstream. I've updated the example project to reflect the changes. Proposal for usage instructions suitable for documentation follows below.

@metasoarous feel free to copy the text below where you find it suitable in the Oz docs, and edit as you will. Thanks for the discussion with you and @ivanminutillo! With this, I'm closing the issue.

Using Oz from Shadow-CLJSJS

It is possible to use Oz with Shadow-CLJSJS, but care must be taken that the right Vega and Vega-lite dependencies are provided to Oz. The Oz project depends on Vega and Vega Lite as packaged in CLJSJS. This enables Oz to be able to spin up a web server with Vega loaded from a normal Clojure application. Shadow-CLJS does not support CLJSJS, and requires JavaScript dependencies to be loadable with NPM. To use Oz from Shadow-CLJS,

  1. Ensure that you have Shadow-CLJS version 2.8.37 or later installed. Shadow-CLJSJS before version 2.8.37 did not include CLJSJS shims for Vega and Vega lite.
  2. Check what versions of Vega, Vega-Lite and Vega-Embed your Oz version requires by reading the Oz package CLJSJS dependencies on Clojars
  3. Install the required versions of Vega, Vega-Lite and Vega-Embed from NPM.

Optionally, check out this example project.

@teodorlu teodorlu closed this May 20, 2019

@metasoarous

This comment has been minimized.

Copy link
Owner

commented May 22, 2019

@teodorlu @ivanminutillo Epic! Thank you both so much for hashing things out with each other! Love seeing the open source juices flowing :-)

I'll put these docs up on the README for now I think. May do some more detailed docs organization eventually, but while it can all fit on one page, may as well. (FWIW, the doc dir is currently a bunch of images, output from the examples; Should move these images to the examples dir when it's time to really clean up the docs).

Splitting (1.) and (2.) into separate packages ...

I don't think I'm inclined to do this, at least for now. There's some benefit to having everything in one repo while things are still actively developing. I've burned myself on breaking things into too many subprojects before. And it sounds like using exclusions has solved the issues which came up above. Could perhaps use a note in the docs though.

I'm experimenting with the role of the database as well -- what does it feel like to use a data model in datascript connected to visualizations in Oz?

I've been thinking about this! We've been more or less extending markdown with Vega-Lite & Vega, so why not also extend Vega-Lite & Vega with Datalog? Imagine datalog queries in your various :data, :datasets and :transform args? Please feel free to open an issue if you're inspired, but know it's on my mind.

Thanks again!

metasoarous added a commit that referenced this issue May 22, 2019

@teodorlu

This comment has been minimized.

Copy link
Author

commented May 22, 2019

Sounds good, keep up the good work!

On the splitting issue, I'm not so sure after a few nights of sleep. Checking out the source code for your reagent component, it's really minimal. Currently, users who would like just that part could copy out that part of the code.

@metasoarous

This comment has been minimized.

Copy link
Owner

commented May 22, 2019

Thanks for the update. You're right, those components are pretty minimal and can be copied pretty easily for now. I do have some plans for extending the configurability of the plots via the vega-view api (plugging into Vega signal chans and the like), but also expect all of that to be pretty minimal. Once that's smoothed out, if there's enough substance there and demand for pulling out a separate lib, I'm open to revisiting.

Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.