Update dependencies

[michielbdejong]

Step 1

Using the latest npm versions of:

• rdflib@1.2.3
• solid-auth-cli@1.0.14
• solid-auth-client@2.4.1
• solid-auth-tls@0.1.2

I'm updating npm dependencies for:

• solid-namespace ()
• pane-registry (rdflib)
• solid-ui (pane-registry, rdflib, solid-auth-client, solid-auth-tls, solid-namespace)
• chat-pane (solid-ui)
• contacts-pane (solid-ui)
• folder-pane (solid-ui)
• issue-pane (solid-ui)
• meeting-pane (solid-ui)
• source-pane (solid-ui)
• solid-panes (chat-pane, contacts-pane, folder-pane, issue-pane, meeting-pane, pane-registry, rdflib, solid-namespace, solid-ui, source-pane)
• mashlib (rdflib, solid-namespace, solid-panes, solid-ui)

[michielbdejong]

Hm, seems I broke something now - try:

npm install
npm run build
cd dist
npx serve

Then open http://localhost:5000/databrowser with your browser and open the developer console.
You'll see an error "panes is not defined". This is probably due to the build steps, not converting TS to JS when pushing to npm? Should hopefully not be too hard to fix, will debug.

[michielbdejong]

It seems to be due to SolidOS/chat-pane#40

[michielbdejong]

Should also include nodeSolidServer/solid-auth-client#150

[michielbdejong]

Step 2

Using the latest npm versions of:

• @babel/runtime@7.10.5
• @sinonjs/text-encoding@latest
• async@latest
• auth-header@1.0.0
• base64url@latest
• commander@6.0.0
• concat-stream@latest
• cross-fetch@latest
• fs-extra@latest
• isomorphic-fetch@2.2.1
• isomorphic-webcrypto@latest
• jsonld@3.1.1
• mime-types@latest
• node-fetch@latest
• n3@1.4.0
• standard-http-error@latest
• whatwg-url@latest
• xmldom@0.3.0

I'm updating npm dependencies for:

• @solid/jose (@sinonjs/text-encoding, base64url, isomorphic-webcrypto)
• @solid/oidc-rp (@solid/jose, base64url, isomorphic-webcrypto, node-fetch, standard-http-error, whatwg-url)
• solid-cli (@solid/oidc-rp)
• solid-rest (concat-stream, cross-fetch, fs-extra, mime-types)
• solid-auth-cli (@solid/cli, async, cross-fetch, jsonld, n3, solid-rest)
• solid-auth-client (@babel/runtime, auth-header, commander, isomorphic-fetch, @solid/oidc-rp)
• solid-auth-tls ()
• rdflib (jsonld, n3, solid-auth-cli, solid-auth-client, xmldom)
• pane-registry (rdflib)
• solid-ui (pane-registry, rdflib, solid-auth-client, solid-auth-tls, solid-namespace)
• chat-pane (solid-ui)
• contacts-pane (solid-ui)
• folder-pane (solid-ui)
• issue-pane (solid-ui)
• meeting-pane (solid-ui)
• source-pane (solid-ui)
• solid-panes (chat-pane, contacts-pane, folder-pane, issue-pane, meeting-pane, pane-registry, rdflib, solid-ui, source-pane)
• mashlib (rdflib, solid-panes, solid-ui)

[michielbdejong]

Step 3 would be:

• solid-multi-rp-client (oidc-rp)
• @solid/oidc-op ()
• oidc-auth-manager (oidc-rp, oidc-op, solid-multi-rp-client)
• node-solid-server (rdflib, oidc-auth-manager, mashlib)

[michielbdejong]

Current status:
I can compile solid-auth-client with @solid/oidc-rp@0.10.0 from npm but not when I link it using Lerna. Then npm run build just fails saying it can't find the imports. I'll try what happens when I install it as a git dependency. -> seems I fixed this with npx lerna bootstrap --force-local.

[michielbdejong]

Full-stack build is successful in mashlib-dev now.
Tests are failing in node-solid-server though.
I think this can be worked around by not linking to local rdflib in NSS , but maybe I should just include step 3 now, so that it all works like it's supposed to in mashlib-dev, and we can all test properly.
I could also maybe set up the whole tree with git dependencies so it can be tested outside mashlib-dev.

[michielbdejong]

I ran npm outdated on all repos up to solid-auth-client, will continue from solid-auth-tls next week.

[michielbdejong]

Hm, https://github.com/solid/solid-auth-tls was archived and not forked anywhere. See https://github.com/solid/solid-ui/issues/324

[michielbdejong]

[michielbdejong]

Also node-solid-ws, @solid/acl-check.

[michielbdejong]

Update complete and tests passing locally in mashlib-dev:

• @solid/jose (@sinonjs/text-encoding, base64url, isomorphic-webcrypto)
• @solid/oidc-rp (@solid/jose, base64url, isomorphic-webcrypto, node-fetch, standard-http-error, whatwg-url)
• solid-cli (@solid/oidc-rp)
• solid-rest (concat-stream, cross-fetch, fs-extra, mime-types)
• solid-auth-cli (@solid/cli, async, cross-fetch, jsonld, n3, solid-rest)
• solid-auth-client (@babel/runtime, auth-header, commander, isomorphic-fetch, @solid/oidc-rp)
• solid-auth-tls ()
• rdflib (jsonld, n3, solid-auth-cli, solid-auth-client, xmldom)
• pane-registry (rdflib)
• solid-namespace
• solid-ui (pane-registry, rdflib, solid-auth-client, solid-auth-tls, solid-namespace)
• chat-pane (solid-ui)
• contacts-pane (solid-ui)
• folder-pane (solid-ui)
• issue-pane (solid-ui)
• meeting-pane (solid-ui)
• source-pane (solid-ui)
• solid-panes (chat-pane, contacts-pane, folder-pane, issue-pane, meeting-pane, pane-registry, rdflib, solid-ui, source-pane)
• mashlib (rdflib, solid-panes, solid-ui)
• solid-multi-rp-client (oidc-rp)
• @solid/oidc-op ()
• oidc-auth-manager (oidc-rp, oidc-op, solid-multi-rp-client)
• node-solid-ws ()
• @solid/acl-check (rdflib, solid-namespace)
• node-solid-server (rdflib, oidc-auth-manager, mashlib)

Exception: stayed on bootstrap@3 in NSS because it seems bootstrap@4 no longer ships fonts in dist/

[michielbdejong]

Now running into nodeSolidServer/solid-auth-client#163 (comment), investigating.

[michielbdejong]

Making progress! :) Now able to log in, but seeing these errors:

[michielbdejong]

Note you need to comment out

# webcrypto-shim.mjs
# webcrypto-liner.js

in the .gitignore of mashlib-dev/workspaces/isomorphic-webcrypto, otherwise the build of for instance mashlib will fail with:

ERROR in ../isomorphic-webcrypto/src/browser.mjs
Module not found: Error: Can't resolve './webcrypto-shim.mjs' in '/Users/michiel/gh/solid/mashlib-dev/workspaces/isomorphic-webcrypto/src'
 @ ../isomorphic-webcrypto/src/browser.mjs 1:0-30
 @ ../oidc-rp/src/AuthenticationResponse.js
 @ ../oidc-rp/src/RelyingParty.js
 @ ../oidc-rp/src/index.js
 @ ../solid-cli/src/SolidClient.js
 @ ../solid-auth-cli/src/index.js
 @ ../rdflib/esm/fetcher.js
 @ ../rdflib/esm/index.js
 @ ./src/index.ts
 @ multi ./src/index.ts

[michielbdejong]

I should look into https://github.com/interop-alliance/oidc-rp / https://github.com/anvilresearch/oidc-rp / https://github.com/solid/oidc-rp situation, thanks @SharonStrats for pointing me to https://github.com/solid/oidc-rp#deprecation-notice ! CC @dmitrizagidulin

[michielbdejong]

Also nodeSolidServer/keychain#4

[michielbdejong]

Merged and published:

• @solid/jose (@sinonjs/text-encoding, base64url, isomorphic-webcrypto)

• @solid/oidc-rp (@solid/jose, base64url, isomorphic-webcrypto, node-fetch, standard-http-error, whatwg-url)

• solid-auth-client (@babel/runtime, auth-header, commander, isomorphic-fetch, @solid/oidc-rp)

• rdflib (jsonld, n3, solid-auth-cli, solid-auth-client, xmldom)

• pane-registry (rdflib)

• solid-namespace

• solid-ui (pane-registry, rdflib, solid-auth-client, solid-namespace)

• chat-pane (solid-ui)

• contacts-pane (solid-ui)

• folder-pane (solid-ui)

• issue-pane (solid-ui, pane-registry, rdflib)

• meeting-pane (solid-ui)

• source-pane (solid-ui)

• solid-panes (chat-pane, contacts-pane, folder-pane, issue-pane, meeting-pane, pane-registry, rdflib, solid-ui, source-pane)

• mashlib (rdflib, solid-panes, solid-ui)

• solid-multi-rp-client (oidc-rp)

• @solid/oidc-op ()

• @solid/keychain

• oidc-auth-manager (oidc-rp, oidc-op, solid-multi-rp-client)

• node-solid-ws ()

• @solid/acl-check (rdflib, solid-namespace)

• @solid/solid-auth-oidc (@solid/oidc-rp)

• node-solid-server (@solid/acl-check, @solid/oidc-auth-manager, rdflib, mashlib, solid-auth-client, solid-namespace, solid-ws, @solid/oidc-op, @solid/solid-auth-oidc)

• solid-cli (@solid/oidc-rp)

• solid-rest (concat-stream, cross-fetch, fs-extra, mime-types)

• solid-auth-cli (@solid/cli, async, cross-fetch, jsonld, n3, solid-rest)

[michielbdejong]

Will do the rest once I can publish https://www.npmjs.com/package/solid-auth-client.

@jeff-zucker do you want to tag and publish versions of solid-cli, solid-rest and solid-auth-cli? I started updating those because I thought I needed them in the dependency tree, but in the end I was able to drop them thanks to linkeddata/rdflib.js#361. Btw, we should also do a dependency update for https://github.com/solid/data-kitchen, right? Do you want to work on that together?

[michielbdejong]

Done! :)

[jeff-zucker]

Will do the rest once I can publish https://www.npmjs.com/package/solid-auth-client.

@jeff-zucker do you want to tag and publish versions of solid-cli, solid-rest and solid-auth-cli? I started updating those because I thought I needed them in the dependency tree, but in the end I was able to drop them thanks to linkeddata/rdflib.js#361. Btw, we should also do a dependency update for https://github.com/solid/data-kitchen, right? Do you want to work on that together?

I am just now seeing this message ... my github notifications must be fubared ... absolutely I want to work together on this ... I have recently made progress with the newest electron and mashlib in Data Kitchen ... I have no problem with removing solid-auth-cli from the rdflib dependency tree BUT I would like to see a way to easily substitute a different fetcher so that solid-rest can be used to fetch non-http requests.

[michielbdejong]

CC @jaxoncreed ^
Do you want to coordinate this between the two of you? Let me know if I can help.

[jeff-zucker]

@jaxoncreed and @michielbdejong - I m not sure which one of you I need to be talking with, could you clarify your respective roles in relation to rdflib/mashlib/SolidOS? I'd like chat/zoom with one or both of you when you are available.

My issues in relation to Data Kitchen and other uses of rdflib/mashlib outside a standard browser context: 1) how to plan for transition from solid-auth-cli to solid-client-authn. 2) How to use an alternate fetch if default_fetcher is removed. 3) whether solid-rest should remain a dependency even when solid-cli and solid-auth-cli are removed.

@michielbdejong - I also would like to talk with you about using the crud tests (and other parts of the test suite) against file:// URIs - AFAIK solid-rest should be able to pass most or all of the tests other than ACL related things.

[michielbdejong]

My role is purely ad-hoc and volunteering, nobody is paying me to maintain these libraries.
I do use these libraries myself though (for instance, when I was writing Solid's Test Suite), and whenever I need to fix something for my own stuff, I also upstream my fix for others to benefit from.

I think Jackson is in mostly the same position.

If you have fixed something then if it works for you and doesn't break the tests, and it feels like the right change to make, then it's probably good to merge.

1. One thing I learned while working in the Solid stack is don't plan for future transition, always stick to what works now. :)

2. You can specify the fetcher when you instantiate rdflib. Check out how solid-logic does it: https://github.com/solid/solid-logic/blob/471cdc01babfa397b440aa38a2f658f9985c7c72/src/index.ts#L27
   Another option is to specify window.solidFetcher as you can see here:
   linkeddata/rdflib.js@752b740#diff-eb275dd515bd68f45e5e14b12d3f4ff97dc9b1e52d742f0e66f56f67dccb21b0R755

3. up to you! :)

re crud tests against file:/// yeah, go for it! You can fork the solid-crud-tests repo and replace all the lines where is uses solid-auth-fetcher, like
https://github.com/solid/solid-crud-tests/blob/b9bb2ab/test/surface/create-container.test.ts#L3
Then just insert your file:///-aware library there and run npm run jest.

[jeff-zucker]

Thanks @michielbdejong, I've more or less figured these things out since posting, but it helps to have confirmation. I also work adhoc as a volunteer so really appreciate all the work you both have put in.

As it turns out, once we add global.solidFetcher in addition to window.solidFetcher (which isn't available in nodejs, see my PR #464) it is indeed easy to sub in an alternate fetcher in rdflib. In terms of the future changes, I believe in supporting what works now while also keeping as open to what might change as possible. I'm releasing a new solid-node-client that does this by giving the user a choice as to whether they want solid-auth-fetcher or solid-client-authn-node or something else as the auth fetcher. I am hoping for a flexible plugin architecture that supports users bringing their own just about everything. I am currently running into a snag with that in that if I import both solid-auth-fetcher and solid-client-authn-node in the same process, they clobber each other somehow. Must dig! But I have it working that user can use either one, just not both.

As soon as I have time (yeah, right!) I'll dig into the crud-tests.

In terms of dependencies, I am now on to trying to figure out how to include data-kitchen in Solidos. Mashlib, panes, and solid-ui are so integral to data-kitchen that it makes sense to have it live connected inside SolidOS. For now I am doing manual npm linking but it will be nice to eventually have it within the amazing generation framework you have built. Data-kitchen is not as ready for prime time as the other parts of SolidOS, so politically, I am not sure when is the right time to include it. Would value your thoughts.

[michielbdejong]

@jeff-zucker You can include data-kitchen at the end of https://github.com/solid/solidos/blob/master/scripts/release so that it gets built with the latest dependencies whenever someone runs that script. For that, it doesn't need to be read for prime time, as long as the build doesn't error.