Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

If this project is dead, what's the alternative? #86

Closed
fkleedorfer opened this issue Feb 26, 2015 · 31 comments
Closed

If this project is dead, what's the alternative? #86

fkleedorfer opened this issue Feb 26, 2015 · 31 comments

Comments

@fkleedorfer
Copy link

I suppose other rdfstore-js users are wondering about this as I am.

I'm working on a project that requires an RDF store in the browser, I chose rdfstore-js as it seemed superior to the other options I checked (and there weren't many). So, above all: Thanks @antoniogarrote for building this! It's great, and, of course, it has its problems and bugs, but the most important problem is: it's abandoned.

  • @antoniogarrote - you are working on the 1.1 branch, but I don't see any documentation to help me figure out if I can use that code. Where is it going? Will it eventually replace the abandoned version in the master branch?
  • To those currently working on forks and waiting on pull requests (@trueg, @JuniperChicago, @bensinober, @RubenVerborgh, @jandrieu... ) - Is any of you planning to continue working on rdfstore-js in the near future? Could we find some way of collaborating to get this code to work the way we need it? Maybe you want to share what your modifications were about? Maybe we could find a way to combine the forks into one we continue to maintain? It seems to me that the potential for continuing is there - or am I missing something?
  • If we have to give up on rdfstore-js: which RDF store would you recommend for use in the browser?
@RubenVerborgh
Copy link
Contributor

Since rdfstore-js uses my N3.js parser, I created pull requests to incorporate the latest version of this library. Maintaining a library of my own, I have however no intention of working on rdf-store.js. My personal plans are as follows:

  • Maintain the N3.js library for parsing, writing, and storing Turtle, TriG, N-Triples, and N-Quads in a streaming way.
  • Maintain the SPARQL.js SPARQL parser.
  • Maintain the Linked Data Fragments client, which executes SPARQL queries against a Triple Pattern Fragments server (using the above two libraries). Eventually, this project might or might not contain a SPARQL query executor for other back-ends.

In general, while I applaud @antoniogarrote's efforts to make this feature-complete rdfstore-js library, the source code is not in optimal shape. It is my personal opinion that it's not the best way to continue; but no other library is as feature-complete as this one (but recent efforts, such as TriG, are missing).

Depending on what features you need, I highly recommend N3.js for the browser. We build cool applications with it, including this one.

@kidehen
Copy link

kidehen commented Feb 26, 2015

On 2/26/15 4:48 PM, Ruben Verborgh wrote:

Since rdfstore-js uses my N3.js
https://github.com/RubenVerborgh/N3.js parser, I created pull
requests so incorporate the last version of this library. Maintaining
a library of my own, I have however no intention of working on
rdf-store.js. My personal plans are as follows:

In general, while I applaud @antoniogarrote
https://github.com/antoniogarrote's efforts to make this
feature-complete rdfstore-js library, the source code is not in
optimal shape. It is my personal opinion that it's not the best way to
continue; but no other library is as complete as this one.

Depending on what features you need, I highly recommend N3.js
https://github.com/RubenVerborgh/N3.js for the browser. We build
cool applications with it, including this one
http://client.linkeddatafragments.org/.


Reply to this email directly or view it on GitHub
#86 (comment).

We have worked with rdf-store.js and to date, there's no other
alternative that we are aware of.

We will continue to use this effort. Naturally, any bug fixes or new
features we introduce will certainly be contributed back to this project.

Our soon too be released RDF Editor does make use of rdf-store.js, for
instance.

Regards,

Kingsley Idehen
Founder & CEO
OpenLink Software
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

@melvincarvalho
Copy link
Contributor

Might not be everyone's cup of tea, but I am currently using :

https://github.com/linkeddata/rdflib.js

It's an active project with 20 commits in the last month from 5 different authors (judge that as you will!)

I would say it requires quite a bit of web literacy to dive into, as the documentation relatively light. But I've been very happy with the results so far.

@fkleedorfer
Copy link
Author

@melvincarvalho Thanks for the pointer - as I extract from open issues, there is no support for JSON-LD, and I don't see any named graph handling (I might have overlooked something), which I need, too. So there seems to be an arms race in terms of features, which rdflib.js will be winning in the long run, given developer activity, but if you need these (and maybe other) features right now, you're stuck with rdfstore-js. Having said that - does anyone think there is potential to join efforts with the people working on/with rdflib.js?

@kidehen It's great to hear that such a strong player in the community as OpenLink is backing this project. Would you be willing to maintain a fork for as long as nobody merges pull requests into the main project?

Having said that, @antoniogarrote as people seem to be actively working on the code, might you be interested in creating an organization for rdfstore-js and adding those actively developing so we get unstuck?

@RubenVerborgh Thanks for that pointer, too - I'll look into it. However, as my server doesn't support RDF Fragments, I suppose it is probably not what I need (reading linked data, preferably in quads - TriG, n-quads or JSON-LD) and querying them in SPARQL - however, that last part may be achieved differently)

@trueg
Copy link

trueg commented Feb 27, 2015

@antoniogarrote first of all: thank you for rdfstore-js.

@RubenVerborgh and thanks to you for N3.js. We actually use it in addition to rdfstore.js since our fork does not contain your patches.

@fkleedorfer I certainly do not have the time to maintain a fork of rdfstore.js all by myself but I am very open to the idea of doing it collaboratively. A first step would be to create a merged branch from all our patches and test the result in our individual projects.

@bblfish
Copy link

bblfish commented Feb 27, 2015

@antoniogarrote has helped integrate rdfstore.js into banana-rdf but I think he is very busy, and so things have not moved very far. I have tried fixing a few bug, but without the lack of progress on this branch has slowed my enthusiasm. We may have to remove it from banana-rdf now. But if a project presents itself that continues where rdfstore.js left off, then this will give us a good reason to continue maintaining that code.

@kidehen
Copy link

kidehen commented Feb 27, 2015

On 2/27/15 3:44 AM, Sebastian Trueg wrote:

@antoniogarrote https://github.com/antoniogarrote first of all:
thank you for rdfstore-js.

Yes, thanks for this effort.

@RubenVerborgh https://github.com/RubenVerborgh and thanks to you
for N3.js. We actually use it in addition to rdfstore.js since our
fork does not contain your patches.

Yes, thank you too!

@fkleedorfer https://github.com/fkleedorfer I certainly do not have
the time to maintain a fork of rdfstore.js all by myself but I am very
open to the idea of doing it collaboratively. A first step would be to
create a merged branch from all our patches and test the result in our
individual projects.

Sebastian:

We have to ensure all our patches are available to both of these projects.

All:

I think we can collectively bring more attention to these important
projects, thanks to this thread.

Regards,

Kingsley Idehen
Founder & CEO
OpenLink Software
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

@kidehen
Copy link

kidehen commented Feb 27, 2015

On 2/27/15 2:58 AM, fkleedorfer wrote:

@melvincarvalho https://github.com/melvincarvalho Thanks for the
pointer - as I extract from open issues, there is no support for
JSON-LD, and I don't see any named graph handling (I might have
overlooked something), which I need, too. So there seems to be an arms
race in terms of features, which rdflib.js will be winning in the long
run, given developer activity, but if you need these (and maybe other)
features right now, you're stuck with rdfstore-js. Having said that -
does anyone think there is potential to join efforts with the people
working on/with rdflib.js?

We should always collaborate, at every turn. We need to build an
ecosystem of tools that build bridges into other specialty realms e.g.,
UI and generic controllers etc..

@kidehen https://github.com/kidehen It's great to hear that such a
strong player in the community as OpenLink is backing this project.
Would you be willing to maintain a fork for as long as nobody merges
pull requests into the main project?

We are happy to work with @antoniogarrote
https://github.com/antoniogarrote in regards to this effort.
Basically, we can assist where needed in regards to ensuring there's a
Javascript based SPARQL compliant RDF DBMS.

Regards,

Kingsley Idehen
Founder & CEO
OpenLink Software
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

@fkleedorfer
Copy link
Author

Given that rdflib.js might be the better alternative in the long run, and not knowing how many of the feature requests that we care about we can actually implement, I'd say the safest way to proceed is to follow @trueg's suggestion and merge our fixes/extensions, and we'll see where we go from there.

So if you agree we:

  • create an organization for the project, add all intersted parties as committers
  • fork @trueg's version
  • start a branch for merging the forks
  • each fork owner merges their changes with that branch
  • Finally, we see if we want to collaborate on some of the open issues.

That should at least leave us with a more stable version of what there is already and maybe somehing like a community around it, just in case.

I have already created an organization to save time, so if you agree, please post here if you'd like to be added to the organization. Also, if someone else wants to be the owner of the organization, please step up :)

Who's in?

@kidehen
Copy link

kidehen commented Feb 27, 2015

On 2/27/15 12:34 PM, fkleedorfer wrote:

Given that rdflib.js https://github.com/linkeddata/rdflib.js might
be the better alternative in the long run, and not knowing how many of
the feature requests that we care about we can actually implement, I'd
say the safest way to proceed is to follow @trueg
https://github.com/trueg's suggestion and merge our
fixes/extensions, and we'll see where we go from there.

So if you agree we:

That should at least leave us with a more stable version of what there
is already and maybe somehing like a community around it, just in case.

I have already created an organization to save time, so if you agree,
please post here if you'd like to be added to the organization. Also,
if someone else wants to be the owner of the organization, please step
up :)

Who's in?

Okay.

I'll leave Sebastian to deal with the rest of the matter, from our side.

Regards,

Kingsley Idehen
Founder & CEO
OpenLink Software
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

@l00mi
Copy link
Contributor

l00mi commented Feb 27, 2015

Hi, sorry I'm late to this conversation.

Thanks also from my side for all the effort from all participants put into this project. We used rdfstore.js in the beginning quite intensively for our projects, but eventually shifted because of the lack of progress a bit.

I had some exchanges with @antoniogarrote ca. a year ago. My interpretation is that after finishing his PhD he wanted to restart rdfstore.js with lessons learned in a complete rewrite (https://github.com/antoniogarrote/rdfstore-js/tree/1x). It seems though that he can not invest as many time he would like.

In general I have the impression he was never against involving other developers, that is one of the reasons he was ok two switch to the MIT license as we asked him back then (a894cd7).

I support the idea of @fkleedorfer to have at least a store where the patches get merged in a regular basis and also to have versioning. We should assure in this that @antoniogarrote is mentioned always as seed developer. Also please add him as an owner of the organization? Further I am happy to become a commiter of the fork. (Though not sure if of any use .. (-, )

@fkleedorfer
Copy link
Author

@l00mi done.
So I guess the next step is asking the people with unmerged pull requests to direct them at our new fork, and do ask people with parallel forks to make pull requests.. Right?

Also, do we agree to start with @trueg's fork?

@antoniogarrote
Copy link
Owner

Hi all.

Sorry for not having been able to answer before.
It's great to see that there's still interest in the project.

Unfortunately I haven't had much time in the last months to dedicate to this project and my perception was that interest on it was very limited so I have been unable to make any progress with the project.

Any changes you want to make with the project or the code are more than welcome. If you need me to change the license, create an organization or transfer ownership of the code, please, just tell me.
Having a true community of developers moving the project forward was always my intention.

I started a rewrite of the code on Christmas. It's in the '1x' branch. It cleans the code, add much needed support for tools like gulp and dependency management, test automation, etc. I's currently 60% complete. I would use it as the basis of any further development.

Scope of the ptoject needs to be reduced and focused. Part of the complexity of the code base originates on its research project nature where multiple engines are tried that need to be removed.

I'll try to follow the conversation to see if I can be of any help.

@kidehen
Copy link

kidehen commented Feb 28, 2015

On 2/28/15 2:28 AM, Antonio Garrote wrote:

Hi all.

Sorry for not having been able to answer before.
It's great to see that there's still interest in the project.

Unfortunately I haven't had much time in the last months to dedicate
to this project and my perception was that interest on it was very
limited so I have been unable to make any progress with the project.

Any changes you want to make with the project or the code are more
than welcome. If you need me to change the license, create an
organization or transfer ownership of the code, please, just tell me.
Having a true community of developers moving the project forward was
always my intention.

Great to hear! As a community we really need to learn how to work
together. Your contribution has been invaluable. We need intelligent
clients and servers in the LOD ecosystem. Relying solely on intelligent
servers is a flawed pursuit, from any conceivable perspective.

I started a rewrite of the code on Christmas. It's in the '1x' branch.
It cleans the code, add much needed support for tools like gulp and
dependency management, test automation, etc. I's currently 60%
complete. I would use it as the basis of any further development.

Okay, so we somehow need to figure out the base for the pending merge
via the new organization taking shape etc..

Scope of the ptoject needs to be reduced and focused. Part of the
complexity of the code base originates on its research project nature
where multiple engines are tried that need to be removed.

I'll try to follow the conversation to see if I can be of any help.

From my vantage point, the goal should be to have a SPARQL 1.1 DBMS for
Javascript. This will enable the development of much smarter clients
re., Linked Open Data exploitation.


Reply to this email directly or view it on GitHub
#86 (comment).

Regards,

Kingsley Idehen
Founder & CEO
OpenLink Software
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

@JuniperChicago
Copy link

My efforts on using this code has centered on the ability to persistently store a small graph locally while using the federation features. The most significant stumbling blocks in the existing code base for me has been the few missing components of SPARQL 1.1, and inability to query all named graphs by default (without UNION).

Antonio Garrote's 1x branch, imho, has set up the right direction in parsing (though incomplete) and indicates a possible abstraction layer or API between the query-engine and any kind of store one may choose to use... which is a great point of experimentation and development.

Antonio Garrote's rdfstore.js is a solid body of work and I think a good foundation for any revitalized effort... I would be very happy to see it continued.

@trueg
Copy link

trueg commented Mar 2, 2015

While I very much like the idea of using the cleaned up 1.x branch as a basis I fear that none of us have the time to finish the missing 40% of it.

That is why I support the idea by @fkleedorfer to start with my branch and merge anything any of us might have in their branches.

Then at least we have a working version to maintain.

Thus, I propose this:

  1. I will create a branch that extends the README to include the points that have been decided here
  2. You will all create pull requests for your features on the new tree.

@RubenVerborgh
Copy link
Contributor

About 2: With a tool such as hub, it is easy to get the code from all pull requests and merge it. Redoing the pull requests might not be realistic.

@trueg
Copy link

trueg commented Mar 2, 2015

Very well, then I guess we need to gather the list of branches that we want to merge rather than re-do pull requests.

@ktk
Copy link

ktk commented Mar 3, 2015

As it was not mentioned yet, one of the more feature complete projects is probably RDF-Ext by @bergos which is using RDFStore for the SPARQL store at least but that will be replaced with @RubenVerborgh SPARQL parser one day.

We basically add features as needed so pull requests are very welcome as well. Note that @bergos tries to focus on the parts no one else did. Turtle is done by Rubens parser, JSON-LD from the JSON-LD guys, SPARQL currently as mentioned from Antonio and there is a LDP implementation as well which Thomas implemented himself. It also provides Promise interfaces where appropriate.

@fkleedorfer
Copy link
Author

@JuniperChicago

The most significant stumbling blocks in the existing code base for me has been the few missing components of SPARQL 1.1, and inability to query all named graphs by default (without UNION).

The inability to query all graphs is also the issue I have with rdfstore-js. Did you find a solution? Or does someone have a suggestion how this could be implemented in rdfstore-js?

@fkleedorfer
Copy link
Author

@trueg I'm in favor of your suggestion - When I find the time, I'll try your version, most probably my modest changes are part of yours (adapting the JSON-LD implementation to changes in the JSON-LD specification). If this is not the case, I'll make a pull request.

@trueg
Copy link

trueg commented Mar 5, 2015

@fkleedorfer I did not touch the JSON-LD code. So just point me to where those patches are and I will merge them.

@antoniogarrote
Copy link
Owner

Hi all.
I have gone through the pending pull requests these last days, merged them and released a new version of the store: 0.8.4.
It also inlcudes the latest version of the N3 parser from @RubenVerborgh.

You can use it as the basis for any future branch/fork.
If somebody is interested in following development of this branch and help with pull requests in this github repository, please just send me an email to antoniogarrote@gmail.com and I can add him/her as a collaborator.

Some advice to follow development of this branch:

  • Run the tests and the smoke tests for all the platforms
  • Never fix or change generated code (e.g. the parser)
  • Be careful with parsers and other dependencies. There's no proper support for them in the current branch so fixing bugs there is probably repeating the work already done in the original project for the dependency. It's better to update the code or report the bug in the original project.

Since there seem to be still some interest on the project, I will try to continue development of the 1.x branch that addresses a lot of the problems of the 0.x branch and should make the code easier for new developers.
Any help is welcomed.

Regards.

@cygri
Copy link

cygri commented Mar 5, 2015

Great work, @antoniogarrote. Good to see you back!

@satra
Copy link

satra commented Mar 5, 2015

thank you all for getting back and reviving this.

re: tests - one suggestion would be to start putting a travis.yml file so that every pr starts getting tested.

@JuniperChicago
Copy link

@fkleedorfer

The inability to query all graphs is also the issue I have with rdfstore-js. Did you find a solution? Or does someone have a suggestion how this could be implemented in rdfstore-js?

I have not coded a solution. I think the quickest and most definitive answer to the question as to how named graphs could be nested into a default graph, in the context of existing rdfstore-js code, would come from @antoniogarrote. If he could point to the best place to start, that would be ideal.

From what I can tell, the ability to traverse the contents of named graphs by default would both be contained in the query-engine (pattern used to discover named graphs within the triplestore) and the lexicon (system tracking of named graphs). Ideally, provision for enhanced treatment of named graphs could be provided for in the design of the triple store, likely benefiting from a quad-store (system level graph partitioning), or even quint-store (enhanced access control), but minimally speaking, the system could treat a named graph as a blank node.

I know I am oversimplifying this, and I have moments of clarity and then get lost in it. But I believe the traversing named graphs issues is another good reason for abstracting the query engine to enable a wider variety of persistent stores to be used and more features to be bolted on.

@antoniogarrote
Copy link
Owner

I've published the first version of the rewrite: 0.9.2. The codeis alread merged in master.
Still not finished but integration tests are passing.
The old code can be found in the 0x branch. Latest version published is 0.8.5.

I have tried to add some of the suggestions in the thread like, adding the project to Travis.

@antoniogarrote
Copy link
Owner

About the datasets for queries, this is a small summary of how the store works. I think the question is what should be the default behaviour:

Any query to the store accepts two sets of graphs as arguments.

  • The first list is used as the default graph for queries without an explicit graph variable.
  • The second argument is the collection of named graphs that will be used to resolve explicit graph variables.
    Additionally, the store mantains a default graph for the cases where no default graph is specified.

Provided the following insertion:
INSERT g1 { t1 }

The following scenarios can happen:

  1. query:
    select * { ? }
    default graph: []
    named graph: []
  • output: []
  1. query:
    select * { g1 ? }
    default graph: []
    named graph: []

-output: []

  1. query:
    select * { ? }
    default graph: [g1]
    named graph: []

-output: [t1]

  1. query:
    select * { g1 ? }
    default graph: [g1]
    named graph: []

-output: []

  1. query:
    select * { ? }
    default graph: []
    named graph: [g1]

-output: []

  1. query:
    select * { g1 ? }
    default graph: []
    named graph: [g1]

-output: [t1]

@trueg
Copy link

trueg commented Mar 13, 2015

@antoniogarrote Thank you for taking this up again. It is very much appreciated. I am now trying to use the new version in my projects but I am running into some trouble:
I did the build for the browser using
$ gulp browserify
but the resulting build does not define "rdfstore". Is any other configuration required?

@trueg
Copy link

trueg commented Mar 13, 2015

I now created a simple file to enforce the "rdfstore" namespace. The file contains only one line and is the new input for browserify:

window.rdfstore = require('./store.js');

Not sure if that makes much sense - I am new to browserify and all its friends.

@antoniogarrote
Copy link
Owner

Hi everyone.
This issue was becoming a pseudo-chat about the project.

I have created a gitter room to continue the conversation. You can join from the README of the project.
Please go on with the conversation there, since it is really valuable.

I have a number of things almost ready, like the bower packaging, but things are going to slow a little bit for a few weeks since I'm looking for a new job.

Hope to start working on new functionality and making all the official W3C tests pass soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests