How to continue maintenance of EasyRdf? (and the way to 1.0.0?)

[k00ni]

Its sad to see another PHP/RDF project being not maintained anymore. First of all, @njh and all contributors did an amazing job of creating EasyRdf. Because it is widely used (Github reports "Used by": 14.5k), there should be a way to continue its maintenance and even add further improvements.

I would love to hear from @njh, if he has plans for EasyRdf.

In case, he has no plans or doesn't react, i would suggest the following: we create a new organization and fork the repository. This would allow a wider audience to work on the code.

I can help with organizing issues, check code and other stuff from time to time.

Based on the network view there are projects with improvements, fixes, i assume. (Just to name a few @SchemaApp, @acdh-oeaw, @frmichel)

What do you guys think?

Ref: #299, #282

[k00ni]

To bring this issue forward, I created a fork of this repository, which can be a vehicle for further development.

https://github.com/sweetyrdf/easyrdf

For now I intend to setup a stable basement to allow continuous maintenance. But I am open if someone wants to provide new features.

[dogoepp]

I like the idea of moving the development to an organisation, even more if the original developer @njh does not respond any more. Thanks for initiating it. I just hope that others will want to join.

As far as I'm concerned, I just started using EasyRdf, but should I keep using this library in the near future, I think I'll want to join and help on this.

[k00ni]

I released the first stable version of my fork recently. Any feedback is appreciated. Take care.

[njh]

Sorry for the delayed response. Unfortunately, I don't have anything like the amount of time that I used to have when I initially write EasyRdf.

I have been wondering what to do about EasyRdf for little a while now. I have lost a lot of my initial motivation:

• I no longer use PHP at work and rarely for personal projects, so I am not up to date with the latest developments and approaches in the PHP world
• After putting a lot of effort into RDF and the semantic web technology and seeing very little come out of it, I have lost some enthusiasm for it. We keep waiting for the killer application and it doesn't come.
• I have lots of other projects that I spend time on
• I have been a bit disappointed by the lack of community around EasyRdf. Rarely does anyone respond to other people's problems or questions on the mailing list or GitHub. This is a bit demotivating and it is hard to gauge how important it is to people.
• EasyRdf wasn't intended to be a general-purpose library for solving all of your RDF needs in PHP. It was just supposed to be a way of getting RDF in and out of your presentation layer, with hard-core processing happening somewhere else. I am not sure if that mission has got lost along the way.
• Naming it EasyRdf was always a mistake. I don't think RDF will ever be easy 🙁

However, on the other hand, there are reasons I haven't given up on it completely:

• I am pretty proud of what I achieved
• I put a huge amount of time and effort into EasyRdf and have been disappointed I didn't release version 1.0 and get to to a good stable point for modern PHP
• I am a big fan of the Wikidata project and it was amazing when they started using EasyRdf in it. Although I am not sure if they still do or not.
• I still see data systems naively evolving into RDF. I still believe in the original mission of trying to make consuming and producing RDF simpler

I don't think just having a number of people to help maintain the project by making a few tweaks and accepting Pull Requests is enough. I have had trouble in other projects 'blindly' accepting Pull Requests - it has introduced bigger problems or led the project in a non-strategic direction, mainly due to contributors not seeing the bigger picture. There needs to be someone with the motivation and vision for the project to make sure it heads in the right direction.

Getting to a 1.0 release

A lot of the composer problems that people are reporting are due to the lack of an official release with modern namespaces. I never got that far because:

• Need to sort out some of the documentation problems by upgrading to new version of Sami, so that a new version of the website can be released
• Finish tidying up the changes when upgrading to PHP 5.3 namespaces

Because of the breaking API change in changing to modern PHP namespaces, the next release should follow semver and increment the major version number to 1.x.x.

Beyond version 1.x

• Replace the Zend Framework 1 based HTTP client and replace it with something modern (Guzzle / PSR-17 / PSR-18)
• Wikidata took a snapshot of the 'core' of EasyRdf because they didn't want to security code review the whole library, only the bits they were using. Would it be better if EasyRdf was lots of smaller libraries - like rdf.rb. That might work for people using composer - but might make it harder to get started for some who just wants to put a library in the same directory as their project.
• Perhaps a good candidate for moving out of the main repo is the RDFa parser, which is a bit of a beast

Also

• Modernise for PHP 7?
• Add types to all the parameters?
• Add support for array access
  • Graph
  • Resource
• Add $resource->addCollection('property', $values...);
• Add $resource->setCollection('property', $values...);
• Double-check that extra namespaces aren't being added to SPARQL queries
• Check for incorrect usage of empty()
• Correct the URIs for formats
• Fix error messages when rapper / dot isn't available
• Fix graph comparison support
• Add caching support
• Add more tests or the parsers and serialisers
• Refactor namespace creation support
• Support for SPARQL 1.0 Update
• Perform memory profiling
• Sparql query shortcuts for common queries

Next Steps

• I have created an easyrdf organisation and move www.easyrdf.org repo there
• I am a bit nervous about moving the easyrdf repo in case it breaks stuff, but I think Github's redirects should work. If anyone has experience of this, I would love to know
• I would like to switch to no commits directly to master and move to all Pull Requests to be reviewed by at least one other person. Volunteers, please?

[k00ni]

Hey @njh, thank you for reaching out here in such a detail.

I worked a couple of years in the RDF world (Linked Open Data, ETL, database) and can approve most of your points. But i think its not only about RDF anymore, but also about how to maintain a library which became part of an ecosystem and about the people around it.

State of RDF in PHP world

Before i continue i want to make some statements about the state of RDF libraries in the PHP world. That is the situation people are facing, if they wanna use PHP for RDF projects.

Unfortunately, PHP never had a stable and feature-complete RDF stack like in the Java or Python world. We "only" have:

• ARC2 by @semsol
• hardf by @pietercolpaert
• Erfurt (maintained by @white-gecko mostly AFAIK)
• librdf (using a PHP extension). There is no PHP 7 support yet as stated in https://github.com/sweetyrdf/easyrdf/issues/1.
• jkphl/micrometa - parsers for RDFa 1.1 lite, JSON-LD etc.
• EasyRdf

ARC2 is only maintenance mode. I stepped in to help semsol so it doesn't die and added few features to bring it to PHP 7 era (like using PDO instead of mysql extension).

hardf is kinda new and was brought from the nodejs world. It may need some refinement but it has fast parsers and does its job well.

Erfurt aims to be a feature-complete framework which not only has parsers but also a triple store implementation like ARC2, for instance. It is the basement of the OntoWiki which basically provides a wiki-like UI to manage RDF data. Status: both are not maintained anymore and not ready for PHP 7.

I am one of the core developers of the Saft framework, which back then aimed to be some kind of an integration layer. The PHP RDF ecosystem was already stuck in 2014-2015, many libraries had weak support (by maintainers or the community) and began to die. Our goal was to provide a basement and integrate different foreign libraries such as EasyRdf or ARC2 to enable users to use them together. One could focus on the use case, we took care of the data transformation etc. We stopped due to the lack of time and because no one really cared, but that's OK.

My suggestions

Your feature-bullets sound nice, but i am afraid there might be only a few people who are not only capable of doing it but are also willing to invest the time needed.

As far as i am concerned i would suggest the following:

• Use a basic set of classes/interfaces to model RDF entities. Might wanna have a look at https://github.com/rdfphp/baseline (collection of PHP interfaces and classes). The idea was to use the same interfaces to "describe" RDF things in the PHP and JS world (interoperability). Or we stay with arrays, which is also fine. Remember Would you support PSR-standardized RDF interfaces? #289?
• I support the point of having specialized libraries rather than a big monolith. Therefore:
  • hardf for parsing (supports most formats except RDF/XML AFAIK)
  • ARC2 to still have some kind of a triple store using a RDBMS
  • Split EasyRdf and use only the parts that cover uncovered areas
• use a single repository for development (like Symfony does) and split up packages using Git subtree-feature. It leads to separate ready-only Git repositories. Development takes place at a central point and is not split between different locations. But due to separate repositories, install only the packages you need.
• Don't aim for the sky but setup for the long run. As long as there are open pull requests or un-answered issues, we shouldn't be talking about the reworking the whole library. We should clean house and automate as much as possible for the maintainer(s) (e.g. external service for code reviews). No one gained anything, if the work stops half the way.

I did my fork https://github.com/sweetyrdf/easyrdf to save EasyRdf's "legacy" and enable others to keep using this library and have a stable basement, even though there might be no new features in the future. I'd be happy to switch course (e.g. remove my fork) and support you, depending on the plans.

[osma]

Hi everyone!

Thank you @k00ni for taking the initiative and creating a plausible basis for a sustainable EasyRdf project, and to @njh for not only creating and maintaining EasyRdf for so long but also explaining your views on the situation and the future in the above comment. Your insight is really valuable.

I am one of the maintainers of Skosmos, a web publishing tool for SKOS/RDF vocabularies. We've been using EasyRdf since many years back. We've also occasionally contributed something back to EasyRdf, for example a faster Turtle parser (#295) and HTTP timeout support (#234, #235). We've obviously noticed that EasyRdf is not very actively maintained anymore and had occasional issues, for example lack of PHP 7.4 compatibility. Other than that, we are quite happy with the current status of EasyRdf and see no need for big overhauls from our perspective.

Here are the parts that we use:

• EasyRdf\Sparql\Client for access to a SPARQL endpoint (usually Jena Fuseki)
• loading external Linked Data resources via EasyRdf\Graph::newAndLoad
• EasyRdf\Graph traversal operations to get information out of retrieved Linked Data and SPARQL CONSTRUCT results
• Turtle parser (for reading the Skosmos configuration file)
• serializers for Turtle, RDF/XML, JSON-LD (via ML\JsonLD)
• EasyRdf\Namespace

Parts that we don't use:

• EasyRdf\GraphStore (we do use the HTTP Graph Store protocol, but not from PHP code)
• EasyRdf\TypeMapper (we just look at individual triples, don't need ORM style mapping)
• EasyRdf\Isomorphic
• parsers and serializers that rely on external tools (ARC2, rapper, Redland, GraphViz...)

Regarding the future, I agree with the other comments above that it would be good to transition the maintenance to a GitHub organization, not an individual account. This gives more flexibility in assigning permissions and roles and is more of a neutral space. The changes by @k00ni in the new fork look very reasonable to me, so I hope these could form a basis for a future maintenance model.

[njh]

Thanks for your comments @k00ni and @osma.
Would you both like to become PR approvers in the new organisation?

I think for now I am going to focus on releasing version 1.0.0.
Sorting out the documentation generation turned out to be pretty easy.
I have also fixed the broken examples, so the tests pass.

The make command in my CI system is working for the first time in a long time now:
make clean lint cs test-lib docs dist

I am going to try and get the website build working again, and then should be pretty close to making a release.

[k00ni]

Hi @njh,

Would you both like to become PR approvers in the new organisation?

Thank your for the invite.

I saw that you did a lot in the last 2 days. Solving issues and updating code is great, but i am wondering what the actual plan is?

It would be great if you could give me some feedback on the following questions:

• Is your agenda fixed now? I am refering to the list in your detailed post above (e.g. with Modernise for PHP 7?)
• Do you consider one of my suggestions i made in my post before?
• Do you plan to use code from my fork https://github.com/sweetyrdf/easyrdf?
• How would you organize the work of us "approvers"? Do we only approve/decline PR's, do we have commit rights, set labels, can we shape the agenda?

[demiankatz]

Thanks, everyone, for moving this forward -- these are developments I've been hoping to see for several years now, and I really appreciate the time and effort spent! My time is already spread very thin across multiple open source projects, but I'm certainly willing to contribute a little effort here now and then to keep the project alive, since I depend upon it pretty heavily for https://github.com/demiankatz/Geeby-Deeby. If it would be helpful to put me on the list in the new organization, I'm willing to help out when I can... but since I'm unlikely to offer more than occasional assistance, I leave it to your discretion. :-)

[neclimdul]

Great news! look forward to helping move this forward.

[njh]

Hi @k00ni,

Is my agenda fixed? Sort of. My current plan is to focus on releasing version 1.0, including releasing a new version of the website that documents that new version. I don't want to release a version with (in some cases completely) broken examples. Once version 1.0 is out, I would hope to do a 1.1 release with more fixes and minor enhancements. I have been going through some of the issues and Pull Requests to check the seriousness of the problems. I do keep getting tempted to fix more stuff but the focus should be to get something released.

Beyond that I don't have any strong plans, other than really wanting to replace the HTTP Client. The list of stuff was more of a brain dump than anything else.

Responding to your suggestions

• Your first two suggestions sound interesting, but do seem like quite big and wide-reaching changes. Would be good to consider this again once more of the basic stuff has been sorted out. I don't know how many PHP devs aren't using composer but I quite like that it is currently possible to download a single tarball and get up and running.

• Using a single repository for development and split up packages using the Git subtree-feature sounds like a good solution. I didn't know that was possible.

• Totally Agree with this: "As long as there are open pull requests or un-answered issues, we shouldn't be talking about the reworking the whole library."

Do you plan to use code from my fork

I am sorry you did lots of work on the assumption that the project was completely dead, before I responded. I am hoping that it will be possible to cherry-pick / merge your changes in.

How would you organise the work of us "approvers"

To keep things simple, I am going to get on with releasing version 1.0 myself in the current setup.

Once that is released:

• move the repo into the easyrdf organisation (still looking for some advice on how safe that is)
• nobody (including myself) will commit direct to master, changes can only be merged via a Pull Request
• somebody other than the author will then need to review / accept the Pull Request
• it would be great to have some help setting labels and responding to/closing issues. Would be good to assign issues to releases too
• work out if the www.easyrdf.org website can be moved somewhere where multiple people can easily release to it
• I would love some help with shaping the future. I would appreciate some advice on the best way to plan / come to a consensus on the direction and roadmap for the project.

nick.

[demiankatz]

@njh, to respond to a couple points out of that long message:

1.) Regarding HTTP library, I wonder if it would be least intrusive to switch to Laminas\Http, as that is the still-maintained successor to Zend_HTTP and is comparable to it in much the way that EasyRDF 1.x will compare to EasyRDF 0.x. I think it also implements some of the interop interfaces, which could make it easier to eventually back out into something more generic.

2.) In my experience, moving GitHub repositories is pretty seamless. I wouldn't be too worried about breaking anything.

[njh]

Thanks @demiankatz.

1. Hm, ok. Worth considering if moving to a PSR interface / Guzzle is looking like a lot of work.
2. Thanks! Just made the move! 🎉

[njh]

Progress Update:

• Move to new organisation ✅
• Update / fix packagist ✅
• Re-enable Travis for new repo location ✅
• Fix and deploy new website: ✅

https://www.easyrdf.org/

[k00ni]

Just in case,

• did you synced your account on Travis side?
• in settings > applications: you can select Travis and manage organization access.

Maybe it helps.

[njh]

Yeah, whenever I press the 'Sync Account' button, it spins for a while then nothing happens.
The first time I pressed it, the list of organisations I am a member of disappeared.
I suspect that there are too many repos for it to load.

However I have just had success with the New way of using Travis as an App. Marketplace -> Travis -> Open Source -> Install for Free.

The difference is that is is on travis-ci.com not travis-ci.org:
https://www.travis-ci.com/github/easyrdf/easyrdf ✅

[k00ni]

And what about my second point? (settings > applications, grant access for organization easyrdf)

[k00ni]

What needs to be done before you release 1.0.0? In the milestone https://github.com/easyrdf/easyrdf/milestone/10 there is only this issue and #318 which seems solved.

[njh]

And what about my second point? (settings > applications, grant access for organization easyrdf)

Permissions were set correctly. I think I am happy with using the new 'apps' way of using Travis CI. I understand that everything will eventually be migrated there. Will just have to remember to go to the right URL: https://www.travis-ci.com/github/easyrdf/easyrdf

[njh]

What needs to be done before you release 1.0.0?

Very little I think. Doing a git log 0.9.0...HEAD and ensure that all the changes are documented in CHANGES.md, I think is the only thing.

I am currently trying to fix the www.easyrdf.org website. Hopefully not too much more to do. I want to do get that done before releasing, incase there are any small changes require to the published .tar.gz - which the website generates itself from.

[osma]

Wow, so much progress here during the last few days. Great work @njh on the new release and everyone else for the support! Really looking forward to the 1.0 release.

I hope that some of @k00ni:s work on the fork could be merged back upstream, perhaps for 1.1. I haven't looked at the specifics in very much detail but both @njh and @k00ni seem to have improved the unit test and CI setup. Maybe @k00ni could create PRs for some items?

One further suggestion: Would it be helpful to set up integrations with quality analysis tools such as Code Climate, Codacy, SensioLabs Insight, Scrutinizer and/or SonarQube/SonarCloud, in addition to Travis CI and Codecov? In my experience, they can provide valuable insight into bugs and other quality issues (e.g. dead code) that are otherwise hard to discover and pinpoint sections in the code base that would be good candidates for refactoring. They are especially valuable for the near-instant review of PRs: they give feedback on quality issues in the new code. I'm finding that with these tools in use, I tend to open draft PRs very early in the process just to get the automated feedback. All of the tools I mention are free (as in beer) for open source projects and support PHP - we use them all in Skosmos development. They all have their strenghts and weaknesses (e.g. false positives) but the combination is powerful.

This was addressed to me:

Would you both like to become PR approvers in the new organisation?

Thanks for asking. I'm a bit torn on this. As I said above I'm mostly happy with the current state of EasyRdf and don't have a need for major improvements - I just want the project to stay viable. I'd be happy to just remain as a user and occasional contributor. But if you are aiming for a broad maintainership model, I wouldn't object to becoming a PR reviewer.

[k00ni]

I wait until 1.0 is released so there is stable basement for further development. Afterwards i surely can create PRs and "move" back some code of my fork.

One further suggestion: Would it be helpful to set up integrations with quality analysis tools ...

That would be great. Scrutinizer is helpful and covers multiple areas, like tests, coding styles and code smells. We could also include tools like PHPStan (https://github.com/phpstan/phpstan) which provides static code analysis. If included in Travis it will mark PRs which violate certain rules.

I ran PHP-CS-Fixer on my fork and it changed over 130 files (see https://github.com/sweetyrdf/easyrdf/pull/12/files). If we deploy tools like that, ALL pending PRs should be merged or canceled, otherwise we will run into a merging-hell.

[njh]

Sorry, slow progress. But I am making little steps each day. Today I fixed the formatting errors on the documentation pages by switching to parsedown.

https://www.easyrdf.org/docs/getting-started

[njh]

Sorry, I have not found time to work on any personal projects in the past couple of weeks.

However I have just spent time to reviewing everything that has changed since version 0.9.0 (a lot!) and have now updated the CHANGELOG:
https://github.com/easyrdf/easyrdf/blob/master/CHANGELOG.md

Slowly getting there...

[sanduhrs]

Also waiting for the 1.0 release @ https://www.drupal.org/project/drupal/issues/3110972
Anything we can do? Thanks for all the work!

[k00ni]

@njh: Can you tell us what is left to do?

As you stated before

To keep things simple, I am going to get on with releasing version 1.0 myself in the current setup.

you want to work on 1.0.0 alone so i assume we can't help to speed it up? You/we should not let pass the opportunity to be part of the Drupal 9 setup.

[njh]

@k00ni I think it is done now.

The key things I wanted to sort out are now done:

• Fixed/removed broken examples and tests
• Sorted out formatting in RDF/PHP and RDF/JSON specification
• Updated Changelog ready for 1.0.0 release
• Sorted out the API documentation generation
• Ensure that things work properly in PHP 7.1+
• Updated the README

I have just tagged version 1.0.0-rc.1.

The things I want to sort out on the website:

• Sort out page title index in the documentation
• Formatting error on the example on the homepage
• Fix API documentation index displaying recursive embedding of frames
• Disable converter outputs that depend on Graphviz
• Finish deploying to new server (that will have HTTPS)

I have just found lots of (unanswered) questions on StackOverflow:
https://stackoverflow.com/questions/tagged/easyrdf

If anyone has to time to help answer some of them, that would be great.

[k00ni]

So, is now a good time to prepare new pull requests?

In my fork I reworked the tooling and rescued some of the pull requests from this repository. Lately PHPStan was added (to enable static code analysis on a basic level) and refined test suite as a whole (https://github.com/sweetyrdf/easyrdf/pull/19).

@zozlak helped a lot by adding further information, fixes and tests (https://github.com/sweetyrdf/easyrdf/pull/18, https://github.com/sweetyrdf/easyrdf/pull/16, https://github.com/sweetyrdf/easyrdf/pull/11).

Any preferences about how to bring back code from my fork?

[njh]

Yes, now would be a good time to prepare Pull Requests, if you have time. Hopefully 1.0.0 won't be very different to 1.0.0-rc.1 but I am trying to fix the broken API reference, which is generated from the main easyrdf repo:
https://new.easyrdf.org/docs/api

If it is possible, it would be good to merge things in small chunks, as it is much easier to review.
But I appreciate that might be a lot of work.

[k00ni]

I created pull requests for all the code parts which can be merged independently from https://github.com/sweetyrdf/easyrdf.

What is left on my list are changes in Travis, tooling, composer dependencies and the test environment (https://github.com/sweetyrdf/easyrdf/pull/19). In my opinion these only makes sense when grouped in one PR. But lets see how my current PRs are first and if there are any problems. After we cleared and merged them, i will create further PRs.

[pfrenssen]

It seems the new 1.0.0-rc.1 version is not being picked up by Packagist: https://packagist.org/packages/easyrdf/easyrdf

It reports though that auto-updates are working, and the reported check has taken place after 1.0.0-rc.1 has been released. I wonder what could be the problem?

[demiankatz]

@pfrenssen, are you able to try logging into packagist and forcing a manual check from that end? If you haven't already done that, it might help narrow down whether something is glitchy about the GitHub integration, or it's something to do with repo URLs, tag naming conventions, or something else.

[pfrenssen]

@demiankatz I had done a manual check earlier today, but now I see that the version has been added! Possibly this works with some delay?

[demiankatz]

@pfrenssen, that's great news! I don't publish to Packagist all that often, so I'm not sure what qualifies as a "normal" delay, but it does seem it's not totally instantaneous sometimes.

[njh]

@pfrenssen thanks for testing and for your feedback.

Packagist can be a bit tricky to diagnose sometimes - I wasn't seeing any error messages, or logs and pressing the Update button wasn't doing anything. Then I realised that the version in composer.json was different to the tag (1.0.0-rc.1. vs. 1.0.0-alpha.1) - whoops. I corrected it, re-tagged and packagist quickly started picking it up.

Thanks again for letting me know about this.

[zozlak]

@njh packagist doesn't need the version number in the composer.json so the easiest way to avoid such issues is to remove it from composer.json and use only github releases.

[njh]

@zozlak The reason why I put the version number in composer.json is so that EasyRdf itself know its own version number and can use make dist to build a tarball for release.

I still think it is useful for people (novices?) to be able to download a .tar.gz which includes pre-built documentation and generated files; everything that they need to get started...

[zozlak]

My personal opinion is:

• You can read the release tag using git. This way you have a single source of truth regarding the current release number and you don't have to assure they are in line with each other by hand (btw after forking the EasyRdf it took me some time to figure out why composer doesn't pick up my github releases - it's definitely not developer friendly)
• Unpacking a whole tar.gz might be the easiest way 10 years ago but nowadays the easiest way is to use a package manager. And there are countless other reasons for preferring installation with a package manager over downloading a .tar.gz by hand.
• I'm betting 9 in 10 people googles for the documentation instead of searching in the downloaded source (especially newbies).

[njh]

Thanks for your feedback @zozlak. Not working after forking is definitely not a good thing.

It is is slightly terrifying that the first commits were in 2009 and that the project is over 10 years old. And indeed a lot has changed in that time. However it is also worth remembering that still not everyone is using package managers, or understands Github. There is still a world of people uploading to shared servers. One of the good things about PHP is that it is really easy to get up and running, without having much development experience.

I totally agree about Googling documentation. At the moment the website build process uses that documentation that was generated as part of the release, but maybe I should change that.

[njh]

Version 1.0.0 is released: https://groups.google.com/g/easyrdf/c/MokuMeJJbEs

• Release tagged and registered with Packagist
• Updated website launched

I will start thinking about how to continue with this project tomorrow but for now, I am going to close this issue! 🎉

[sanduhrs]

That's great, thank you!

[k00ni]

I will maintain EasyRdf in my fork at https://github.com/sweetrdf/easyrdf and welcome anyone to use it and give feedback.

[njh]

Just to be clear that I intend to continue to maintain EasyRdf here at https://github.com/easyrdf/easyrdf and https://www.easyrdf.org/

@k00ni is very welcome to create a fork - but it will be something new and different to EasyRdf.