Future plans for wallabag

[tcitworld]

I open here so we can discuss it.

API

The API is certainly the most blocking thing now with the features we want to see in wallabag. It will make (nearly) everything possible. Third-party apps could have exactly the same features than the webapp has, so I consider it on top of our todo-list.

Why can't an API be in wallabag right now ?

Because wallabag is not written for it. Actions are most of the time not handled by a unique function. It's not very oriented-object too, and this is much better for building an API.

Our options

• change the major part of wallabag (the poche.inc.php file) to make it clean and usable. Use more oriented-object, create more classes. The rest can stay as it is.
• rewrite all from scratch (good luck)
• use a framework (Silex, Symphony, etc.) and rewrite wallabag on top of it.
  I think the best option would be to use a framework, but it needs devs who know it a little.

wallabag v2 - A framework ?

Previous paragraph leads us here. I've read somewhere that over 100 (or was it 1000 ?), lines of code, using of a framework was starting to be better for your app. wallabag is way over this limit, and the way it works suits to the use of a framework (MVC).
I leave up to you to discuss the framework we should use, I've no experience with that.

Third-party apps

Even though we have no API, it seems to me they could be improved. But there's no need to do it now if we focus on the API, because we will have to rewrite them (at least a large part) anyway for the API.
Yup, I'm really not saying anything constructive here.

Things that can be done before

I think the best thing to do before starting work on v2 (whatever are the plans) is to deliver a stable release.
Can we be stable right now or should we add more features ? These features could be :

• epub export It's detached of the rest of wallabag's code, not much influence, the library handles most of it. I should merge this soon.
• Change database structure (Change database structure #620) This can lead in a lot of improvements, not all necessary directly implemented :
  • Save metadata (Better Metadata: Harvest and display source, author and date #629)
  • Saving PDFs/plain text files (Saving a PDF document #116)
  • Comments about articles (Add comment about wallabaged page #592)
  • public (or restricted) articles
  • Find articles based on estimated reading time (Find articles with reading time = X #393)
  • Edit title, content ? (~Display better title than "Untitled" #564)
  • ...
• Display/appearance/theme changes Apart from huge changes, should not be a problem.

Conclusion

Should we start the big changes ? when ? with what method ?
Please give your advice. And as always, help is always welcomed (now it would be very welcomed)

[pVesian]

I agree with the API & v2 points.
For the use of a framework, that has to be discussed, since it's a very
trollish subject. However, if the use of a framework is necessary, it
should be a framework supporting "popular" PHP versions. Some free host
services haven't upgraded or don't allow their users to change the config /
install modules, ...

In order to make a clean refactoring, there shouldn't be any feature
additions : KISS, so the migratiion goes with less trouble.

2014-05-13 16:54 GMT+02:00 tcitworld notifications@github.com:

I open here so we can discuss it.
API

The API is certainly the most blocking thing now with the features we want
to see in wallabag. It will make (nearly) everything possible. Third-party
apps could have exactly the same features than the webapp has, so I
consider it on top of our todo-list.
Why can't an API be in wallabag right now ?

Because wallabag is not written for it. Actions are most of the time not
handled by a unique function. It's not very oriented-object too, and this
is much better for building an API.
Our options

• change the major part of wallabag (the poche.inc.php file) to make
  it clean and usable. Use more oriented-object, create more classes. The
  rest can stay as it is.
• rewrite all from scratch (good luck)
• use a framework (Silex, Symphony, etc.) and rewrite wallabag on
  top of it. I think the best option would be to use a framework, but it
  needs devs who know it a little.

wallabag v2 - A framework ?

Previous paragraph leads us here. I've read somewhere that over 100 (or
was it 1000 ?), lines of code, using of a framework was starting to be
better for your app. wallabag is way over this limit, and the way it works
suits to the use of a framework (MVC).
I leave up to you to discuss the framework we should use, I've no
experience with that.
Third-party apps

Even though we have no API, it seems to me they could be improved. But
there's no need to do it now if we focus on the API, because we will have
to rewrite them (at least a large part) anyway for the API.
Yup, I'm really not saying anything constructive here.
Things that can be done before

I think the best thing to do before starting work on v2 (whatever are the
plans) is to deliver a stable release.
Can we be stable right now or should we add more features ? These features
could be :

• epub export It's detached of the rest of wallabag's code, not much
  influence, the library handles most of it. I should merge this soon.
• Change database structure (Change database structure #620Change database structure #620)
  This can lead in a lot of improvements, not all necessary directly
  implemented :
  • Save metadata (Better Metadata: Harvest and display source, author and date #629Better Metadata: Harvest and display source, author and date #629
    )
  • Saving PDFs/plain text files (Saving a PDF document #116Saving a PDF document #116
    )
  • Comments about articles (Add comment about wallabaged page #592Add comment about wallabaged page #592
    )
  • public (or restricted) articles
  • Find articles based on estimated reading time (Find articles with reading time = X #393Find articles with reading time = X #393
    )
  • Edit title, content ? (~Display better title than "Untitled" #564Display better title than "Untitled" #564
    )
  • ...
    • Display changes Apart from huge changes, should not be a problem.

##Conclusion##
Should we start the big changes ? when ? with what method ?
Please give your advice. And as always, help is always welcomed (now it
would be very welcomed)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/687
.

[vjousse]

We started to work on it some time ago using Silex. You can have a look at #414 and https://github.com/wallabag/wallabag/tree/v2-silex

I think that Silex is a good choice, but in order for a v2 to emerge, I think that the v1 should be frozen (only bug fixes added to it). Everybody should switch to working on it.

I was the one starting it but I don't think I will have any time to put into it in a near future. I hope that @nicosomb or somebody else can continue the development.

[a-zen]

change the major part of wallabag (the poche.inc.php file) to make it clean and usable. Use more
oriented-object, create more classes. The rest can stay as it is.

I think it would be possible to integrate the api in the current code, without cleaning that much.
The features are sort of hidden in the "action" function. We could extract the actions from the switch case statement to functions to make them reusable. After that we could implement the api as described in issue 414 and make use of these functions. I think the biggest thing missing therefore is a routing component.

use a framework (Silex, Symphony, etc.) and rewrite wallabag on top of it. I think the best option
would be to use a framework, but it needs devs who know it a little.

This is a serious question: What do you think to get from using framework? Sorry but I'm not really seeing it, besides permanently binding to the framework.

wallabag is way over this limit, and the way it works suits to the use of a framework (MVC).

Again this is a serious question: What do you think would be gained by using MVC? From my understanding there is a benefit, when you have data which you want to display in different formats, e.g. tables, charts. But with wallabag you just want to display text with some pictures.

[vjousse]

If you want Wallabag to be a serious project and thus, integrate automatic testing, using a Framework and MVC seems a good choice.

[nicosomb]

I agreee with @vjousse.
If we want to be serious (and to find some contributors too), a framework is necessary.

@tcitworld I'll try to answer soon.

[EliasZ]

I once started porting Wallabag to Laravel but due to lack of time/interest I abandoned it prematurely. Feel free to use it as a base and if I am on a time surplus I will be sure to contribute.

[mariroz]

As for me, I think, that if we don't want to deal with "second-system syndrome" (http://en.wikipedia.org/wiki/Second-system_effect), will be good to froze v1 (except of bugfixes) like it's proposed by @vjousse above. I.e. no more new features in v.1 and clear plan about what to add to v.2 for the beginning: in theory NO new visible features - only deep refactoring and API. Will be good to make it as soon as possible and only then start to add more features.
Every new feature in v.1 introduces entropy and makes development of v.2 more difficult and delayed in time.

I also think, that framework is absolutely necessary.
Framework will give us next advantages:

1. unit testing
2. db migrations
3. code guidelines (i.e. project structure)
4. sameness in handling of request data, form validation
5. sameness in handling of db queries (DBA, Active Record, may be not necessary - don't know)
6. I18N support
7. and more...

What framework to use? Hard question.
Preferable is, of course, something lightweight, as wallabag has to stay easy to install/deploy and minimalistic like now. Maybe here we need to make some investigation.
I have an experience in ZF and Yii. ZF can't recommend for wallabag - too heavy. Yii seems also to be too advanced for wallabag, but if you will decide to test it, I can prepare some prototype quickly.
Main problem with fully OOP frameworks is in huge amount of files/classes and related to it slowness: most requires some cache techniques to speed up.

What, I think is really NOT needed, is Bootstrap (http://getbootstrap.com/) for site design: it is very fashionable for the moment but isn't really usable for clear, silent projects. However, maybe some css pre-processor (maybe "less") may be helpfull for our themes as will bring more order in css (this, especially can help a lot with generating of css filed for all default-based themes). But, anyway, this can be next step after v.2 release: refactoring of themes.

[versvs]

Hi,

really interesting thread here. I fear I will not be the one contributing a lot of commits (though I can help keeping up-to-date the es_ES translation), so technical decisions like what framework to use are far from my reach: I will recommend to use whatever framework the guys maintaining the core of wallabag are comfortable with.

From an users perspective, however, I have an opinion:

• Since I have blogged about Wallabag a few times, some people I know have their own instances running. This is so because Wallabag can be installed and run even in "shared servers". A lot of people wouldn't be using Wallabag if it required a dedicated server.
• The benefits of using a framework probably outweight the drawbacks. Being forced to use some "good practices" will probably help the project to be more maintenable, and also more easy to learn for whoever that finds time enough to commit an improvement/fix.
• I support the "general consensus" of not adding new features for the moment, and focus on the port to the brand new v2.
• Regarding the API question, but also touching the framework issue, we may find a crazy idea: what about using Drupal as base? It is a "CMS-that-wants-to-be-framework". Drupal 8 comes with a RESTful API inside. (Possible drawback, though it is definitely lighter than ZF, maybe it is not what we would call a "light framework".) I think that Wallabag is better if kept simple and light, but if using a heavy base is mandatory, Drupal maybe provides some help.
• Have we discarded the good old XML-RPC API? As I said, there are possibly technical issues above my understanding, but for an API that don't require a lot of time being developed and maintained, and that is also widely known for many developers, this could be a good issue. I know it is not like using JSON and all these things that are now trendy and cool to have (they put a lot of the app in the client side, thus allowing Wallabag to be more of a webapp itself, which is perhaps a nice achievement).

As I said, though I can code some PHP, I'm not a PHP-jedi, and also I'm not finding a lot of time lately, but you can count me on for small tasks (maybe localization, maybe some really simple PHP fixes, and also some brainstorming like now :D).

[tcitworld]

Hey.
@xppppp has send a PR for an (very) simple implementation for API (it only permits to add an URL). What do you think about going, a little, in this direction ?

In other news, @Debetux started taking a look at what @vjousse already made on v2-silex. This may be what is the most likely to become wallabag's v2.

[versvs]

Agree with both ideas: the API should be really simple (the notion of "Second system" is really powerful, a path to avoid :D) and not reinventing the wheel seems also a good idea (siles, alas, is lighter than many other options).

[nicosomb]

Like @mariroz @ @vjousse said, we have to freeze v1 development.

But, if @xppppp developed a little API which can help us for Firefox / Android / etc. applications, why not?

[vjousse]

@nicosomb exactly for the reason why the v1 has to be frozen: if it's not, there will be no v2.

If you provide an API for v1, people will start using it, asking for missing features, and then you'll stay with v1 for a long time and there will be no more motivation for the v2. This is fine for me, but you need to be aware of it.

[tcitworld]

That's the kind of answer I was expecting at first. I agree.

[xppppp]

One could make two arguments here. The first is that you already have half of an API, which is the retrieval of content via RSS. The programmatic availability of the data allows for the ecosystem of apps to be developed. Adding a concise programmatic way to add content would essentially complete V1.

The second argument would be that if you provide an v1 API that you can architecturally support in v2, you allow for the ecosystem to grow instead of saying "wait for v2." Then when v2 rolls out, as long as it externally supports the v1 API (i.e. a POST to api.php with parameters X yields Y result), everybody's happy.

[vjousse]

Supporting backward compatibility is a pain in the ass, believe me. You should do it only if you have to. Here we have the possibility to not do it by starting with a fresh v2.

To me, it's just a timing problem: if we define milestones and a rough deadline for v2, the API may be soon available.

[xppppp]

I understand about becoming "mired in antiquity". The other side of that coin is building and maintaining momentum in your ecosystem. If you can get app developers to add first class (at the same level as Pocket) support for wallabag/poche now instead of later, maybe that makes that pain in the ass worthwhile. To do that, all you really need is the ADD verb. Then you can ask the developer of your reader of choice to add first class wallabag support now. That certainly broadens your audience for v2.

[tcitworld]

Well, It's been more than one year since wallabag was created, and since nearly the beginning (remember #31) we realised we needed to provide an API. We waited until now, I guess we can wait for a few more weeks (perhaps less, why not ?).

However, I also think as long it does not take much of our time (which is now devoted to v2 :) ), we could consider merging some of your changes into the wallabag 1.x branch. But the first condition would be for it to respect the format stated at #414, so that all the 3rd party apps could be ready without changes.

[xppppp]

I personally don't mind if you pull in my changes or not. I thought you'd find them useful.

In any case, the full API implementation for v1 would be just reinventing the wheel so I'll just wait for you to finish off v2.

[tcitworld]

Well, not whole of it, only what you propose, the add article feature. API implementations can be started in apps even if it's not ready on the server side (but it is certainly uncomfortable).

[xppppp]

That would make sense, except for authentication. You don't want to say you're going to support username and password authentication and then pull it out and say, "now go use OAuth."

[tcitworld]

Hum, I didn't thought about that. In that case, my advice is we better work only on v2.

[nicosomb]

Well. I understand I have to make a decision.

If we want to have a more robust application, we have to work on 2.x. We don't have to waste any time to develop features on wallabag 1.x.
@Debetux wants to help us on 2.x. We'll use v2-silex branch, from @vjousse.
Everyone is welcome on 2.x: @Debetux, @vjousse, @mariroz, @tcitworld, etc.

We have to release a first beta this year, with basic features of wallabag.

In next days, @tcitworld and I will make a clean up in issues: all new features will be set for 2.x.
Next releases of wallabag 1.x will only concern security / bugs fixes.

[anarcat]

one of the thing the Mythical Man-month teaches us is that the "team will design a throw-away system (whether it intends to or not). This system acts as a "pilot plant" that reveals techniques that will subsequently cause a complete redesign of the system".

In other words, if the current code infrastructure is too tricky to implement an API, maybe it should be discarded. I know this may feel like an insurmontable challenge, but I have had experience dealing with old codebases and nothing short of a rewrite could fix some fundamental problems with the code.

Also, I am personally very critical of PHP in general (see this article for why so I would suggest considering another programming language framework, especially now that hosting other frameworks has become more accessible.

I would also like to point out the very interesting Bookie project which I had the chance to try out. It is very similar to poche in terms of functionality and objectives, and is well designed. Maybe all that would be needed would be to implement the Poche features missing from Bookie there? So far, i have found that only the "read/unread" status is missing, something which should be fairly easy to implement.

To conclude, I am sorry in advance if my comments seem a little too radical, but I read the blog post and it said to not hesitate to give feedback. :) I like both Wallabag and Bookie, and I feel it's unfortunate to see the projects duplicate their efforts like this: united we are stronger!

[tcitworld]

Also, I am personally very critical of PHP in general

I'm sure that a lot of people agree with this (I totally prefer Python and Django as a framework), however PHP stays the default language for all of shared hosting plans, and it assures maximum compatibility.

About Bookie, I'll say again what I've just said : it needs a dedicated server, so it's not really for everyone*. And judging from the documentation (I didn't test it) I don't see much features wallabag doesn't have, knowing the projects have different goals. The main goal of wallabag isn't bookmarks management, even though it could be used like this.

* I know VPS are becoming really cheap, but novices will always prefer to start with shared hosting. I'm sure there's stats somewhere about this.

[anarcat]

with hosting platforms like heroku popping up for accessible prices, i am not sure the argument for PHP holds so much anymore. i know that we at koumbit, as a hosting provider, are certainly looking at hosting more than PHP applications exactly for this reason.

but i understand your position, i tried. :)

[tcitworld]

i know that we at koumbit, as a hosting provider, are certainly looking at hosting more than PHP applications exactly for this reason.

And this makes you a good hosting provider. :) Unfortunately, some providers are still crap.

[fireflystorm]

Was wondering what's the situation with API v2...
Issue 414 seems quiet and I can't figure out if there is active development on API v2.

[tcitworld]

And is now planned for v1.8 and seems pretty good so far. We have big changes coming (at last!).

[fireflystorm]

Nice. What's the planned release date and where can I find the v 1.8 API documentation?

[brunelli]

v1.8 released. What is the API status?

[nicosomb]

We failed ... because we (re) planned it for v2, which is available here https://github.com/wallabag/wallabag/tree/v2-symfony
I have to put API doc online. Stay tuned here.

[nicosomb]

We are not dead :) https://framasphere.org/posts/367022 v2 is coming.

[ptah-alexs]

How about php7 support in newest versions of wallabag?

[tcitworld]

PHP7 is already supported and even advised.

[ptah-alexs]

Ok,although, I didn't found it in google.

[tcitworld]

About ?

[ptah-alexs]

Search query "wallabag php7" doesn't clearify about wallabag's support php 7 .

[tcitworld]

I'll clarify it in the documentation.

[nicosomb]

@ptah-alexs you can also have a look to our testsuite: https://travis-ci.org/wallabag/wallabag/builds/115307113 all is green with PHP7.

[ptah-alexs]

Ok thanks.

[nicosomb]

I close this issue: we are in the future today :-) and wallabag v2 is born. I think this issue is useless, you can reopen it if you think it's necessary.