Improving development process and quality of the app

[fschrempf]

I'm just a regular NC user/admin and don't know much about NC development and internals, but I made a few observations, especially with the groupfolders app that kind of make me feel like there should be some improvements.

1. There are a lot of open and unhandled issues (no replies, no labels, etc.).
2. There are a lot of duplicate issues.
3. There are quite a few severe bugs that exist for years and haven't been fixed.
4. Some features (e.g. ACL) are quite complex and there's not enough docs.

I guess that this app is used by many people and should get the same level of attention and maintenance as the NC server repository.

Please note, that I don't mean to blame anyone. I really appreciate the huge amount of work all of you put into this.
Still I would like to ask if anyone sees some way to improve things? Is there even an officially assigned maintainer or team of maintainers for this app?

I would be even willing to add some small bounties on long-standing issues that affect me if there is any reliable platform to do this.

[fschrempf]

#1201 is somewhat related.

[ChristophWurst]

Thanks for the feedback @fschrempf 🙏

I agree with you on these points. As I see it the app is extremely important to the community and enterprise users, but it's also complex and still under a certain level of development. Right now most of this is in the hand of a few individuals. So that is where the scaling hits a limit.

I would be even willing to add some small bounties on long-standing issues that affect me if there is any reliable platform to do this.

I'm afraid this won't solve the problem. We need more people with some domain knowledge to help triage tickets, close duplicates, collect the most important tickets, hunt bugs down until there is a clear pattern that leads to an issue and so on. This would then take some load off the main developers, and they can then focus on more features and fixes. This isn't a recipe for guaranteed success but from what I personally experienced with other apps/projects that's usually what helps the most.

I hope that makes sense and creates some perspective why it looks like this app isn't maintained, while some people are simply too busy with other things ✌️

[ChristophWurst]

Is there even an officially assigned maintainer or team of maintainers for this app?

Btw this also won't magically improve anything. Nextcloud (the community) is loosely structured and I think we live the collective code ownership. Some people know more about this app, others less.

[fschrempf]

@ChristophWurst Thanks for joining the discussion!

As I see it the app is extremely important to the community and enterprise users,

We need more people with some domain knowledge to help triage tickets, close duplicates, collect the most important tickets, hunt bugs down until there is a clear pattern that leads to an issue and so on.

You mentioned the level of importance this app has and in contrast the lack of knowledgeable contributors. This situation is exactly what puzzles me. My expectation would be, that the interest for investing time and money here should be out there somewhere. At least that's the impression I get when I see all the media coverage and announcements of how NC is or will be used in public and private infrastructure, etc. How should NC compete with other products if things that people expect to be core features are - to put it in slightly exaggerated terms - buggy and unreliable over years?

This isn't a recipe for guaranteed success but from what I personally experienced with other apps/projects that's usually what helps the most.

Sounds reasonable. So the question is where are these people? 😉

Is there even an officially assigned maintainer or team of maintainers for this app?

Btw this also won't magically improve anything. Nextcloud (the community) is loosely structured and I think we live the collective code ownership. Some people know more about this app, others less.

Collective code ownership is all right up to a certain degree. I still think that to reach a satisfying level of quality in a sub-project like this, there needs to be someone who feels responsible and dedicated. Even if this someone does not have the resources to do the actual work, at least he/she could try to advertise the needs to the community or look for sponsors and contributors.

For someone not being part of the core community or even the NC GmbH a lot of things like this and if/how they are supposed to work are not really obvious.

Sorry for taking this on a wider level. Please don't feel pushed to pick up my arguments/questions. It's just my random thoughts I'd like to add here.

[ChristophWurst]

I can't comment on priorities of the company, I'm just here to shed some light from my personal perspective.

Sounds reasonable. So the question is where are these people? wink

hehe, this is a community project so either people show up and help or nobody does anything or even worse: expects other to do the hard work for them (for free).

Collective code ownership is all right up to a certain degree. I still think that to reach a satisfying level of quality in a sub-project like this, there needs to be someone who feels responsible and dedicated. Even if this someone does not have the resources to do the actual work, at least he/she could try to advertise the needs to the community or look for sponsors and contributors.

I agree but it's again easier said than done. If you are lucky you find people who fit well but you have to have some luck.

Sorry for taking this on a wider level. Please don't feel pushed to pick up my arguments/questions. It's just my random thoughts I'd like to add here.

Nono, I really appreciate some constructive input :)

[pierreozoux]

Ok, I did some stuff, hope it helps!
(I never really maintained software, and I don't know yet how Nc is using labels, and I took some freedom to answer some.)

So I took a look at the last page of issues, and marked some as to develop.

I'd love to offer a beer or any drink to both of you to discuss how can we improve here :) (and I mean, as a non paying user, of course, it would be just some comments, I have of course no rights to tell Nc what to do :) )

Still :) If I may, from the outside, I'd prefer Nc to put a bit more effort on stability of the existing before developing new features (like chat, which php is a terrible choice :) another debate). But again, I'm just saying out loud my preference, and nobody probably care :)

We are also actually looking to hire a freelance to do some PR on Nc (if you want, feel free to contact me PM).

@fschrempf do you also want to help triage issues? One thing is for sure, Nc is good with helping community members to be independent to help where they can :) I guess if you want to help it would be welcomed :) (It took me several hours just to look at the first page of issues and label them.. no code involved :) )

[fschrempf]

Ok, I did some stuff, hope it helps!
(I never really maintained software, and I don't know yet how Nc is using labels, and I took some freedom to answer some.)

So I took a look at the last page of issues, and marked some as to develop.

Great, thanks a lot! Getting things done is always the best to tackle such issues. 😄

Still :) If I may, from the outside, I'd prefer Nc to put a bit more effort on stability of the existing before developing new features (like chat, which php is a terrible choice :) another debate).

Exactly what I'm thinking, too. In general I think pushing out plenty of new fancy features without really caring enough about the quality and stability of the code that the "average user" will run is not a good idea.

In my daytime job I'm occasionally involved with some Linux kernel development. And while I know that it's difficult to compare such projects and there are other downsides of their development process, when it comes to quality and stability they really know how to do it right.

We are also actually looking to hire a freelance to do some PR on Nc (if you want, feel free to contact me PM).

I'm currently not in a position to generally fund NC development, but as said above I can imagine to add (rather small) out-of-pocket bounties on specific issues/PRs to support people working on this. But this only works if there is a platform to do this and others are willing to join.

@fschrempf do you also want to help triage issues? One thing is for sure, Nc is good with helping community members to be independent to help where they can :) I guess if you want to help it would be welcomed :) (It took me several hours just to look at the first page of issues and label them.. no code involved :) )

I can't offer to dedicate a constant amount of time for this, but I could imagine to do some issue triaging occasionally when I have a bit of time to spare. Though I don't really now much about the NC community so far, or even how to become a member if this is what is needed to get access to the repositories/issues.

One other thing I would like to get done somehow is to communicate very clearly what major known bugs are existing in the app. Triaging the according issues is one thing, but I think we should go even further and include a clearly visible section in the docs that informs users about what they might be up to when they use the app. And this will hopefully help to draw attention to the issues and help get them solved eventually. Not communicating this openly and "selling" this as a stable and featured app is not the right way in my opinion.

[icewind1991]

The help with triaging bugs is greatly appreciated!

Things have been very busy with support related work but I'll try to work trough some of the triaged tasks soon

[pierreozoux]

@fschrempf thanks :) I think together, we managed to triage them all :)

I created this issue:
#1414

And I'll remove the note from the README in a PR. (If you agree, just merge)
#1415

Added issue template to facilitate triage.
#1416

A bot would be nice, but I also find them annoying..

we do host quiet some Nextcloud instances (like a dozen), and we are wondering if we should continue to use it or not.
(https://git.indie.host/indiehost/tech/plateforme/-/issues/105 in french, sorry )

When I see this list and especially this bug that we investigated, it looks like GroupFolder is reimplementing some logic from core.
It looks like (i'm no developer) that either core should expose more API so that developer can develop app like group folders, or group folder is too tightly integrated with files and should just be part of core.
I have a feeling that this decision has to be made, if not, it is/will be a nightmare to maintain.

I'm now really interested to have the feedback of the dev team on this question. And the answer will help us take a decision.

[fschrempf]

@fschrempf thanks :) I think together, we managed to triage them all :)

Thanks! Great job! 🎉 There is still a lot of work to be done, but cleaning those issues is a good first step.

I created this issue:
#1414

And I'll remove the note from the README in a PR. (If you agree, just merge)
#1415

I already added some points to the pinned issue and the PR is merged.

Added issue template to facilitate triage.
#1416

Let's discuss this in the PR.

A bot would be nice, but I also find them annoying..

Hm, yes I'm not really sure if a bot is helpful and I don't have any experience with setting up one. We could try it, though.

we do host quiet some Nextcloud instances (like a dozen), and we are wondering if we should continue to use it or not.
(https://git.indie.host/indiehost/tech/plateforme/-/issues/105 in french, sorry )

My french is a bit rusty 😕, but I get what your discussion is about.

When I see this list and especially this bug that we investigated, it looks like GroupFolder is reimplementing some logic from core.
It looks like (i'm no developer) that either core should expose more API so that developer can develop app like group folders, or group folder is too tightly integrated with files and should just be part of core.
I have a feeling that this decision has to be made, if not, it is/will be a nightmare to maintain.

I can't really tell as I don't have time to really dive into the codebase, but from looking at all the different kind of issues I tend to support your conclusions.

I'm now really interested to have the feedback of the dev team on this question. And the answer will help us take a decision.

Yes, it would be great indeed, to have some of the developers join the discussion and get some perspective for the future.

Apart from the issue about code complexity and separation between the core and the app, probably also some more effort for testing would be needed for reaching a satisfying quality. I guess if we would have a good test coverage, we could prevent a lot of bugs and regressions and spend more of the valuable development efforts on better things like new features, etc.

[fschrempf]

We get new reports for all kinds of problems and bugs almost on a daily basis. Any ideas how to continue the improvement process?

My ideas currently are:

• Being more blunt about the current status of the app: README.md: Be more blunt about the current status of the app #1459
• Announcing a feature-freeze until ...
  • ... a big part of the major bugs have been resolved
  • ... (unit) testing has been improved

Also here are some things I noticed/wondered:

• How do regressions like this happen and how can they be prevented? Shouldn't CI be able to catch this?
• We have multiple bugs that have fixes in master, that haven't been released. How are fixes for stable branches released?
• There is a large group of users and reporters of issues. Is there some way to better include them in the process? Maybe there is a way to do alpha/beta releases to receive more testing before releasing?

[ghost]

@fschrempf @pierreozoux First of all, I'd like to thank both of you for starting to tackle the most annoying issues!

I'm not a developer, but became interrested in supporting OpenSource since using and administrating Nextcloud at work and private. I've quit a lot of experience in coding in VisualBasic (formost excel and acces), so I know how to read code. I'm now trying to understand the code of this app as we really need this functionality.

Therefore I'd like to offer some time for testing. I think, setting up a nightly builds, beta and rc channel will be a helpfull aproach.

Another thought: on trying to understand the code, I noticed there are 49 branches, 62 tags and 60 labels. Most of them probably of no relevance anymore. Would a cleanup be usefull? And maybe use a stale bot?

Another thing, as you allready noticed, improvements are backported but not released. Can using of milestones be helpfull? This is really good practice on nc-development. And who is responsible for releasing? I seems to me that @juliushaertl and @rullzer don't have the time to do the consequently releasing.

Anyway, let me know if I can be of assistance.

[ghost]

@fschrempf Another thought: can we pin this issue (or create one) as a basic collecting area for ideas and help offers?

[pierreozoux]

We actually decided that we'll no longer recommend using it, and use a "virtual" group user instead that creates the folder and share it. So I'll be less present in triaging bugs.

I also though about the feature freeze, I think it is a good idea, but then, it is always complicated to say no to one shot contributor that come to only add one feature..

Good luck to the Nc team :) I send you a lot of love! Choose wisely your battles :) (I think the chat and visio was not a wise battle to pick :) )

[fschrempf]

I agree but it's again easier said than done. If you are lucky you find people who fit well but you have to have some luck.

In the context of nextcloud/server#34351 I want to come back to this point from @ChristophWurst:

I don't think finding capable people to join the development efforts is only about luck. The main aspect in my opinion is maintaining a healthy community that is welcoming and having a healthy development process.

People considering to contribute and join the community will see issues like these, the sheer amount of issues without response, the many unresolved bugs, etc. and simply move on without ever looking at NC again. And I can't blame them.

[vasyugan]

Is there even an officially assigned maintainer or team of maintainers for this app?

Btw this also won't magically improve anything. Nextcloud (the community) is loosely structured and I think we live the collective code ownership. Some people know more about this app, others less.

Is groupfolders a community maintained app? It is among the featured apps and included in the "education" app package. If it is actually an app maintained by random volunteers, that's unexpected.