clarify motivation for alternative to unofficial docker-mautic

[aspiers]

(Also posted to mingfang/docker-mautic#6)

Hi there, firstly thanks a lot for sharing this great work!

I'm new to mautic so please forgive me if this is a newbie question, but I was wondering if you could clarify why you decided to build your own docker-mautic? I appreciate that there are problems with the official mautic/docker-mautic (e.g. it is not properly stateless as described in mautic/docker-mautic#2) so I am wondering why you didn't try to fix the problems there? Or did you try to contribute to the official project but encountered problems getting your contributions accepted?

I am asking because each time the community provides a different approach to the same thing, it causes fragmentation in the community, where development effort gets split between the different but overlapping approaches. This results in multiple groups independently trying to fix the same problems, and this slows down the community as a whole.

I think it would be great if the community could try to converge on a single solution. Then each problem only needs to get fixed once in a single place, and newcomers aren't faced with confusion over which solution to pick.

Does this sound reasonable to you? Thanks again!

[pjeby]

Sorry, I don't understand why you're asking this question. If you can tell me what goal you personally are trying to achieve, perhaps I can be of more assistance. Are you going to be using Mautic yourself, or supporting someone who is?

[aspiers]

Hi @pjeby, thanks for the reply. I've just started using Mautic for myself (deployed via the official Docker image before I discovered this repository), and as a long-time developer and supporter of Free and Open Source Software, I really want to see Mautic succeed!

After many years of contributing to various FOSS projects, I've learned (sometimes the hard way) what ingredients create really successful FOSS projects.

So my main goal in asking this question is to see if it's possible to achieve more unity within the Mautic community, especially regarding those of us using Docker to deploy Mautic. I really believe that everyone will benefit if we can converge on a single solution rather than fragmenting efforts across multiple solutions for the same thing. So I filed this issue to see if others agree with me, and if there's an appetite for this convergence. I hope that makes it clearer?

[pjeby]

Not really, no. I don't really see myself as part of "the Mautic community", in the sense of having any ability to steer that project or direct any resources other than my own. So it's sort of like you're coming and asking me if it would be a good idea for Canada's government to do something or other, when I live in the U.S. and don't even get involved in my own local politics. ;) I just made something to scratch my company's itches, and if it helps other people too, then great.

At the time I created this image, the mingfang image was the only thing close to what I needed, but rather than reuse it I chose to build on my dirtsimple/php-server image because it already provided the base features and the additional feature of privilege separation -- i.e. running the webserver under a different user ID that can't overwrite any code files.

Integrating my changes into either the mingfang or mautic upstreams is nontrivial because this project is 100% wedded to dirtsimple/php-server and its security features, let alone the templating features, etc. Even if they chose to take advantage of some of the non-docker-specific bits, I'd still have to maintain this image in order to run it under dirtsimple/php-server, and I neither have the need nor the political capital within the larger "Mautic community" to try and make such a thing happen.

Beyond the mere technical detail of what base image is used, there's a bigger issue of goals and directions in play: Mautic is built with the intention of being able to support running in 1990's era primitive shared hosts that do a single document root that basically covers the entire filesystem available to a single user ID, by dropping the whole thing in a directory and then installing/configuring it via the web UI.

This design philosophy is pretty much 100% diametrically opposed to good docker practices, and it can't be trivially worked around, either. Mautic, in its core directory layout and other design choices, is fundamentally broken for use in a properly secured environment and has to be worked around at every step to have a solid docker deployment. Even with what I've done in this project, I keep finding new ways in which it's broken. (For example, its log files are written using .php extensions, and it has the webserver writing PHP code to cache files that it will later execute! I hope to fix these issues, too, eventually.)

But mostly, you can't fix Mautic's security problems and still keep the ability to do "unzip and access via the web" installs on primitive shared hosting, so I don't expect the core developers to have any interest in PRs that address such issues... at least, ones that I'd be willing to submit. The problem is that I probably have less interest in supporting the unzip-and-run scenario than the core devs have interest in supporting privilege separation, so the motivation for me to create the kind of PR that might get accepted for this kind of thing is extremely low.

If somebody from the core team reached out and asked me for help or advice on making Mautic work better in docker, or with planning the architecture for Mautic 3 to do so, I would be open to discussing it. But without active interest in this topic from the core team, I don't think that it would go anywhere, and I'm better off tending my own garden. (It is, after all, how I make a living so I can do FOSS in my off-time. :) )

[aspiers]

Wow, thanks a lot for the long reply! And sorry for being slow to respond. This conversation is actually super useful for me, since it's helping crystallise certain thoughts and concepts in my head regarding code reuse / collaboration forking in FOSS communities, which I'm hoping to mold into a new FOSS manifesto at some point :-)

@pjeby commented on May 27, 2019 8:44 PM:

Not really, no. I don't really see myself as part of "the Mautic community", in the sense of having any ability to steer that project or direct any resources other than my own. So it's sort of like you're coming and asking me if it would be a good idea for Canada's government to do something or other, when I live in the U.S. and don't even get involved in my own local politics. ;)

Hah :-) I don't think the analogy holds up though, because a regular U.S. citizen would not (generally, dual-citizenship or political careers aside) have the right to try to steer the government of another country. In contrast, anyone is free to join a FOSS project at any time. That is of course one of the great beauties of FOSS.

Having said that, equally noone can be forced to join a FOSS project, and I certainly wasn't trying to do that to you. I think a better analogy would be that I was part of a local volunteer group and I saw that you were interested in the same topic so I invited you to join the fun. And of course you would be welcome to decline the invitation with no hard feelings :-)

I just made something to scratch my company's itches

... which is another great thing about FOSS :-)

and if it helps other people too, then great.

Well, that's the question - does it help other people? I think it certainly does in many ways, because you've incorporated some fantastic best practices, such clear separation of code / config / data, configuration via environment variables, and sequential cron jobs. You've added some totally awesome extra features, such as applying PRs and custom patches. And dirtsimple/php-server looks like a really nice piece of work too.

And unlike the vast majority of micro-forks, you've even documented this stuff really nicely!

This is really great, and in fact with hindsight I kind of wish I'd used your repo rather than the official one.

But even with all that awesome work, I still worry that it bears a risk of hindering the community more than it helps it, because a) it fragments the community, and b) from what you stated, it sounds like you don't have time to maintain or support this fork for anything other than your own personal (or company) use. For example if I started submitting PRs, would you have time to review them and collaborate? What if it ended up becoming the most popular choice for deploying Mautic via Docker?

Now, you're under no obligation to help anyone at all - of course you have the right to use FOSS purely for your own purposes, even without publishing anything at all. But I think in the long run it can potentially be a win for you, as well as the community, if you give back the itch-scratching work you've done in a way which can be reused and maintained long-term. Because then the community version will work perfectly for you long-term without you having to maintain your own approach, and then you would get other benefits in return.

At the time I created this image, the mingfang image was the only thing close to what I needed, but rather than reuse it I chose to build on my dirtsimple/php-server image because it already provided the base features and the additional feature of privilege separation -- i.e. running the webserver under a different user ID that can't overwrite any code files.

I understand the motivation for that choice, but you're not the only one who would want or benefit from this privilege separation. I certainly want it, and in fact the whole community wants it (even those who don't know they want it), because following security best practices is always a great idea. By ensuring that the upstream version has it, it's one less thing for each security-conscious individual to worry about.

Similarly, I'd argue that the community probably wants the majority of the cool stuff you've done here, whether they know it or not; the letsencrypt/certbot support is one of several more examples.

Integrating my changes into either the mingfang or mautic upstreams is nontrivial because this project is 100% wedded to dirtsimple/php-server and its security features, let alone the templating features, etc. Even if they chose to take advantage of some of the non-docker-specific bits, I'd still have to maintain this image in order to run it under dirtsimple/php-server, and I neither have the need nor the political capital within the larger "Mautic community" to try and make such a thing happen.

I don't know enough about dirtsimple/php-server or your particular use case to comment accurately, but after looking closer, I'm getting the impression that dirtsimple/php-server is a way better foundation to build on than what the official docker-mautic repo has done. I just think it would be a real shame and wasted opportunity if the wider Mautic community didn't benefit from this great work.

Beyond the mere technical detail of what base image is used, there's a bigger issue of goals and directions in play: Mautic is built with the intention of being able to support running in 1990's era primitive shared hosts that do a single document root that basically covers the entire filesystem available to a single user ID, by dropping the whole thing in a directory and then installing/configuring it via the web UI.

I totally acknowledge this problem, but I don't think Mautic was deliberately trying to avoid good practices - I think it's just one of those things which happens organically through lack of time and/or awareness.

This design philosophy is pretty much 100% diametrically opposed to good docker practices, and it can't be trivially worked around, either. Mautic, in its core directory layout and other design choices, is fundamentally broken for use in a properly secured environment and has to be worked around at every step to have a solid docker deployment. Even with what I've done in this project, I keep finding new ways in which it's broken. (For example, its log files are written using .php extensions, and it has the webserver writing PHP code to cache files that it will later execute!

Yeah, those issues are kinda horrible. But maybe not too hard to fix.

I hope to fix these issues, too, eventually.)

Great, but maybe you don't have to fix them all by yourself ... ;-)

But mostly, you can't fix Mautic's security problems and still keep the ability to do "unzip and access via the web" installs on primitive shared hosting, so I don't expect the core developers to have any interest in PRs that address such issues... at least, ones that I'd be willing to submit. The problem is that I probably have less interest in supporting the unzip-and-run scenario than the core devs have interest in supporting privilege separation, so the motivation for me to create the kind of PR that might get accepted for this kind of thing is extremely low.

Genuine question - why do you think the core devs care so much about the unzip-and-run scenario? I'm new to the Mautic community so I'm not familiar with the make-up of the core team yet, but it is clear that containers are already hugely popular and I very rarely see any developer pushing back and saying that containers should be avoided. Also, I don't imagine that the unzip-and-run approach is particularly relevant to any core devs on Acquia's payroll, since they have their own SaaS offering which is presumably more sophisticated, but either way would surely benefit from all the technical advantages your approach offers.

If somebody from the core team reached out and asked me for help or advice on making Mautic work better in docker, or with planning the architecture for Mautic 3 to do so, I would be open to discussing it. But without active interest in this topic from the core team, I don't think that it would go anywhere, and I'm better off tending my own garden. (It is, after all, how I make a living so I can do FOSS in my off-time. :) )

OK, good to know! Like I said, I'm very new to Mautic, but I've already spoken to a few key folks and even met a couple of them in person at Acquia Engage earlier this week, so I'm optimistic that the community can start working together more efficiently in the future.

[pjeby]

If you actually submit PRs I'll certainly review them, but of course whether they get in will depend on how well they help or hinder my own goals for the project.

Regarding the rest, I think that our viewpoints are too disjoint for any useful communication to happen. I'm continually tempted to dissect and debunk the abstractions you're using, but I'm not actually getting paid for that service, so I just end up irritated. (For example, your reference to "fun": I personally find nothing about Mautic to be fun; working with it has been a constant PITA on virtually every level.)

I am not interested in further discourse on this topic, so please limit any future issues or PRs to specific problems or feature requests in the technical operation of this repository. I am not open to discussion about how I administer this project or how it ought to be part of the official organization's work, except to the extent I noted previously: i.e., if the administrators of the official image contact me themselves expressing interest.

IOW, I have zero interest in lobbying for any change to the Mautic official images in relation to this project, and I wish to explicitly disavow any lobbying you may wish to do on that subject. I'm not spending what virtually-nonexistent political capital I have on it, and I don't want you spending it for me, either, so please, if you are attempting to lobby for such a thing, do not indicate or imply that I have anything to do with it, but instead make clear that it is your desire and yours alone. As I stated before:

If somebody from the core team reached out and asked me for help or advice on making Mautic work better in docker, or with planning the architecture for Mautic 3 to do so, I would be open to discussing it. But without active interest in this topic from the core team, I don't think that it would go anywhere, and I'm better off tending my own garden.

As I said, please feel free to submit on-topic, concrete issues or PRs for this project, but discussion of abstractions around "community" is off-topic for this repository or any other repository I manage, as it simply creates extra work that I am not being compensated for. Thank you for your understanding.