PDA Project Update

[AzorianMatt]

PDA Project Update

Introduction

Hello PDA community members,

I know it has been quite awhile since the last formal announcement like this. Things have been quite busy and difficult
for me both professional and personally. While I try hard to never make my problems someone else's problems, I do
believe it's important to be transparent with the community. I'm not going to go into details, but I will say that I
have been dealing with some mental health issues that have been quite challenging. I'm not one to give up though,
so I'm pushing through and trying to get back on track.

With that being said, let's jump into the announcements.

Project Maintenance

Granted I haven't been nearly as active on the project as I would like to be, I have been keeping an eye on things and
trying to keep up with the maintenance. I know there are a lot of issues and feature requests that have been piling up,
and I'm sorry for that. Even if I had been more active in recent months, it would have not changed the true root cause
of the issue.

This project was started out of a need for an individual's own use-case. I don't believe it was ever intended to be a
commercial quality product nor a community project. It did however gain traction quickly and the community grew. This
is a great thing, but it also comes with some challenges. The biggest challenge is that the project was never designed
to be a community project. This means that the project lacks many of the things that are required to effectively manage
a community project. This is not to say that the project is doomed, but many of the fast-paced changes combined with
the lack of standards has lead to a difficult situation that makes implementing certain changes incredibly unproductive
and quite often, entirely counter-productive.

After many years of accepting contributions from those who are not professional developers, the project has become quite
difficult to maintain. This is not to say that I don't appreciate the contributions, but it's important to understand
that the state of the code-base for the project is not in a good place. This is not uncommon with projects born how PDA
was born, so it's unfortunate but not unexpected.

As of today, there are so many dependencies and a large amount of very poorly implemented features that it's difficult
to make any changes without breaking many other pieces. This is further complicated by the fact that the current code
base is harder to follow naturally since it largely lacks uniformity and standards. This lack of uniformity has lead to
a situation where automated regression testing is not possible. This is a very important aspect of any project that
expects to be able to make changes without breaking things. This is also a very important aspect of any project that
expects to be able to accept contributions from the community with minimum management resources.

The hard reality is that the majority of stakeholders in the project are not professional developers. This naturally
means the amount of people that can offer quality contributions is very limited. This problem is further aggravated by
the poor quality feature implementation which is very hard to follow, even for seasoned developers like myself. So many
seemingly small issues that have been reported, have lead to finding that the resolution is not as simple as it seems.

New Direction

As I previously stated in my last formal announcement, we would be working towards a total replacement of the project.
Unfortunately, this is not a simple task, and it's not something that can be done quickly. Furthermore, with
increasingly limited capacity in our own lives to work on this, we are essentially drowning in a sea of technical debt
created by the past decisions of the project to accept all contributions. We have essentially reached a point where
far too much time and resources are being wasted just to attempt to meet the current demand of requests on the current
edition of PDA. This is a tragedy because the efforts that are invested into the current edition, really aren't
creating true progress for the project, but instead merely delaying the inevitable.

As I have stated before to many community members, one aspect of taking over management of this project to ultimately
save it and keep it alive, would involve making hard decisions that many will not agree with. It's unfortunate that
many of those who are less than supportive of these decisions, often lack the appropriate experience to understand the
importance of these decisions. I'm not saying that I'm always right, but I am saying that it's not hard to see where
this is headed without some drastic changes.

With all of that being said, it's time for me to make some hard decisions. I have decided that the best course of
action is to stop accepting contributions to the current edition of PDA. At this point, due to the aforementioned
issues that lead to breaking the application with seemingly simple changes, it's just not worth the effort to try to
keep up with the current edition. This is not to say that I'm giving up on the project, but instead I'm going to
re-focus my efforts on the new edition of PDA. This is the only way to ensure that the project will survive and
hopefully thrive in the future.

I will not abandon the current set of updates that were planned for the next release of 0.4.2 however. I have
re-scheduled that release to be out by the end of the year. This will be the last release of the current edition of
PDA. The consensus from some users is that the current edition is stable enough to be used in production environments.
I don't necessarily agree with that, but I do believe that it's stable enough to be used in production
environments with the understanding that it's not a commercial quality product.

Future Contributions

For those of you wondering about contributions to the new edition of PDA, the answer for now is simple. I won't be
accepting any contributions to the new edition until I can achieve a stable release that delivers the core features of
the current edition. This is not to say that I won't be accepting any contributions at all, but instead that I will be
very selective about what contributions I accept. I believe this is the only way to ensure that a solid foundation not
only takes shape, but remains solid.

It is well understood that many developers have their own ways of doing things, but it's important to understand
that this project is not a personal project. This project is a community project and therefore must be treated as such.
This means that the project must be engineered in a way that allows for the community to participate in the development
process. This is not possible if the project is not engineered in a way that is easy to follow and understand.

Project Enhancements

It should be understood that one of the greatest benefits of this pivot is that it will allow for a more structured
development process. As a result of that, the project could potentially see a future where it adopts a whole new set of
features that weren't previously imagined. One prime example of this could be integration with registrar APIs. This
could make easy work of tasks such as DNSSEC key rotation, which is currently a very manual process.

I am still working on final project requirements for additional phases of the new PDA edition, but these additions
won't receive any attention until the core features are implemented. I will be sure to make announcements as these
requirements are finalized. It is my intention to follow a request for proposal (RFP) process for these additional
features. This will allow the community to participate in the decision-making process for future expansion of the
project.

Closing

I hope that by the time you have reached this point in the announcement, that I have elicited new hope for the
long-term future of the project. I know that many of you have been waiting for a long time for some of the features that have been
requested. I know that many of you have been waiting for a long time for some of the issues to be resolved, for
requested features to be implemented, and for the project to be more stable. It's unfortunate that it has taken this
long to get to this point, but this is the nature of life itself. I hope that you can understand that this is the only
reasonable gamble that the project survives and thrives in the future.

[CodeMeZone]

TL;DR:

The PDA project is facing challenges due to its rapid growth and lack of professional development practices. The codebase has become difficult to maintain, leading to issues and delays. The project will not accept contributions to the current edition anymore and will focus on developing a new, more stable version. The last release for the current edition (0.4.2) will happen by the end of the year. Future contributions to the new edition will be selective to ensure a solid foundation, and a more structured development process will be implemented, potentially leading to new features and community involvement in decision-making.

[AzorianMatt]

@CodeMeZone well said! I'll try to remember to provide a TL;DR in future updates as long.

[Ramalama2]

I remember the time i asked my company to implement Reverse Zones in a separate Tab/Window/Category, the code was accepted but as i was Reviewing the Code, i thought already at that time how crappy the Code is that we contribute xD

But the issue was, there wasn't a better way. A better way required changing half Powerdns-Admin and that was out of question, because my Boss would have killed me, because our Dev had a lot other things todo.

In the end, i believe that most of the Contributions are Quick & Dirty like ours, which simply leads over time to this state.
So thats the right Choice that you do.

But i don't think that you can avoid that in the future either, the only solution would be some sort of Plugin Capability, but i have no clue how to realize that and it will get very complex either.

Cheers

[AzorianMatt]

@Ramalama2 and thus you clearly understand the issue at hand. I do intend to prevent it in the future by outright rejecting contributions that don't meet the standard and can't be easily mutated to it.

[nulluserid]

Please also do consider having some way to contribute financially, be it bounties, or Patreon style gifting etc.

[AzorianMatt]

@nulluserid I doubt we will see anything like that occur on the project unfortunately. One has to appreciate that the target audience for this project is fairly limited in comparison to large scale open source projects of wider popularity.

That is to say, it would be fairly difficult to collect the funding for this given the status quo.

[mrimann]

@AzorianMatt Not with the goal of "having each and every hour paid" in mind - that won't work and I'm with you.

But having a possibility to (financially) support the project and your work as a low-barrier option to help would still be nice (more like giving back a gift).
I think some companies that use the product and thus have a benefit from it, could consider giving something back.

[AzorianMatt]

@mrimann agreed. Unfortunately benevolence is not typically in the business model. I've had the ability set up for the project to receive donations for just over a year now, and not one. I received a short-term sponsorship directly through my organization at one point, but that has been it.

Granted, I haven't done any marketing specifically geared toward the angle of topic here.

[murphy83]

Sad to see what happend, but health is something you are absolutely right to give priority to. Best wishes!

I am currently doing some work related to PDA and PowerDNS when it comes to running and orchestrating the containers for a full DNS Service with some added features - I am not going to change the code of the application as my current focus is clearly far more on the infrastructure-scope.

Basic things I am currently developing are:

1. native IPv6 support in Container
2. native use of TLS/SSL/HTTPS by application (instead of a reverse proxy)

Reason is that we are trying to get rid of the more and more expensive IPv4-Adresses in many of our scenarios.
My current development and some explanations can be found here:
https://github.com/murphy83/PowerDNS-Admin

To clarify: I am not going to handle pull requests when it comes to proposed changes in the code itself, any contributions with regards to Dockerfile, docker compose definitions, etc. are welcome. (I am not a native Python developer and I am currently not able to learn and support yet another language or framework).

[jsenecal]

Hello @murphy83,

I reviewed your fork and I am wondering why you created separate run statements here:
master...murphy83:PowerDNS-Admin:master#diff-f34da55ca08f1a30591d8b0b3e885bcc678537b2a9a4aadea4f190806b374ddcR40-R43

Doesn't this make the image much larger ?

[murphy83]

I see what you are thinking about... looks like a remainder from debugging.

But as you probably noticed, the Dockerfile is a so called "Multistage" file, in which the build-steps are done up to line 70. Starting from Line 71, you find the final stage which is the set of File-Layers that will be published.
Starting from a small python/alpine variant, this image is "enriched" by build artifacts from the rest of the file.

A Multistage Dockerfile allows to have additional stuff / tooling for building (compiling) your application, while only copying over the results / artifacts needed to another stage which has no need to include all the tooling. You may think of it as a "mini"-package-management on your own: You will need some additional libs and tooling to compile a piece of software, but you do not need e.g. a fully fledged C-Compiler once you compiled your binary and just want to run it.

So for the time being: The final and exposed version stored in a registry is not affected by the amount of run commands and layers, so there is no urgent need to do something about it. I will most likely fix it once I find the time to do so.

Anyway: Thanks a lot for your comment and finding.

[AzorianMatt]

@jsenecal I concur with @murphy83's explanation. The project should have switched to a multi-stage image long ago, but it has just not been a priority due to other major stability problems.

This is indeed a more ideal way to handle images that fit this type of scenario.

Nevertheless, I have a maintenance transition planned in early June to hand the project over to another project member to finally take the reins and start pushing progress once again.

@AgentTNT would be a good person to work with to get these changes into the primary project.

[jsenecal]

I completely overlooked the multistage aspect of the docker file. Thanks for the explanation nonetheless!

Great to see the project moving forward once again, I was an early adopter many years ago and I was sad when I rediscovered it in the current state!

Keep up the good work guys! Hopefully the refactor will warrant me to use this in production again and I may be able to contribute then.

[Xsmael]

Hey @AzorianMatt I hope you're health has gotten better since. Please take care. Health and wellbeing comes first. That's basic ingredients which will enable to do anythings productive or meaningful. So take the time you need to get better. Life ends, but work never ends.

[naveedpash]

@AzorianMatt I hope that life is much better...

Just wanted to make a suggestion regarding the project structure (even though I'm not a contributer): Reflex is a comprehensive web development framework which focuses on keeping as much code as possible in Python while providing a rich set of UI components as well as backend integration with database.

[Justinzobel]

Is new development starting on a new branch to start fresh? Or is it happening on master? If so, is it possible to get an update on this restart of the project on this thread, so people can determine if they want to invest time in this project, thank you.

[DaanSelen]

From my experience its in a very slow (due to not many devs) pace to renewing.