PDA Project Update

[AzorianMatt]

PDA Project Update

THE LATEST PROJECT UPDATE CAN BE FOUND HERE: #1888

[CodeMeZone]

TL;DR:

The PDA project is facing challenges due to its rapid growth and lack of professional development practices. The codebase has become difficult to maintain, leading to issues and delays. The project will not accept contributions to the current edition anymore and will focus on developing a new, more stable version. The last release for the current edition (0.4.2) will happen by the end of the year. Future contributions to the new edition will be selective to ensure a solid foundation, and a more structured development process will be implemented, potentially leading to new features and community involvement in decision-making.

[AzorianMatt]

@Ramalama2 and thus you clearly understand the issue at hand. I do intend to prevent it in the future by outright rejecting contributions that don't meet the standard and can't be easily mutated to it.

[aeeeeeew]

The project will not accept contributions to the current edition anymore and will focus on developing a new, more stable version. The last release for the current edition (0.4.2) will happen by the end of the year.

Some of us work very well with the still current version v0.4.2, but a small fix regarding the comment section would be helpful, as with a larger number of zone entries, the entries shift by a random number of lines after a comment entry, instead of staying as originally defined.

[AzorianMatt]

The project will not accept contributions to the current edition anymore and will focus on developing a new, more stable version. The last release for the current edition (0.4.2) will happen by the end of the year.

Some of us work very well with the still current version v0.4.2, but a small fix regarding the comment section would be helpful, as with a larger number of zone entries, the entries shift by a random number of lines after a comment entry, instead of staying as originally defined.

We very much understand the challenges presented when running larger deployments like this. Unfortunately, nothing about the features related to what you're asking is a small fix, hence how we got here. I'm not saying the current maintainer won't consider it, but it's unlikely since the new replacement has been well underway in development by a professional team that has donated a very helpful amount of resources to get it done. Hopefully great things to come!

[aeeeeeew]

We very much understand the challenges presented when running larger deployments like this. Unfortunately, nothing about the features related to what you're asking is a small fix, hence how we got here. I'm not saying the current maintainer won't consider it, but it's unlikely since the new replacement has been well underway in development by a professional team that has donated a very helpful amount of resources to get it done. Hopefully great things to come!

Thank you very much for the quick reply!
I’d like to correct my last message. I didn't mean the zone entry that shifts, but the entry for the comment on it.
Speaking on our behalf, I can say that we do not take your work for granted. Regardless of whether the project is currently facing difficulties, the effort you have put into it remains highly valuable to many of us. We are patiently waiting for any potential positive developments.

We wish you all the best going forward!

[AzorianMatt]

We very much understand the challenges presented when running larger deployments like this. Unfortunately, nothing about the features related to what you're asking is a small fix, hence how we got here. I'm not saying the current maintainer won't consider it, but it's unlikely since the new replacement has been well underway in development by a professional team that has donated a very helpful amount of resources to get it done. Hopefully great things to come!

Thank you very much for the quick reply! I’d like to correct my last message. I didn't mean the zone entry that shifts, but the entry for the comment on it. Speaking on our behalf, I can say that we do not take your work for granted. Regardless of whether the project is currently facing difficulties, the effort you have put into it remains highly valuable to many of us. We are patiently waiting for any potential positive developments.

We wish you all the best going forward!

You're welcome! Thank you as well for the kind sentiments! Everyone involved appreciates feedback like that from time to time to help remind them that not everyone is out for themselves.

[nulluserid]

Please also do consider having some way to contribute financially, be it bounties, or Patreon style gifting etc.

[AzorianMatt]

@nulluserid I doubt we will see anything like that occur on the project unfortunately. One has to appreciate that the target audience for this project is fairly limited in comparison to large scale open source projects of wider popularity.

That is to say, it would be fairly difficult to collect the funding for this given the status quo.

[mrimann]

@AzorianMatt Not with the goal of "having each and every hour paid" in mind - that won't work and I'm with you.

But having a possibility to (financially) support the project and your work as a low-barrier option to help would still be nice (more like giving back a gift).
I think some companies that use the product and thus have a benefit from it, could consider giving something back.

[AzorianMatt]

@mrimann agreed. Unfortunately benevolence is not typically in the business model. I've had the ability set up for the project to receive donations for just over a year now, and not one. I received a short-term sponsorship directly through my organization at one point, but that has been it.

Granted, I haven't done any marketing specifically geared toward the angle of topic here.

[murphy83]

Sad to see what happend, but health is something you are absolutely right to give priority to. Best wishes!

I am currently doing some work related to PDA and PowerDNS when it comes to running and orchestrating the containers for a full DNS Service with some added features - I am not going to change the code of the application as my current focus is clearly far more on the infrastructure-scope.

Basic things I am currently developing are:

1. native IPv6 support in Container
2. native use of TLS/SSL/HTTPS by application (instead of a reverse proxy)

Reason is that we are trying to get rid of the more and more expensive IPv4-Adresses in many of our scenarios.
My current development and some explanations can be found here:
https://github.com/murphy83/PowerDNS-Admin

To clarify: I am not going to handle pull requests when it comes to proposed changes in the code itself, any contributions with regards to Dockerfile, docker compose definitions, etc. are welcome. (I am not a native Python developer and I am currently not able to learn and support yet another language or framework).

[jsenecal]

Hello @murphy83,

I reviewed your fork and I am wondering why you created separate run statements here:
master...murphy83:PowerDNS-Admin:master#diff-f34da55ca08f1a30591d8b0b3e885bcc678537b2a9a4aadea4f190806b374ddcR40-R43

Doesn't this make the image much larger ?

[murphy83]

I see what you are thinking about... looks like a remainder from debugging.

But as you probably noticed, the Dockerfile is a so called "Multistage" file, in which the build-steps are done up to line 70. Starting from Line 71, you find the final stage which is the set of File-Layers that will be published.
Starting from a small python/alpine variant, this image is "enriched" by build artifacts from the rest of the file.

A Multistage Dockerfile allows to have additional stuff / tooling for building (compiling) your application, while only copying over the results / artifacts needed to another stage which has no need to include all the tooling. You may think of it as a "mini"-package-management on your own: You will need some additional libs and tooling to compile a piece of software, but you do not need e.g. a fully fledged C-Compiler once you compiled your binary and just want to run it.

So for the time being: The final and exposed version stored in a registry is not affected by the amount of run commands and layers, so there is no urgent need to do something about it. I will most likely fix it once I find the time to do so.

Anyway: Thanks a lot for your comment and finding.

[AzorianMatt]

@jsenecal I concur with @murphy83's explanation. The project should have switched to a multi-stage image long ago, but it has just not been a priority due to other major stability problems.

This is indeed a more ideal way to handle images that fit this type of scenario.

Nevertheless, I have a maintenance transition planned in early June to hand the project over to another project member to finally take the reins and start pushing progress once again.

@AgentTNT would be a good person to work with to get these changes into the primary project.

[jsenecal]

I completely overlooked the multistage aspect of the docker file. Thanks for the explanation nonetheless!

Great to see the project moving forward once again, I was an early adopter many years ago and I was sad when I rediscovered it in the current state!

Keep up the good work guys! Hopefully the refactor will warrant me to use this in production again and I may be able to contribute then.

[Xsmael]

Hey @AzorianMatt I hope you're health has gotten better since. Please take care. Health and wellbeing comes first. That's basic ingredients which will enable to do anythings productive or meaningful. So take the time you need to get better. Life ends, but work never ends.

[naveedpash]

@AzorianMatt I hope that life is much better...

Just wanted to make a suggestion regarding the project structure (even though I'm not a contributer): Reflex is a comprehensive web development framework which focuses on keeping as much code as possible in Python while providing a rich set of UI components as well as backend integration with database.

[Justinzobel]

Is new development starting on a new branch to start fresh? Or is it happening on master? If so, is it possible to get an update on this restart of the project on this thread, so people can determine if they want to invest time in this project, thank you.

[AzorianMatt]

@Justinzobel @DaanSelen is correct. The project has continued to see an overwhelming lack of reliable help, unfulfilled promises, etc.

[DaanSelen]

@Justinzobel @DaanSelen is correct. The project has continued to see an overwhelming lack of reliable help, unfulfilled promises, etc.

I would love some people offering to help but right now I am trying to do something with what I have.

[Xsmael]

@Justinzobel @DaanSelen is correct. The project has continued to see an overwhelming lack of reliable help, unfulfilled promises, etc.

seemingly @Justinzobel question's wasn't answered he needed to know where are people who want to actually help can start doing it.

[AzorianMatt]

seemingly @Justinzobel question's wasn't answered he needed to know where are people who want to actually help can start doing it.

I don't want to be that guy, but the project is plastered with large link references to contribution guidelines and information, as well as direct references to the discord server.

I think it's a reasonable expectation that if someone finds their way to this post, they should be at least reviewing the landing README of the repo / project.

[DaanSelen]

@Justinzobel @DaanSelen is correct. The project has continued to see an overwhelming lack of reliable help, unfulfilled promises, etc.

seemingly @Justinzobel question's wasn't answered he needed to know where are people who want to actually help can start doing it.

@Xsmael @Justinzobel If you want to help the project, please mail me on dselen@nerthus.nl I can add you to the people I know that want to help.

[Justinzobel]

I really would love to help make this project, but I have 932 projects and time for about 2. Maybe one day my plate will free up more and I will get time. Happy to get updates on this thread though.

[AzorianMatt]

For all those following this thread, a new project update is now available here: https://github.com/orgs/dnsmin/discussions/1

[aloislecomte]

PDA Project Update

THE LATEST PROJECT UPDATE CAN BE FOUND HERE: https://github.com/orgs/dnsmin/discussions/1

Introduction

Hello PDA community members,

I know it has been quite awhile since the last formal announcement like this. Things have been quite busy and difficult for me both professional and personally. While I try hard to never make my problems someone else's problems, I do believe it's important to be transparent with the community. I'm not going to go into details, but I will say that I have been dealing with some mental health issues that have been quite challenging. I'm not one to give up though, so I'm pushing through and trying to get back on track.

With that being said, let's jump into the announcements.

Project Maintenance

Granted I haven't been nearly as active on the project as I would like to be, I have been keeping an eye on things and trying to keep up with the maintenance. I know there are a lot of issues and feature requests that have been piling up, and I'm sorry for that. Even if I had been more active in recent months, it would have not changed the true root cause of the issue.

This project was started out of a need for an individual's own use-case. I don't believe it was ever intended to be a commercial quality product nor a community project. It did however gain traction quickly and the community grew. This is a great thing, but it also comes with some challenges. The biggest challenge is that the project was never designed to be a community project. This means that the project lacks many of the things that are required to effectively manage a community project. This is not to say that the project is doomed, but many of the fast-paced changes combined with the lack of standards has lead to a difficult situation that makes implementing certain changes incredibly unproductive and quite often, entirely counter-productive.

After many years of accepting contributions from those who are not professional developers, the project has become quite difficult to maintain. This is not to say that I don't appreciate the contributions, but it's important to understand that the state of the code-base for the project is not in a good place. This is not uncommon with projects born how PDA was born, so it's unfortunate but not unexpected.

As of today, there are so many dependencies and a large amount of very poorly implemented features that it's difficult to make any changes without breaking many other pieces. This is further complicated by the fact that the current code base is harder to follow naturally since it largely lacks uniformity and standards. This lack of uniformity has lead to a situation where automated regression testing is not possible. This is a very important aspect of any project that expects to be able to make changes without breaking things. This is also a very important aspect of any project that expects to be able to accept contributions from the community with minimum management resources.

The hard reality is that the majority of stakeholders in the project are not professional developers. This naturally means the amount of people that can offer quality contributions is very limited. This problem is further aggravated by the poor quality feature implementation which is very hard to follow, even for seasoned developers like myself. So many seemingly small issues that have been reported, have lead to finding that the resolution is not as simple as it seems.

New Direction

As I previously stated in my last formal announcement, we would be working towards a total replacement of the project. Unfortunately, this is not a simple task, and it's not something that can be done quickly. Furthermore, with increasingly limited capacity in our own lives to work on this, we are essentially drowning in a sea of technical debt created by the past decisions of the project to accept all contributions. We have essentially reached a point where far too much time and resources are being wasted just to attempt to meet the current demand of requests on the current edition of PDA. This is a tragedy because the efforts that are invested into the current edition, really aren't creating true progress for the project, but instead merely delaying the inevitable.

As I have stated before to many community members, one aspect of taking over management of this project to ultimately save it and keep it alive, would involve making hard decisions that many will not agree with. It's unfortunate that many of those who are less than supportive of these decisions, often lack the appropriate experience to understand the importance of these decisions. I'm not saying that I'm always right, but I am saying that it's not hard to see where this is headed without some drastic changes.

With all of that being said, it's time for me to make some hard decisions. I have decided that the best course of action is to stop accepting contributions to the current edition of PDA. At this point, due to the aforementioned issues that lead to breaking the application with seemingly simple changes, it's just not worth the effort to try to keep up with the current edition. This is not to say that I'm giving up on the project, but instead I'm going to re-focus my efforts on the new edition of PDA. This is the only way to ensure that the project will survive and hopefully thrive in the future.

I will not abandon the current set of updates that were planned for the next release of 0.4.2 however. I have re-scheduled that release to be out by the end of the year. This will be the last release of the current edition of PDA. The consensus from some users is that the current edition is stable enough to be used in production environments. I don't necessarily agree with that, but I do believe that it's stable enough to be used in production environments with the understanding that it's not a commercial quality product.

Future Contributions

For those of you wondering about contributions to the new edition of PDA, the answer for now is simple. I won't be accepting any contributions to the new edition until I can achieve a stable release that delivers the core features of the current edition. This is not to say that I won't be accepting any contributions at all, but instead that I will be very selective about what contributions I accept. I believe this is the only way to ensure that a solid foundation not only takes shape, but remains solid.

It is well understood that many developers have their own ways of doing things, but it's important to understand that this project is not a personal project. This project is a community project and therefore must be treated as such. This means that the project must be engineered in a way that allows for the community to participate in the development process. This is not possible if the project is not engineered in a way that is easy to follow and understand.

Project Enhancements

It should be understood that one of the greatest benefits of this pivot is that it will allow for a more structured development process. As a result of that, the project could potentially see a future where it adopts a whole new set of features that weren't previously imagined. One prime example of this could be integration with registrar APIs. This could make easy work of tasks such as DNSSEC key rotation, which is currently a very manual process.

I am still working on final project requirements for additional phases of the new PDA edition, but these additions won't receive any attention until the core features are implemented. I will be sure to make announcements as these requirements are finalized. It is my intention to follow a request for proposal (RFP) process for these additional features. This will allow the community to participate in the decision-making process for future expansion of the project.

Closing

I hope that by the time you have reached this point in the announcement, that I have elicited new hope for the long-term future of the project. I know that many of you have been waiting for a long time for some of the features that have been requested. I know that many of you have been waiting for a long time for some of the issues to be resolved, for requested features to be implemented, and for the project to be more stable. It's unfortunate that it has taken this long to get to this point, but this is the nature of life itself. I hope that you can understand that this is the only reasonable gamble that the project survives and thrives in the future.

Hello. Is this still ongoing or is it an abandoned project?

[AzorianMatt]

It's about to be abandoned if I have to keep telling people to actually read the things they comment on.

Right at the top of this I placed an update:

THE LATEST PROJECT UPDATE CAN BE FOUND HERE: https://github.com/orgs/dnsmin/discussions/1

The lack of effort to find information (especially when it's posted in many places), is bar none one of the most demotivating things about creating work in the FOSS community these days. You may take this personally and so be it if so, but this happens way too much.

[aloislecomte]

Oh, I'm so sorry to have bothered and annoyed you. I didn't see it; I'm not yet comfortable with GitHub, I'm really sorry.

I understand. I wish you the best of luck with the rest of your project, and I know that this kind of massive undertaking requires a lot of investment. My apologies.

[DaanSelen]

Oh, I'm so sorry to have bothered and annoyed you. I didn't see it; I'm not yet comfortable with GitHub, I'm really sorry.

I understand. I wish you the best of luck with the rest of your project, and I know that this kind of massive undertaking requires a lot of investment. My apologies.

No worries! Its just a stacking annoyance. You are still very welcome to try out the project!

[AgentTNT]

Please see #1888 for update!