Capifony V3 (POC + Discussion) #437
I thinks it's a good reason to get rid of legacy code
Initially: Think of a sequentiell
If you want to manually reinstall the assets it is still required, but I would keep it out of the "root" and push it instead into
I once thought, that it maybe makes sense to strip out the doctrine-tasks into a separate cookbook (but also pointed out, that I don't want to do it :D) Maybe thats interesting?
I still have the opinion, that it isn't the job of capifony, to ensure, that the tools are all available on the server. I'd drop it completely and would only keep
Nice. Would like it.
Hi @nifr at the time of writing SSHkit was not complete (or just not documented) and so there was no DSL ready for uploading files to the remote server, or for running local commands.
Both these pieces of functionality have been since implemented so we are good to continue.
That aside I have frozen a little in continuing this PR because I am rethinking the approach to the update.
The core functionality for deploying is somewhat implemented as is (ignoring the lack of tests, or any bugs that may be present in my work). The maintainers of Capistrano have worked hard to allow modular libraries on top of capistrano core and I think this project and the community would be better served by taking advantage of that.
Ideally I would like to have
Would be interesting to hear what you guys think! (cc @willdurand) If this sounds good then I can start a dialog with the maintainers of Capistrano.
I had started writing a Capistrano 3.x compatible version of Capifony from scratch a few days ago before i came across this issue when researching for things that could be improved.
separation of concerns like ...
... were the first thing that came into my mind.
Configuring capistrano has always been googling for "capistrano+xxx" ...
A rewrite should respect and encourage others to implement the advanced modularity that Capistrano 3.x provides.
It's exactly what you're refering to i guess? let's make it right this time. I have been gathering a lot of information already like collecting all used variables in capifony, tasks in great need of improvement and other things to optimize in my private issue tracker. I wanted to create some structure in my collection before publishing it here.
If you're interested in collaborating we should do a quick hangout, mix up our ideas and maybe create a structured roadmap together.
My reason to start the rewrite was the missing
@nifr I have already done as you suggest and re-written (essentially from scratch) the functionality - this is what this PR is about. Definitely take a look at what I have done so far, and contribute/improve from there (https://github.com/peterjmit/capifony/tree/capistrano-v3/lib/capistrano).
The composer stuff could (should?) be separated out, but apart from that the code I have implemented so far only handles the following core parts of a symfony deployment:
All other unrelated tasks/variables have been removed.
I am suggesting that this list of functionality should form a core library (capistrano/symfony), Capifony could continue as a library dependent on that core, adding "convenience" tasks, pretty output etc.
I would suggest moving permission handling ( chmod, chown, acl, ... ) and basic file operations like removing front-controllers to an external gem and only making use of it in capifony.
Setting permissions correctly and/or removing files for production occurs for every (web)-application deployment and a well written implementation could be useful for many other developers ( not limited to a framework like symfony2 or even PHP in general ).
We would totally benefit in the long run if a bigger target-group could use these file/permission tasks and in turn helps improve it. Things like the missing debian support for
What do you think ?
I have been using a simple self-written pure ruby implementation of IncenteevParameterHandler to deploy my parameters for quite a while.
Lately i removed
We should discuss a solid solution for dealing with those parameter files - either create/use an external gem that provides parameter-handling in general (which would be my recommendation) ... or at least integrate IncenteevParameterHandler with capifony by default if possible as it is part of the symfony2 standard edition.
Those logfile-handling tasks like tailing should be moved out of the capifony/symfony/whateveryoucallit core aswell in my oppinion.
Testing the application should be moved to an external phpunit/testing gem, too.
Let's take a look into the great future of pain-free deployments that everybody can and will use ...
... where someone, somewhen will come up with a nice capistrano configurator
... where we can click together our modular tasks/recipes (capifony being one of them)
... of which each only provides the stuff we really need just at that moment
... or we have a really nice CLI ( <3 ncurses ) or symfony/console or ... that guides us safely through our deployment process
... a symfony2 bundle to deploy our application interactively from the web-profiler
... maybe even a nice repository for deployment recipes/tasks.
Efforts have been made towards all of these points in the past ... none of these projects really survived because the amount of work needed to put it all together was too much...
We can have all that ... but we need smaller, widely used and well tested modules first ... lets walk our first steps into the right direction with capifony and start decoupling.
That's true but it would be nice to be able to tag feature proposals with
I have like 30+ things in the pipe and i'm pretty sure they will get lost in here ...
Now It's time to come up with some clear questions ...
Who will be the core members and decision makers for the rewrite?
Do we drop Symfony1 support in the new version? [ yes / no ]
Should we decouple composer operations into a separate gem? [ yes / no ]
Which features are the most important? [ => separate issue ]
We should aswell discuss a generic structure for the rewrite. coding standards, documentation, release process, testing frameworks...
@willdurand has already said that he will comment in due time, we have already agreed that dropping Symfony 1 support (and bin/vendors) is the path we will take.
I have also agreed to take the lead on this v3 rewrite - thats why I opened this PR in the first place!
Like you say, the separate gem idea is something that needs discussing - not only here but eventually with the Capistrano folks (because if it is the path we decide to take, that would be the best place for it to live).
Calm down folks!
First, keep commenting on what you want/wish/don't want. I'll take all suggestions and come up with a roadmap that will be the best compromise IMO. We will have to agree on every single point before working on the v3.
Also, symfony 1.x support will be dropped. Same thing for the bin/vendors thing.
Last but not the least, I told @peterjmit to work on the v3, and therefore he became the defacto lead developer.
I need new people to work on capifony. Just let me know what you want to do.
And, again, be nice together please.
Thanks for your statement @willdurand. That clears things up.
@peterjmit I dropped my comment aswell - sorry, didn't mean to be rude. Maybe we misunderstood each other a bit. I just wanted to push things forward a bit too quick maybe.
Let's focus and get this going. I'm willing to help wherever i can. I have a few suggestions for the general structure.
@peterjmit could you open an issue tracker in your repo or shall we go for the
By the way, I have several ideas for the v3. Basically, capifony v2 provides way too much features, it would be nice to drop as much features as we can, it will easier to maintain. For instance, Composer runs scripts that are useful to install assets, warm up cache, and so on. I don't think we need to keep such tasks in the v3. People use Composer and the script handlers provided by the Symfony standard edition most of the time. So let's rely on Composer. However, we might introduce some tasks in order to ease our life.
I like the database backup feature, but it is partially implemented. We will need to work on it at some point.
I introduced many optional settings in the v2, and I think it would be nice to do the same in the v3. Being able to set custom options while running Composer, PHP, or any other command is important IMO.
Oh, and we will have to focus on tests.
This would enable us to keep them together with a decent documentation for each option in that file. For doc-comments i suggest going for yard. It is mostly backward compatible with rdoc but provides a more complete featureset. Additionally http://rubydoc.info updates the yard documentation on every push.
New users could get a quick overview of all options and what's possible this way. What do you think?
composer & post-install/post-cmd hooks
I usually don't compile my assets on the production server for a simple reason. Depending on the frontend complexity you need a lot of dependencies on your production server for this one single task ( ruby, nodejs,minimizers, css preprocessors, image optimizers, ... ). Further there are situations where you don't have access to these on the production server and don't have the permissions to install them.
A workaround but not a clean solution would be adding the assets i.e. to the production branch of the repository.
Being able to compile / install the assets ( additionally in a configurable/temporary location when working with a headless repository ) before deployment would be very useful.
Capifony v2 task summary
57 (+1 default) tasks provided by capifony (symfony2)
29 commands execute/proxy symfony2 commands
Should all be included using one helper function ( @peterjmit already did something like this in his PR )
We could introduce auto-registering all possible commands and their options provided by our application using the output of
What do you think about this?
3 deprecated tasks
bye, bye ..
6 tasks composer integration
Should be moved to an external gem/plugin!
There are several things on my list for composer. Will be covered in a separate post.
4 tasks database management
Very practical but should be moved to an external gem/plugin.
I.e. getting the parameters from a local-install / environment variables needs improvement.
2 tasks for maintainance
Should be moved to an external gem/plugin!
1 task for shared folder download
Should be moved to an external gem/plugin - something for generic remote file handling i.e. !
2 tasks tailing logfiles
should be moved to an external gem/plugin - i.e. remote file handling ( see above ).
1 phpunit task
[x] lacks check requirements
should be moved to an external gem/plugin.
7 file/permission handling tasks
These should use a well-defined file-handling gem/plugin that is maintained externally.
We should introduce a few more flags for the setup command aswell:
Capistrano 3 doesn't have the dependency to
While providing a class for configuration questions the old
We should encourage(force?) users not to store passwords in their deployment configurations.
What do you think?
I originally implemented exactly that, and then I decided to split variables across different files - I couldn't decide which was best so am happy to have it either way!
DB & backups
Testing (of deployed code)
Personally my main desire is to keep this as simple as possible and aim to prioritize to aiding a user of the Symfony standard edition in deploying their project. I also would like to aim to keep this layer on top of Capistrano as transparent as possible, so as to help us keep up to date with the main project - and be accessible to Capistrano users.
I currently use a conditional.rb file that has the following checks:
Would be great to have this in core.
There has still been no feedback / proposed roadmap from @willdurand in 3 months now.
... while we're still waiting for feedback from him or @everzet here.
Sadly both of them don't seem to notice (or care about) new comments to this thread anymore.
... i'm not quite sure anymore if we'll ever get the chance to collaborate on a capistrano v3 version of capifony over here. Not to mention active maintenance afterwards. That's somewhat disappointing as 4 months have passed since this issue was created.
@peterjmit and i both have working alpha-state versions of a symfony2 deployment plugin for capistrano v3 that (if merged somehow and improved together with others) many people could benefit from.
The main reason for starting this thread was Peter's (and my) vision that this repository and the popular name capifony paired with @willdurand's and @everzet's experience with capifony would make it easier for the community to find this plugin and possibly contribute to it. We wanted to make use of existing infrastructure (i.e. capifony is already mentioned in the symfony docs) and not re-invent the wheel.
Now we're somewhat stuck. I would suggest that we'll try to contact @willdurand an @everzet once more ... and if we don't still don't get a response / roadmap within a reasonable timeframe (of let's say two weeks) we should consider taking a different approach ...
Be it asking @leehambley to create a
The coolest thing would still be to hear from @everzet or @willdurand ... even if they only gave green light in terms of "we don't have the time to maintain a v3 branch but we're not willing to hand out permissions for this repo to someone else"
What do you think Peter?
Well, I am following and still maintaing capifony. Writing a roadmap is not easy as none of us knows all use cases. While what we have right now here sounds good, I'd like to not miss important stuff.
And yeah, you noticed I managed quite a lot of other projects, thank you for your support... Please go read my last blog post. At some point I need to prioritize tasks, capifony works, hence the low priority.
I know that sometimes priorities shift due to changes in life or other projects taking more time. And as capifony is open source it is probably also build and maintained in the contributors free time.
But as this project is really popular and probably has a lot of users, I think it is sad that the project has only little progress. As @peterjmit and @nifr have done a lot on v3 and seem really to understand what capifony is doing under the hood, wouldn't it make sense to have on of them or both as additional maintainer on the project? (Given that they would want to do this.)
Thanks for the feedback and life-sign @willdurand
Yes, I happened to read your blog-post quite a while ago and i'm the last person on earth that doesn't understand your situation ... seriously!
We all understand that you're currently busy with other (awesome) things and that you've got a long list of stuff with higher priorities. Personally i'm more than grateful for all the nice, high-quality stuff you push out to the community ... and to me you're a miracle in terms of productivity.
My personal time-frame for open-source contributions is limited, too. It's just been somewhat frustrating that you said you'd come up with a timetable in a week or so ... and we - as the ones really wanting to help out, parring the ground - didn't get a response for 3 months afterwards.
The thing is ... our hands are tied right now. We need something to start with - some infrastructure that we can work with ... to get this
There are obvious benefits in creating a new
A lot could be improved in a v3 version - better, mandatory tests for features, project-structure, documentation, ... and we're here to help ! But if we continue like this there will be no capifony v3 in 2020 ...
This repository is - not only due to its popularity ... with no simple symfony2 deployment alternatives around - THE place for building something rock-solid
From my personal experience even 3/4 agencies are still not making use of automated deployments due to a lack of knowledge ( and fear of diving into another programming language like ruby ) ... and as long as we don't have ssh-agent support around in PHP in simple library like Seclib - without the need of installing that old-fashioned SSH2 extension... there will be nothing like capistrano in PHP. And i'm not the only person thinking like that,
@willdurand ... take your time ... think about how you can enable us to kick this off without you getting overwhelmed by a "new", additional project that needs maintainance, guidance ... and tell us what we can do to assist.
Thanks in advance :)
So due to the 5 month wait, and some personal/work related requirements I have decided to release https://github.com/capistrano/symfony.
It is going to remain very simple so there is scope to have capifony build upon that and perhaps provide extra "sugar" for those wanting more functionality, or more out of the box config for those less comfortable with writing ruby.
I will close this PR shortly unless anyone has any objections.
RIP capifony v3
I'm cool with that as I'm quite sure there will be no capifony v3 ... ever. We've waited long enough.
I had a different vision for a new start ... team-up with you and probably a few more guys. Build a group that would collectively take decisions, then collaboratively prepare the infrastructure (homepage,docs, ...) and create a release plan ... to finally release a solid application that fits some enterprise needs and is worth spreading the word at the point of it's initial release.
I offered a hangout , was willing to contribute some code and ideas for the generic structure of the new project to prevent releasing just another highly immature project to the public.
At least by now we just have another one-man-show going on here. This thread is a wonderful example where this is leading.
There's no CI, no codestyle rules, no contribution guidelines, no examples that would show new users how to install and configure it to quickly deploy their first application - there's basically nothing that tries to ensure a decent level of quality by now.
I truly hope the new project matures with time, gets some attention and can at some point replace capifony but i have strong doubts about that ... anyways good luck with it Pete.
I'll focus on other things now.
@nifr Weird attitude, it is under the Capistrano organisation (and so gains their community and input/oversight) and it is v0.1.0. It is just a start and if you want to contribute you are more then welcome.
I figured it was better to do something rather than nothing and I was fed up of having to custom install the capifony gem from my v3 branch.
This is not how things work. I took the lead of capifony because I asked @everzet in person, explaining what I planned to do, and so on.
Here, this is not exactly the same thing. I thought I had enough time to prepare the v3, but it seems that you are really in a rush. Hence the proposal here..
Hey, let's try and keep this civil. FOSS maintainer ship is a burden, anyone who's spent weeks waiting for me knows how much pressure I can be under. As a maintainer it's painful to know you are disappointing people, but have to choose that, or burnout.
A fork doesn't need to mean the end to collaboration, and I hope that people will all rally behind _the best experience for Symphony users_, I can't say who that will be, and my supporting any fork/project under
For whatever it's worth, I think that the project exists at all is great, and regarding tests, perhaps the work I was starting in https://github.com/capistrano/packer might constitute a good place to start integration testing/tutorials/etc, which is how I envisioned that being used.
I am honestly happy to do whatever people think is best - I just want to do something, and people have been asking me what the status of this PR is semi-regularly. Unfortunately I have received no communication or indication on next steps for this PR from @willdurand (I have reached out both here, and on twitter) making it very difficult to know whether I should continue to work on this.
From what I can see a lot of work has gone into building a test environment for capistrano v3, and I would like to benefit from that with the Symfony integration.
I think this repository suffers from being monolithic which I feel must have been a factor in the lack of progression here. There is also lot of functionality within this project that I feel could benefit both non Symfony, and non PHP projects (database tasks, backup tasks, etc).
I am sorry if this appears to be an move towards fragmentation, that is not my intention My hope was that under capistrano/* the project could gain from that community (in addition to everyone contributing here).