New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add command to setup legacy settings/designs symlinks #107

Merged
merged 6 commits into from Oct 3, 2017

Conversation

6 participants
@andrerom
Member

andrerom commented Sep 7, 2017

This is done for doc effort in: ezsystems/developer-documentation#28
Which is from issue: https://jira.ez.no/browse/EZP-27142
Part of Epic: https://jira.ez.no/browse/EZEE-1455

Without this feature, each and every user would have to find out the hard way that
Composer sometimes wipes out the ezpublish_legacy folder, and also that versioning
this folder is not really recommended so you somehow need to deal with settings and designs.

So this command setups and symlinks the following into legacy on compser install and update:

  • src/legacy_files:
    • designs/*
    • settings/override
    • settings/siteaccess/*

The feature in this PR is primary aiming at upgrade/migration use cases, where
people are coming from 5.4 or 2014.x and similar and need to move over settings and designs.

Review wanted from partners that have their own way of doing this, as you will know
better what you need. /cc @emodric @joekepley @peterkeung ..

Why no extension folder handling?

Extensions are handled by composer, and also by legacy bundles command by this bundle,
so if you need an extension in src folder you can make app bundle a legacy bundle.
Hence it felt easily confusing to give another option for this.

Why no storage and storage/packages folder handling?

As site storage folder is not typically versioned in git, I guess this should be handled by something else then src/. Input on how people tend to deal with this is more then welcome!

NOTE: But merged version of this PR contains hints to user during setup on manually handling storage folder, and with similar pointers in documentation this is somewhat covered for now, even if it is not automated (yet).

Why place them directly in src/legacy_files?

Opted for this mainly because placing it in bundle is more of a legacy bundles feature, and also makes sure we are forward compatible with Symfony 4/Flex's single bundle src layout.

[Feature] Add command and composer command to setup legacy settings/d…
…esigns symlinks

Without this feature, each and every user would have to find out the hard way that
Composer sometimes wipes out the ezpublish_legacy folder, and also that versioning
this folder is not really recommended.

The feature in this PR is primary aiming at upgrade/migration use cases, where
people are coming from 5.4 or 2014.x and similar and need to move over settings and designs.

Review wanted from partners that have their own way of doing this, as you will know
better what you need.

*Why no extension folder handling?*
Extensions are handled by composer and also by legacy bundles, so if you need an extension in
src folder you can make app bundle an legacy bundle.
Hence it felt wrong to give another option for this which could be confusing.

*Why no storage and storage/packages folder handling?*
As site storage folder is not typically versioned, I guess this should be handled by something else.
Input on how people tend to deal with this more then welcome.

@ezsystems ezsystems deleted a comment from ezrobot Sep 7, 2017

@emodric

This comment has been minimized.

Show comment
Hide comment
@emodric

emodric Sep 7, 2017

Collaborator

I will go through the code in couple of days since I'm on a short break right now, but this looks okay to me.

At Netgen, we do something similar, with the difference that legacy_files is stored in a bundles' Resources folder, making it possible for every bundle to add it's own legacy stuff (i.e. legacy kernel patches). We also use different folders for different environments (e.g. legacy_files (used in every env), legacy_files_dev, legacy_files_prod) so we can have different configuration for different servers.

Collaborator

emodric commented Sep 7, 2017

I will go through the code in couple of days since I'm on a short break right now, but this looks okay to me.

At Netgen, we do something similar, with the difference that legacy_files is stored in a bundles' Resources folder, making it possible for every bundle to add it's own legacy stuff (i.e. legacy kernel patches). We also use different folders for different environments (e.g. legacy_files (used in every env), legacy_files_dev, legacy_files_prod) so we can have different configuration for different servers.

@andrerom

This comment has been minimized.

Show comment
Hide comment
@andrerom

andrerom Sep 7, 2017

Member

Cool, looking forward to it 😃 Btw reason why I opted for not putting legacy files in bundle was among other things because symfony 4 has single bundle layout with no bundle dir afaik, and legacy bundles feature already allow you to place extensions and legacy stuff inside it even if it takes a bit more config + extension structure.

Member

andrerom commented Sep 7, 2017

Cool, looking forward to it 😃 Btw reason why I opted for not putting legacy files in bundle was among other things because symfony 4 has single bundle layout with no bundle dir afaik, and legacy bundles feature already allow you to place extensions and legacy stuff inside it even if it takes a bit more config + extension structure.

@brookinsconsulting

This comment has been minimized.

Show comment
Hide comment
@brookinsconsulting

brookinsconsulting commented Sep 8, 2017

+1

@andrerom

This comment has been minimized.

Show comment
Hide comment
@andrerom

andrerom Sep 13, 2017

Member

@emodric had a chance to have a look yet? :)

Member

andrerom commented Sep 13, 2017

@emodric had a chance to have a look yet? :)

@emodric

This comment has been minimized.

Show comment
Hide comment
@emodric

emodric Sep 13, 2017

Collaborator

@andrerom Nope, but will do it ASAP and let you know!

Collaborator

emodric commented Sep 13, 2017

@andrerom Nope, but will do it ASAP and let you know!

*/
$filesystem = $this->getContainer()->get('filesystem');
if (!$filesystem->exists($srcArg)) {

This comment has been minimized.

@emodric

emodric Sep 13, 2017

Collaborator

What's the use creating the source folder structure?

@emodric

emodric Sep 13, 2017

Collaborator

What's the use creating the source folder structure?

This comment has been minimized.

@andrerom

andrerom Sep 13, 2017

Member

Migration/Upgrade use case:

  • install legacy bridge (will execute this step)
  • manually move files over from old to new structure

So no manual steps needed for creating the folders, and it can be explicit referred to in doc where you move them as a default convention

@andrerom

andrerom Sep 13, 2017

Member

Migration/Upgrade use case:

  • install legacy bridge (will execute this step)
  • manually move files over from old to new structure

So no manual steps needed for creating the folders, and it can be explicit referred to in doc where you move them as a default convention

This comment has been minimized.

@emodric

emodric Sep 14, 2017

Collaborator

How hard is it to type mkdir -p src/legacy_files/settings ? :) Doesn't seem that much useful to me if it's documented right, but okay, if you think it can help, sure, but I still don't like that script is running with -c automatically every time.

@emodric

emodric Sep 14, 2017

Collaborator

How hard is it to type mkdir -p src/legacy_files/settings ? :) Doesn't seem that much useful to me if it's documented right, but okay, if you think it can help, sure, but I still don't like that script is running with -c automatically every time.

This comment has been minimized.

@wizhippo

wizhippo Sep 22, 2017

Contributor

I'm with @emodric here on having -c as default. Leaving off -c will throw the exception and help the users who have not read the docs be aware they can and should migrate their existing config to the new location.

@wizhippo

wizhippo Sep 22, 2017

Contributor

I'm with @emodric here on having -c as default. Leaving off -c will throw the exception and help the users who have not read the docs be aware they can and should migrate their existing config to the new location.

This comment has been minimized.

@andrerom

andrerom Oct 2, 2017

Member

PR updated, also added much more output to try to explain the concepts better for user.

better? (changes done in new commit)

@andrerom

andrerom Oct 2, 2017

Member

PR updated, also added much more output to try to explain the concepts better for user.

better? (changes done in new commit)

Show outdated Hide outdated INSTALL.md
Show outdated Hide outdated bundle/Command/LegacySrcSymlinkCommand.php
Show outdated Hide outdated bundle/Command/LegacySrcSymlinkCommand.php
Show outdated Hide outdated bundle/Command/LegacySrcSymlinkCommand.php
Show outdated Hide outdated bundle/Command/LegacySrcSymlinkCommand.php
Show outdated Hide outdated bundle/Command/LegacySrcSymlinkCommand.php
@emodric

This comment has been minimized.

Show comment
Hide comment
@emodric

emodric Sep 13, 2017

Collaborator

@andrerom Left a couple of remarks, but in general, this looks okay.

What I would like to definitely see is symlinking based on Symfony environment used when running the script (src/legacy_files_dev, src/legacy_files_prod and src/legacy_files (for all environments)).

Collaborator

emodric commented Sep 13, 2017

@andrerom Left a couple of remarks, but in general, this looks okay.

What I would like to definitely see is symlinking based on Symfony environment used when running the script (src/legacy_files_dev, src/legacy_files_prod and src/legacy_files (for all environments)).

@peterkeung

This comment has been minimized.

Show comment
Hide comment
@peterkeung

peterkeung Sep 13, 2017

Contributor
  • "Composer sometimes wipes out the ezpublish_legacy folder" What are the conditions that cause this? (I'm pinging @ernestob.)

  • We don't depend on Composer for deployments and don't use it on production environments. We use it only on dev to pull in code. So we do end up versioning ezpublish_legacy.

  • Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

Contributor

peterkeung commented Sep 13, 2017

  • "Composer sometimes wipes out the ezpublish_legacy folder" What are the conditions that cause this? (I'm pinging @ernestob.)

  • We don't depend on Composer for deployments and don't use it on production environments. We use it only on dev to pull in code. So we do end up versioning ezpublish_legacy.

  • Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

@emodric

This comment has been minimized.

Show comment
Hide comment
@emodric

emodric Sep 14, 2017

Collaborator

Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

Symlinking design folder is simpler than creating and activating the extension together with design.ini.append.php. For simpler usecases, symlinking designs to ezpublish_legacy/design is a welcome feature.

Collaborator

emodric commented Sep 14, 2017

Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

Symlinking design folder is simpler than creating and activating the extension together with design.ini.append.php. For simpler usecases, symlinking designs to ezpublish_legacy/design is a welcome feature.

@andrerom

This comment has been minimized.

Show comment
Hide comment
@andrerom

andrerom Sep 14, 2017

Member

"Composer sometimes wipes out the ezpublish_legacy folder" What are the conditions that cause this? (I'm pinging @ernestob.)

From slack #ezplatfomr-use on september 5th leading up to this PR:

One issue I know of is if the composer.json setting ezpublish-legacy-dir was missing, which can happen when you switch between branches to one not setting it. However as of legacy-installer v2.0.4 that is set by default now, so unless you set it to other setting it should always be set: https://github.com/ezsystems/ezpublish-legacy-installer/releases/tag/v2.0.4 (edited)

[16:48]
Oh, there is one other case. When composer removes the folder and does a new checkeout, for instance moving from dist to source or opposite. Also might happen on dist updates somtimes

Long story short: use symlinks or similar for var, settings, extensions and design folders. And add a step in your composer post-update / post-install step which makes sure they are setup.

Not experienced it? I guess if you don't use composer in prod (which is a good thing, I guess you then have deployment pipeline for instance involving composer when rolling out new code, anyway) it won't be an issue for you.

Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

I assumed some people use it given it was never deprecated in favour of extensions, as for extensions they are as said already handled by other features of this bundle.

Maybe I should get the folder creation step to place a readme in legacy_folder directory to explain what it is for and not.

Member

andrerom commented Sep 14, 2017

"Composer sometimes wipes out the ezpublish_legacy folder" What are the conditions that cause this? (I'm pinging @ernestob.)

From slack #ezplatfomr-use on september 5th leading up to this PR:

One issue I know of is if the composer.json setting ezpublish-legacy-dir was missing, which can happen when you switch between branches to one not setting it. However as of legacy-installer v2.0.4 that is set by default now, so unless you set it to other setting it should always be set: https://github.com/ezsystems/ezpublish-legacy-installer/releases/tag/v2.0.4 (edited)

[16:48]
Oh, there is one other case. When composer removes the folder and does a new checkeout, for instance moving from dist to source or opposite. Also might happen on dist updates somtimes

Long story short: use symlinks or similar for var, settings, extensions and design folders. And add a step in your composer post-update / post-install step which makes sure they are setup.

Not experienced it? I guess if you don't use composer in prod (which is a good thing, I guess you then have deployment pipeline for instance involving composer when rolling out new code, anyway) it won't be an issue for you.

Why the "design" folder? I guess people are still putting their templates here. We always put our templates in extensions.

I assumed some people use it given it was never deprecated in favour of extensions, as for extensions they are as said already handled by other features of this bundle.

Maybe I should get the folder creation step to place a readme in legacy_folder directory to explain what it is for and not.

@andrerom

This comment has been minimized.

Show comment
Hide comment
@andrerom

andrerom Sep 20, 2017

Member

updated. Regarding folder creation, maybe create on install? the point is to have a meaningful output to users and also to avoid additional steps. Also for the sake of trying to keep the documentation on this straight forward without to much if's and buts :)

Member

andrerom commented Sep 20, 2017

updated. Regarding folder creation, maybe create on install? the point is to have a meaningful output to users and also to avoid additional steps. Also for the sake of trying to keep the documentation on this straight forward without to much if's and buts :)

@emodric

This comment has been minimized.

Show comment
Hide comment
@emodric

emodric Sep 20, 2017

Collaborator

@andrerom If you think it can help, you can leave it as is. Since you don't have to use the command in your stack if empty folder structure bothers you, this should be okay :)

Collaborator

emodric commented Sep 20, 2017

@andrerom If you think it can help, you can leave it as is. Since you don't have to use the command in your stack if empty folder structure bothers you, this should be okay :)

@andrerom

This comment has been minimized.

Show comment
Hide comment
@andrerom

andrerom Sep 20, 2017

Member

Ok, then who else should we get to review this then?

Member

andrerom commented Sep 20, 2017

Ok, then who else should we get to review this then?

@emodric

This comment has been minimized.

Show comment
Hide comment
@emodric

emodric Sep 22, 2017

Collaborator

Ok, then who else should we get to review this then?

@wizhippo maybe ?

Collaborator

emodric commented Sep 22, 2017

Ok, then who else should we get to review this then?

@wizhippo maybe ?

@wizhippo

Besides my thought on the default creation of of the src directory structure above I'm fine with this as is.

I too am curious how others are handling storage and should it be addressed, perhaps in another PR. I'm glad I have had backups a couple of times.

@andrerom

This comment has been minimized.

Show comment
Hide comment
@andrerom

andrerom Sep 28, 2017

Member

I too am curious how others are handling storage and should it be addressed, perhaps in another PR. I'm glad I have had backups a couple of times.

yes, at the very least would need to alert the user when we detect that folder being removed (overwritten by symlink) is not empty, like now reported in #110.

Member

andrerom commented Sep 28, 2017

I too am curious how others are handling storage and should it be addressed, perhaps in another PR. I'm glad I have had backups a couple of times.

yes, at the very least would need to alert the user when we detect that folder being removed (overwritten by symlink) is not empty, like now reported in #110.

@ezsystems ezsystems deleted a comment from ezrobot Oct 2, 2017

@andrerom andrerom requested a review from glye Oct 3, 2017

@glye

glye approved these changes Oct 3, 2017

(more nitpicks)

@andrerom

This comment has been minimized.

Show comment
Hide comment
@andrerom

andrerom Oct 3, 2017

Member

Thanks all for reviews!

Merging as I assume the only remaining concerns by @emodric and @wizhippo is now handled given structure is no longer automatically created, but instead instructions for how to do it is outputted (and it is covered in migration doc).

Member

andrerom commented Oct 3, 2017

Thanks all for reviews!

Merging as I assume the only remaining concerns by @emodric and @wizhippo is now handled given structure is no longer automatically created, but instead instructions for how to do it is outputted (and it is covered in migration doc).

@andrerom andrerom merged commit 5dd5ecf into master Oct 3, 2017

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

@andrerom andrerom deleted the legacy_src_symlink branch Oct 3, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment