[WIP] added migration namespaces to options, added alias syntax support

[schmunk42]

Q	A
Is bugfix?	it depends ;)
New feature?	yes
Breaks BC?	no
Tests pass?	tbd
Fixed issues	#13356

[samdark]

I don't see any pros in merging it so it's up to @cebe who has more experience with this topic.

[schmunk42]

One thing is migrationNamespaces - is there a reason why this should NOT be available on the CLI?

---

The second change is about supporting migrations in global namespaces, like how they had to be used until 2.0.9.

If there's no support, we'll end up in including an extension which does that, because otherwise ALL extensions with non-namespaced migrations would be useless in 2.1 (if migrationPath will be deprecated).

As said, there might be nicer options for the syntax or a parameter to enable the fallback, ideas welcome.

CC: @cebe @klimov-paul

[klimov-paul]

One thing is migrationNamespaces - is there a reason why this should NOT be available on the CLI?

In general I can see no reason for exposing this param at CLI level as it is unlikely to be changed frequently. But this is debatable of course.

The second change is about supporting migrations in global namespaces, like how they had to be used until 2.0.9.

Support of the global namespace is still present via migrationPath. In the future major version support for non-namespaced classes will be dropped as they make no sense. Modern PHP application should not use classes without namespaces in general. I can see no problem in this.

[klimov-paul]

Both migrationPath and migrationNamespaces can be used at the same time without any troubles.

[samdark]

Having it available via parameter makes no harm and is in line with migrationPath which is available.

[schmunk42]

In general I can see no reason for exposing this param at CLI level as it is unlikely to be changed frequently. But this is debatable of course.

Since migrationPath was available and it's widely used from my experience - every extension with migrations has to be installed using it, I would definitely enable it.

[schmunk42]

Support of the global namespace is still present via migrationPath

It supports only one, not multiple - that was the reason why I created yii2-migration-command in the first place. Cebe implemented this in Yii 1. That's what the very initial feature request was all about.

As said in my first posting, we'll end up in using an extension, which should actually be obsolete with the changes from 2.0.10.

[klimov-paul]

It supports only one, not multiple - that was the reason why I created yii2-migration-command in the first place. Cebe implemented this in Yii 1. That's what the very initial feature request was all about.

I am already tired of this argument appearing over and over again. We have made a decision at #384 in favor of namespaces, which are modern way to resolve such task. If you prefer to code your project without namespaces like it was several years ago - it is your choice. I can see absolutely no reason for supporting old-fasion coding approach.

[schmunk42]

In the future major version support for non-namespaced classes will be dropped as they make no sense. Modern PHP application should not use classes without namespaces in general. I can see no problem in this.

It would render all migrations not upgrading their migrations obsolete, just for no real reason. The chance of conflicts is still pretty low - do you know of any conflict with a migration name in Yii 1 or 2?

For sure it should not be enabled by default. You may think about a full deprecation in 2.2. It makes no sense to enforce this policy in such a strong way, since people will work around that and we'll end up in the same situation, needing a 3rd party extension to run multiple migrations of 2.0 extensions.

And let's be honest, nobody wants this discussion again.

[schmunk42]

If you prefer to code your project without namespaces like it was several years ago - it is your choice. I can see absolutely no reason for supporting old-fasion coding approach.

You're just breaking things for the sake of it. I also really tired of this.

[samdark]

@schmunk42 but nothing is broken, isn't it? Namespaces are optional, extension works as before if these are not enabled, right?

[schmunk42]

@samdark Not in 2.1, as planned.

[samdark]

What do you mean?

[samdark]

Namespaces are a way to solve the same problem you're trying to solve. There's no point in having two similar ways to solve it. Additionally, you're not forced to use these. You can stick to no-namespace approach which works exactly the same as it did before. So extension will still work well.

[schmunk42]

How would you run multiple migrations in 2.1, like some from @app/migrations, @vendor/a/b and @vendor/c/d. If the vendor packages are not updated already and if they can't be run one after each other.

The whole point about this feature is not isolation - that has never been a real world problem with migrations.
People struggle with running migrations from different locations and bringing them into the correct order.

While our command solves this problem, whether they are namespaced or not - the current implementation is completely ignoring how things out there work. Instead of making a migration-extension obsolete with one fallback parameter and half a dozen lines of code.

[samdark]

OK. So the real issue is the migration from no namespaces to namespaces in the middle of the project with 200 migrations?

[schmunk42]

That would be a huge issue yes, especially if they are located in vendor.

[samdark]

So it's not really about aliases at all. It's about reading from both namespaces and non-namespaces at the same time, right?

[schmunk42]

So it's not really about aliases at all. It's about reading from both namespaces and non-namespaces at the same time, right?

Yes. It would be fair enough if you need to adjust your migration config after a change to 2.1. I have no-problem with that. The code in the PR shows this.

[samdark]

The fallback to old behavior should be optional.

[samdark]

As for migrating from no-namespaces to namepsaces, the case itself is important for existing projects.

[klimov-paul]

Converting namespaced classes to non-namespaced is inconsistent.
Consider following PHP class autoloading example:

$obj = new \app\migrations\Foo();

At this code I expect $obj to be populated by instance of \app\migrations\Foo. If such class does not exist Iexpect to get en error message.
Now following proposed concept, this code should create me instance of class \Foo in case \app\migrations\Foo does not exist but \Foo does exist.

Providing such fallback makes code to be risk prone. If migrationNamespaces has been specified it should seek for namespaced classes as well as in example above.

Second concern is migration history. Using namespaces was a solution to the history loss problem. Fully qualified class name is stored in migration history table. Thus while reverting or re-applying migration you have no need to seek for the correct migration source path - it is already present in the class namespace.
Even if particular namespace has been removed from migrationNamespaces list already applied migration can still be reverted as it will be found using native PHP class autoloading mechanism.

With this change migration applied via non-namespace fallback will be unable for revertion as there is no iteration over namespaces during this process.

This is the point why I consider exposing migrationNamespaces to the CLI to be redundant. While operating migrationPath there were no other way to use different source locations, migrationNamespaces can contain as much values as needed including every extension you use.
You can configure it just once at console application config and then use always. Switching its value seems to be completely redundant work to me in such scope.

[klimov-paul]

the real issue is the migration from no namespaces to namespaces in the middle of the project with 200 migrations?

You can refactor your migrations source code adding a namespace for them.
Then you can use yii migrate/mark command to clear the old migration history and then apply the new one containing namespaces.
It does not sounds so bad to me.

or sure it should not be enabled by default. You may think about a full deprecation in 2.2.

This has not been yet decided. There is no due date even for 2.1 milestone. And there is no final decision is migrationPath will be removed or not. In fact several years may pass before this matter will become today issue. I can see no reason to be worry.

[klimov-paul]

While our command solves this problem, whether they are namespaced or not - the current implementation is completely ignoring how things out there work. Instead of making a migration-extension obsolete with one fallback parameter and half a dozen lines of code.

I am not sure if I understand this correctly.
Using of namespaces for migrations will make necessity of operating migrationPath obsolete. All we need to do is just wait for extensions to pick up the improvement.
That is why I consider we can drop migrationPath in the future as it will make no sense.
Still I do not like the idea of supporting 2 different approaches for the same sake.

If particular extension wishes to support both namespaced and non-namespaced migrations following tricks might provide it:

1. You can declare migration classes twice: one namespaced with actual code and another one non-namespaced, which just extends it:

// file `migrations/namespaced/M123...Init.php`
namespace ext\foo\migrations\namespaced;

class M123...Init extends \yii\db\Migration
{
    public function up() {}

    public function down() {}
}

// file `migrations/m123...init.php`
class m123...init extends \ext\foo\migrations\namespaced\M123...Init
{
    // empty: code inherited
}

2. You may also pick both those classes into a single file declaring namespaced and non-namespaced classes at same time:

// file `migrations/m123...init.php`

namespace ext\foo\migrations
{
    class m123...init extends \yii\db\Migration
    {
        public function up() {}

        public function down() {}
    }
}

// same file, out of `namespace` block
class m123...init extends \ext\foo\migrations\m123...init
{
    // empty: code inherited
}

Although both ways are little inconsistent they allows you usage of new feature keeping a BC.

[cebe]

this step is done by the Yii autoloader in case you enable second parameter in class_exists, so it is not needed here.

[cebe]

So it would be

} elseif (!class_exists($class, true)) {
    $class = StringHelper::basename($class);
}

[schmunk42]

Will update that if all concerns are sorted out.

[samdark]

Providing such fallback makes code to be risk prone.

Not really. Migration classes have very special names so it's unlikely that there will be a collision. Or do you mean another risk?

With this change migration applied via non-namespace fallback will be unable for revertion as there is no iteration over namespaces during this process.

Why? Current non-namespaced migrations are OK with reverting, aren't they?

You can configure it just once at console application config and then use always. Switching its value seems to be completely redundant work to me in such scope.

It is similar to ./yii message ability to use either config or options. One may want to write a shell script setting these options depending on various conditions.

You can refactor your migrations source code adding a namespace for them.
Then you can use yii migrate/mark command to clear the old migration history and then apply the new one containing namespaces.
It does not sounds so bad to me.

It is quite bad. First of all, some migrations may be irreversible and already applied. Then there are extensions providing non-namespaced migrations which you can't really modify yourself. At the same time there could be extensions w/ namespaced migrations.

[schmunk42]

Converting namespaced classes to non-namespaced is inconsistent.

It is, I am not happy about that either.

Providing such fallback makes code to be risk prone.

How big is that risk really, since all migrations carry a name and a date-time-second?
There have been around 100 million seconds since the existance of Yii2 and there are around 5000 packages, not even modules for Yii2 - and we're talking only about legacy support here.

That's the reason why the syntax was chosen - now we have to live with this, somehow.

There is not one example showing that this has ever been a problem, even with Yii1 - but I am repeating myself. Saying this would be a greater risk than refactoring hundreds of files is simply not true.

Second concern is migration history ... With this change migration applied via non-namespace fallback will be unable for revertion as there is no iteration over namespaces during this process.

This has not to be a concern, since aliases and namespaces are both essentially paths, not taking the 1:100.000 chance from above into account, but that's all.

This has not been yet decided. There is no due date even for 2.1 milestone. And there is no final decision is migrationPath will be removed or not. In fact several years may pass before this matter will become today issue. I can see no reason to be worry

"Also BaseMigrateController::$migrationPath is considered as deprecated and will be removed in future versions. Why we should extend its functionality?" -- @klimov-paul a day ago

[removed paragraph] Sorry bad wording

You can refactor your migrations source code adding a namespace for them.

No, I can not, since we're using a lot of 3rd party extensions - which would have to be forked for this task, added manually to composer.json etc..., completely unfeasible.

Using of namespaces for migrations will make necessity of operating migrationPath obsolete.

[removed] Sorry bad wording

Using of namespaces for migrations will make necessity of operating migrationPath obsolete. All we need to do is just wait for extensions to pick up the improvement.

[removed] Sorry bad wording

You can declare migration classes twice: one namespaced with actual code and another one non-namespaced, which just extends it

[removed] Sorry bad wording

Yes, that's something to think about. I was thinking about registering different autoload paths for the global namespace (<- damn ugly). Maybe we can find a solution starting from here, I'd really like to.

[cebe]

@schmunk42 @klimov-paul maybe it is about time to take a break, get some fresh air or sleep and come back tomorrow with a calm mind? :)

[klimov-paul]

Not really. Migration classes have very special names so it's unlikely that there will be a collision. Or do you mean another risk?

Program behavior should be consistent: in case you expect namespaced class - the namespaced class should be used, in case you are expecting non-namespaced class - non-namespaced class should appear.
Developer may forgot to add namespace operator at the migration class declaration by mistake. He should be notified about this mistake instead of silent fallback to non-namespaced class.

Why? Current non-namespaced migrations are OK with reverting, aren't they?

This has not to be a concern, since aliases and namespaces are both essentially paths, not taking the 1:100.000 chance from above into account, but that's all.

We are talking about fallback from namespaced class to non-namespaced one. The migration history will be broken: instead of some\module\migrations\M...Foo it will write up just M...Foo. Then when you run yii migrate/down it will search for M...Foo.php inside migrationPath, which value is app/migrations and obviosly will fail to find it and thus migration can not be reverted.

It is similar to ./yii message ability to use either config or options. One may want to write a shell script setting these options depending on various conditions.

I am not against exposing migrationNamespaces to the CLI interface. I just say I never considered it as necessary, but if will make you feel better - we can add it - I have no solid objections here.

It is quite bad. First of all, some migrations may be irreversible and already applied. Then there are extensions providing non-namespaced migrations which you can't really modify yourself. At the same time there could be extensions w/ namespaced migrations.

Pay attention: you should use migrate\mark not migrate\down! Mark command simply clears the migration history without execution of the migration code: no up() or down() methods will be ever called, no database changes expect the values in migration table will be performed.

"Also BaseMigrateController::$migrationPath is considered as deprecated and will be removed in future versions. Why we should extend its functionality?" -- @klimov-paul a day ago

Yes, but I have not said anything about it being removed tomorrow or at 2.1 release. 'Future' is a relative entity.

No, I can not, since we're using a lot of 3rd party extensions - which would have to be forked for this task, added manually to composer.json etc..., completely unfeasible.

Noone forces you to abandon exiting code - it will work jsut fine with current code.
Switching to namespaces is a long shot. I do not expect it to appear tomorrow and I do not understand why you do.
The newly created project may gain a benefit from it, existing one may migrate to them or stay with old implementation. I still can not see any problem here.

Please do not mix up 2 different terms: 'file path' and 'namespace' - they are not equal. File path can be calculated by namespace, however it is not always so. Each namespace has a file path related to it, but not every file path corresponds to some namespace.

We have chosen namespaces because they contain a file path, while being more flexible resolving more problems.

[cebe]

Switching to namespaces is a long shot.

Do we have documentation on how to adjust an existing migration history to the new, namespaced version? If not, we should create that.

[samdark]

Program behavior should be consistent

That is why I'm insisting this fallback to be optional and disabled by default. It's indeed could get cryptic to find out migration w/o namespace but it is essential if you have to support both at the same time.

We are talking about fallback from namespaced class to non-namespaced one. The migration history will be broken

For migrations applied using --migrationPath only and that's the same as it was before namespaces so we're not breaking more than it was. Migrations in the default path are still OK.

you should use migrate/mark not migrate/down

So:

1. Use mark to remove migrations info from applied history.
2. Introduce namespaces to all existing migrations.
3. Use mark to add new migrations infor to applied history.

One thing. You can't use mark to get to state of no migrations applied.

[samdark]

@cebe no, we don't have it.

[schmunk42]

He should be notified about this mistake instead of silent fallback to non-namespaced class.

He should even be forced to enable this setting.

We are talking about fallback from namespaced class to non-namespaced one. The migration history will be broken: instead of some\module\migrations\M...Foo it will write up just M...Foo.

Just save it with its alias.

Noone forces you to abandon exiting code - it will work jsut fine with current code.

That's fragmenting the framework as a whole, leaves old versions running forever - and therefore creates security problems.

I didn't respond to the migration mark things, cause they are completely unrelated and don't help in any way.

Please do not mix up 2 different terms: 'file path' and 'namespace' - they are not equal.

I am only talking about aliases and namespaces.

"An alias is automatically defined for each extension that is installed via Composer. Each alias is named after the root namespace of the extension as declared in its composer.json file, and each alias represents the root directory of the package. For example, if you install the yiisoft/yii2-jui extension, you will automatically have the alias @yii/jui defined during the bootstrapping stage,..."

http://www.yiiframework.com/doc-2.0/guide-concept-aliases.html#extension-aliases

That's the reason why everyone who is using multiple migrations, was using aliases for that. Because it was the closest, actually equivalent, match.

If it would have been possible to use namespaces in migrations or if it would have been recommended people would have done that.

We have chosen namespaces because they contain a file path, while being more flexible resolving more problems.

Which ones? Isolation is none.

[klimov-paul]

I am only talking about aliases and namespaces.

'Path alias' and 'namespace' are different terms just as well as 'file path' and 'namespace'.
Most namespace has an path alias associated with them, this is valid for every yii2-extension composer package. However not every path alias has corresponding namespace. For example: there are @vendor, @runtime, @webroot and so on.
You may specify path to the same directory using different path aliases. For example following path aliases will lead to the same directory:

• @some/extension/migrations
• @vendor/some/extension/migrations

Which ones? Isolation is none.

While using PHP namespaces particual class can not belong to 2 different namespaces at the same time ever.

The fact that 'path alias' matches 'namespace' in some situation does not justify substitution of namespace by path alias.
Terms 'path alias' and 'namespace' are not equal, thus they can not substitute each other.

That's the reason why everyone who is using multiple migrations, was using aliases for that. Because it was the closest, actually equivalent, match.

Original usage of migrationPath at Yii2 is anachronism. While migrating code from Yii1 to Yii2 we have not reconsidered the migration source logic. File path should been converted to namespace back at 2.0.0 release. Now this mistake has been fixed up, so the future Yii2 based projects gain better migration support.

Creating a fallback from namespaced to non-namespaced class name is inconsistent from the programming point of view. Its only value is making people happy to use old walk-around they got used to.

I do not like the idea of putting inconsitent behavior inside the Yii core, even if has its own benefit (even sugnificant one). In this situation I would prefer modified MigrateCommand, with 'namespace to path alias' fallback, being released and maintained as a separated extension, which it is now already.

Anyway, I am in minority here, and my voice have a little value in this discussion. The rest of the core team are free to proceed with this PR as they consider best - I will not perform any attempts to stop them as it makes no sense.

[schmunk42]

However not every path alias has corresponding namespace. For example: there are @Vendor, @runtime, @webroot and so on.

That's not the point in this discussion, it's only about yii2-extensions from composer, all have corresponding aliases <-> namespaces.

Creating a fallback from namespaced to non-namespaced class name is inconsistent from the programming point of view. Its only value is making people happy to use old walk-around they got used to.

We're making a 6-digit turnover with Yii projects a year, I simply don't want to throw out a huge amount of money out of the window.

In this situation I would prefer modified MigrateCommand, with 'namespace to path alias' fallback, being released and maintained as a separated extension, which it is now already.

The reality is, that there are hundreds of projects using the alias system, because there was no other alternative. Aliases are 99% as-good-as-namespaces when it comes to migrations.

The current solution does only partially help the community and a decent support in the core would save 100s of work hours.

There were mistakes made, forget it! I agree to most programatically or technical statements you made. But what I say is about efficiency not philosophy.

---

One more thing, which might lead to a solution (don't know 100%).

I just tried this, I added namespace app\migrations; to some migrations in my app.
I thought they would not matter, since I am not using the new feature at all. But this also fails.

While this is also not technically 100% perfect, wouldn't it make sense to allow correct namespaces, even if we're still using migrationPath - this might lead to a much smoother transition.

[klimov-paul]

Support for field migrationNamespaces setup from CLI added, since we agreed on it:
862a5dd

[cebe]

My proposal of solving this by allowing to use aliases in $migrationNamespaces: #14241
I hope this is more clear and a sane approach of solving this issue, as it does not confuse namespaces and alisases anymore. alisases are for specifying directories, namespaces for namespaces.
Closing this PR as the change is included in #14241.