[PHP.NEXT] Add support for package private class, trait and interface

[guilhermeblanco]

Consumes PR #931

This PR allows this to be possible:

namespace Foo 
{
    private class Bar
    {
        // ...
    }
}

namespace Woo {
    class Fuzz extends Bar {}
}

To not be allowed as library developer of Foo may not have planned extension points in class Bar.

RFC to be written yet...

Implementation progress:

• Add lexical support for public classes, traits and interfaces
• Make sure public classes, traits and interfaces are supported correctly
• Add lexical support for private classes, traits and interfaces
• Implement compile time checks for private extension and implementation
• Implement run time check for private class instantiation
• Find a sponsor to review and help polish implementation (Nikita Popov)
• Implement ReflectionClass::isPublic() and ReflectionClass::isPrivate()
• Implement ReflectionClass::setAccessible()

Add more tests coverage for:

ReflectionClass:

• Add test for ReflectionClass::isPublic() with implicit value
• Add test for ReflectionClass::isPublic() with explicit value
• Add test for ReflectionClass::isPrivate() with explicit value
• Add test for ReflectionClass::setAccessible()

Classes:

• Top level private class, top level instantiation
• Top level private class, top level class instantiation
• Top level private class, top level function instantiation
• Top level private class, top level closure instantiation
• Top level private class, top level clone instantiation
• Top level private class, top level class extension
• (FAIL) Top level private class, different namespace instantiation
• Top level private class, different namespace class instantiation
• Top level private class, different namespace function instantiation
• Top level private class, different namespace closure instantiation
• (FAIL) Top level private class, different namespace clone instantiation
• Top level private class, different namespace class extension
• Namespaced private class, top level instantiation
• Namespaced private class, top level class instantiation
• Namespaced private class, top level function instantiation
• Namespaced private class, top level closure instantiation
• (FAIL) Namespaced private class, top level clone instantiation
• Namespaced private class, top level class extension
• (FAIL) Namespaced private class, same namespace instantiation
• Namespaced private class, same namespace class instantiation
• Namespaced private class, same namespace function instantiation
• Namespaced private class, same namespace closure instantiation
• Namespaced private class, same namespace clone instantiation
• Namespaced private class, same namespace class extension
• (FAIL) Namespaced private class, different namespace instantiation
• Namespaced private class, different namespace class instantiation
• Namespaced private class, different namespace function instantiation
• Namespaced private class, different namespace closure instantiation
• (FAIL) Namespaced private class, different namespace clone instantiation
• Namespaced private class, different namespace class extension
• Any of the above test using autoloading

Interfaces:

• Top level private interface, top level class implementation
• Top level private interface, different namespace class implementation
• Namespaced private interface, top level class implementation
• Namespaced private interface, same namespace class implementation
• Namespaced private interface, different namespace class implementation
• Any of the above test using autoloading

Traits:

• Top level private trait, top level class usage
• Top level private trait, different namespace class usage
• Namespaced private trait, top level class usage
• Namespaced private trait, same namespace class usage
• Namespaced private trait, different namespace class usage
• Any of the above test using autoloading

[laruence]

hmm, classname are case-insensitive in php, so maybe zend_binary_strcasecmp here?

[guilhermeblanco]

@laruence first time I saw this function... yes, it makes sense. I'll update the patch with this usage. =)

[kinncj]

Maybe it's time to call for case-sensitive classname... After all, universal ClassLoader will appreciate it.

\Namespace\User; src/whatever/Namespace/User.php !== src/whatever/Namespace/USER.php

[Majkl578]

@kinncj: That would be a massive BC break.

[kinncj]

That's why it's called "new generation" :-)
And yes, I agree... it will be.

[shiroyuki]

It might break if internally it uses case-insensitive to differentiate class names. However, only people who really know how PHP would consider problems. For instance, some filesystems, e.g., EXT4, allow files named User.php, uSer.php and user.php respectively to exist in the same folder. How would developers implement autoloader to pick? Which one is the chosen one?

[Kubo2]

@kinncj No, that's not so good idea. :-(

[kinncj]

@Kubo2 , if it's not related to BC break, can you explain why? (only if it's not related to BC break).

[Kubo2]

@kinncj No, it isn't related to BCb. But in PHP world, there is a big number of programmers, who does not all have the same conventions (especially if one writes Class_With_Underscored_Name, other ClassWithCamelCasedName but someone else Classwithfulllowercasenameexceptfirstcharacter) and if you are instantiating such third-party classes, it is a good thing to at least partially keep your own or your core's conventions (because the code looks much better).

[kinncj]

Extremely disagree.
PHP is evolving, and developers _must_ do so...

It was the same with the regex \e operator or even ereg_*

It's natural that Class_With_Underscored_Name represents a
package name and folders + class name are logically uppercase, as @shiroyuki
said: . "For instance, some filesystems, e.g., EXT4, allow files named User.php, uSer.php and user.php respectively to exist in the same folder. How would developers implement autoloader to pick? Which one is the chosen one?"
The same works with ClassWithCamelCasedName

Nowadays vendors are really committed to quality, we are not talking about
orphan libraries in PEAR...

Anyway, it's out of scope of this PR.

[Majkl578]

Nice. Do you also plan to add some support to ReflectionClass?

a) visibility reflection:

• ReflectionClass::isPrivate
• ReflectionClass::isProtected
• ReflectionClass::isPublic
• expanding ReflectionClass::getModifiers

b) accessibility reflection:

• ReflectionClass::setAccessible (like ReflectionMethod::setAccessible), not for direct instantiation in user code, but to legally call newInstance* methods.

[guilhermeblanco]

@Majkl578 I do plan to support ReflectionClass, and also some more test scenarios. I got it working the hard way and I needed some review such as the one @laruence started, since I'm not a core expert.
Now about setAccessible(), it would demand more research on my side.

[Ocramius]

You may want to add new tests for these cases that involve class autoloading:

spl_autoload_register(function () { eval('namespace C { class D implements \A\B {} }'); });

[flaupretre]

1. Does it mean that access is allowed from the same namespace or a sub-namespace, or just exactly the same namespace. In other words, if \name\space\myclass is declared private, could it be instantiated from \name\space\sub ?
2. Why do you just block instanciation, and not any access to the class' methods and properties from anywhere outside of its namespace ? this is what I would naively expect from a restriction on 'class visibility'.

[guilhermeblanco]

@flaupretre

1. Does it mean that access is allowed from the same namespace or a sub-namespace, or just exactly the same namespace. In other words, if \name\space\myclass is declared private, could it be instantiated from \name\space\sub ?

Only same namespace. As I said earlier, it is inline with package private from other languages.

1. Why do you just block instanciation, and not any access to the class' methods and properties from anywhere outside of its namespace ? this is what I would naively expect from a restriction on 'class visibility'.

This reasoning requires a deeper explanation. Consider a public Factory class inside of a namespace with many private classes. Upon instantiation, Factory will instantiate the private class which can be a dependency of another class outside of that namespace. It is perfectly fine to consume a private class outside of its namespace.

[Majkl578]

@guilhermeblanco: Is there a reason for not introducing protected visibility? I see some use cases for it, e.g. when splitting a class/subsystem to a smaller components, which should stay internal (object composition)

One more question, how would you propose to properly test private classes? Tests usually reside in a completely separate namespace (e.g. App vs. AppTests).

[Kubo2]

@guilhermeblanco:

Why would classes outside current namespace depend on a private class from this namespace? I suggest new visibility modifier: package-internal, to introduce along with package-private one, and it would have slightly different behavior: package-internal classes would be visible along the namespace and all its sub-namespaces and absolutely not visible outside the namespace.

[guilhermeblanco]

@Majkl578 you can test private classes by using ReflectionClass::setAccessible.
Now how do you see "protected class" behaviour?

[guilhermeblanco]

@Kubo2 that would be the first language to have such thing. It also would "-" to become a valid identifier which can be a BC break and also 2 new keywords (another potential BC break)

[Kubo2]

@guilhermeblanco:

that would be the first language to have such thing.

No, Java has something similar, I think, but they implemented it as „nested classes“. I think potential package-internal implementation in PHP would be more descriptive than the Java's one.

It also would "-" to become a valid identifier (...)
and also 2 new keywords

Why? It could be like a internal class Foo { // ..., which would introduce only one new keyword, internal, because private modifier will be reused for package-private classes.

which can be a BC break
(another potential BC break)

I don't see a BC break here, exactly. But if so, PHP 7 is a „new generation“ and thus BC breaks are welcome, aren't they? :-)

---

I would try to implement it, when I learn C one time.

[kinncj]

@Kubo2 , how can you contradict your self?
This is what you said few days ago:

No, it isn't related to BC. But in PHP world, there is a big number of programmers, who does not all have the same conventions (especially if one writes Class_With_Underscored_Name, other ClassWithCamelCasedName but someone else Classwithfulllowercasenameexceptfirstcharacter) and if you are instantiating such third-party classes, it is a good thing to at least partially keep your own or your core's conventions (because the code looks much better).

In the end, yes: making it case-sensitive is a BC, and will eventually break some vendors...

That's what you said today:

I don't see a BC break here, exactly. But if so, PHP 7 is a „new generation“ and thus BC breaks are welcome, aren't they? :-)

How can you be against case-sensitive in php-ng, and ask for an internal modifier? not only ask, but accept BC breaks for that in specific? :-)

FYI: Internal occurrences

Without case-sensitive you may break things like:

class Internal {}
interface Internal {}
define ("Internal", "foo");// Nice, you can define, but not use...
function Internal() {}
const Internal = "foo";
/** etc, etc, etc... **/

[Kubo2]

@kinncj:

define ("Internal", "foo");// Nice, you can define, but not use...

Yes, you can use it, with only restriction, that you can not directly echo Internal but echo constant('Internal'). :-)

FYI: Internal occurrences

Ah. Yes, you are right. It is really not as good idea as I thought. But instead of introducing internal keyword (which I will definitely miss) we could make package-private classes' behavior adopt package-internal, described above, and currently presented package-private behavior to be adopted by package-protected, requested by @Majkl578.
(ping @guilhermeblanco)

@Kubo2 , how can you contradict your self?

Sorry for that. Few days passed, I have read another large amount of pull-requests for PHP 7 during that time and on that base I little changed my mind.

Without case-sensitive you may break things like (...)

If so, I am no longer against implementing case-sensitive indentifiers in PHP (but thus for introducing package-internal visibility, as that could be the solution for it — following on this).
But I'll never understand, why keywords (visibility modifiers and such things) are case-insensitive in PHP (I had never seen any code using for example PUBLIC Function Foo() { /*...*/ }). Thus, making PHP keywords only lowercase and case-sensitive would be probably better solution and probably less backwards compatibility break.

[lt]

@guilhermeblanco I haven't seen any activity on this in internals. Is this something you still want to work on and get RFC'd before the feature freeze? How can I help? :)

[guilhermeblanco]

@lt I need to address one bug that is still active and the RFC that needs to be written.
If you want to give me hand, feel free to write the RFC (as I have an almost working solution to the bug).
We could open for voting as soon as this is finalized. Nikita already reviewed the patch and sounds reasonable.

[lt]

I will draft the RFC today and submit to you + a couple of others this evening (UTC :)) for review.

[lt]

Just thought of something. clone!

[lt]

I see tests for expected failures during autoloading, but none for successful autoloading.

[Fleshgrinder]

@flaupretre Having a private or protected class would mean that they are not part of the public interface of a package, hence, they are not subject to BC. The friendship concept is interesting but it solves other use cases in my opinion.

@guilhermeblanco Do you still intend to go forward with this?

[brzuchal]

I'm looking at your examples for better understanding of your feature and I think there is a missconception, for eg. if we have:

<?php
namespace MyVendor\MyLibrary
{
    private class MyInternalStuff {}
}
namespace AnotherVendor\AnotherLibrary
{
    class MyLibraryApi
    {
        public function __construct()
        {
            $privateClassInstance = new \MyVendor\MyLibrary\MyInternalStuff();
            echo 'Should NOT work'; // I think it shouldn't be allowed to work
        }
    }
}

Providing such private class can cause missconcept usage out of my package - which is MyVendor\MyLibrary

[HallofFamer]

I think a private class is more like a private inner/nested class, whose instantiation and usage are restricted by the outer class. What you are proposing is more like an internal class in C#, although C#'s internal keyword is to restrict usage within an assembly, not just one namespace.

https://msdn.microsoft.com/en-us/library/7c5ka91b.aspx

[guilhermeblanco]

@brzuchal It is exactly what your example described.
I'm a library developer, so I create a class inside of Doctrine\ORM and I don't want anyone else outside of this package to use it. By marking it as private, people that is consuming it on MyApp\Domain shouldn't be able to instantiate this class at all.

[brzuchal]

Ok so how about example with library. Assuming I'm develoing complex library with other 10 developers and need to hide my specific class drivers to awoid misusage:

<?php
namespace MyVendor\MyLibrary
{
    class PublicApi
    {
        public function __construct()
        {
            $this->widget = new InternalStuff\NonPublicWidget();
        }
    }
}
namespace MyVendor\MyLibrary\InternalStuff
{
    private class NonPublicWidget {} // I don't want anyone else use this class only my library package visible
}

Can it be handled properly? I'm not assuming that package-private classes in PHP should work exactly like in Java because there they are causing plenty of classes in same package/namespaces. I want to keep the right order and place my classes in folders.

[guilhermeblanco]

@brzuchal Personally I can't remember if I allowed sub-namespaces or parent-namespace access.

[brzuchal]

I think not: https://github.com/php/php-src/pull/947/files#diff-5459151fc119e0faf6f26162999c0cfdR2411

[Fleshgrinder]

It is correct that private is restricted to the same namespace, the feature you want requires package scoping and another form of modifier. I once coined the idea of including yet another keyword called assembly to achieve exactly that, since it would solve the package scope problem and it would not result in a BC.

<?php

assembly brzuchal {

  namespace MyVendor\MyLibrary {
    public class PublicApi {
      public function __construct() {
        $this->widget = new InternalStuff\NonPublicWidget();
      }
    }
  }

  namespace MyVendor\MyLibrary\InternalStuff {
    class NonPublicWidget {}
  }

}

http://marc.info/?t=145838845200002&r=1&w=2

[brzuchal]

@Fleshgrinder I was thinking about the same but called it package keyword like here: https://gist.github.com/brzuchal/8411a9b299f521e0186a2077be8f5321

[Fleshgrinder]

From my mailing list messages:

Usage of the word package could be a blessing and a curse at the same
time because the word is already heavily in use in the PHP world, e.g.
@package in DocBlocks or in composer package.

The same holds true for module because it is sometimes used to refer
to PHP extensions and PEAR uses it.

Hence, assembly might be the best choice here and is my proposal.

— http://marc.info/?l=php-internals&m=145838834927862&w=2

---

And here is a list of additional keywords I collected that could be used
instead of assembly (which I do not like that much):

• group
• crate (used by Rust)
• unit
• segment
• kit
• container
• bin
• …

— http://marc.info/?l=php-internals&m=146005064030910&w=2

[Fleshgrinder]

Just noticed your proposal to combine package and namespace. This is actually a very nice idea and would help to keep repetition low. How feasible it is would be another question but I really dig the idea.

[brzuchal]

@Fleshgrinder Yes I wanted to use package instead of namespace to provide this what you calles assembly (that keyword is also fine for me) but that's no further purpose of this PR. I'll keep reading internals at your thread and try there to continue discussion. Have you got any implementation started yet?

[Fleshgrinder]

Nope, mainly because I need time to get into all the Bison stuff.

[brzuchal]

@guilhermeblanco What is the status of this feature? When can I expect it could be in PHP?

[guilhermeblanco]

@brzuchal First, let's talk about what is left:

• Rebase with latest master

• Address a fundamental issue in this patch, where namespace level execution have no scope defined. Problem arrises when you do something like this:

  namespace Foo {
      $bar = new \OtherNamespace\PrivateClass();
  }

  While processing this code at runtime, class instantiation have no scope (it doesn't know it's inside a namespace). The reason comes due to how namespaces are implemented in the language, where they only exist at compile time. This is a blocker to fully finalize this patch. Addressing it can come from 2 main alternatives:

  • Decouple namespace into its own structure. This approach is the easiest from a coding standpoint, but heaviest in terms of amount of places to change (I'm guessing several thousands LOC).
  • Create two new Zend VM handlers whenever you start/end namespace. This approach is the hardest, because we require someone with deep knowledge of Zend VM. However, the amount of code is minimal (I'd argue a couple hundred LOC).

• Review and make sure all "private" class/interface/trait tests pass

• Discuss about "protected" structures and implement them.

On first part, I hit a dead-end. If we take the first approach, it's an entire RFC itself and could potentially invalidate all the implemented approach took here (PHPT tests, BNF would still be usable though). Also, I have no extensive knowledge of all internals tricks... onboarding any approach would be steep learning curve for me. Also, I have academic knowledge in C, which is good enough to read code, but not to do extensive writes of new functionality. =(

The second part should be ok on itself. The third part I haven't put any thoughts, but conceptually I envisioned it doing something similar to "private", but also I would like to see the ability to access protected members of classes (properties and methods). As said before, I didn't invest a lot of time in this because it could easily be another RFC after this one.

[nikic]

@guilhermeblanco It's been a while since I last looked at this, so not sure if I remember things right. Some comments on the namespacing issue:

First, note that there may be control-flow that crosses namespace boundaries in the form of jumps. For example, this is valid code:

namespace A {
    echo "In A";
    goto label;
    echo "Still in A";
}

namespace B {
    echo "In B";
label:
    echo "Still in B";
}

The goto here will jump from code in namespace A to code in namespace B without "directly" crossing the boundary. For this reason just inserting opcodes when a namespace starts or ends will quite cut it.

This leaves your other suggestion of representing namespaces in their own structure. In principle this should be relatively simple: Introduce a struct _zend_namespace { zend_string *name, uint32_t start; uint32_t end; } structure (where start/end are opline offsets), and store an array of these in the op array. Usually this array will have only a single element (or be empty). Generating and storing this data should be relatively simple. The most difficult part is probably adding support for this structure in the opcache optimizer (particularly the block pass), but this can later be done by someone else, who is familiar with the component.

[krakjoe]

@guilhermeblanco this generated a very decent amount of conversation, there is obviously much interest in the feature.

It also provoked other contributors to raise concerns and they haven't yet been addressed by comments or updates to the patch: Can I ask for an update on the status of this RFC please ?

If you consider this work abandoned, please close this PR yourself.

[krakjoe]

I hate this, but since it's been 6 months since I asked for feedback, and since there are unresolved issues in this very out of date implementation, I'm going to close this PR :(

Please please please take this as encouragement to open a clean PR, and get the RFC process started.

[lt]

@krakjoe This feature is not possible without severe engine tinkering. I have some ideas on how to resolve the prerequisites but it will involve changes to opcache - a blocker for many of us.

When code in the top level scope can be aware of the namespace it is executing in, we have grounds to re-open :)

Thanks for attending to this issue!

[chris-kruining]

I have some serious intrest in this feature, I come from the rfc page (last update atm is 22-09-2017(dd-mm-yyyy)) and this PR is musch more out of date. Is there a place where this feature request is still alive?

[ravanscafi]

Can we revive the discussion?

[brzuchal]

As @lt said:

When code in the top level scope can be aware of the namespace it is executing in, we have grounds to re-open :)

This isn't done so there are no chances.

[stanvass]

@brzuchal

Or another approach is to forbid top-level code when visibility is used. We don't really need top-level code in namespaces, we only need it in one single file per project: the bootstrap.

[carusogabriel]

Looks like it's back by @mdwheele https://wiki.php.net/rfc/namespace-visibility

[brzuchal]

This is fantastic that someone is willing to raise up the subject again. Last time there was a question about how it should work with goto: <?php namespace Example { private class A {} $a = new A(); goto out; } namespace OtherVendor { out: var_dump($a); } This is a valid PHP code and the question is should there be no access to $a inside OtherVendor namespace? An additional question is what about functions? Because this RFC is all about classes only. I keep my fingers crossed for this feature. That might be a reason to update codebase from 7.2 because now I don't see that much to upgrade to 7.3 at all. niedz., 9 wrz 2018 o 04:00 Gabriel Caruso <notifications@github.com> napisał(a):

…

Looks like it's back by @mdwheele <https://github.com/mdwheele> https://wiki.php.net/rfc/namespace-visibility — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#947 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADAPuWNrKwUcVKivB_6ef-Ce9TPaM3LEks5uZHZQgaJpZM4DF47g> .

[offdev]

This is a valid PHP code and the question is should there be no access to $a inside OtherVendor namespace?

It should not work, for obvious reasons. Nor should the current implementation work (which it unfortunately does...). If I declare something in a namespace, I expect the variabled to be namespaced too, not written to the global scope.

I hope someone picks up this pull request, it would be a really great feature to have!

#revivingdeadpullrequests

[olleharstedt]

Another interesting feature this PR could make way for is namespace-private class properties, for example using the "internal" keyword (as in Swift: https://docs.swift.org/swift-book/LanguageGuide/AccessControl.html).

[Tony-Sol]

I really want to see progress in this RFC, because, despite the presence of @psalm-internal annotations or modulite-like tools, it seems to me, that it would be a great to have such functionality natively.
It would be a super awesome.