Skip to content
This repository

RFC: Implementation of Stackable::import() #78

Closed
olekukonko opened this Issue · 13 comments

4 participants

Oleku Konko Joe Watkins chibisuke Bob Weinand
Oleku Konko

Introduction

From the #76 it shows that Stackable is a different data type with [] that looks like array which create the need for conversion

Example

    class Storage extends Stackable {
        function run() {
        }
    }

    $verLargeArray = range(1, 10);
    $s = new Storage();

    for($i = 0; $i < count($verLargeArray); $i ++) {
        $s[] = $verLargeArray[$i];
    }

    var_dump($s);

Propose Import method

    class Storage extends Stackable {

        function run() {
        }

        public static function import($mixed) {
            if (! is_array($mixed) and ! is_object($mixed))
                throw new ErrorException("Only Traversable Types accpeted ");

            $s = new Storage();
            foreach ( $mixed as $k => $value ) {
                $s[$k] = $value;
            }
            return $s;
        }
    }


Simple Usage

    $veryLargeArray = range(1, 10);
    $s = Storage::import($veryLargeArray);
    var_dump($s);

Advantages

  • Benefits from Internal loop
  • Ease of Use
  • Avoid having to loop all the time working with other common datatypes such as json or xml
Joe Watkins
Owner

This is not easy to implement in a uniform and compatible manner.

Remembering that any amount of contexts can hold references to any amount of objects in either the collection being merged into or from, how best should locking of these objects be handled while traversal takes place. Should edits be allowed to happen while it occurs, should the objects be locked down ... it depends on the implementation.

This kind of thing in my opinion belongs firmly in userland where you can make the choices for your specific collection. I can't implement this well at the level of pthreads.

As we're calling it an RFC, I'd like some more opinions ...

chibisuke

The problem of deciding to lock or not to lock doesn't exist in the example above imo.

The method proposed above is basically like a copy-constructor. So the target object is new and can only have one reference... the one returned from the function.

For the source object I don't see any problems ether since its an array which - if I'm not mistaken - cannot be referenced from multiple contexts at the same time. For objects that the array might hold a reference to it doesn't matter either as you're only creating an additional reference to it.

However I tend to agree that this is something that belongs into userland.

Joe Watkins
Owner

What if two contexts decide to import different objects with matching keys from two or more contexts at the same time ... you always have the problem of locking, somewhere, if not destination then source, always, forever, welcome to multi-threading :)

Even if you do not know java, you will be comfortable reading it's docs ...

http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/package-summary.html

In particular: ConcurrentHashMap, ConcurrentLinkedQueue, SynchronousQueue, BlockingQueue, BlockingDeQueue

Note: CopyOn* is irellevant, and if anything on the page appears irrelevant then it probably is, most of Collections.* is irrelevant too, but there's some methods like Collections.synchronizedList you might want to look at if you take these ideas seriously )

The plan should be I think, to implement and perfect the implementation of these objects in userland, where everyone can take part in their development and implementation.

You should not be scared of executing PHP, you should remove yourselves from the mindset that it is more efficient to execute in C than it is in PHP, while that is true, it's relevance is almost entirely diminished by what we are doing here, you are scripting C, in a multi-threaded environment, the overhead of most php processes, as we know is compilation which only occurs one time in a pthreads application ... we won't go into memory usage, I have plans for that, see apcu/apcup ... what applies to others using PHP doesn't apply to us, execute the hell out of PHP to get what you want .... perfect those implmentations and I will write them properly, on my own if I have too, I simply do not have the time to come up with such a vast amount of code from scratch.

Of course, we could hack together a few methods that emulate some of this behaviour, but trust me when I say, we should not, pthreads is not written like that, it will not respond well, look at the classes ( in /classes ), compare those to any other extension in existence ... this was the only way to write such a vastly complicated idea down. It exposes enough to give you all the power, and equally, give all the power to more C, another extension, whatever, extending it properly is something I relish, writing concurrent.so based on real world tested ideas and design is also something I relish ... it will take more than one of me ( which there isn't, I check, regularly ) to achieve what you all need at the C level if I am forced to work from nothing, so please, don't make me do that, and don't make me remove the power of choice from you in order that you can execute some very specific tasks that work well only for your current idea.

Someone start a hub and lets start collecting our ideas where we can be properly productive, together ...

Oleku Konko

I used the name$veryLargeArray to asume the aarray is a large dataset from database , json or xml sorce. We all know issue relating to how foreach works http://stackoverflow.com/questions/10057671/how-foreach-actually-works and How big are PHP arrays (and values) really? (Hint: BIG) http://nikic.github.com/2011/12/12/How-big-are-PHP-arrays-really-Hint-BIG.html

Having to loop multiple times to add values , looping working working , store values in multiple times is more than enough burden and can lead to performance issues.

A lot of PHP Datastructures have similar issue.. The only implementation which solves the issue is SplFixedArray::fromArray

@nikic once said PHP is simple array oriented language( lol) and since we can not support array directly we need to offer a flexible alternative.

I can easily find my way around but strongly believe this is not just a userland issue but usability of Stackable as a Datastructure.

I totally agree with the new implementations of Concurrent Collections but would it support this importation ?. Has you have clearly stated this would involve vast amount of code from scratch --- not clearly visible

Another Suggestion is Instead of messing with Stackable create a new class StackHash (Striped down version of Stackable) with strictly for storage and also ability to import from array.

This are all suggestions but i think usability should strongly also be looked at ...

Joe Watkins
Owner

An array is simple because it is used simply, anything being manipulated by multiple contexts is inherently complex, not simple at all, an array is no where near enough, and we should not try to make it so. There is no, one alternative, is what I am saying .... java is VERY mature, if there were one solution to this problem you would have seen it on the page I linked too.

The only way to move forward is to design these objects properly, each with their own specific goals, I am not able to commit the time required to design all of what is required in C, which is why I suggest the design phase takes place in PHP. I am aware they will be better written in C, which is why I said I'd do it, but pthreads exposes enough in every object for everyone that is using pthreads to be involved in their design, and for their designs to be fully formed, and tested before I have to commit the time to writing them in C.

If you look at the code in classes, you will see your execution of PHP methods results in the execution of the pthreads API directly, when these objects exist in C, this is how they will exist, by executing the API that exists now. Everything you need is exposed, and by going through the design phase of such a framework in PHP we are sure to find (m)any bugs, it'll be good for everyone from every angle ... There is nothing I have at my disposal in C that you do not have at your disposal in PHP.

To clarify, this is not a userland issue, this is a framework issue. I suggest that the design of the framework that solves the issues inherent in multi-threaded environments occurs in PHP where we can be productive, I admit it belongs in C, but it's a matter of logistics, I cannot do it on my own in C, and I would be for all intents and purposes, on my own. Together, in PHP we can put together and start being productive in no time at all ...

Joe Watkins krakjoe referenced this issue from a commit
Joe Watkins help to support decent solutions to #78 and #79, native count()
superior handling of anonymous members
0db6ec0
Oleku Konko

Nice casting Implementation :+1:

    class Test extends Stackable {
        public function run() {
        }
    }

    $t = new Test();
    $t[] = "one";
    $t[] = "two";
    $t["three"] = "three";

    /* get a normal array */
    $normal = (array) $t;
    var_dump(is_array($normal));

If Stackable::import() is not implemented how do you convert array back to Stackable

Joe Watkins
Owner

You can't because a stackable requires a run method.

Casting and counting make it easier to properly implement collections in pthreads. I don't aim to cover this functionality here as I said already.

Bob Weinand

Then create a Stackable with an empty run method? Is exactly the same for the user?

Joe Watkins
Owner

See the post in this issue with the link to javadocs, what I am trying to impress on you all is that no single object is suitable, it might seem it to you at the moment but trust me, and a million years of java development, it is not. A stackable ( or any other object ) can be used as a vessel for storage and it will function how the language wants it too ( ability to foreach and such ), but on it's own it cannot possibly cover all the different needs of the programmer in this environment. It just can't be done, I can do things like allow you to cast and count and loop, but that's very generic, in the real world, in real applications far more flexible means of manipulating your data should be designed, pthreads will only provide the tools to make that possible. I simply do not have the time to design, debug and test every pattern required during execution, I can only make those patterns programmable in userland.

I feel strongly that this is something that should be done by as many people as can take part, in PHP.

Joe Watkins
Owner

Encompassed in ::merge, in progress ...

Joe Watkins krakjoe closed this
Oleku Konko

Well done .. this would be a huge plus ...

Oleku Konko

Any update on this ?? can't wait to start getting my hands dirty ...

Joe Watkins
Owner

This is part of merge, ::merge handles arrays, pthreads objects and standard objects ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.