From a54e784ec61b3a0c6de151afdddc7a3f9984e879 Mon Sep 17 00:00:00 2001 From: Whiteknight Date: Wed, 25 Apr 2012 20:10:58 -0400 Subject: [PATCH] Delete two more old drafts. Move one to stage --- drafts/tag_update_path.md | 184 ----------------------------- drafts/upcoming_branches.md | 79 ------------- {drafts => stage}/context_reuse.md | 0 3 files changed, 263 deletions(-) delete mode 100644 drafts/tag_update_path.md delete mode 100644 drafts/upcoming_branches.md rename {drafts => stage}/context_reuse.md (100%) diff --git a/drafts/tag_update_path.md b/drafts/tag_update_path.md deleted file mode 100644 index 76d80b6..0000000 --- a/drafts/tag_update_path.md +++ /dev/null @@ -1,184 +0,0 @@ -I've been doing a lot of work recently on various new features, various -deprecations, and various other changes in Parrot. The goal of all these -these things is clear: A more sane, more understandable, cleaner, more -performant Parrot. Whether the path I'm pursuing is going to achisve all these -things in the long run or whether they are the best method of achieving them -is certainly open to debate, but I think it's a pretty good plan and I'm -putting my energies behind it. But, what is the plan, exactly? - -It turns out that the plan is pretty big, and has a few more steps than I -would like. This blog post is as much an attempt for me to explain myself to -the readers as it is an attempt for me to commit the various ideas in my head -down someplace safe for reference. If I've learned anything in my life so far, -it's that if I don't write a thought down, it does not and never has existed. - -Last weekend I added `:tag()` syntax to IMCC. Now, instead of writing this: - - .sub 'foo' :load - ... - .end - -You would instead write something like this: - - .sub 'foo' :tag("load") - ... - .end - -And when you want to get a library and initialize it, you used to do something -like this: - - load_bytecode "foo.pbc" - -But now you're going to be doing something like this: - - $P0 = load_bytecode "foo.pbc" - $P1 = $P0.'subs_by_flag'("load") - $P2 = iter $P1 - loop_top: - unless $P2 goto loop_bottom - $P3 = shift $P2 - $P3() - goto loop_top - loop_bottom: - -I know this all looks like extra PIR code to write, but I have two tricks up -my sleeve: The first is that this new approach is *faster* than the old one, -even if it looks like more code to write and therefore to be executed. The -second is that people don't, and shouldn't, be writting PIR by hand -anymore. It's trivial for a compiler like Winxed or NQP to generate this code -automatically, without the user having to do anything special. Libraries like -Rosella are also going to be providing tools to manage, cache, and initialize -libraries. It's only a few lines of code, so any other library, framework, or -compiler can implement it too, if you don't want to rely on Rosella. - -The big benefit here is flexibility. Let's say that the user has a library -which absolutely requires 2-step initialization. One set of functions must be -executed first (such as functions which register class definitions and create -space for global variables) and one set of functions must be executed second. -Now, we can simply give them two different tags: - - .sub 'first' :tag("load-phase1") - ... - .end - - .sub 'second' :tag("load-phase2") - ... - .end - -And when you load the library, you can do something like this: - - $P0 = load_library "foo.pbc" - $P1 = $P0.'subs_by_flag'("load-phase1") - 'execute_all_subs'($P1) - $P1 = $P0.'subs_by_flag'("load-phase2") - 'execute_all_subs'($P1) - -OR, let's say you have a language like Perl6, which offers `BEGIN` and `END` -blocks, among others. Currently PIR doesn't have any flags whatsoever to be -used for specifying which subs are to be executed at the end of the program -run for cleanup. With the new system, this is trivially easy to do. Currently -PIR doesn't have a way to mark with metadata any arbitrary group of subs, only -the `:load` and `:init` ones. In the new system, you do. - -We run into a few problems, however. First is that the load `load_bytecode_s` -opcode automatically executes the `:load` functions while the new -`load_bytecode_p_s` variant does not. Also, the two are using different -caches, so we cannot currently intermingle them. The old `load_bytecode_s` -does the old `do_sub_pragma` song and dance: Every time you `load_bytecode_s`, -it loops over all constants in the constant table, checks if each is a Sub, -checks if each has the `:load` flag, executes it if so (in a new, expensive, -nested runloop), and then *clears the flag*. That way, if you re-load the same -packfile somehow, it will attempt to re-initialize but all the flags would -have been cleared. Nevermind the fact that we really need packfiles and their -contents to remain constant at runtime for a variety of performance and -concurrency reasons, and never mind the fact that the real solution is to -not automatically intermingle library initialization with other unrelated -operations like program execution, packfile loading, or even compilation. -Part of the problem also is that the old `load_bytecode_s` mechanism loads a -new PackFile from the file and *merges it into the single global packfile*, -again violating our need for immutable packfiles and adding all sorts of -unnecessary complexity. - -There's no way to cache the individual PackFile that was loaded, because it -isn't kept separate. There's no way to know what Subs were tagged with what -pragmata, because the flags are cleared. The only way you have of protecting -yourself from harmful collisions is a patchwork of quick fixes and magical -coupled behavior. Libraries avoid re-initialization not because we are -careful to make initialization a separate and explicit operation, but because -we attempt to magically and automatically re-initialize things all over the -codebase, clearing flags in otherwise immutable data structures. - -If you think about it hard enough, you might start to see why I've been so -energetic and enthusiastic about getting some of this crap cleaned up. - -Of course, some of the problems and idiosyncracies are the reasons why an -upgrade path is neither going to be easy nor straight-forward. - -First things first, we need to start replacing `load_bytecode_s` with -`load_bytecode_p_s`. That deprecation notice is already in, although it's not -a straight-forward or easy upgrade. Remember that `load_bytecode_s` -automatically initializes the bytecode by executing `:load` Subs. The new -op does not automatically initialize, and does not not keep track of what -has been initialized, or how. Separation of concerns is a very good thing in -general, but in this case we do need to create a new mechanism to replace this -behavior before we can start to move forward. I haven't quite decided on a way -to do this yet, but I'm looking at several options. What is most likely is to -add a cache to the PackfileView PMC type of arbitrary user data. That way we -can get a reference to the PackfileView, check if it has been initialized yet, -and initialize it if not. - -The next thing we need to do is rip out all the calls to `do_sub_pragmas` and -all the calls to the subs which call it, directly and indirectly. Library -initialization becomes explicit and performed in PIR code, not implicitly -performed throughout the C code base. We *can* add a wrapper that does it from -C code to ease the transition, but we lose all our performance gains if we go -that route. One place where we currently *need* to call `:init` Subs from C -is during program execution from the C frontend. Of course, we only need to do -it that way until we have an alternate frontend which is written in PIR. - -If we have a frontend that bootstraps to a PIR entry function earlier, we can -do packfile loading and packfile initialization from PIR code. This way we -can avoid overhead of creating multiple runloops before we've even started -executing our program. In the `whiteknight/frontend_parrot2` branch I've -started working on this very thing. The only stumbling block I am running -into is finding a good and inexpensive way to separate out the arguments which -need to be handled in C from the arguments we can safely handle from PIR, and -then handling those arguments in the correct places. This isn't technically -difficult, I just need time (and feedback!) to make sure I do it in a sane, -performant, and maintainable way. - -Right now, in the `whiteknight/imcc_tag` branch, the two flags `:load` and -`:tag("load")` function almost identically (same with `:init` and -`:tag("init")`), except the second is not found or executed automatically by -`do_sub_pragma()`. Once we have the new frontend in place, I can make the -switchover so that `:load` internally is treated exactly the same as -`:tag("load"), and we can get rid of that portion of `do_sub_pragma()` -completely. At that point, we can slowly phase out the `:load` and `:init` -syntax. - -To recap, here's the basic road forward: -1. Get the basic `:tag()` syntax well-tested and merged into master. This - should be non-destructive and most people won't even notice it has been - added. People who want it can start to make immediate use of it. Plobsing - has also suggested some further possible performance improvements here, so - we can be making it even better. -2. Implement some kind of a cache for keeping track of which libraries have - been initialized, and in which ways. -3. Start moving towards a new frontend, which bootstraps to PIR as early as - possible. -4. Remove `load_bytecode_s` -5. Update `:load` and `:init` to be thin aliases over `:tag("load")` and - `:tag("init")` respectively. As far as the user of PIR code is concerned, - this step should be completely transparent. -6. Rip out much of the `do_sub_pragma()` logic. -7. Deprecate and remove `:load` and `:init` flags. -8. Try to unify even further, switching `:main` to `:tag("main")` (which - should be both simple and possibly have other benefits) and `:postcomp` to - `:tag("postcomp")`. Other flags like `:immediate` and `:anon` will have to - be re-thought and possibly removed in later, separate refactors. -9. Profit! - -It's a long way forward, but there's a very good chance that most of this -could be done before the 4.0 release in January, and the remainder shortly -thereafter. - diff --git a/drafts/upcoming_branches.md b/drafts/upcoming_branches.md deleted file mode 100644 index d7b5b2b..0000000 --- a/drafts/upcoming_branches.md +++ /dev/null @@ -1,79 +0,0 @@ -We've got a couple big branches which could be merged before the next release -in the works, and I want to give a quick overview of each. If all of these -merge in before the 3.9 release, it could end up being a pretty awesome -release overall. - -## `whiteknight/frontend_parrot2` - -The `whiteknight/frontend_parrot2` branch is one I started to look into the -reoccuring idea I and others have had for some time: Can we bootstrap into PIR -earlier and rewrite the frontend executable to be at least partially written -in PIR? - -The answer is a resounding "yes!". The only things that are done in C are -creating the interpreter, creating the IMCC compiler object instances, and -parsing a handful of the lowest-level commandline options. Everything else -is done by a new PIR program called 'prt0.pir'. The prt0 program parses the -rest of the commandline options and is in charge of doing things like reading -in and compiling PIR, loading PBC packfiles, writing out PBC files, handling -uncaught exceptions, showing usage, version and help text, and doing a few -other things. In the future when IMCC is dynamically loadable, the prt0 -frontend will also be responsbile for loading it and registering the compiler -objects. - -What does it mean for you? Well, startup performance is affected in a small -way. For simple programs, startup is slightly slower. For more complicated -programs, startup is slightly faster. It gets faster as the program gets -more complicated, but we're only talking about startup times. Once we start up -and get to running the main program logic, there should be no change in -performance. - -In the bigger sense, we're running PIR code *before* we execute the `:main` -subroutine. This means that we can start defining things in PIR, like classes -or global functions, or whatever. It means we also have complete control over -things like packfiles from PIR, so we can start changing around some of the -IMCC internals to make better assumptions about where it's being called from, -etc. That should help to simplify some code and provide us with new features. - -Parrot hacker plobsing has been doing some excellent fixing and cleanup work -in this branch, and barring a few last minute problems we should be good to -merge it sometime this week. Most users won't even notice the change, but it -will enable us to start working on other cool new projects. - -## `whiteknight/kill_threads` - -We've finally hit our patience limit as a community, and we're ripping out -threads. At least, we're doing it in a branch. It's not all parts of the work -are straight-forward, but already I've been able to clean up a lot of terribly -ugly code and accidentally fix a few bugs along the way. - -We've been living with a lot of problems in the threading system, and alot of -problems tangentially caused by its implementation for a long time now. -Ripping it all out is like a breath a fresh air. You almost start to get -hopeful and think "hey, we could really have new concurrency stuff implemented -here, if we don't have to work around all this other junk". We've got a lot -of ideas floating around for a replacement, and a few hackers who have -expressed an interest in implementing a replacement. I'm hoping that as soon -as we get the crufty old code out of the way we will be able to break ground -on the shiney new stuff. - -This branch has a ways to go. There are still a few broken tests, including -a few which aren't entirely straight-forward to fix. Also, I'm, not entirely -certain where we stand with respect to deprecation boundaries, so we might -not be able to merge it until after 3.9 in any case. - -## `whiteknight/6model` - -I started this branch last weekend, to start porting the 6model infrastructure -from the NQP repo to Parrot. Right now, it's basically a copy+paste of the -code to Parrot, plus fixes to make it build and start passing some of our -code formatting tests. This isn't necessarily the way we want to proceed in -the long run, we may want to do a proper port and rewrite the functionality -from the ground up with our own requirements in mind. - -At the moment the branch builds but has a few problems with regards to -initialization. Also, I haven't done any testing to see if the 6model code -actually works. I suspect there will be some problems. We've ported over all -the PMC types and all the ops, so now it's a matter of seeing how things work -and fit together. I've heard jnthn, the author of 6model, is away for a few -days, so this branch might be on hold until we have a chance to talk to him. diff --git a/drafts/context_reuse.md b/stage/context_reuse.md similarity index 100% rename from drafts/context_reuse.md rename to stage/context_reuse.md