IrcLog2008 12 03

17:18:19  *      stevenknight (n=stevenkn@nat/google/x-ff6fd1a26f0686e6) has joined #scons 
17:25:16  *      Jason_ (n=[Jason@bementil-116.illinois.prairieinet.net](mailto:Jason@bementil-116.illinois.prairieinet.net)) has joined #scons 
17:29:26  *      garyo-home (n=[chatzill@209.6.158.38](mailto:chatzill@209.6.158.38)) has joined #scons 
17:31:26  *      [GregoryNoel](GregoryNoel) is here 
17:31:38  <[GregoryNoel](GregoryNoel)>  Hi, guys, are we ready? 
17:31:51  <garyo-home>   Hi Greg.  Ready as I'll be. :-/ 
17:32:23  <[GregoryNoel](GregoryNoel)>  I only see limited comments in the spreadsheet; it's gonna be a slow night. 
17:32:53  <garyo-home>   yes, sorry.  Just doing some now. 
17:33:02  <stevenknight> hey 
17:33:07  <stevenknight> me too 
17:34:15  <[GregoryNoel](GregoryNoel)>  1945? 
17:34:42  <garyo-home>   steven to research & pick one option. 
17:34:47  <garyo-home>   w/ Ludwig. 
17:34:57  <garyo-home>   IMHO. 
17:35:25  <[GregoryNoel](GregoryNoel)>  I'll go with that, at least for now.  We need to come up with something better in the long run. 
17:36:17  <garyo-home>   Not sure about your comment re: rewriting implicit dependency logic, but won't go there now. 
17:36:44  <stevenknight> agree w/what you guys are saying 
17:37:17  <[GregoryNoel](GregoryNoel)>  I think we can do more than what we are now by caching negatives, but it's not the time to redesign it. 
17:37:36  <garyo-home>   right. 
17:38:00  <[GregoryNoel](GregoryNoel)>  so research, Steven + Ludwig? 
17:38:11  <garyo-home>   good. 
17:38:19  <[GregoryNoel](GregoryNoel)>  And once the decision is reached, Ludwig to implement? 
17:38:28  <stevenknight> done 
17:38:31  <[GregoryNoel](GregoryNoel)>  done 
17:38:35  <[GregoryNoel](GregoryNoel)>  2258 
17:38:36  <garyo-home>   2258: stevenknight to add a hook to get help text, 2.x p4? 
17:39:07  <[GregoryNoel](GregoryNoel)>  Hmmmm..... 
17:40:06  <[GregoryNoel](GregoryNoel)>  Yeah, something like that.  His suggestion is awkward, but it should be possible to work out something. 
17:40:38  <[GregoryNoel](GregoryNoel)>  And for 2.x p4, we don't need to volunteer anyone. 
17:40:41  <[GregoryNoel](GregoryNoel)>  yet 
17:40:48  <garyo-home>   agree. 
17:40:52  <stevenknight> okay, i'm back from the spreadsheet 
17:41:04  <stevenknight> 2258:  done 
17:41:09  <stevenknight> 2.x p4, future draft pick 
17:41:10  <[GregoryNoel](GregoryNoel)>  2261 
17:41:38  <stevenknight> it's a google guy, so i'd like a crack at it 
17:41:58  <[GregoryNoel](GregoryNoel)>  We have another issue looking at this already; one should be closed. 
17:42:00  <garyo-home>   Shouldn't he be using the target filename as the target?  (And not rm/mkdir either) 
17:41:59  <stevenknight> if i can't come up with a real case, i'll mark it invalid 
17:42:35  <garyo-home>   ok, that's good enough for me.  Steven to find testcase or mark invalid. 
17:43:05  <[GregoryNoel](GregoryNoel)>  or dup to the other one 
17:43:16  <stevenknight> yeah, i guess i view it similar to unpacking a .tar.gz into a directory 
17:43:22  <stevenknight> i think you should be able to specify the directory 
17:43:30  <stevenknight> and have it do the right thing w.r.t. changes to the source(s) 
17:43:41  <stevenknight> but i'll mark it invalid if I'm just dreaming 
17:43:47  <[GregoryNoel](GregoryNoel)>  done 
17:44:00  <[GregoryNoel](GregoryNoel)>  2262 
17:44:12  <stevenknight> i think it should be trivial 
17:45:26  <[GregoryNoel](GregoryNoel)>  If you think it's trivial, I'll let you have it. 
17:45:20  <stevenknight> 2262:  anytime, anyone 
17:45:29  <[GregoryNoel](GregoryNoel)>  anytime is fine 
17:45:39  <garyo-home>   The OP says he'd like "not .py" but I think .py makes the most sense. 
17:45:41  <stevenknight> okay, 2262 anytime stevenknight 
17:46:15  <Jason_>       I like the .scons ,myself 
17:46:17  <stevenknight> while we're at it, i'd go for both .py and .scons 
17:46:36  <stevenknight> and draw the line there 
17:46:38  <[GregoryNoel](GregoryNoel)>  I've also seen .sc used. 
17:46:45  <garyo-home>   ok, why not.  Just pick an order, seems fine to me. 
17:47:04  <garyo-home>   Greg: I think .sc is too short. 
17:47:17  <[GregoryNoel](GregoryNoel)>  Sigh.  Seems too complex; I'd still wontfix it as I indicated. 
17:47:02  <stevenknight> isn't .sc for [SuperCalc](SuperCalc) files...? 
17:47:05  *      stevenknight shows his age... 
17:47:12  <garyo-home>   [SuperCalc](SuperCalc)?!! 
17:47:38  <[GregoryNoel](GregoryNoel)>  Yeah, I remember it, too. 
17:47:46  *      [GregoryNoel](GregoryNoel) _really_ shows his age. 
17:47:47  <stevenknight> 2262:  anytime, stevenknight 
17:47:52  <[GregoryNoel](GregoryNoel)>  done 
17:48:03  <[GregoryNoel](GregoryNoel)>  2264 
17:48:27  <garyo-home>   Greg, you looked at this most -- is the order reversal the only thing going on here? 
17:48:28  <stevenknight> 2.x p2 gregnoel? 
17:49:16  <[GregoryNoel](GregoryNoel)>  No, the dependencies aren't reversed, reversing the order of the names on the command line produces different trees. 
17:49:43  <garyo-home>   ok, I believe you :-) 
17:49:47  <[GregoryNoel](GregoryNoel)>  No matter which way you list the names on the command line, it should show the same dependencies. 
17:49:57  <garyo-home>   do you mind taking it? 
17:50:45  <[GregoryNoel](GregoryNoel)>  I know almost nothing about that part of the code, so I'd rather not, but if you push, I'll consider it. 
17:51:04  <stevenknight> 2.x p2 can be future draft pick 
17:51:16  <stevenknight> or else put my name on it 
17:51:16  <[GregoryNoel](GregoryNoel)>  I'll go with that 
17:51:27  <stevenknight> okay, 2.x p2 future draft pick 
17:51:28  <stevenknight> done 
17:51:40  <[GregoryNoel](GregoryNoel)>  2265 
17:51:50  <garyo-home>   ah, yes. 
17:51:56  <Jason_>       just so it is known this is not new 
17:52:15  <stevenknight> 2265:  1.2 p1 stevenknight 
17:52:16  <garyo-home>   Jason_, you mean 2265? 
17:52:23  <Jason_>       Yes 
17:52:27  <stevenknight> like I said, I'm just in Taskmaster lately 
17:52:28  <Jason_>       I filed it 
17:52:30  <garyo-home>   thanks, Steven. 
17:52:47  <garyo-home>   Ah, you're *that* Jason -- welcome! 
17:52:55  <Jason_>       Thanks 
17:53:13  <[GregoryNoel](GregoryNoel)>  stevenknight, done 
17:53:19  <[GregoryNoel](GregoryNoel)>  and thanks 
17:53:18  <stevenknight> Jason_:  glad to have you here 
17:53:24  <Jason_>       hope we can talk latter :-) 
17:53:38  <stevenknight> yep 
17:53:46  <stevenknight> i have a few non-issue topics i'd like to discuss as well 
17:53:54  <garyo-home>   ok 
17:54:12  <[GregoryNoel](GregoryNoel)>  2266: I'll let you guys fight it out 
17:54:17  <stevenknight> 2266:  1.x p3 stevenknight 
17:54:36  <Jason_>       we have seen this .. it depends on the python used 
17:54:43  <garyo-home>   2266: not sure why this doesn't work for him.  I do this all the time.  What python is he using? 
17:54:58  <garyo-home>   Do NOT use cygwin python. 
17:55:15  <Jason_>       That follows what we see here as well 
17:55:29  <Jason_>       cygwin python will mess up a windows based build 
17:55:37  <garyo-home>   2266: let me take it and suggest that to him. 
17:55:43  <[GregoryNoel](GregoryNoel)>  done 
17:55:54  <[GregoryNoel](GregoryNoel)>  2267 
17:56:47  <stevenknight> either 1.2 and defer if investigation shows we can live with it 
17:56:56  <[GregoryNoel](GregoryNoel)>  This is the new Action.py code I put in, but I can't see how it's not initialized. 
17:56:57  <stevenknight> or resesarch with the option of making it a 1.2 blocker 
17:58:19  <[GregoryNoel](GregoryNoel)>  Seeing no hands, I'll research it.  It's a bad week for that, but I'll try to make time. 
17:58:22  <garyo-home>   Greg: possible exception issue? 
17:58:27  <[GregoryNoel](GregoryNoel)>  garyo-home, huh? 
17:58:35  <garyo-home>   never mind. 
17:59:15  <[GregoryNoel](GregoryNoel)>  so 2267 research me? 
17:59:20  <stevenknight> done 
17:59:47  <stevenknight> [GregoryNoel](GregoryNoel):  let me know if I can help, I've had to debug things like this many times 
17:59:53  <[GregoryNoel](GregoryNoel)>  thanks 
18:00:05  <[GregoryNoel](GregoryNoel)>  Before we go on, can I insert a topic? 
18:00:17  <garyo-home>   sure 
18:00:34  <stevenknight> sure, let's do GN's first, then Jason_, then me 
18:00:30  <[GregoryNoel](GregoryNoel)>  I'd like to change the agenda slightly.  I'd like to add a permanent item to discuss the current status of the release schedule and how we're doing on it. 
18:00:30  <[GregoryNoel](GregoryNoel)>  Discussion points would be what's expected in the next release(s) (whether checkpoint, candidate, or public), adjustments to the schedule, whether we should do any retriage for current or future releases, things like that. 
18:00:30  <[GregoryNoel](GregoryNoel)>  I'd put it between the current issues and the backlog.  (Aside: we're almost to the home stretch for finishing off the backlog (hopefully we'll kill off 2005 this time or next time); after that, the time now spent on backlog will be taken up by bickering about retriage of issues, so we should discuss the release schedule strategy before the tactics.  This positioning puts it in the right place for the future.) 
18:01:13  <stevenknight> +1 to discussing release schedule regularly 
18:01:24  <garyo-home>   greg: +1 to your agenda change.  I always review the schedule before the meeting anyway. 
18:02:16  <[GregoryNoel](GregoryNoel)>  OK, I'll do it.  Since the slot would be now, should we discuss the schedule now? 
18:02:21  <stevenknight> yes 
18:02:23  <garyo-home>   yes. 
18:02:31  <[GregoryNoel](GregoryNoel)>  Schedule:  1.2 was due out 24 November.  Needless to say, we didn't make it.  And issues 2265 and 2267 could be a blockers; we may need to slip 1.2 until that's resolved. 
18:02:31  <[GregoryNoel](GregoryNoel)>  We were supposed to have a checkpoint out 1 December.  We didn't make that, either. 
18:02:31  <[GregoryNoel](GregoryNoel)>  Assuming we get the release out this week, we have some decisions to make.  Should we slide the rest of the schedule by a week, so the next checkpoint is 8 December?  Cancel the 1 December checkpoint and keep the current schedule, with the next checkpoint 15 December?  Or something else? 
18:02:31  <[GregoryNoel](GregoryNoel)>  I lean toward dropping the 1 December checkpoint, but I could be easily persuaded differently. 
18:02:59  <stevenknight> i put out a release candidate 11/25? 
18:03:33  <[GregoryNoel](GregoryNoel)>  I think that's about right 
18:03:28  <garyo-home>   I have three 1.2 tickets on my plate.  None serious except maybe #2048 which I'm working on but fails in weird ways 
18:04:42  <stevenknight> oh, wait, I see--the release schedule is 12/1 checkpoint that was supposed to be *after* 1.2 was out 
18:04:45  <stevenknight> and i'm late with 1.2 
18:04:51  <stevenknight> right? 
18:05:10  <garyo-home>   I think so.  I'd like to get 1.2 out soon then put vs_revamp in for next checkpoint after that. 
18:05:40  <[GregoryNoel](GregoryNoel)>  stevenknight, yes 
18:05:37  <stevenknight> right, and I actually have batched Visual C/C++ compilation working on a private branch that I'd like to go in 
18:05:56  <garyo-home>   wow! 
18:05:57  <Jason_>       I would like to add stuff to the revamp 
18:06:12  <garyo-home>   Jason: good.  You have an intelc.py version of it too I think. 
18:06:12  <stevenknight> okay, so let's review 1.2 issues and push anything that's not a potential/likely blocker in 1.3 
18:06:18  <Jason_>       We have some work not finished that will improve it and make it faster 
18:06:38  <garyo-home>   OK.  Let's get it into the trunk, then review & apply your changes 
18:06:46  <Jason_>       yep... still working on that part 
18:06:48  <garyo-home>   ok? 
18:07:02  <Jason_>       plus we have early support for vs 2010 and intel 11 
18:07:02  <stevenknight> garyo_home:  "it" into the trunk:  "it" == vs_revamp? 
18:07:14  <stevenknight> Jason_:  sweet 
18:07:31  <garyo-home>   steven: yes, it==vs_revamp. 
18:07:36  <Jason_>       I have "people" that know "people" 
18:07:38  <stevenknight> okay, agreed 
18:08:01  <Jason_>       Well my fixes to the revamp.. are a bit different from the work David did 
18:08:02  <garyo-home>   There are 40 open 1.2 issues. 
18:08:05  <[GregoryNoel](GregoryNoel)>  Almost 40 issues scheduled for 1.2 still 
18:08:13  <[GregoryNoel](GregoryNoel)>  garyo-home, jinx 
18:08:47  <garyo-home>   Jason_: send a summary of your ideas & differences to the list, we'll discuss it there. 
18:08:55  <Jason_>       ok 
18:09:13  <garyo-home>   It's a bigger forum, David will be present, etc. 
18:09:16  <stevenknight> a lot of the 40 issues are readily rollable to 1.3 
18:09:19  <stevenknight> doc issues, e.g. 
18:09:57  <[GregoryNoel](GregoryNoel)>  Mine are all doc and testing (I haven't figured out how to test Action._subproc yet) 
18:10:05  <garyo-home>   15 P2s, the rest P3 & P4. 
18:10:49  <stevenknight> oh, and 40 doesn't count the two potential blockers we discussed tonight 
18:11:15  <stevenknight> quick, quick glance suggests all P3 & P4 are automatically deferrable 
18:12:07  <garyo-home>   agreed. 
18:12:09  <stevenknight> and I think the cournape P2 issues are because we once thought vs_revamp was going to make it into 1.2 
18:12:54  <garyo-home>   Is Benoit around?  Any likelihood of his 4 issues getting done? 
18:13:06  <stevenknight> slim, he's been AWOL for a while 
18:13:21  <garyo-home>   thought so. 
18:13:31  <stevenknight> worth considering reassigning his issues so they don't languish 
18:14:04  <garyo-home>   Yes.  I suggest moving to 1.3 for now, then reassigning. 
18:15:02  <[GregoryNoel](GregoryNoel)>  garyo-home, agree 
18:15:13  <stevenknight> okay, so it sounds like the only two issues that don't go to 1.3 are the two new potential blockers 
18:15:32  <[GregoryNoel](GregoryNoel)>  I can do that with a mass move. 
18:15:41  <stevenknight> [GregoryNoel](GregoryNoel):  thanks 
18:15:33  <stevenknight> we get patches for those (or research suggests deferring to 1.3) and then I cut another 1.2 candidate 
18:16:02  <garyo-home>   What's up w/ 2249 though? 
18:16:44  <stevenknight> dunno... 
18:16:47  <garyo-home>   (Not that it should be in 1.2 anyway, but how'd it get there?) 
18:17:07  <stevenknight> vs_revamp, we thought it would be in 1.2 
18:17:08  <garyo-home>   It can move to 1.3 too since it's vs_revamp related. 
18:17:59  <garyo-home>   ok, so given all that, what's the proposed 1.2 release date? 
18:18:10  <[GregoryNoel](GregoryNoel)>  ASAP? 
18:18:13  <stevenknight> stake in the ground:  12 December, a week from Friday 
18:18:20  <stevenknight> candidate 5 December, two days 
18:18:32  <stevenknight> for the two potential blockers 
18:18:41  <[GregoryNoel](GregoryNoel)>  works; I'll update the roadmap 
18:18:44  <garyo-home>   ok, so I have some time to sneak in some little things before the candidate, then we freeze, right? 
18:19:00  <stevenknight> yep 
18:19:04  <garyo-home>   good. 
18:19:20  <stevenknight> okay, shall we move on to Jason_'s topic? 
18:19:37  <garyo-home>   yes. 
18:19:57  <garyo-home>   I looked a little at the doc for Parts; it is quite serious! 
18:20:22  <Jason_>       Well we have some new tool coming out... it has to be a bit serious 
18:20:41  <garyo-home>   Jason, I wonder if some of it couldn't be done with SCons Tools instead, but basically it looks useful. 
18:20:49  <garyo-home>   What's the new Intel tool coming out? 
18:21:05  <Jason_>       well I have a lot to say on all of this 
18:21:21  <Jason_>       Idea the doc i sent is about extending Scons... 
18:21:31  <garyo-home>   The basic idea is a way to make a scalable composition of libs etc., right? 
18:21:32  <Jason_>       here i think tools needs many improvements 
18:22:05  <Jason_>       well yes... but also about making large project sain to deal with 
18:22:33  <garyo-home>   Right, by imposing certain conventions that make it subdividable nicely. 
18:22:36  <Jason_>       having a unified build system is very important 
18:22:55  <Jason_>       but also trying to be flexible 
18:23:07  <stevenknight> sorry, I haven't had time to look before now 
18:23:09  <Jason_>       it is a hard balance... 
18:23:12  <garyo-home>   Of course; lots of people have large unified build systems with SCons.  Your way is one way of formalizing a structure. 
18:23:18  <stevenknight> is the doc the .7z file you attached? 
18:23:29  <garyo-home>   Steven: yes, it's in there. 
18:23:37  <Jason_>       yes.. the problem with the team i work with is that we are all over the place 
18:23:48  <Jason_>       having a "make system" does not work well 
18:23:56  <Jason_>       yes 
18:24:16  <stevenknight> agh, i don't have 7zip on this system 
18:24:24  <Jason_>       it is in word 2007 format.. I can give you an update in something else like html 
18:25:25  <Jason_>       so the first question for me is how can we best stare this code 
18:25:54  <Jason_>       My boss would like this to be part of SCons instead of it own project... 
18:26:27  <Jason_>       however it might be best to have it as a seperate project with heavy SCons involment 
18:27:11  <garyo-home>   I've long thought it would be good for SCons to have a contrib/ subtree, so that might be one way to go.  The downside is you're tied to SCons release schedule and other stuff though.  Separate is probably easier. 
18:27:36  <Jason_>       well we have our own svn branch here at Intel 
18:27:48  <Jason_>       ideally we would try to move to a public one 
18:28:15  <Jason_>       but we have some stuff in a subdir call "pieces" that i cannot ship 
18:28:33  <garyo-home>   Not sure what the best way to host a project like that is, but there are many places to look. 
18:28:44  <Jason_>       as it is internal add on that have knowleage of internal Intel networks 
18:28:59  <garyo-home>   You'll have to make it work "out of the box" from the public repo, of course. 
18:29:10  <Jason_>       yep that is the idea 
18:29:22  <Jason_>       the sample in the code i gave you should work out of the box 
18:29:33  <garyo-home>   I think everyone will be happier if it is separate from scons but closely related, at least for now. 
18:29:37  <Jason_>       the full sample will need svn to be installed :-) 
18:30:09  <garyo-home>   But we should spend some time at least reading the doc to get an idea of the scope and goals of the project. 
18:30:17  <Jason_>       If this agreed, I will start the process to make a tigris Parts project 
18:30:39  <garyo-home>   How about calling it SConsParts or something to show its relation to SCons? 
18:30:58  <Jason_>       that is fine with me 
18:31:12  <Jason_>       any other votes? 
18:31:58  <garyo-home>   Let's talk about the tech details and direction more at the next bug party meeting in 2 weeks. 
18:32:11  <stevenknight> sorry, still trying to open things up 
18:32:28  <stevenknight> got the .7z open, but i don't have Office on the system I switched to... 
18:32:28  <Jason_>       in 2 week I will be in florida *not* working 
18:32:31  <garyo-home>   btw, is this going to appear in a public tool from Intel? 
18:32:35  <Jason_>       so after that is fine 
18:33:01  <garyo-home>   Jason: ok, whenever you are around.  But give us a week or so to look it over. 
18:33:10  <Jason_>       sure 
18:33:14  <garyo-home>   Esp. given 1.2 release schedule :-) 
18:33:29  <stevenknight> Jason_:  there's an early effort here at Google to do something that sounds similar to Parts 
18:33:38  <stevenknight> but it looks like you're much farther along 
18:33:46  <stevenknight> when I look, I 
18:33:47  <Jason_>       so the parts stuff will be public.. MIT under Intel with the intent to be move as much as possible to SCons 
18:34:02  <Jason_>       you work at Google 
18:34:14  <stevenknight> I'm going to look for areas of possible cooperation 
18:34:27  <stevenknight> yes, almost 11 months now 
18:34:38  <Jason_>       oh .. I thought it was VMware 
18:34:45  <stevenknight> that was before, and on contract 
18:34:56  <Jason_>       well you should contact me... INtel and google have a lot going on 
18:35:18  <stevenknight> the problem you're addressing is what we're running into too 
18:35:46  <Jason_>       so are many projects at Intel .... 
18:35:56  <stevenknight> right, SCons is pretty good low-level plumbing 
18:36:08  <Jason_>       very powerful stuff however 
18:36:15  <stevenknight> but there's no consistency for higher layer plug-and-play between components 
18:36:35  <stevenknight> esp. in the open source world it all depends on how the person wrote the SCons config 
18:36:37  <Jason_>       I think as i under scons better and you parts.. I think there is a lot of value here to improve on 
18:36:47  <stevenknight> agreed 
18:37:10  <Jason_>       the plug and play in very important 
18:37:30  <stevenknight> right, it's the real value add of Autotools (IMHO) 
18:37:31  <Jason_>       when I was at a start up before intel.. one person controled the make scripts 
18:37:58  <stevenknight> being able to approach any package and know how to at least build it consistently 
18:38:12  <Jason_>       in there larger cross geo.. developer fight over details and perfection 
18:38:14  <stevenknight> sounds like Parts goes even further though 
18:38:23  <Jason_>       so everything breaks . 
18:38:26  <stevenknight> right 
18:38:32  <stevenknight> so following up on what Gary said 
18:38:42  <Jason_>       cross geo poltics i have found can make it hard to get simple stuff done 
18:38:56  <stevenknight> I'm going to take a closer look after 1.2 is out 
18:39:03  <Jason_>       ok 
18:39:24  <stevenknight> and perhaps touch base with you off line to compare first impressions 
18:39:26  <Jason_>       I will send out friday an update to the code and document ( small tweaks and fixes) 
18:39:33  <stevenknight> thanks 
18:39:41  <Jason_>       so FYI 
18:39:41  <stevenknight> if you can convert doc to some non-Word format that'd help me 
18:39:51  <garyo-home>   same here 
18:39:49  <Jason_>       WW 51 i am out of town 
18:40:05  <garyo-home>   What is "WW 51"? 
18:40:08  <Jason_>        I will be around other wise and will keep track of the e-mails 
18:40:23  <Jason_>       week of Dec 15 
18:40:34  <garyo-home>   ah yes. 
18:40:43  <stevenknight> okay, i'll look for your update and we'll go from there 
18:40:50  <Jason_>       ok 
18:40:57  <Jason_>       thank you for your time 
18:41:12  <garyo-home>   ok, Steven I think it's your turn now! 
18:41:17  <stevenknight> ok 
18:41:21  <garyo-home>   Jason_: yr welcome. 
18:41:40  <stevenknight> one is the Visual C/C++ batch builder change I mentioned 
18:41:58  <stevenknight> took a while, but it's close to solid 
18:42:09  <stevenknight> some unit tests need attention 
18:42:27  <garyo-home>   Impressive, can't wait to try it.  We have a lot of vc++ compiled into a few big libs. 
18:42:54  <stevenknight> i'm going to try to get it in right away after 1.2 is out so it gets soak time 
18:43:16  <stevenknight> it has some changes to the Taskmaster that aren't terribly extensive 
18:43:23  <stevenknight> but may have unintended side effects 
18:43:31  <[GregoryNoel](GregoryNoel)>  (BRB.  My wife just arrived with food and I've got to eat it while it's hot.) 
18:43:56  <garyo-home>   Do users have to do anything or does it just work? 
18:43:59  <stevenknight> darn, bad timing, I was going to ask Greg something about Taskmaster NG 
18:44:03  <Jason_>       If you need someone to help test the taskmaster on a large project we are happy to help 
18:44:24  <stevenknight> it's controlled by setting MSVC_BATCH=True (or 1 or whatever) 
18:44:59  <garyo-home>   sounds great.  Does it generalize to other batch-buildable things? 
18:45:06  <stevenknight> yes 
18:45:10  <[GregoryNoel](GregoryNoel)>  (I'm still reading, but I have to put down the burger to type.) 
18:45:18  <stevenknight> okay, cool 
18:45:21  <Jason_>       if you can give me the detail of what to do i will give it a run on a few new i7 systems 
18:45:41  <stevenknight> the Taskmaster issues is that this suggests the Taskmaster is handling the wrong granularity of input 
18:45:45  <stevenknight> it's looking at individual Nodes 
18:45:55  <stevenknight> and I'm beginning to think it should be looking at Executors 
18:46:13  <stevenknight> [GregoryNoel](GregoryNoel), just chew on that a little and let me know if it sounds crazy 
18:46:16  <stevenknight> based on your TNG work 
18:46:37  <stevenknight> you can do your own batch builder at the Action() level 
18:46:51  <stevenknight> simplest case is Action('$FOOCOM', batch_key=True) 
18:47:02  <[GregoryNoel](GregoryNoel)>  That's what TNG does: the transverse graph is in (a wrapper around) the Executor. 
18:47:14  <stevenknight> sweet, so I'm not entirely crazy, at least 
18:47:21  <stevenknight> maybe this helps move us toward TNG 
18:47:52  <stevenknight> batch_key=True will separate all calls to the given Builder using that action 
18:48:59  <stevenknight> into grouped batches for the four-tuple (Action, env) 
18:49:05  <stevenknight> excuse me, two-tuple 
18:49:22  <stevenknight> batch_key can also be a function that takes (action, env, target, source) 
18:49:39  <stevenknight> and returns the key to be used for separating batches 
18:50:37  <stevenknight> so the MSVC_BATCH support uses the four-tuple (action, env, target.dir, source.dir) as the key 
18:50:56  <garyo-home>   That's very clever. 
18:51:05  <stevenknight> with the upshot that you get batched compilation of all object files in a given directory 
18:51:26  <stevenknight> because $CPPPPATH can affect your #includes based on the directory 
18:51:36  <garyo-home>   ... and the same compiler options etc. 
18:51:49  <stevenknight> right, out of the same env 
18:52:14  <garyo-home>   Is it a lot faster? 
18:52:31  <stevenknight> for our config (Google Chrome) only about 10%-15% 
18:52:41  <stevenknight> which is nothing to sneeze at, but still, we were hoping for more 
18:52:50  <garyo-home>   That's nothing to sneeze at (jinx) 
18:53:04  <stevenknight> on the other hand, i haven't gone in and profiled and optimizd it yet 
18:53:17  <stevenknight> i'll probably upload to a subversion branch so people can try it out early 
18:53:20  <garyo-home>   For us, we use -O3 with the Intel compiler so all the time goes into the link step, but I'm sure this will help us even in that case. 
18:53:21  <stevenknight> and send out email+doc 
18:53:50  <stevenknight> (i've been doing this work experimenting with mercurial as a front end private branch development) 
18:54:07  <stevenknight> okay, enough of that 
18:54:24  <garyo-home>   what else? 
18:54:34  <stevenknight> next:  Google contributors to SCons 
18:54:41  <Jason_>       so i have a question 
18:54:49  <stevenknight> yes? 
18:55:03  <Jason_>       is there any plans to improve the depend checker for C/C++ 
18:55:16  <Jason_>       there is a bit of an issue with BOOST headers for use 
18:55:21  <stevenknight> i.e. make it like a real C preprocessor? 
18:55:34  <Jason_>       We have a work around... but it seem to cause more trouble than it solves 
18:55:39  <stevenknight> there's an experimental module that will do that 
18:56:08  <Jason_>       well one that is smart enought to get the #include FOOBAR stuff 
18:56:16  <stevenknight> yes 
18:56:27  <stevenknight> take a look at (IIRC) Scanner/C.py 
18:56:31  <Jason_>       how can we try it out 
18:56:59  <Jason_>       branch in SCons? 
18:57:01  <stevenknight> there's some commented out code that uses a cpp.py module to do enough "real" CPP stuff to handle boost 
18:57:05  <Jason_>       or in the main code 
18:57:09  <stevenknight> no, it's checked in trunk 
18:57:33  <stevenknight> just hasn't been productized by giving it the right user-configurable hook 
18:57:43  <stevenknight> if you want to finish that piece and contribute it back, it'd be very welcome 
18:58:04  <stevenknight> next:  google contributors 
18:58:30  <stevenknight> i have more people here starting to look at SCons internals 
18:58:39  <stevenknight> especially w.r.t. performance 
18:59:08  <stevenknight> which is good 
18:59:23  <stevenknight> but we're finding that a lot of the flexibility we prize in SCons 
18:59:53  <stevenknight> is antithetical to the kind of caching we'd need to do to really speed up things 
19:00:08  <stevenknight> i'm wrestling with how best to channel this interest and energy 
19:00:21  <garyo-home>   that's hard. 
19:00:34  <stevenknight> in ways that help SCons overall as well as Google's own purposes 
19:00:50  <stevenknight> a Google fork of SCons is unattractive 
19:00:58  <garyo-home>   Right, certainly. 
19:01:12  <garyo-home>   Some kind of "mode"? 
19:01:17  <stevenknight> but taking some of this work back into main line runs serious risk of breaking backwards compatibility 
19:02:05  <garyo-home>   and also making some existing idioms just plain not work, possibly. 
19:02:12  <stevenknight> right 
19:02:31  <garyo-home>   How invasive is it? 
19:02:57  <stevenknight> well, we're in early stages 
19:03:07  <stevenknight> so more hypthetical than not 
19:03:12  <garyo-home>   ok. 
19:03:27  <stevenknight> one patch i have queued tries to avoid starving child worker threads 
19:03:39  <stevenknight> (actually doesn't try, it does avoid it) 
19:03:57  <stevenknight> by just feeding as many ready Tasks as possible into the queue 
19:04:00  <stevenknight> that doesn't break compatibility 
19:04:20  <garyo-home>   That seems great.  In general, I think if it's really valuable, breaking simple back compat is ok (i.e. people have to do some simple recoding), but forcing larger changes is problematic. 
19:04:23  <stevenknight> but it's in that pesky Taskmaster area where unintended affects abound 
19:04:25  <[GregoryNoel](GregoryNoel)>  Sounds like TNG 
19:05:00  <stevenknight> mm, maybe I should send this patch to you to see what you think 
19:05:36  <stevenknight> we were able to do some visualization (using Incredibuild) to see the starvation at work without the patch, and no starvation with it 
19:06:32  <[GregoryNoel](GregoryNoel)>  Sure, I can take a look.  You should also look at TNG. 
19:06:33  <stevenknight> tradeoff is more CPU cycles in the master thread searching the whole DAG for work more frequently 
19:06:55  <[GregoryNoel](GregoryNoel)>  TNG avoids that. 
19:07:09  <stevenknight> okay, let's swap code and see how close we are 
19:07:33  <stevenknight> are there issues holding up TNG? 
19:08:02  <[GregoryNoel](GregoryNoel)>  No, not really.  Just lack of time. 
19:08:08  <stevenknight> okay 
19:08:12  <stevenknight> i'll send you the patch from here 
19:08:37  <stevenknight> so now that I'm articulating this 
19:08:38  <[GregoryNoel](GregoryNoel)>  (here?) 
19:08:43  <stevenknight> Google 
19:09:30  <stevenknight> the meta issue is that this could bring a lot more impactive changes quickly into SCons 
19:09:42  <stevenknight> with attendant possibility for destabilizing things 
19:10:04  <stevenknight> in order to not hold up stuff, I'm going to have to spend (more) work time actually, oh, integrating patches 
19:10:07  <[GregoryNoel](GregoryNoel)>  (my fingers are still sticky, but I'm otherwise back.) 
19:10:08  <stevenknight> in general, not just from Google 
19:10:34  <stevenknight> and to do that I'm having to draft people here to cover some of the front-line work i've been trying to do on the Google Chrome build 
19:11:12  <stevenknight> one other possibility I considered was to have a "google branch" in our SVN repository 
19:11:29  <stevenknight> and pluck changes from that into trunk 
19:11:32  <[GregoryNoel](GregoryNoel)>  opposed: branches by features, not source 
19:11:49  <stevenknight> yeah, that makes sense 
19:12:23  <stevenknight> okay, so i just need to become johnny-on-the-spot with integration 
19:12:33  <stevenknight> and maybe recruit additional committers / integrators 
19:12:40  <stevenknight> from Google or elsewhere 
19:12:50  <garyo-home>   can you do that from Google?  Would be useful. 
19:13:05  <garyo-home>   We haven't had much luck from the users list to date. 
19:13:20  <stevenknight> can I do...?  get more people from here working on SCons-the-project? 
19:13:57  <garyo-home>   Yes. 
19:14:18  <stevenknight> there's certainly no institutional barrier 
19:14:57  <garyo-home>   Maybe get another googler or two on the dev list, tag along to a bug party... 
19:15:01  <stevenknight> client-side projects here are getting behind SCons pretty heavily 
19:15:09  <[GregoryNoel](GregoryNoel)>  Doesn't Google have a rule that you can spend up to 20% of your time "doing good"?  Recruit in that space. 
19:15:43  <stevenknight> that still ends up being driven by whether people have an itch to scratch 
19:15:51  <stevenknight> that they want to spend their 20% on 
19:16:07  <garyo-home>   but SCons is *sooo* *coool*! 
19:16:07  <[GregoryNoel](GregoryNoel)>  Of course.  But fixing their build system to support themselves better is a strong itch. 
19:16:08  <stevenknight> the potential itch here is speeding up builds for their projects 
19:16:31  <garyo-home>   That makes sense. 
19:16:53  <Jason_>       you need faster startup times? 
19:17:01  <stevenknight> faster null build times 
19:17:14  <garyo-home>   steven: agreed, same here. 
19:17:18  <[GregoryNoel](GregoryNoel)>  ditto 
19:17:24  <garyo-home>   hence your caching ideas. 
19:17:28  <Jason_>       ya... big issue here.. I have a work around... but it a hack 
19:17:37  <stevenknight> Jason_:  what's your workaround? 
19:17:48  <Jason_>       we have a DB file we make 
19:18:03  <Jason_>       we use this to figure out if we need to define soemthing to SCons 
19:18:13  <Jason_>       this speeds up builds a great deal 
19:18:26  <Jason_>       we woudl have NULL build of 8-10 minutes 
19:18:39  <stevenknight> so you only conditionally load parts of the configuration based on this DB? 
19:18:47  <Jason_>       got it down to 13-63 
19:18:53  <Jason_>       depending on the target 
19:19:06  <Jason_>       13-63 secs 
19:19:22  <Jason_>       yes... but we have to have one good run 
19:19:33  <stevenknight> right, to prep the DB with legitimate info 
19:19:35  <stevenknight> that makes sense 
19:19:39  <Jason_>       else we don't know what is defined 
19:20:01  <stevenknight> kind of related to this is making the DAG in SCons a real DAG 
19:20:08  <stevenknight> (something I know Greg is very much in favor of) 
19:20:18  <stevenknight> with separate tuples representing the edges 
19:20:19  <Jason_>       Same here 
19:20:35  <[GregoryNoel](GregoryNoel)>  Er, I thing you mean making the .sconsign a real DAG, but yes, I do. 
19:20:42  <Jason_>       then you can throw some well known task processing code at it 
19:20:53  <Jason_>       such as stuff used by a compiler 
19:20:59  <stevenknight> right 
19:20:59  <garyo-home>   That scares me a little unless you explicitly allow runtime modification of the DAG 
19:21:16  <[GregoryNoel](GregoryNoel)>  white papers on request. 
19:21:27  <stevenknight> and be able to infer what to build from known changed files 
19:21:43  <garyo-home>   ... but maybe users have to explicitly identify when they're modifying the DAG at runtime to invalidate and/or rescan 
19:21:48  <stevenknight> instead of always having to walk from targets down, searching for what might have changed 
19:21:57  <stevenknight> [GregoryNoel](GregoryNoel):  request, send white papers 
19:22:19  <stevenknight> garyo-home:  you have a use case in mind? 
19:22:31  <garyo-home>   Dynamic source generators. 
19:22:42  <Jason_>       Agreeded 
19:22:46  <garyo-home>   Where you don't know all the generated source names up front. 
19:22:48  <Jason_>       we have that as well 
19:22:51  <stevenknight> that should still be possible 
19:23:05  <garyo-home>   Good, I rely heavily on that. 
19:23:21  <stevenknight> i think the nodes would still have .sources, .implicit, .explicit etc. 
19:23:23  <Jason_>       a simple example if a Doxygen builder 
19:23:31  <[GregoryNoel](GregoryNoel)>  In point of fact, I'd like to optimize the null case first, when the SConscripts haven't been changed, and work from there. 
19:23:33  <stevenknight> but instead of pointing directly to ohter Nodes 
19:23:40  <Jason_>       it can't really know what it will build up front 
19:23:41  <stevenknight> they'd be lists of DAG edges 
19:24:00  <stevenknight> [GregoryNoel](GregoryNoel):  agree in general 
19:24:03  <garyo-home>   Jason_: a very good use case there. 
19:24:18  <stevenknight> but how do we know when the null case is optimized "enough" ? 
19:24:30  <garyo-home>   Steven: I have to think about that before I understand you. 
19:24:34  <[GregoryNoel](GregoryNoel)>  Zero time in parsing. 
19:24:50  <stevenknight> wow, configurations are so different 
19:25:00  <stevenknight> parsing is a really small part of our null build time 
19:25:32  <[GregoryNoel](GregoryNoel)>  I wonder... 
19:26:02  <stevenknight> profiling shows we're heavily dominated by string (re-)expansion 
19:26:46  <[GregoryNoel](GregoryNoel)>  True, and working the bug list yesterday I found a case where a string was expanded *six* times to the same effect. 
19:26:35  <Jason_>       idea... you might want to look at the parts mapper.py file 
19:26:59  <Jason_>       we sort of replace expantions.. to get around the time issue here 
19:27:07  <stevenknight> Jason_;  aha 
19:27:27  <stevenknight> I'll look at what you did as a source of ideas 
19:27:40  <stevenknight> one of my big problems is I've spent so much time focusing on making corner cases work 
19:27:44  <Jason_>       part of this is me hacking a wrapper around the scanner object to force a few expansions 
19:27:59  <stevenknight> that I'm prone to ignore effective solutions that really help the common case(s) 
19:28:03  <Jason_>       5-10% difference in speed 
19:28:32  <stevenknight> we got a pretty good improvement just running big long CPPPATH lists through env.subst() when assigning them 
19:28:58  <Jason_>       the problem was i did not see it when you added it :-( 
19:29:13  <stevenknight> ?  not in SCons itself, in our configuration 
19:29:32  <Jason_>       oh... never was able to use them 
19:30:29  <stevenknight> anyone else about ready to turn into a pumpkin too? 
19:30:36  <stevenknight> I should head for home... 
19:30:39  <garyo-home>   yes, time for me to go. 
19:30:41  <[GregoryNoel](GregoryNoel)>  Long past time for me 
19:30:48  <stevenknight> okay, many thanks guys 
19:30:52  <garyo-home>   Thanks a lot as usual, guys! 
19:30:54  <[GregoryNoel](GregoryNoel)>  two weeks? 
19:30:59  <garyo-home>   ok for me. 
19:31:00  <stevenknight> hopefully we'll see some increased activity coming up 
19:31:06  <stevenknight> two weeks is good for me 
19:31:07  <Jason_>       latter! 
19:31:12  <[GregoryNoel](GregoryNoel)>  Next meeting 17 Dec then?  1.2 should be out, as well as the first checkpoint toward 1.3.  Agreed? 
19:31:12  <[GregoryNoel](GregoryNoel)>  Next meeting 17 Dec then?  1.2 should be out, as well as the first checkpoint toward 1.3.  Agreed? 
19:31:12  <[GregoryNoel](GregoryNoel)>  Next meeting 17 Dec then?  1.2 should be out, as well as the first checkpoint toward 1.3.  Agreed? 
19:31:13  <stevenknight> should have 1.2 out by then... 
19:31:27  <[GregoryNoel](GregoryNoel)>  oops... 
19:31:26  <stevenknight> what I tell you three times is true... 
19:31:29  <garyo-home>   greg: yes, yes, yes. 
19:32:04  <stevenknight> for the snark *was* a boojum, you see... 
19:32:08  <garyo-home>   :-) 
19:32:17  <[GregoryNoel](GregoryNoel)>  snicker-snack 
19:32:17  <garyo-home>   ok, g'night all. 
19:32:22  *      Jason_ has quit ("Leaving") 
19:32:25  <[GregoryNoel](GregoryNoel)>  cul 
19:32:29  <stevenknight> later 
19:32:39  *      garyo-home has quit ("[ChatZilla](ChatZilla) 0.9.84 [Firefox 3.0.4/2008102920]") 
19:47:01  *      stevenknight has quit ("Leaving")