Skip to content

Templify Here Slow #23

Closed
ronnieoverby opened this Issue Oct 26, 2011 · 34 comments

4 participants

@ronnieoverby

I templified a project. When I "Templify Here"'d the process was very slow and after 20 minutes, I just killed it.

@alecwhittington

How large was the project you Templified? With the S# Templify package, it should take no longer than 5 mins. Can you provide more details that "the process was very slow"?

@ronnieoverby
@HowardvanRooijen
endjin member

I think I've located a way to do much faster compression and extraction. I'll spike it and see if it makes any difference. Sorry for the delay - GitHub isn't sending me notifications of issues on this project.

@ronnieoverby

Any update on this? I just fresh installed templify and templified the included sharp arch. project. It took 6 minutes. 6 MINUTES on a lightning fast machine is Windows 8 Pro RTM, 8 Cores @ 4.15ghz, 32gb of DDR3, with an extremely fast SSD.

What does this thing do that takes so long?

@HowardvanRooijen
endjin member

I believe it's the 7Zip api that takes the time, or should I say the call I have to make. VNext will more than likely be nuget compatible - so you'll see similar perf.

@ondesertverge

This looks like such a great tool 👍 But, if it takes 2 hrs. to setup a package it just is unsuable like this. Is there a simple solution to this issue?

@HowardvanRooijen
endjin member

Solution is to replace the 7Zip implementation with a standard Zip implementation which should offer a faster package / unpackage speed.

@ondesertverge
@HowardvanRooijen
endjin member

Hi,

as mentioned above - I think the problem is in the 7Zip API we use - to get access to specific files (to tokenise, or move into a location specified in the manifest file) - it seems to be quite slow. It's looking more like V2 will be nuget compatible - and will drop support for the manifest file / 7Zip so things should be super speedy. Only problem is my lack of time to make it happen :(

@ondesertverge
@HowardvanRooijen
endjin member

Not off the top of my head - the whole extraction piece needs to be rethought - I'd love to have time to do it, but I have 0 hobby coding time at the moment - I feel bad because it could really do with a couple of days of my time to get v2 spiked and working - then others could take it and run with it. A few people have asked if they can contribute - maybe I'll pass this quick fix by them?

@ondesertverge
@HowardvanRooijen
endjin member

That'll be great. Much appreciated.

@HowardvanRooijen
endjin member

SevenZipProcessor & SevenZipBuilder are the two bits you need to look at and replace with a ZipProcessor and ZipBuilder.

@HowardvanRooijen
endjin member

Hopefully the rest of the architecture is sufficiently abstracted that it'll "Just Work"(TM)

@ronnieoverby

btw... why compress at all? Would archiving in storage mode (no compression) solve this problem? I, for one, have plenty of hard drive space.

@HowardvanRooijen
endjin member

It was more a distribution problem. The test case was always the Sharp Architecture Package - that didn't seem to take an excessively long time to process.

I'm open to all solutions - it's just time to implement them that's the problem!

@ondesertverge
@ondesertverge
@HowardvanRooijen
endjin member

In v2 I'd really like all these internal bits to be more pluggable and orderable - so you can insert your own plugin into what ever bit of the processing pipeline.

As for storage - you still need to add them to some kind of container as the delivery package - it just depends whether you need / want to compress it.

Thanks - I look at the code and grimace a little - it's the first and only time I've used MEF and have to say when V2 happens - MEF wont be part of it! The MVVM bits are nice, as it the TPL.

I refactored the code base severely - because in the 1st spike every time I didn't follow the SRP - it caused a bug / issue somewhere - so pretty much everything is SRP in there now - should make it easier to replace bits!

@HowardvanRooijen
endjin member

I'm not sure this is in the notes anywhere - but if you set the start up project to be Endjin.Templify.Client then open the properties pane and set the command line arguments to be (adjusting for your local path):

templify-vs-options

Makes debugging a lot easier.

@ondesertverge
@HowardvanRooijen
endjin member

Only think I can think of is passing it down the stack. I was thinking about having a more generic processing context available - but you know what the story is there :(

@ondesertverge
@ondesertverge
@HowardvanRooijen
endjin member

Nice! Good work!

@HowardvanRooijen
endjin member

Have you kept the project as a VS2010 project (and there for can build the installer) or did you upgrade to VS2012? If you can produce an installer - I'll upload it for distribution. Also does it clean up after itself?

@ondesertverge
@HowardvanRooijen
endjin member

No - don't worry - I've only got 2012 too! Was just hoping to save myself some work - but that's OK - I'll migrate it to an AdvancedInstaller template - that wont take two ticks.
1. ok - makes sense (I'll try and control my OCD)
3. If you've forked it - just push your changes and issue me a pull request.

@ondesertverge
@ondesertverge
@HowardvanRooijen
endjin member

Excellent. If I try and sprain my brain to remember - I think there may have been a reason for the 2 step unpacking process - some edge case that I caught during testing - but I can't remember what it was! I'll see if I can create the AdvancedInstaller on the train into work this morning - then we might have something we can push out.
And there was much rejoicing.

@HowardvanRooijen
endjin member
@HowardvanRooijen
endjin member

I've just pushed up a new release with the performance fix, a fix for file encoding error and a new installer https://dl.dropboxusercontent.com/u/5892928/Templify/Templify-Installer.exe

@HowardvanRooijen HowardvanRooijen referenced this issue Sep 21, 2013
Closed

Error #35

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.