Allow more convenient use of ghc profiling options -auto and -auto-all #193

Closed
bos opened this Issue May 24, 2012 · 17 comments

Comments

Projects
None yet
6 participants
@bos
Contributor

bos commented May 24, 2012

(Imported from Trac #200, reported by guest on 2007-12-18)

Add Setup.(l)hs options that: 1) tells ghc to make a profiling library; 2) tells ghc to make a profiling executable 3) pass on the (a) -auto or (b) -auto-all option 4) Let you do (1)-(2)-(3a) or (1)-(2)-(3b) with a single option, instead of three.

@ghost ghost assigned kmcallister May 24, 2012

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by @dcoutts on 2007-12-18)

Replying to guest:

Add Setup.(l)hs options that: 1) tells ghc to make a profiling library;

This is covered by the existing configure --enable-library-profiling

2) tells ghc to make a profiling executable

This is covered by the existing configure --enable-executable-profiling

3) pass on the (a) -auto or (b) -auto-all option

This is currently missing.

4) Let you do (1)-(2)-(3a) or (1)-(2)-(3b) with a single option, instead of three.

This is covered by cabal-install since the install command does configure, build and install and it takes the above two options.

So what we're currently missing is any way of handling the -auto or -auto-all options. I'm changing the summary to reflect that.

See also the cabal-devel thread starting with this message: http://www.haskell.org/pipermail/cabal-devel/2007-January/000379.html

Does anyone have any good suggestion about how to handle -auto or -auto-all ? We cannot make them default for profiling (see above thread) and they are not features that make sense for other haskell implementations.

Currently one can put them in the .cabal file, but it's not really appropriate to do that since they're not static options.

Contributor

bos commented May 24, 2012

(Imported comment by @dcoutts on 2007-12-18)

Replying to guest:

Add Setup.(l)hs options that: 1) tells ghc to make a profiling library;

This is covered by the existing configure --enable-library-profiling

2) tells ghc to make a profiling executable

This is covered by the existing configure --enable-executable-profiling

3) pass on the (a) -auto or (b) -auto-all option

This is currently missing.

4) Let you do (1)-(2)-(3a) or (1)-(2)-(3b) with a single option, instead of three.

This is covered by cabal-install since the install command does configure, build and install and it takes the above two options.

So what we're currently missing is any way of handling the -auto or -auto-all options. I'm changing the summary to reflect that.

See also the cabal-devel thread starting with this message: http://www.haskell.org/pipermail/cabal-devel/2007-January/000379.html

Does anyone have any good suggestion about how to handle -auto or -auto-all ? We cannot make them default for profiling (see above thread) and they are not features that make sense for other haskell implementations.

Currently one can put them in the .cabal file, but it's not really appropriate to do that since they're not static options.

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by guest on 2007-12-18)

I think the best way is to pretend -auto and -caf-all don't exist. They are relatively rarely used, and when they are they can be passed with --ghc-option=, right?

For whether or not to use -auto-all, the portable distinction is whether we are compiling in order to support profiling something else, or to profile this code in particular. How about --enable-library-profiling for -prof -auto-all, and --enable-library-profiling-support for -prof? Better (shorter!) names welcomed; bonus marks if you can make --enable-library-profiling keep its current meaning without sounding too forced.

For other implementations both flags might have the same effect.

Ian Lynagh

Contributor

bos commented May 24, 2012

(Imported comment by guest on 2007-12-18)

I think the best way is to pretend -auto and -caf-all don't exist. They are relatively rarely used, and when they are they can be passed with --ghc-option=, right?

For whether or not to use -auto-all, the portable distinction is whether we are compiling in order to support profiling something else, or to profile this code in particular. How about --enable-library-profiling for -prof -auto-all, and --enable-library-profiling-support for -prof? Better (shorter!) names welcomed; bonus marks if you can make --enable-library-profiling keep its current meaning without sounding too forced.

For other implementations both flags might have the same effect.

Ian Lynagh

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by @dcoutts on 2007-12-19)

Replying to guest:

I think the best way is to pretend -auto and -caf-all don't exist. They are relatively rarely used, and when they are they can be passed with --ghc-option=, right?

Possibly, I've not tried it but I fear that we call ghc for other purposes too and those extra flags might mess things up.

For whether or not to use -auto-all, the portable distinction is whether we are compiling in order to support profiling something else, or to profile this code in particular.

Yes. That is a good portable distinction.

How about --enable-library-profiling for -prof -auto-all, and --enable-library-profiling-support for -prof? Better (shorter!) names welcomed; bonus marks if you can make --enable-library-profiling keep its current meaning without sounding too forced.

Yes, if we can find a decent name and not change the meaning of the existing flag, which is already the right default.

For other implementations both flags might have the same effect.

Right.

Contributor

bos commented May 24, 2012

(Imported comment by @dcoutts on 2007-12-19)

Replying to guest:

I think the best way is to pretend -auto and -caf-all don't exist. They are relatively rarely used, and when they are they can be passed with --ghc-option=, right?

Possibly, I've not tried it but I fear that we call ghc for other purposes too and those extra flags might mess things up.

For whether or not to use -auto-all, the portable distinction is whether we are compiling in order to support profiling something else, or to profile this code in particular.

Yes. That is a good portable distinction.

How about --enable-library-profiling for -prof -auto-all, and --enable-library-profiling-support for -prof? Better (shorter!) names welcomed; bonus marks if you can make --enable-library-profiling keep its current meaning without sounding too forced.

Yes, if we can find a decent name and not change the meaning of the existing flag, which is already the right default.

For other implementations both flags might have the same effect.

Right.

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by @dcoutts on 2007-12-19)

This can't be that hard. We just need to figure out what behaviour we want.

Contributor

bos commented May 24, 2012

(Imported comment by @dcoutts on 2007-12-19)

This can't be that hard. We just need to figure out what behaviour we want.

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by @bos on 2008-06-19)

I'm all for just a single knob to control -auto-all. I've never heard of someone caring about -auto or -caf-all.

Contributor

bos commented May 24, 2012

(Imported comment by @bos on 2008-06-19)

I'm all for just a single knob to control -auto-all. I've never heard of someone caring about -auto or -caf-all.

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by @dcoutts on 2008-06-19)

So we want something like --enable/disable-profiling-this-package, though with a less silly name. It would modify the meaning of
--enable-library-profiling and --enable-executable-profiling to add -auto-all. For libs the default would be disabled and for exes the default would be enabled.

Hmm, if our defaults our different then we'd need different enable/disable flags for libs vs exes.

I agree with bos and Ian that we can basically ignore -auto and -caf-all. On the other hand I think that if we use -auto-all by default for executables then we do need a way to let people not do that for the rare cases (though we don't need to support those cases specifically). The easy way to do that is just to have a corresponding disable flag.

Contributor

bos commented May 24, 2012

(Imported comment by @dcoutts on 2008-06-19)

So we want something like --enable/disable-profiling-this-package, though with a less silly name. It would modify the meaning of
--enable-library-profiling and --enable-executable-profiling to add -auto-all. For libs the default would be disabled and for exes the default would be enabled.

Hmm, if our defaults our different then we'd need different enable/disable flags for libs vs exes.

I agree with bos and Ian that we can basically ignore -auto and -caf-all. On the other hand I think that if we use -auto-all by default for executables then we do need a way to let people not do that for the rare cases (though we don't need to support those cases specifically). The easy way to do that is just to have a corresponding disable flag.

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by dolio on 2008-06-20)

It may be that having annotations on all the libraries one uses is too much information. However, I'd argue that annotations on library calls isn't useful merely for profiling said libraries.

So, my program is slow, and it spends most of the time in function f.

f x = ... includes calls to some libraries ...

What is slow about f? Is it doing some of its own stuff that takes time/memory, or if we unfolded one level, would we find that 90% of f's work is done in the library calls it makes? Now, at some level, I suppose you can get carried away with that (f doesn't do any work, it's all GHC primops :)), but I think that a bit of such information can be useful, to tell me that I need to swap in a different data structure, or contact the library's maintainer and see if anything can be done to speed it up. Maybe you can do some of that with manual SCC annotations, but I've had trouble getting those to work well for me.

At least, it seems to me that there are situations where libraries aren't simply a profiling black hole would be useful when not profiling that particular library (even if only to tell you that that library needs to be profiled :)). See the somewhat recent BLAS optimization question on haskell-cafe for a potential example (is it his use of lists and non-fused operations? Is BLAS itself slow? etc.).

It'd be nice if there were a way to turn -auto-all on and off for libraries when compiling an application that uses those libraries. Of course, I don't suspect that's an option at the moment (I suppose you could build two sets of profiling files for each library, and have some way of selecting them, but I imagine that'd be hairy).

Anyhow, just my 2 cents on the issue.

Contributor

bos commented May 24, 2012

(Imported comment by dolio on 2008-06-20)

It may be that having annotations on all the libraries one uses is too much information. However, I'd argue that annotations on library calls isn't useful merely for profiling said libraries.

So, my program is slow, and it spends most of the time in function f.

f x = ... includes calls to some libraries ...

What is slow about f? Is it doing some of its own stuff that takes time/memory, or if we unfolded one level, would we find that 90% of f's work is done in the library calls it makes? Now, at some level, I suppose you can get carried away with that (f doesn't do any work, it's all GHC primops :)), but I think that a bit of such information can be useful, to tell me that I need to swap in a different data structure, or contact the library's maintainer and see if anything can be done to speed it up. Maybe you can do some of that with manual SCC annotations, but I've had trouble getting those to work well for me.

At least, it seems to me that there are situations where libraries aren't simply a profiling black hole would be useful when not profiling that particular library (even if only to tell you that that library needs to be profiled :)). See the somewhat recent BLAS optimization question on haskell-cafe for a potential example (is it his use of lists and non-fused operations? Is BLAS itself slow? etc.).

It'd be nice if there were a way to turn -auto-all on and off for libraries when compiling an application that uses those libraries. Of course, I don't suspect that's an option at the moment (I suppose you could build two sets of profiling files for each library, and have some way of selecting them, but I imagine that'd be hairy).

Anyhow, just my 2 cents on the issue.

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by @dcoutts on 2008-07-25)

Come on, this is easy to implement. All we need is to agree on the user interface! What command line options should we use?

To be concrete, the proposal is:

For libraries we have:

cabal configure --enable-library-profiling
Keeps its current meaning which is to compile this package to allow dependent packages to be profiled, but not with the intention of profiling this package. For ghc it implies -prof.
cabal configure --enable-library-profiling-this-package-yes-this-one!!!1
This enables profiling but with the intention of profiling this package. For ghc it implies -prof -auto-all. This is where we obviously want a sensible suggestion for the flag name.

For executables we have:

cabal configure --enable-executable-profiling
Which currently, for ghc, builds the exe just with basic profiling. Unlike with libraries, executable have no deps, so generally the only reason to build an exe with profiling support is to profile that exe. The suggestion is that the default be changed to use -auto-all because that is the most common method people use.

However some advanced uses might need to use -prof on it's own with manually annotated SCCs, or with flags -auto or -caf-all.

cabal configure --enable-executable-profiling-but-without-auto-all!!1
Similarly we need a sensible name for this. People would use this in combination with --ghc-option=-caf-all if necessary.

Now it looks like we need different flags to turn on/off the use of -auto-all for libs vs exes. The default values are different for exes vs libs, so having two flags makes things simpler. We could possibly use one flag to toggle both settings away from their defaults. It would mean that for a package like darcs with both a lib an an exe, that we could not build the lib with -auto-all but the exe without.

Having said all that, I'm not sure we can get this into Cabal-1.6.0.2 because it changes the API by changing an exposed type (the type of command line settings which is passed to the user hooks). So perhaps we have to punt for 1.8. That's probably ok as nobody seems to be shouting about this one.

Contributor

bos commented May 24, 2012

(Imported comment by @dcoutts on 2008-07-25)

Come on, this is easy to implement. All we need is to agree on the user interface! What command line options should we use?

To be concrete, the proposal is:

For libraries we have:

cabal configure --enable-library-profiling
Keeps its current meaning which is to compile this package to allow dependent packages to be profiled, but not with the intention of profiling this package. For ghc it implies -prof.
cabal configure --enable-library-profiling-this-package-yes-this-one!!!1
This enables profiling but with the intention of profiling this package. For ghc it implies -prof -auto-all. This is where we obviously want a sensible suggestion for the flag name.

For executables we have:

cabal configure --enable-executable-profiling
Which currently, for ghc, builds the exe just with basic profiling. Unlike with libraries, executable have no deps, so generally the only reason to build an exe with profiling support is to profile that exe. The suggestion is that the default be changed to use -auto-all because that is the most common method people use.

However some advanced uses might need to use -prof on it's own with manually annotated SCCs, or with flags -auto or -caf-all.

cabal configure --enable-executable-profiling-but-without-auto-all!!1
Similarly we need a sensible name for this. People would use this in combination with --ghc-option=-caf-all if necessary.

Now it looks like we need different flags to turn on/off the use of -auto-all for libs vs exes. The default values are different for exes vs libs, so having two flags makes things simpler. We could possibly use one flag to toggle both settings away from their defaults. It would mean that for a package like darcs with both a lib an an exe, that we could not build the lib with -auto-all but the exe without.

Having said all that, I'm not sure we can get this into Cabal-1.6.0.2 because it changes the API by changing an exposed type (the type of command line settings which is passed to the user hooks). So perhaps we have to punt for 1.8. That's probably ok as nobody seems to be shouting about this one.

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by ganesh on 2009-01-22)

I suggest cabal configure --profile-this-library to add -auto-all to a library's profiling options, and cabal configure --custom-profiling to remove it from an executable's profiling options.

If the GHC ways system ever supports it, I would also like to be able to build a library both -prof and {{{-prof -auto-all}} for future use. This would then require some interface to select which profiling versions of libraries to use downstream.

Contributor

bos commented May 24, 2012

(Imported comment by ganesh on 2009-01-22)

I suggest cabal configure --profile-this-library to add -auto-all to a library's profiling options, and cabal configure --custom-profiling to remove it from an executable's profiling options.

If the GHC ways system ever supports it, I would also like to be able to build a library both -prof and {{{-prof -auto-all}} for future use. This would then require some interface to select which profiling versions of libraries to use downstream.

@bos

This comment has been minimized.

Show comment
Hide comment
@bos

bos May 24, 2012

Contributor

(Imported comment by @kmcallister on 2009-01-25)

Patch is attached.

Contributor

bos commented May 24, 2012

(Imported comment by @kmcallister on 2009-01-25)

Patch is attached.

@sseefried

This comment has been minimized.

Show comment
Hide comment
@sseefried

sseefried Nov 17, 2014

Has there been any further work on this issue? Do we still think it's a good idea? I think it would be useful.

Has there been any further work on this issue? Do we still think it's a good idea? I think it would be useful.

@iustin

This comment has been minimized.

Show comment
Hide comment
@iustin

iustin Apr 24, 2015

Contributor

I also happened to get confused by this when migrating a project from Makefile/ghc --make to pure cabal. What's the status? Is the still at the "we need to decide on the command line arguments"?

Contributor

iustin commented Apr 24, 2015

I also happened to get confused by this when migrating a project from Makefile/ghc --make to pure cabal. What's the status? Is the still at the "we need to decide on the command line arguments"?

@ttuegel

This comment has been minimized.

Show comment
Hide comment
@ttuegel

ttuegel Apr 24, 2015

Member

As of Cabal-1.22, the command line options to Cabal have changed. Library profiling is enabled through --enable-library-profiling. Executable profiling is enabled by --enable-profiling and it implies --enable-library-profiling. Each of these implies the GHC option -prof during the appropriate phase, but no other options. GHC now has a wider range of profiling options. I think it would make the most sense to apply -fprof-auto; it doesn't generate too many SCCs like -fprof-calls, but it also generates enough SCCs that stack traces are useful. No one is working on this at the moment.

Member

ttuegel commented Apr 24, 2015

As of Cabal-1.22, the command line options to Cabal have changed. Library profiling is enabled through --enable-library-profiling. Executable profiling is enabled by --enable-profiling and it implies --enable-library-profiling. Each of these implies the GHC option -prof during the appropriate phase, but no other options. GHC now has a wider range of profiling options. I think it would make the most sense to apply -fprof-auto; it doesn't generate too many SCCs like -fprof-calls, but it also generates enough SCCs that stack traces are useful. No one is working on this at the moment.

@ttuegel ttuegel added the meta: easy label Apr 24, 2015

@ttuegel ttuegel modified the milestones: Cabal-1.24, _|_ Apr 25, 2015

@ttuegel ttuegel self-assigned this Apr 25, 2015

@ttuegel

This comment has been minimized.

Show comment
Hide comment
@ttuegel

ttuegel Apr 25, 2015

Member

As documented on this haskell-cafe thread, -fprof-auto can have very high costs, giving misleading profiles. On the other hand, -fprof-auto-exported only generates SCCs for exported bindings, which is low cost and more generally useful. I recommend that we make that the default for libraries. For executables, which generally do not have exported bindings, I recommend we use -fprof-auto-top by default. It generally has a higher cost that -fprof-auto-exported, but one only builds executables with profiling if one really wants the profiling information.

Member

ttuegel commented Apr 25, 2015

As documented on this haskell-cafe thread, -fprof-auto can have very high costs, giving misleading profiles. On the other hand, -fprof-auto-exported only generates SCCs for exported bindings, which is low cost and more generally useful. I recommend that we make that the default for libraries. For executables, which generally do not have exported bindings, I recommend we use -fprof-auto-top by default. It generally has a higher cost that -fprof-auto-exported, but one only builds executables with profiling if one really wants the profiling information.

@iustin

This comment has been minimized.

Show comment
Hide comment
@iustin

iustin Apr 27, 2015

Contributor

Thanks for the information, I was not aware of the recent GHC changes, and neither of the fact that auto-all is that expensive. Looking forward to have this merged in!

Contributor

iustin commented Apr 27, 2015

Thanks for the information, I was not aware of the recent GHC changes, and neither of the fact that auto-all is that expensive. Looking forward to have this merged in!

@23Skidoo 23Skidoo modified the milestones: Cabal 1.24, Cabal 1.26 Feb 21, 2016

@ttuegel

This comment has been minimized.

Show comment
Hide comment
@ttuegel

ttuegel Mar 10, 2016

Member

I think this issue is half-solved in its current state. We have reasonable default options. I would like to add an optional argument to --enable-profiling to allow the user to select from the available profiling options, e.g. --enable-profiling=auto to enable -fprof-auto, etc.

Member

ttuegel commented Mar 10, 2016

I think this issue is half-solved in its current state. We have reasonable default options. I would like to add an optional argument to --enable-profiling to allow the user to select from the available profiling options, e.g. --enable-profiling=auto to enable -fprof-auto, etc.

@ttuegel ttuegel modified the milestones: Cabal 1.24, Cabal 1.26 Mar 10, 2016

@ttuegel

This comment has been minimized.

Show comment
Hide comment
@ttuegel

ttuegel Mar 10, 2016

Member

Oh, my mistake, it looks like this was already done.

Member

ttuegel commented Mar 10, 2016

Oh, my mistake, it looks like this was already done.

@ttuegel ttuegel closed this Mar 10, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment