Improve support of make autocompletion for huge Makefiles

[philipp-classen]

I noticed that if Makefiles become very large, bash completions becomes
very slow. To some degree that cannot be easily avoided, for instance, just the
call to make can take several seconds.

This patch is an attempt to improve the support of huge Makefiles as they are
generated by Automake. I do not expect that you merge it back as it is but
I thought it might be useful for others to see and play around with.
Maybe someone finds a less hacky solution to the underlying problem.

Anyway, here is the idea behind my changes:

If the files get too large (> 50000 lines of code in my example),
the calls to make and sed become very slow (make takes about 4 seconds
while sed takes up to 11 seconds, again in my example).

Additionally, there is a lot of noise in the recommendations.
I get about 2800 results, mostly uninteresting. Instead I am mainly
interested in the binaries or the test commands.

As a hack, I added a simple cache for expensive calls to make and sed, respectively.
I also filter away the search results referencing source files or intermediate files
like ".o" and ".obj" files, thus reducing the number of recommendations to 160.

In sum, it is a "work-for-me" solution, but what I rather want is to encourage a
discussion of a more thoughtful approach:

1. I lack understanding of the sed expression. As it is the biggest bottleneck, it would be interesting to know if it could be improved in terms of performance.
2. So far, I have not seen that bash-completion uses caches. Caches bring extra complexity, of course, so I am not sure whether it is a good idea to go in that direction. A compromise could be to make it an opt-in feature. For instance, it could be disabled unless an environment variable is set.
3. Would you agree that recommended completions for source files and ".o" and ".obj" files should be filtered out? Or are there use cases where you actually want to see them?

[philipp-classen]

As I just read in the "Contributing" guide, writing to the file system is to be avoided. That basically answers my second question. However, even a perfectly fast sed would than not be enough, as make itself will make the completion almost unusably slow (response times of multiple seconds).

As far as I see it, the options for making it responsive without using the filesystem, are fairly limited.

[scop]

I've forwarded this to Tristan Wibberley who has been mostly working on the make completion, let's see what he things of this.

[philipp-classen]

I have created two follow-up issues (#65 and #66). One for the aspect of the relevance of results, and one for the performance problem.

I agree with Tristan that we should not merge the patch. Here are the arguments against it:

Make is used for many more purposes than ".c/cpp -> executable
binaries" so hardcoding that filter into make completion will add new
problems to the system over time more than taking them away so I'd
like to strongly recommend against it, except maybe as a "GNU automake
filter" for backward compatibility only.

Caching is very difficult to implement correctly in and of itself, one
must make a list of the files that impact the completion list (eg,
multiple makefiles, non-makefiles that list data that GNU Make uses
when directed to do so by makefiles, etc).

I would recommend to close the pull request and continue the discussion on the individual issues. If I can help in providing better examples or testing some changes, please feel free to ask.

[scop]

Closing as recommended.