Allow ability to override slim defaults

[mikepm35]

(Note: I am packaging individually with zip: true, and am using a branch from @felipe-augusto to deal with the issue as described in #203.)

In order to meet lambda size limitations I am using slim: true, however I was not able to successfully run due to one my modules (Sagemaker SDK) not being able to find a dependency (in this case urllib3). What I found is that Sagemaker is using pkg_resources to pull the SDK version (below), and then pkg_resources throws an exception because it is not able to identify the distribution information since all of the dist-info files have been removed by slim.

SDK_VERSION = pkg_resources.require('sagemaker')[0].version

Setting slim: false is not an option for me since I need that additional trimming, but there is no easy workaround since custom slim options only append to the defaults. I could create specialized packages that I trim myself, or create a branch that removes that default option. I went with the latter, but I think it would be a handy feature to allow for overriding the defaults. If defining custom slim options, it might be easier to remove the defaults altogether, and if a user wants them they can add them back in manually or use a special default option.

[dschep]

Huh. Is the issue a file that's remove or a file that's striped? If it's the former, I think you should be able to use the slimPatterns option to specify a custom set of patterns to be deleted?

[johnf]

I have a similar issue with strip'd files breaking. e.g. opencv
gives the following when slim is enabled

Unable to import module 'handler': /tmp/sls-py-req/cv2/cv2.so: ELF load command address/offset not properly aligned

[dschep]

What OS are you on and are you using docker? @johnf

[bweigel]

I have found the same problem using sqlalchemy with the sqlalchemy_redshift plugin/dialect.
pkg_resources is used internally to find the plugins, which itself uses egg-info and dist-info to determine some information.

Huh. Is the issue a file that's remove or a file that's striped? If it's the former, I think you should be able to use the slimPatterns option to specify a custom set of patterns to be deleted?

Unfortunately no, because of the way slimPatterns is implemented it concatenates to the list instead of overwriting:

  let patterns = ['**/*.py[c|o]', '**/__pycache__*', '**/*.dist-info*'];
  if (options.slimPatterns) {
    patterns = patterns.concat(options.slimPatterns);
}

[mikepm35]

Maybe an solution would be to create a new customPatterns option that would remove all defaults to give complete control.

[dschep]

Tha would be reasonable @mikepm35

[bweigel]

I see the point, however I would argue one doesn't need an extra option (i.e. customPatterns) in addition to slimPatterns, because you wouldn't use them at the same time (i.e. you either want to overwrite or not).

I propose to use a flag for the overwrite option, which let's you disable (or rather enable overwriting) the default patterns. This way backwards compatibility is also kept. I have updated my PR accordingly.

Cheers.

[dschep]

Yup. no worries, i'm accepting your change @bweigel. @mikepm35 & others, see #276 for how this will be implemented.