x2cpg: extracted file exclusion by regex and file path

[max-leuthaeuser]

This lives in x2cpg now and may be used by all frontends. jssrc2cpg and c2cpg already made use of it.

To all frontend maintainers:
If your frontend should support file exclusion please use def determine(inputPath: String, sourceFileExtensions: Set[String], config: X2CpgConfig[_]): List[String] from x2cpg.SourceFiles to retrieve a file list for your extension(s) in the input path. X2CpgConfig provides three fields for the filter process:

• ignoredFilesRegex: a regex that the user may supply via --exclude-regex. Filters out files or folders (path relative to the input is matched against the regex)
• ignoredFiles: a list of files that the user may supply via --exclude. Filters out files or folders (paths relative to the input dir as well as absolute paths)
• defaultIgnoredFilesRegex: a list of regex that the frontend maintainer may provide. Filters out files or folders (path relative to the input is matched against the regex) by default (e.g., unwanted test folders).

--exclude-regex and --exclude are now available for all frontends making use of X2Cpg.parseCommandLine.

[johannescoetzee]

the absolute file path is matched against the regex

Why is the absolute used for the regex methods? This could lead to surprising (at least to me) results, for example when using a directory structure like /tmp/test/<test_repo>/.., scanning <test_repo> and adding an exclude for the test directory (which would lead to an empty CPG in this case.

Using the relative (to the project root) path when filtering would avoid this issue, while still giving users control over excluding files outside of the project dir by not providing those as input files.

[max-leuthaeuser]

Yes, you are right. I will change this.

[johannescoetzee]

Looks good overall! This is very useful for source-based frontends in general :-)