cargo package excludes hidden files that are explicitly included

[teythoon]

Problem
I'm packaging the source of a C library to "vendor" it. The build system of said library uses hidden files (UNIX style, beginning with a dot). These files are excluded by cargo package even if I explicitly include them. The missing file cause problems with the build system.

The crate in question is nettle-src. Resulting build problems: https://gitlab.com/sequoia-pgp/sequoia/issues/328

Furthermore, the cargo manifest documentation (https://doc.rust-lang.org/cargo/reference/manifest.html) does not mention any special handling of hidden files.

Steps
This problem can be reproduced using a minimal example:

% cargo init da-hidden-filez
     Created binary (application) package
% cd da-hidden-filez
% touch .hidden
% emacs Cargo.toml
[...]
% cat Cargo.toml
[package]
name = "da-hidden-filez"
version = "0.1.0"
authors = ["Justus Winter <teythoon@avior.uberspace.de>"]
edition = "2018"
include = ["/Cargo.toml", "/src/main.rs", "/.hidden"]

[dependencies]
% cat .gitignore
/target
**/*.rs.bk
% cargo package --allow-dirty
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See <http://doc.crates.io/manifest.html#package-metadata> for more info.
   Packaging da-hidden-filez v0.1.0 (/tmp/da-hidden-filez)
   Verifying da-hidden-filez v0.1.0 (/tmp/da-hidden-filez)
   Compiling da-hidden-filez v0.1.0 (/tmp/da-hidden-filez/target/package/da-hidden-filez-0.1.0)
    Finished dev [unoptimized + debuginfo] target(s) in 0.34s
% tar tf target/package/da-hidden-filez-0.1.0.crate
da-hidden-filez-0.1.0/Cargo.toml.orig
da-hidden-filez-0.1.0/Cargo.toml
da-hidden-filez-0.1.0/src/main.rs

Possible Solution(s)
Please allow me to package hidden files, if necessary by explicitly including them. Also, the documentation needs to reflect the special handling of hidden files.

Notes

Output of cargo version:
cargo 1.34.0

I'm on Debian Buster, the Rust toolchain is as packaged by Debian. However, the same applies to cargo 1.36.0 (c4fcfb725 2019-05-15) I got via rustup.

[teythoon]

FTR, this is my rather ugly workaround: https://gitlab.com/sequoia-pgp/nettle-src/commit/8541b16d739ca01af4c8f065ce18856370a302cb

[stefanhoelzl]

I started looking into this issue and wrote a test case to verify it.

I also was able to write a fix for it but this needs some discussion I think.

my fix will also change the existing behavior (but looks like it does not break any existing tests). Currently all dotfiles are getting ignored during packaging, with my change the dotfiles aren't ignored anymore. So my questions is what is the desired behavior?

I see two ways for the implementation

1. change the behavior and include dotfiles by default
2. keep the behavior and exclude dotfiles by default, but allow it to include in Cargo.toml

there is a existing comment regarding this, but it is unclear to me

Skip dotfile directories.

Is it intention to only skip directories starting with a dot, or also dotfiles?

[ehuss]

This has been fixed by #7680.