Keep directory structure / folder hierarchy when copying with recursive glob

[mario-d-s]

When using the plugin like this:

copy({
    targets: [
        { src: 'assets/**/*', dest: 'dist/assets/' },
    ]
})

It would be nice to optionally copy the directory structure as-is.
My suggestion: support a flatten option for a target that defaults to true for keeping backwards compatibility.

[vladshcherbin]

@mario-d-s hey, won't this work for your case?

copy({
  targets: [
    { src: 'assets/**', dest: 'dist/assets' },
  ]
})

It will copy the contents of assets folder and keep the folder structure.

[mario-d-s]

@mario-d-s hey, won't this work for your case?

copy({
  targets: [
    { src: 'assets/**', dest: 'dist/assets' },
  ]
})

Hi @vladshcherbin thank you for your quick comment! However this does not work as expected.
It does both things, namely copy all directories and their contents into the same hierarchy, as well as take all files from within those directories and places them flattened at dist/assets. I hope my explanation is making sense.

Also the use case of filtering input is not supported like this:

copy({
  targets: [
    { src: 'assets/**/*.jpg', dest: 'dist/assets' },
  ]
})

This will place all .jpg files under dist/assets and not keep the directories they were in.

[vladshcherbin]

@mario-d-s hm, that's interesting. If I use this code:

// folder structure
// assets/js
// assets/js/inside-one.js
// assets/one.js
// assets/two.js

copy({
  targets: [
    { src: 'assets/**', dest: 'dist/assets' },
  ]
})

The output for me will be

// dist/assets/js
// dist/assets/js/inside-one.js
// dist/assets/one.js
// dist/assets/two.js

So for me it just copies all directories and their contents and keeps same hierarchy but doesn't flatten them. Interesting if it works differently for you.

The *.jpg example is a good one.

I'll think about this one, thank you the suggestion :)

[mario-d-s]

@vladshcherbin I'm willing to help you by debugging our case where it strangely goes wrong, and also open a pull request for implementing the filtering use case, when I get some time (might not be soon though, don't take this as a commitment 😄)

[vladshcherbin]

@mario-d-s yeah, it would be nice if you can run it with verbose option to see the output and to know your OS.

[vladshcherbin]

@mario-d-s actually, I was able to reproduce it too. So no need for any debug info, will be investigating the issue :)

[vladshcherbin]

@mario-d-s ok, so this was my misunderstanding how globstar (**) works alone.

For the first example (directory and keep structure) try this one instead:

copy({
  targets: [
    { src: 'assets/*', dest: 'dist/assets' },
  ]
})

Does it work correctly now?

[mario-d-s]

@vladshcherbin yes, using that syntax it does work, but that is not correct - it would be confusing to people who explicitly want to copy only first-level files as that is what such glob would normally do.

I do think it would be better to fix this so that ** is properly supported especially when combining it with a filter like **/*.jpg.

[vladshcherbin]

@mario-d-s that's the correct way how glob works thought. If you want to copy only files, you have different options:

• set extension you need: { src: 'assets/*.js', dest: 'dist/assets' }
• set multiple extensions you need: { src: 'assets/**/*.{js|jpg}', dest: 'dist/assets' }
• use onlyFiles option: { src: 'assets/*', dest: 'dist/assets', onlyFiles: true }

There is also an option to turn off globstar (**) so it works as wildcard (*):

{ src: 'assets/**', dest: 'dist/assets', globstar: false }

As per **/*.jpg glob - it takes all jpg files as expected and currently I can't think of a possible way to preserve their folder structure. Do you need this feature in your project or it's just an example?

[mario-d-s]

@vladshcherbin this is not how globs work in other node projects nor in bash where this form of globbing originated.

In the meantime I found a "competing" (for lack of a better word) plugin to yours that does it right:
https://github.com/paulmelnikow/rollup-plugin-cpy
it uses https://github.com/sindresorhus/cpy under the hood thus supporting the parents option which does what I'm asking.

The possible way is that you recreate whatever ** matches in the destination directory. If you have for example:

src
|-- assets
|   |-- one
|   |   |-- one.md
|   |   |-- some.jpg
|   |-- two
|   |   |-- two.md
|-- other
|   | -- some.md

And you have this target for your plugin:

{ src: 'src/assets/**/*.md', dest: 'dist/markdown' }

In dist you end up with:

dist
|-- markdown
|   |-- one
|   |   |-- one.md
|   |-- two
|   |   |-- two.md

If you put it like this:

{ src: 'src/**/*.md', dest: 'dist/markdown' }

You should get:

dist
|-- markdown
|   |-- assets
|   |   |-- one
|   |   |   |-- one.md
|   |   |-- two
|   |   |   |-- two.md
|   |-- other
|   |   |-- some.md

[vladshcherbin]

@mario-d-s when I wrote about how glob works I meant how globstar works alone, for example assets/** and not assets/**/*.md. It was my misunderstanding, it works correctly and this is why it copied both directories and files from this directories in your comment here.

A simple wildcard glob solution from this comment also works correctly and will copy first level files and folders. So how glob works in this package is the same as other node packages and bash.

I understand what you mean with src/**/*.md example and how you would like it to work. Unfortunately, this plugin uses a simple fs-extra copy function, which doesn't have an option to preserve parent directories.

So this feature can be added either by adding such parent resolution function or by using another copy function (for example cpy you've mentioned).

In your project, do you actually need to copy files with something like assets/**/*.md and preserve parent directory structure or a simple wildcard solution assets/* would be enough?

[mario-d-s]

Hi @vladshcherbin, we do indeed need this function.

[chrisdmacrae]

To weigh in on this issue, we're using this plugin to move non-js assets across with a typescript class library we are building.

It's not doing what we expect w/ the following config:

copy({
  src: [
    "src/**/*",
    [".js", ".jsx", ".ts", ".tsx"].map(e => "!src/**/*" + e);
    // becomes...
    // ["!src/**/*.js", "!src/**/*.jsx", "!src/**/*.ts", "!src/**/*.tsx"]
  ],
  dest: "dist/static"
})

With the follwing src structure:

- src/
-- index.ts
-- utils/
---- example.ts
---- example.html
---- example.css

We expect the following output:

dist/
- index.js
- static/
-- utils/
--- example.html
--- example.css

But we get:

dist/
- index.js
- static/
-- example.html
-- example.css
-- utils/
--- example.ts
--- example.html
--- example.css

[hrodriguez758]

@mario-d-s @vladshcherbin I did a few changes for preserving the parent directories, I can send the pull request with that if you can, what I did? when u get the matchedPath if you replace the base root (src for the destination base dest) you preserve the parent structure.

src -> ./src/**/*.html
dest -> dist
matchedPath -> ./src/components/mycomponent.html

replace the base src for dest we get:

./dist/components/mycomponent.html

I did a couple of tests and works fine

[hrodriguez758]

@vladshcherbin any updates?

[vladshcherbin]

Released in 3.2.0 🎉

Please test in your projects to see if it's working as expected.

[scottdickerson]

this fixed the problem for us, thanks for fixing

[dasa]

Thanks for this fix! I noticed though that the option is missing at https://github.com/vladshcherbin/rollup-plugin-copy/blob/master/index.d.ts#L22

[vladshcherbin]

@dasa nice catch, thanks. Fixed in 3.2.1

[rottmann]

Glob is still/again broken in v3.3.0. Please revisit.

Souce files:

|- assets
   |- file1
   |- dir1
      |- file2
   |- dir2
      |- file3

Settings:
src assets/**/*
dest some/dir

Output is

|- some
   |- dir
      |- file1
      |- file2
      |- file3
      |- dir1
         |- file2
      |- dir2
         |- file3

Currently it copies the files into the dest-dir too.

[nicolasr75]

Same problem here! Exactly the same configuration as @rottmann

[Ballpin]

While waiting you can write it yourself. I went with this approach.

targets: [
                {
                    src: 'src/main-process/api/**/*.{js,mjs}',
                    dest: ".vite/build",
                    rename (name, extension, fullPath) {
                        fullPath = fullPath.replace("src/", "");
                        fullPath = fullPath.replace(`/${name}.${extension}`, "")
                        
                        // Rename .mjs to .js in order to follow rules of build system
                        if (extension === "mjs") {
                            return `${fullPath}/${name}.js`
                        }
                        // Preserve the directory structure when copyng the file.
                        return `${fullPath}/${name}.${extension}`
                    }
                },
            ]

[EddieDover]

Thank you @Ballpin. This satisfies the needs of my current project, but I'm leaving a note for future devs that switching to https://github.com/paulmelnikow/rollup-plugin-cpy may be neccessary.