destination_files behaviour

[mattem]

I am attempting to use copybara to migrate many standalone repos into a monorepo, and use copybara to keep them in step while we are in the process of migration. However I am hitting an issue with the behaviour of destination_files, or perhaps my understanding of it.

The following in the config used

core.workflow(
    name = "default",
    origin = git.origin(
        url = origin_url,
        ref = "master",
    ),
    destination = git.destination(
        url = dest_url,
        fetch = "master",
        push = "copybara/microtime_import",
    ),
    ask_for_confirmation = True,
    destination_files = glob(["libs/microtime/**"]),
    origin_files = glob(["**"]),
    authoring = authoring.pass_thru("Copybara <cc@internal>"),
    transformations = [
	    core.move("", "libs/microtime"),
    ],
)

When this runs, it removes all other files in the destination repo, unless they are matched by an exclude in destination_files, which seems to go against the line in the docs:

A glob relative to the root of the destination repository that matches files that are part of the migration. Files NOT matching this glob will never be removed, even if the file does not exist in the source.

How can I get copybara to import from the the origin repo and only affect the destination folder?
Using copybara built from ea42789

[mattem]

Not sure what I changed, but have now got it working! Awesome tool!

[mikelalcon]

Hi Matt,

Glad to hear that it is working. Your first snippet should work. If this happen again and you have a repo case please tell us. We'll be interested to see what is going on.

Cheers,
Mikel.

[Korolen]

My experience was behavior that depended on whether destination_files was set to either

1. the subfolder containing code for the repo, as you have configured in your OP post, in which case the new folder libs/microtime/ would be created and all the commits from the associated repo copied in, interlaced with whatever commits that had already imported into that branch in the monorepo.

or,

2. just glob(["**"]), in which case all previously present files in the monorepo would be eaten. Even though, oddly enough the commits - and even their associated tree modifications - would still show up in the commit log, same as (1). But when you checked out a commit the files would not actually be there.

(both of these situations using all other settings same as OP, except for ask_for_confirmation not being set, copybara --init-history flag being set, and pushing to branch called master.)

So, the takeaway is to set destination_files to be a subfolder used exclusively for syncing with that particular manyrepo, since anything else contained in it will be eaten. I suppose this allows syncing in the other direction to be totally unambiguous?