Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MergeStrategy seems to be completely broken (mergeWith failure) #11337

Closed
william-lohan opened this issue Jun 21, 2018 · 24 comments · Fixed by #19878
Closed

MergeStrategy seems to be completely broken (mergeWith failure) #11337

william-lohan opened this issue Jun 21, 2018 · 24 comments · Fixed by #19878
Labels
area: devkit/schematics freq1: low Only reported by a handful of users who observe it rarely severity3: broken type: bug/fix
Milestone

Comments

@william-lohan
Copy link

william-lohan commented Jun 21, 2018

Re-post of angular/devkit#745

Bug Report or Feature Request (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request

Area

- [ ] devkit
- [x] schematics

Versions

Node v8.11.1
NPM 5.8.0
Mac OS High Sierra

Repro steps

Use the following schematic

import { Rule, url, apply, mergeWith, chain, MergeStrategy } from '@angular-devkit/schematics';

export function index(): Rule {
  return chain([
    simpleMerge(),
  ]);
}

function simpleMerge(): Rule {

  const source = url('./files');

  return mergeWith(source, MergeStrategy.Overwrite);
}

The log given by the failure

schematics .:boilerplate

ERROR! /src/app/app.component.html already exists.
ERROR! /src/app/app.component.spec.ts already exists.
ERROR! /src/app/app.component.ts already exists.
ERROR! /src/app/app.module.ts already exists.
ERROR! /src/assets/.gitkeep already exists.
ERROR! /src/environments/environment.prod.ts already exists.
ERROR! /src/environments/environment.ts already exists.
ERROR! /src/index.html already exists.

Desired functionality

All files should be overwritten.

Mention any other details that might be useful

I tried every MergeStrategy possible (AllowOverwriteConflict, etc.), same result.

angular/devkit#745
https://stackoverflow.com/questions/48957132/how-to-overwrite-file-with-angular-schematics

@clydin
Copy link
Member

clydin commented Jun 21, 2018

Could you provide the versions used?

I cannot answer with certainty without knowing the versions but this PR (#11274) will most likely correct the behavior.

@william-lohan
Copy link
Author

version 0.6.8

dup repo https://github.com/gatimus/schematics-merge-strategy

@hansl
Copy link
Contributor

hansl commented Jul 19, 2018

Could you try with the RC of 0.7? I fixed a lot of issues by reworking the whole Tree implementation, should have fixed a few of those merging issues (amongst others, e.g. branching).

@hansl
Copy link
Contributor

hansl commented Jul 19, 2018

Dupe of #11246. Let's keep this one for tracking though as it has more information.

@william-lohan
Copy link
Author

Still duplicating on 0.7.0-rc.3 https://github.com/gatimus/schematics-merge-strategy/tree/RC-0.7

@Brocco Brocco added freq1: low Only reported by a handful of users who observe it rarely severity3: broken labels Jul 26, 2018
@cgatian
Copy link

cgatian commented Aug 2, 2018

This is making me cry 😢

@joeson1
Copy link
Contributor

joeson1 commented Aug 10, 2018

Is this already fixed?
I have following workaround:

const templateSource = apply(url('./project-files'), [
      template({ ...strings, ...options }),
      move(options.path),
      forEach((fileEntry: FileEntry) => {
        if (host.exists(fileEntry.path)) {
          host.overwrite(fileEntry.path, fileEntry.content);
        }
        return fileEntry;
      }),
    ]);

@tinesoft
Copy link

Hi,

For me, the issue appears after upgrading my schematics "dependencies" from:

 {
    "@angular-devkit/core": "^0.6.8",
    "@angular-devkit/schematics": "^0.6.8",
    "typescript": "^2.5.1"
  },

to:

 {
    "@angular-devkit/core": "^0.7.3",
    "@angular-devkit/schematics": "^0.7.3",
    "typescript": "^2.9.0"
  },

Version v0.6.8's mergeWith was working fine...

@joeson1 : i tried your workaround, but it didn't help 😞 ...

@william-lohan
Copy link
Author

@joeson1 where do you assign host?

@william-lohan
Copy link
Author

@tinesoft I get the exact same behavior with v0.6.8 as with v0.7.3

@Bored-Bohr
Copy link

Bored-Bohr commented Oct 10, 2018

I believe this is the same issue as #12088.

I originally commented this on #12088:
This issue still exists in 0.8.4.

The issue occurs on HostTree merge. I've detailed the simplified propagation below:

  1. The url function generates actions with kind 'c'
  2. Since there exists no prior actions this._willCreate(path) || this._willOverwrite(path) returns false.
  3. this._record.create triggers.
  4. This throws the fileDoesNotExistException on execute.

@egervari
Copy link

I'm still having this issue, in case someone needs to know it's still not fixed. Using version 0.8.6

@bautistaaa
Copy link

bautistaaa commented Oct 28, 2018

I have the same issue qith 0.8.6, however if I run schematics my-schematic:my-schematic it will work BUT if I run ng g my-schematic:my-schematic it will give me the already exists error message. Can anyone explain why?

@cgatian
Copy link

cgatian commented Nov 9, 2018

@egervari and @bautistaaa could you let me know if this workaround works for you?

https://stackoverflow.com/a/53230034/684869

@mattezell
Copy link

As others have stated, this still seems to be an issue... I've reproduced with 0.6.8 and 0.7.2... Based on one of the suggestions in the thread linked by @cgatian, I was able to formulate a workaround for my specific situation...

In my method, I was attempting to call move() within the rules array of apply()... Essentially the source url() was a subdirectory of my 'files' directory in my schematic - and the destination in move() was a subdirectory of the project tree.

So, I had the schematics source assets in: \schematics\ng-add\files\help\my-help\index.html

And I wanted my schematics source assets to be dropped in the projects source directory: \src\help\my-help\index.html.

When calling url('./files/help') and move('.', '/src/'), I would get "src/help/my-help/index.html already exists. The Schematic workflow failed."

To work around this, I changed my folder structure in my schematics assets to match the project folder structure - so instead of '\schematics\ng-add\files\help....' I used '\schematics\ng-add\files\src\help....'. I then updated url to be "url('./files')" and move to be "move('.')"... So now instead of explicitly setting the URL source as a subfolder containing the assets, it's just using the whole 'files' directory to overlay on top of the root of the project (move('.')).

I ran across this workaround when I by chance notice that I only got the 'already exists' error when operating on subfolders of the project - but in my troubleshooting, I realized that things behaved as expected when operating against the root of the project...

Here's a sample of my updated method which now works as desired in case it assists:

/** Add help assets to application */
// NOTE: work around due to busted MergeStrategy - https://github.com/angular/angular-cli/issues/11337
function addHelpToApplicationRoot(options: Schema) {
  return (tree: Tree, context: SchematicContext) => {
    context.logger.info(`Adding help assets to application help.`);
    const rule = mergeWith(
      apply(url('./files'), [
        forEach((fileEntry) => {
          if (tree.exists(fileEntry.path)) {
            tree.overwrite(fileEntry.path, fileEntry.content);
            return null;
          }
          return fileEntry;
        }),
        move('.')
      ]),
    );
    return rule(tree, context);
  };
}

Seems that the important detail, at least in my case, was to make my 'files' assets be structured in a way that matches the project's folder structure rather than relying on url() and move() to work on specific subfolders.

@ngbot ngbot bot added this to the needsTriage milestone Jan 24, 2019
@denisoby
Copy link

denisoby commented Feb 4, 2019

@mattezell Thank you for workaround example. However, I'd assume that you don't need this "move('.')" - everything should work even w/o it.

@matheo
Copy link

matheo commented Feb 11, 2019

I have the same problem not using a Source but trying to create/override custom stuff in the Tree. There's no simple way via MergeStrategy to avoid the exists check?

My hacky intuition thought that using tree.overwrite with MergeStrategy.AllowOverwriteConflict would default to create the files I'm trying to overwrite. Or create with AllowCreationConflict would overwrite, something like that would be useful.

@shusson
Copy link

shusson commented Feb 11, 2019

There's no simple way via MergeStrategy to avoid the exists check?

I believe this is exactly what MergeStrategy.Overwrite is suppose to do.

@GregOnNet
Copy link

Hi,

I experience the same issue.
I setup a minimal reproduction: https://github.com/GregOnNet/merge-strategie-issue

Setup

  • The repository contains a file .prettierrc
  • My schematic overwrite loads another .prettierrc from ./templates
  • MergeStrategy.Overwrite is configured as 2nd parameter of apply.
  • The repository contains a unit test to check if the template overwrites the origin file.

Observed behaviour

  • The unit test succeeds
  • The integration test using schematics fails

    Please refer to section Repro Steps to see the details.

Versions

Node v11.11.0
NPM 6.9.0
Mac MacOS Mojave 10.14.3

@angular-devkit/core: 7.3.6
@angular-devkit/schematics: 7.3.6

Repro Steps

git clone https://github.com/GregOnNet/merge-strategie-issue.git
cd merge-strategie-issue

npm install
npm test # Everything is fine
npm run build

schematics .:overwrite
# ERROR
ERROR! /.prettierrc already exists.
The Schematic workflow failed. See above.

If I can provide any further information, please let me know. 👍

GregOnNet added a commit to co-IT/schematics that referenced this issue Mar 15, 2019
GregOnNet added a commit to co-IT/schematics that referenced this issue Mar 17, 2019
@ngbot ngbot bot modified the milestones: needsTriage, Backlog May 13, 2019
@tscislo
Copy link

tscislo commented Jul 30, 2019

Is there any plan to fix this? Any current workarounds?

@william-lohan
Copy link
Author

william-lohan commented Jul 30, 2019

@tscislo this is my current workaround:

export function overwriteIfExists(host: Tree): Rule {
  return forEach(fileEntry => {
    if (host.exists(fileEntry.path)) {
      host.overwrite(fileEntry.path, fileEntry.content);
      return null;
    }
    return fileEntry;
  });
}

export function mySchematic(options: MyOptions): Rule {
  return (host: Tree, context: SchematicContext) => {
    /* ...logic... */
    const templateSource = apply(
      url('./files'),
      [
        template({
          ...strings,
          ...options
        }),
        move(options.path),
        options.overwrite ? overwriteIfExists(host) : noop()
      ]
    );

    return mergeWith(templateSource);
  };
}

derived from other variations in answers here https://stackoverflow.com/questions/48957132/how-to-overwrite-file-with-angular-schematics

@gund
Copy link

gund commented Oct 30, 2019

For me even workarounds did not work.
When I apply them - changes are simply ignored and I get message Nothing to be done....

@myin142
Copy link

myin142 commented Jul 22, 2020

A note about the workaround using manual forEach and overwrite.
Make sure that you put the forEach before the move since the file paths that you get are relative to the root project. And also make sure that you create the files first before trying to overwrite them. (e.g calling externalSchematic first).Took me a while to notice that.

Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 26, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 26, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 26, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 26, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 26, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 26, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 27, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 27, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
Brocco added a commit to Brocco/angular-cli that referenced this issue Jan 27, 2021
This fixes angular#11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
alan-agius4 pushed a commit that referenced this issue Jan 28, 2021
This fixes #11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.
alan-agius4 pushed a commit that referenced this issue Jan 28, 2021
This fixes #11337 to allow for merging of a tree with another when the the file already exists in the tree being merged into.

(cherry picked from commit a179828)
@angular-automatic-lock-bot
Copy link

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot bot locked and limited conversation to collaborators Feb 28, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area: devkit/schematics freq1: low Only reported by a handful of users who observe it rarely severity3: broken type: bug/fix
Projects
None yet
Development

Successfully merging a pull request may close this issue.