sfdx force:source:pull tracking changes incorrectly for Profiles since Spring 20 release

[lockholster]

Summary

Prior to the Spring 20 release, if you push field level security or object level security profiles, then later pull (due to changes via ui), the new changes would be retrieved along with what was pushed prior. Now the profiles are returned almost empty, with only the fields changed since last push.

Also, this means if you push profile changes then pull again later, you also lose other fls, ols, and application visibilities, even if they were changed via the UI in the same session.

sfdx-cli/7.45.1-bc18d49798 win32-x64 node-v10.15.3
api v48.0

Steps To Reproduce:

Many ways to produce this but the best way to display why this breaks source driven development would be:

1. Create scratch org
2. Create new custom profile
3. Create new object, and one new field on that object (giving read/write to custom profile)
4. Pull changes (Profile retrieves with new object and new field, no other fls/ols)
5. Create a new scratch org and push project
6. Create a 2nd field on the new object
7. Pull changes (Profile retrieves with previous field removed, only contains new field)
8. Create a new scratch org and push project

Expected result

Profile has read/write access to both fields as configured

Actual result

Read/write access has been removed from the First field.

Additional information

The obvious problem this creates is that for source driven projects, whenever fls/ols changes and a pull is required, all our previous profile metadata (which was pushed) is removed.

Only way to get around this is to forceignore profiles (no longer functional for source driven projects), and use scripting (via mdapi) to replace them so future scratch orgs and CI deployments to sandboxes/production.

SFDX CLI Version(to find the version of the CLI engine run sfdx --version):
sfdx-cli/7.45.1-bc18d49798 win32-x64 node-v10.15.3

SFDX plugin Version(to find the version of the CLI plugin run sfdx plugins --core)
@oclif/plugin-autocomplete 0.1.5 (core)
@oclif/plugin-commands 1.2.3 (core)
@oclif/plugin-help 2.2.3 (core)
@oclif/plugin-not-found 1.2.3 (core)
@oclif/plugin-plugins 1.7.9 (core)
@oclif/plugin-update 1.3.9 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-trust 3.0.7 (core)
analytics 1.7.0 (core)
generator 1.1.2 (core)
salesforcedx 47.18.0 (core)
├─ salesforcedx-templates 47.19.0 (core)
└─ salesforce-alm 47.16.0 (core)

OS and version:
Windows 10, sourceApiVersion: 48.0 (this was updated to most recent in attempt to avoid the above)

[lockholster]

Cannot recreate this today, still happening in existing projects, but not for new projects created today, trying to isolate and will update asap.

[rbklaassen]

Hi, any updates on this @lockholster? We face the problem that sfdx pull just deletes the (custom) profiles...

[lockholster]

sorry @rbklaassen, never came back to this, sometimes we had profiles deleted as well. I haven't been following this too closely as we've restructured our project to pull profiles over mdapi now instead.
Do you have a binary SiteDotCom included in your project? As far as I could tell, the behaviour for pushing/pulling changed just whether this was included in the project or not.

[shetzel]

@lockholster @rbklaassen - This should be fixed now but please let us know if not. Feel free to reopen if it's not working with the latest CLI and plugin versions.

[FabienTaillon]

@shetzel do you think this behavior could be documented somewhere in the sfdx doc ? This is behaving differently from the Metadata API and is not documented anywhere, so I wasn't sure I could rely on it.

[shetzel]

@FabienTaillon - not sure what you mean. Is this still a problem? The source commands just use mdapi under the covers so there shouldn't be a difference. This is an old issue and there have been many changes to the plugins around source tracking since March.

[FabienTaillon]

@shetzel here is what I see and can't find documented.
Having a Profile like this (I removed user permissions to make it more clear):

<?xml version="1.0" encoding="UTF-8"?>
<Profile xmlns="http://soap.sforce.com/2006/04/metadata">
    <custom>false</custom>
    <fieldPermissions>
        <editable>true</editable>
        <field>Account.My_Field_2__c</field>
        <readable>true</readable>
    </fieldPermissions>
    <fieldPermissions>
        <editable>true</editable>
        <field>Account.My_Field__c</field>
        <readable>true</readable>
    </fieldPermissions>
    <userLicense>Salesforce</userLicense>
</Profile>

If I create a new My_Field_3__c in the Scratch Org and add it to the Profile, the 2 metadata that will be pulled are the Profile and the Field, and the resulting Profile metadata is the following:

<?xml version="1.0" encoding="UTF-8"?>
<Profile xmlns="http://soap.sforce.com/2006/04/metadata">
    <custom>false</custom>
    <fieldPermissions>
        <editable>true</editable>
        <field>Account.My_Field_3__c</field>
        <readable>true</readable>
    </fieldPermissions>
    <fieldPermissions>
        <editable>true</editable>
        <field>Account.My_Field_2__c</field>
        <readable>true</readable>
    </fieldPermissions>
    <fieldPermissions>
        <editable>true</editable>
        <field>Account.My_Field__c</field>
        <readable>true</readable>
    </fieldPermissions>
    <userLicense>Salesforce</userLicense>
</Profile>

You can notice that the new field was added to the existing Profile metadata, even though only the new field was pulled.
In a regular org (not scratch org), if I use Metadata API and package.xml to retrieve only the Profile and the field, the resulting metadata file will be the following:

<?xml version="1.0" encoding="UTF-8"?>
<Profile xmlns="http://soap.sforce.com/2006/04/metadata">
    <custom>false</custom>
    <fieldPermissions>
        <editable>true</editable>
        <field>Account.My_Field_3__c</field>
        <readable>true</readable>
    </fieldPermissions>
    <userLicense>Salesforce</userLicense>
</Profile>

So the other field permissions are removed, because Metadata API for Profiles will only retrieve access rights of other metadata included in the same retrieve.
This is this behavior that I can't find any documentation on, so I'm not sure why pulling a profile and a field in a scratch org don't behave the same way that retrieving the same metadata from a sandbox (field permission added instead of replacing what's already in the profile metadata file).
As it's not documented, I don't know exactly how it works, and if I can rely on it or not.

[shetzel]

@FabienTaillon - Ok we're on the same page now. Thanks for the details. Super helpful. I'll test this out sometime today, but if you beat me to it I'd be interested to see if there is a difference in the mdapi request between the 2 commands. If they are the exact same request (i.e., same package.xml) then it's likely a difference between scratch org and non-scratch org and that would be a question for another team. Profiles have never really played nicely with DX. I agree that we should discover the behavior, the reason for it, then doc it. I'll reopen.

[no-response]

This issue has been automatically closed because there has been no response to our request for more information from the original author. Currently, there is not enough information provided for us to take action. Please reply and reopen this issue if you need additional assistance.

[FabienTaillon]

Looks like the behavior is now documented here.