Don't return null vdtp fields

[xanderman]

My attributes map looks like this:

def self.scim_attributes_map
      {
        id: :id,
        externalId: :external_id,
        userName: :email_address,
        active: :is_active,
        name: {
          givenName: :first_name,
          familyName: :last_name
        },
        title: :title,
        timezone: :time_zone,
        emails: [
          {
            match: 'type',
            with: 'work',
            using: {
              value: :email_address,
              primary: true
            }
          }
        ],
        phoneNumbers: [
          {
            match: 'type',
            with: 'work',
            using: {
              value: :phone,
              primary: true
            }
          },
          {
            match: 'type',
            with: 'mobile',
            using: {
              value: :mobile_phone,
              primary: false
            }
          },
          {
            match: 'type',
            with: 'fax',
            using: {
              value: :fax,
              primary: false
            }
          }
        ],
        photos: [
          {
            match: 'type',
            with: 'photo',
            using: {
              value: :photo_url,
              primary: true
            }
          }
        ],
        roles: [
          {
            match: 'primary',
            with: true,
            using: {
              value: :role
            }
          }
        ]
      }
    end

When a user has only some of these mapped fields set, it gets serialized as a Resource like this:

{
  "id": "26eb0668-9717-4f1b-ad99-85cdda9b813e",
  "externalId": null,
  "userName": "admin_123@example.com",
  "active": true,
  "name": {
    "givenName": "Rickie",
    "familyName": "Nolan"
  },
  "title": null,
  "timezone": null,
  "emails": [
    {
      "type": "work",
      "value": "admin_123@example.com",
      "primary": true
    }
  ],
  "phoneNumbers": [
    {
      "type": "work",
      "value": null,
      "primary": true
    },
    {
      "type": "mobile",
      "value": null,
      "primary": false
    },
    {
      "type": "fax",
      "value": null,
      "primary": false
    }
  ],
  "photos": [
    {
      "type": "photo",
      "value": "200x200.gif",
      "primary": true
    }
  ],
  "roles": [
    {
      "primary": true,
      "value": "admin"
    }
  ],
  "meta": {
    "location": "http://localhost:4242/api/scim/v2/Users/26eb0668-9717-4f1b-ad99-85cdda9b813e",
    "created": "2025-10-24T21:28:31-06:00",
    "lastModified": "2025-10-24T21:28:31-06:00",
    "resourceType": "User"
  },
  "schemas": [
    "urn:ietf:params:scim:schemas:core:2.0:User"
  ]
}

Now, an IDP that uses PUT to update a user will do a read-modify-write. The IDP, in the first place, sent me a Resource that didn't have these null vdtp fields. But they dutifully read the user from my SCIM API, replace the fields that they have mapped, and send back the result. If the user, say, has no mobile phone in the IDP, that means I get sent a PUT with a resource that has a vdtp field with "value": null, which gets rejected as invalid.

I see that I can set optional_value_fields_required to false, but what I'd really rather do is not return vdtp fields with null values. For the same reason that option default to true -- "there's not much point being sent (say) an e-mail section which has entries that don't provide the e-mail address." The fault here is not the IDP, it's doing the right thing with a read-modify-write. The fault is in my own server returning a resource that it itself considers invalid. Is it possible to suppress setting null values in output Resources?

[pond]

Not presently, but it shouldn't be too hard to add an option. Earliest I might get to work on Scimitar this week would be maybe Thursday, else Friday NZ time, so I'll keep this in mind.

[pond]

I've implemented a simple version of this in v2.15.0 but it might need iteration as there are known limitations in the simple implementation. Set render_mapped_nil_values_in_response to true in your engine configuration initialiser. Hopefully it'll work well enough for your use case 😄

[pond]

Assuming 2.15.0 was OK -> closing.

[xanderman]

I apologize for how long it took me to test this. Life got busy, it kept falling off my todo list, etc. I just checked it now, and while it is slightly better (no "value" => nil) it doesn't actually solve the problem I was having. Before, I'd get output like this:

...
"phoneNumbers": [
  {
    "type": "fax",
    "value": null,
    "primary": false
  },
  ...
],
...

Now I get output like this:

...
"phoneNumbers": [
  {
    "type": "fax",
    "primary": false
  },
  ...
],
...

If I'm reading the known limitations comment correctly, this is exactly what you expect to happen. Unfortunately, it makes the feature pretty much useless for me. The problem is that IDPs frequently do "read-modify-write", where they GET the User, update only the field that changed (like the mobilePhone), and then PUT that object. I would like to have optional_value_fields_required: true, because that feels like a useful validation. But in the read-modify-write scenario, the PUT will have a vdtp field with value set to null (or missing entirely with this new config), neither of which passes the validation.