REST API Product update url_key doesnt re-gerenate url_rewites (INSANE)

[Seb2nim]

Summary

Updating product url_key via REST API ; Product Url Rewrites are not re-generated. (yeah, i know, thats crazy)

This occurs on actually "Simple" setup, out of the box : Single-store mode and calls on "all" Rest Endpoint {{magento_url}}/rest/all/V1/products/{{sku}}, elementary school level

I use the term "Url Rewrites" (with a "s") because that involves category-path-product-urls also. But i focus on product "direct" url

Preconditions

1. Magento CE 2.3.4 - Single store mode
2. Magento 2.4-develop
3. Calls made on "all" : PUT {{magento_url}}/rest/all/V1/products/{{sku}}
   ⚠️ Note: 2.4-develop doesn't contain POST call for {{magento_url}}/rest/all/V1/products/{{sku}}
   
4. Stores->Configuration->General->Single-Store Mode->Enable Single-Store Mode=Yes

Steps to reproduce

1. Given a standard product (simple product, dont try to overcomplicate things)
   • Said product has id=10 : entity_id=10
   • Current url_key="my-product-url" (In Admin Product View : Section "Search Engine Optimization")
2. Check current Url_Rewrites in (Marketing / SEO & Search / Url Rewrites)
   • Search for target_path like catalog/product/view/id/10
   • Current url_rewrite should exist : "my-product-url.html" (i suppose ".html" as most used case prefix)
3. REST PUT Update on product :
   Sorry, no time to write a one line curl call. If you read this, you're smart enough to use Postman and already have this call ready to go.
   • PUT {{magento_url}}/rest/all/V1/products/{{sku}}
   • Json pload :

{
  "product": {
    "custom_attributes": [
        {
            "attribute_code": "url_key",
            "value": "my-new-url"
        }
    ]
  }
}

4. Check Url_key has been updated (In Admin Product View : Section "Search Engine Optimization")
   • Url_key is my-new-url (whaou)
5. Check Url_Rewrites in (Marketing / SEO & Search / Url Rewrites)
   • Search for target_path like catalog/product/view/id/10(or update tab/search, as you want, take your time)
   • URL rewrites not updated

Expected result

• URL Rewrites should have been updated

Actual result

• URL Rewrites NOT updated

Additional info

• There is absolutely NOTHING related to async, indexing, cache... No, thats really crazy : follow....
• Url rewrites ARE generated on product CREATION, but NOT ON UPDATE
• Passing websites_id on extension_attributes does not help (wait, it should, but wait...)
• Calling update on "default" rest endpoint instead of "all" would work BUT : IT IS A REALLY BAD IDEA :
  You will not see "real value" for the store (1) in Admin product view (and NO way to see it in Single store mode)
  There is another BUG that leads to almost ALL attributes to be copied for store 1 scope, not JUST the one you updated and, sorry but that sucks. I will ckeck if issue is already reported and eventually post. But poor chance to get a fix on that.

Debugging

Actually deep debugged code and i think the issue is on Product Model getStoreIds() even if it HARD to admit on such a central class :
Line $websiteIds = array_keys($websiteIds); gets array_keys()... ???
Does not have much sense... : Website_ids being either an array from json OR retreived from DB : NO WAY that array_keys would be some website_ids !

Checked current 2.4 develop, still there : https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/Catalog/Model/Product.php#L832

Workaround / I would say "Hack"

To congratulate you for your patience reading me, i offer you this hack if you promise to have a good laugh :

The FACT is that the array_keys() "bug" CAN be used as a temporary workaround hack :
You can pass values so that there will be a key for the targeted website id.
If you pass twice (here) the website_id in extension attribute, you'll exploit the bug and pass => Will work.
Hopefully, these arrays are not "cleaned" by any interceptor before being in product data object... Not surprising.

I think you got what you have to do if you want to re-gen website id 4... LOL

{
  "product": {
    "custom_attributes": [
        {
            "attribute_code": "url_key",
            "value": "my-url-key-16"
        }
    ],
    "extension_attributes": {
        "website_ids": [
            1,1
        ]
    }
  }
}

Exract from Magento\Catalog\Model\Product.php

...
public function getStoreIds()
    {
        if (!$this->hasStoreIds()) {
            $storeIds = [];
            if ($websiteIds = $this->getWebsiteIds()) {
                if (!$this->isObjectNew() && $this->_storeManager->isSingleStoreMode()) {
                    $websiteIds = array_keys($websiteIds);
                }
                foreach ($websiteIds as $websiteId) {
                    $websiteStores = $this->_storeManager->getWebsite($websiteId)->getStoreIds();
                    $storeIds[] = $websiteStores;
                }
            }
            if ($storeIds) {
                $storeIds = array_merge(...$storeIds);
            }
            $this->setStoreIds($storeIds);
        }
        return $this->getData('store_ids');
    }
...

[m2-assistant]

Hi @Seb2nim. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[Hexmage]

The same is true if you add a product to a website through the api. The product will not generate any new url rewrites for that website.

After double checking the url rewrites are also not regerenated when the categories change.

The problem seems to be that $product->getIsChangedCategories() and $product->getIsChangedWebsites() always return false when changed through the api.

[Hexmage]

How is this a low priority issue?
It basically means that any shop managing its product catalog through the api with an external system can't reliably generate product urls.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 14 days if no further activity occurs. Is this issue still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? Thank you for your contributions!

[Hexmage]

Still relevant.

[m2-assistant]

Hi @engcom-Delta. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[magento-engcom-team]

✅ Confirmed by @engcom-Delta
Thank you for verifying the issue. Based on the provided information internal tickets MC-40710 were created

Issue Available: @engcom-Delta, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[m2-assistant]

Hi @engcom-Charlie. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if you want to validate it one more time, please, go though the following instruction:

• 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
• 2. Verify that the issue is reproducible on 2.4-develop branch
     Details

     - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
     - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
     - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
• 3. If the issue is not relevant or is not reproducible any more, feel free to close it.

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[MariyaKlimenko]

Still reproduces on multi-store

[stephen-larkbury]

Summary

Updating product url_key via REST API ; Product Url Rewrites are not re-generated. (yeah, i know, thats crazy)

This occurs on actually "Simple" setup, out of the box : Single-store mode and calls on "all" Rest Endpoint {{magento_url}}/rest/all/V1/products/{{sku}}, elementary school level

I use the term "Url Rewrites" (with a "s") because that involves category-path-product-urls also. But i focus on product "direct" url

Preconditions

1. Magento CE 2.3.4 - Single store mode
2. Magento 2.4-develop
3. Calls made on "all" : PUT {{magento_url}}/rest/all/V1/products/{{sku}}
   ⚠️ Note: 2.4-develop doesn't contain POST call for {{magento_url}}/rest/all/V1/products/{{sku}}
   
4. Stores->Configuration->General->Single-Store Mode->Enable Single-Store Mode=Yes

Steps to reproduce

1. Given a standard product (simple product, dont try to overcomplicate things)

   • Said product has id=10 : entity_id=10
   • Current url_key="my-product-url" (In Admin Product View : Section "Search Engine Optimization")

2. Check current Url_Rewrites in (Marketing / SEO & Search / Url Rewrites)

   • Search for target_path like catalog/product/view/id/10
   • Current url_rewrite should exist : "my-product-url.html" (i suppose ".html" as most used case prefix)

3. REST PUT Update on product :
   Sorry, no time to write a one line curl call. If you read this, you're smart enough to use Postman and already have this call ready to go.

   • PUT {{magento_url}}/rest/all/V1/products/{{sku}}
   • Json pload :

{
  "product": {
    "custom_attributes": [
        {
            "attribute_code": "url_key",
            "value": "my-new-url"
        }
    ]
  }
}

4. Check Url_key has been updated (In Admin Product View : Section "Search Engine Optimization")

   • Url_key is my-new-url (whaou)

5. Check Url_Rewrites in (Marketing / SEO & Search / Url Rewrites)

   • Search for target_path like catalog/product/view/id/10(or update tab/search, as you want, take your time)
   • URL rewrites not updated

Expected result

• URL Rewrites should have been updated

Actual result

• URL Rewrites NOT updated

Additional info

• There is absolutely NOTHING related to async, indexing, cache... No, thats really crazy : follow....
• Url rewrites ARE generated on product CREATION, but NOT ON UPDATE
• Passing websites_id on extension_attributes does not help (wait, it should, but wait...)
• Calling update on "default" rest endpoint instead of "all" would work BUT : IT IS A REALLY BAD IDEA :
  You will not see "real value" for the store (1) in Admin product view (and NO way to see it in Single store mode)
  There is another BUG that leads to almost ALL attributes to be copied for store 1 scope, not JUST the one you updated and, sorry but that sucks. I will ckeck if issue is already reported and eventually post. But poor chance to get a fix on that.

Debugging

Actually deep debugged code and i think the issue is on Product Model getStoreIds() even if it HARD to admit on such a central class : Line $websiteIds = array_keys($websiteIds); gets array_keys()... ??? Does not have much sense... : Website_ids being either an array from json OR retreived from DB : NO WAY that array_keys would be some website_ids !

Checked current 2.4 develop, still there : https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/Catalog/Model/Product.php#L832

Workaround / I would say "Hack"

To congratulate you for your patience reading me, i offer you this hack if you promise to have a good laugh :

The FACT is that the array_keys() "bug" CAN be used as a temporary workaround hack : You can pass values so that there will be a key for the targeted website id. If you pass twice (here) the website_id in extension attribute, you'll exploit the bug and pass => Will work. Hopefully, these arrays are not "cleaned" by any interceptor before being in product data object... Not surprising.

I think you got what you have to do if you want to re-gen website id 4... LOL

{
  "product": {
    "custom_attributes": [
        {
            "attribute_code": "url_key",
            "value": "my-url-key-16"
        }
    ],
    "extension_attributes": {
        "website_ids": [
            1,1
        ]
    }
  }
}

Exract from Magento\Catalog\Model\Product.php

...
public function getStoreIds()
    {
        if (!$this->hasStoreIds()) {
            $storeIds = [];
            if ($websiteIds = $this->getWebsiteIds()) {
                if (!$this->isObjectNew() && $this->_storeManager->isSingleStoreMode()) {
                    $websiteIds = array_keys($websiteIds);
                }
                foreach ($websiteIds as $websiteId) {
                    $websiteStores = $this->_storeManager->getWebsite($websiteId)->getStoreIds();
                    $storeIds[] = $websiteStores;
                }
            }
            if ($storeIds) {
                $storeIds = array_merge(...$storeIds);
            }
            $this->setStoreIds($storeIds);
        }
        return $this->getData('store_ids');
    }
...

We are also experiencing this issue in Magento 2.4.6 Commerce with the exact same issues. (including creating multiple duplicate values for Admin store 1 when in single store mode)