ManifestWork does not work correctly if namespace is set for Cluster scoped resource

[tamalsaha]

Describe the bug
We are trying to create ManifestWork with cluster scoped resources where namespace is set. In that case the resource is not properly synced/created on the spoke cluster.

To Reproduce

apiVersion: work.open-cluster-management.io/v1
kind: ManifestWork
metadata:
  name: demo
  namespace: c1
spec:
  workload:
    manifests:
    - apiVersion: v1
      kind: Namespace
      metadata:
        name: demo
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: config-reader
        namespace: demo
      rules:
      - apiGroups:
        - ""
        resources:
        - configmaps
        verbs:
        - get
        - list
        - watch

On the spoke cluster c1, I see

apiVersion: work.open-cluster-management.io/v1
kind: AppliedManifestWork
metadata:
  creationTimestamp: "2023-11-28T13:14:44Z"
  finalizers:
  - cluster.open-cluster-management.io/applied-manifest-work-cleanup
  generation: 1
  name: 43967cf4fa7b6c9c1f4014eb104077ac73d86aabc3d2c4ae6c51babdf5898540-demo
  resourceVersion: "4440"
  uid: d323fbfc-a15d-40d4-9041-506d8d20a6c6
spec:
  agentID: a7abf39a-bf7c-470c-af6b-9cc95ac22884
  hubHash: 43967cf4fa7b6c9c1f4014eb104077ac73d86aabc3d2c4ae6c51babdf5898540
  manifestWorkName: demo
status:
  appliedResources:
  - group: ""
    name: demo
    namespace: ""
    resource: namespaces
    uid: b980ab49-0b9f-4754-ad08-2eb7419cf80f
    version: v1

Expected behavior
In Kubernetes, if users set metadata.namespace field for a cluster scoped resource and then kubectl apply that, everything still works. I am expecting ManifestWork to work the same way. The work controller should auto discover that a given resource is cluster scoped and act accordingly.

Environment ie: OCM version, Kubernetes version and provider:

Additional context
The reason this is important to us is because we are trying to use helm charts to generate addon agent manifests. Some helm charts don't set the metdata.namespace field in the templates and let helm configure that based on --namespace flag. I would like to be able to use such helm charts as is to generate addon manifests. To that end, I created open-cluster-management-io/addon-framework#220 . This pr always sets the namespace field in the generated ManifestWork because there is no way to know if a resource is cluster scoped or not at the hub cluster. But due to the above mentioned issue, such ManifestWorks are not syncing properly. It should be possible for a work agent to use the spoke clusters discovery api to detect if a resource is cluster scoped or not and sync properly.

[qiujian16]

Do you have the status condition of this manifestwork? I think it should correctly synced.

[qiujian16]

/assign