OSDOCS#17158: node replacement procedure #102520

skopacz1 · 2025-11-13T21:41:12Z

OSDOCS-17158

Version(s): 4.19+

This PR takes this KCS and puts it into product docs.

QE review:

QE has approved this change.

Preview: Replacing a failed bare-metal control plane node without DHCP and BMC credentials

openshift-ci-robot · 2025-11-13T21:41:16Z

@skopacz1: This pull request references OSDOCS-17158 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

OSDOCS-17158

Version(s): 4.19+

This PR takes this KCS and puts it into product docs.

QE review:

QE has approved this change.

Preview:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2025-11-13T21:41:28Z

@skopacz1: No Jira issue is referenced in the title of this pull request.
To reference a jira issue, add 'XYZ-NNN:' to the title of this pull request and request another refresh with /jira refresh.

In response to this:

OSDOCS-17158

Version(s): 4.19+

This PR takes this KCS and puts it into product docs.

QE review:

QE has approved this change.

Preview:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

ocpdocs-previewbot · 2025-11-13T21:55:25Z

🤖 Tue Nov 25 16:14:30 - Prow CI generated the docs preview:

https://102520--ocpdocs-pr.netlify.app/openshift-enterprise/latest/nodes/nodes/nodes-nodes-replace-control-plane.html

skopacz1 · 2025-11-14T21:12:21Z

modules/nodes-link-node-machine-bmh.adoc

+. Generate `providerID` lines for control plane nodes by running the following command:
+
+[source,terminal]
+----
+$ oc get -n openshift-machine-api baremetalhost -l installer.openshift.io/role=control-plane -ojson | jq -r '.items[] | "baremetalhost:///openshift-machine-api/" + .metadata.name + "/" + .metadata.uid'
+----
+
+. Identify the cluster by running the following command:


Can I get example output to show for both of these commands, if that would be helpful?

For the second step, is there a more descriptive phrasing I could use for what the purpose of the step is?

The suggestion for these 2 commands:
1, Get providerID for control plane nodes by running the following command:
2, Get cluster information for labels by running the following command:
<cluster_api_cluster> should be the cluster_name, right?

I can update both commands to use the new suggested wording.

As for whether cluster-api-cluster in the second command is the actual name of the cluster, I am not sure. I can try to find the original authors and ask them if that was their intention.

skopacz1 · 2025-11-14T21:13:50Z

modules/nodes-link-node-machine-bmh.adoc

+    -L machine.openshift.io/cluster-api-cluster
+----
+
+. Create a `Machine` object for the new control plane node by creating a yaml file similar to the following:


For this step, should I break it out into two substeps, where they create a YAML file like new-machine.yaml and then run oc create -f new-machine.yaml?

One step is fine to me

zniu1011 · 2025-11-24T06:45:19Z

modules/nodes-link-node-machine-bmh.adoc

+$ oc get bmh -n openshift-machine-api -ojson | jq -r '.items[] | .metadata.name + "   ProvisioningState:" +  .status.provisioning.state'
+----
+
+.. If the provisioning state is not `unmanaged`: Change the provisioning state by running the following command:


Since we have only one note here, maybe we don't need a list?

Suggested change

.. If the provisioning state is not `unmanaged`: Change the provisioning state by running the following command:

If the provisioning state is not `unmanaged`, change the provisioning state by running the following command:

I can change it to an IMPORTANT admonition rather than having it be a substep, sure. The only thing I shouldn't do is make this instruction a second paragraph of the step to "Review the BareMetalHost object's provisioning state"

nodes/nodes/nodes-nodes-replace-control-plane.adoc

openshift-ci · 2025-11-25T16:15:51Z

@skopacz1: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

zniu1011 · 2025-11-26T02:47:49Z

/lgtm

maxwelldb

Left a couple of comments. I think the formatting and .Procedure ones should be addressed prior to merge.

maxwelldb · 2025-11-26T15:45:11Z

modules/nodes-replace-control-plane-prereqs.adoc

+* You have access to the cluster as a user with the `cluster-admin` role.
+* You have taken an xref:../../backup_and_restore/control_plane_backup_and_restore/backing-up-etcd.adoc#backup-etcd[etcd backup] in case you encounter any issues.
+* You have downloaded and installed the link:https://console.redhat.com/openshift/downloads#tool-coreos-installer[`coreos-installer` CLI].
+* Your cluster does not have a control plane `machineset`. You can check for `machinesets` by running the following command:


Could theoretically be a procedure module, particularly if it's a common situation.

maxwelldb · 2025-11-26T15:46:23Z

modules/nodes-add-new-etcd-member.adoc

+$ oc rsh -n openshift-etcd <running_pod>
+----
+
+Where `<running_pod>` is the name of a running pod shown in the previous step.


See SSG for user-replaceable value DL format, i.e. where: and so on.

maxwelldb · 2025-11-26T15:48:04Z

modules/nodes-remove-unhealthy-etcd-member.adoc

+
+[IMPORTANT]
+====
+* Make note of the ID and the name of the unhealthy etcd member because these values are required later.


Given the importance, could make this its own step or a second sentence for this list item. Flow consideration. 🤷

maxwelldb · 2025-11-26T15:50:08Z

modules/nodes-delete-machine-unhealthy-etcd.adoc

+After you remove the `BareMetalHost` and `Machine` objects, the machine controller automatically deletes the `Node` object.
+====
+
+. If deletion of the machine is delayed for any reason or the command is obstructed and delayed: Force deletion by removing the machine object finalizer field.


Does this need a reformat?

maxwelldb · 2025-11-26T15:51:26Z

modules/nodes-create-new-control-plane-node.adoc

+.. Remove the `status` section of the file.
+--
+
+The resulting file should resemble the following:


Suggested change

The resulting file should resemble the following:

The resulting file should resemble the following example:

maxwelldb · 2025-11-26T15:52:20Z

modules/nodes-create-new-control-plane-node.adoc

+
+This command also removes the starting `userData` line of the ignition secret.
+
+. Create an nmstate YAML file titled `new_controlplane_nmstate.yaml` for the new node's network configuration, using the following example for reference:


nmstate or NMState?

maxwelldb · 2025-11-26T15:52:58Z

modules/nodes-delete-machine-unhealthy-etcd.adoc

+
+. Ensure that the Bare Metal Operator is available by running the following command:


Suggested change

. Ensure that the Bare Metal Operator is available by running the following command:

.Procedure

. Ensure that the Bare Metal Operator is available by running the following command:

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Nov 13, 2025

skopacz1 changed the title ~~OSDOCS-17158: node replacement procedure~~ OSDOCS#17158: node replacement procedure Nov 13, 2025

openshift-ci-robot removed the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Nov 13, 2025

openshift-ci bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Nov 13, 2025

skopacz1 added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. branch/enterprise-4.19 branch/enterprise-4.20 branch/enterprise-4.21 and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 13, 2025

skopacz1 added this to the Continuous Release milestone Nov 13, 2025

openshift-ci bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Nov 13, 2025

skopacz1 force-pushed the OSDOCS-17158 branch from 8f930b9 to b79c46f Compare November 14, 2025 21:02

skopacz1 commented Nov 14, 2025

View reviewed changes

zniu1011 reviewed Nov 24, 2025

View reviewed changes

ocpdocs-vale-bot reviewed Nov 24, 2025

View reviewed changes

nodes/nodes/nodes-nodes-replace-control-plane.adoc Outdated Show resolved Hide resolved

ocpdocs-vale-bot reviewed Nov 24, 2025

View reviewed changes

nodes/nodes/nodes-nodes-replace-control-plane.adoc Outdated Show resolved Hide resolved

OSDOCS-17158: node replacement procedure

a11b7ea

skopacz1 force-pushed the OSDOCS-17158 branch from e80bffe to a11b7ea Compare November 25, 2025 16:07

openshift-ci bot assigned zniu1011 Nov 26, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 26, 2025

skopacz1 added the merge-review-needed Signifies that the merge review team needs to review this PR label Nov 26, 2025

maxwelldb self-requested a review November 26, 2025 15:42

maxwelldb added the merge-review-in-progress Signifies that the merge review team is reviewing this PR label Nov 26, 2025

maxwelldb reviewed Nov 26, 2025

View reviewed changes

maxwelldb removed merge-review-in-progress Signifies that the merge review team is reviewing this PR merge-review-needed Signifies that the merge review team needs to review this PR labels Nov 26, 2025

	.. If the provisioning state is not `unmanaged`: Change the provisioning state by running the following command:
	If the provisioning state is not `unmanaged`, change the provisioning state by running the following command:

	The resulting file should resemble the following:
	The resulting file should resemble the following example:


		. Ensure that the Bare Metal Operator is available by running the following command:

OSDOCS#17158: node replacement procedure #102520

Are you sure you want to change the base?

OSDOCS#17158: node replacement procedure #102520

Uh oh!

Conversation

skopacz1 commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Nov 13, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Nov 13, 2025

Uh oh!

ocpdocs-previewbot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

openshift-ci bot commented Nov 25, 2025

Uh oh!

zniu1011 commented Nov 26, 2025

Uh oh!

maxwelldb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

skopacz1 commented Nov 13, 2025 •

edited

Loading

openshift-ci-robot commented Nov 13, 2025 •

edited by openshift-ci bot

Loading

ocpdocs-previewbot commented Nov 13, 2025 •

edited

Loading