DFBUGS-1654: [release-4.18] Delete the succeeded pods with duplicate tolerations to avoid alert

[malayparida2000]

PrometheusDuplicateTimestamps alert is generated due to the presence of duplicate tolerations on the osd-prepare job pods & osd-key-rotation cronjob pods. Although the root of the issue has been fixed with another fix, the existing succeeded pods need to be cleaned up to stop the alert.

[openshift-ci-robot]

@malayparida2000: This pull request references [Jira Issue DFBUGS-1654](https://issues.redhat.com//browse/DFBUGS-1654), which is invalid:

• expected the bug to target the "odf-4.18" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

PrometheusDuplicateTimestamps alert is generated due to the presence of duplicate tolerations on the osd-prepare job pods & osd-key-rotation cronjob pods. Although the root of the issue has been fixed with another fix, the existing succeeded pods need to be cleaned up to stop the alert.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[malayparida2000]

/retest

[malayparida2000]

/hold for testing

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: iamniting, malayparida2000

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [iamniting]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[malayparida2000]

Testing result-

{"level":"info","ts":"2025-02-25T09:21:08Z","logger":"controllers.StorageCluster","msg":"Deleting pod with duplicate tolerations","Request.Namespace":"openshift-storage","Request.Name":"ocs-storagecluster","pod":"rook-ceph-osd-prepare-ocs-deviceset-0-data-0xb5s5-bfgv7"}
{"level":"info","ts":"2025-02-25T09:21:08Z","logger":"controllers.StorageCluster","msg":"Deleting pod with duplicate tolerations","Request.Namespace":"openshift-storage","Request.Name":"ocs-storagecluster","pod":"rook-ceph-osd-prepare-ocs-deviceset-2-data-07bc4n-tfdgl"}
{"level":"info","ts":"2025-02-25T09:21:08Z","logger":"controllers.StorageCluster","msg":"Deleting pod with duplicate tolerations","Request.Namespace":"openshift-storage","Request.Name":"ocs-storagecluster","pod":"rook-ceph-osd-prepare-ocs-deviceset-1-data-05cdqr-h859p"}

osd-prepare pods with duplicate tolerations have been cleaned up.

[malayparida2000]

/unhold

[malayparida2000]

/jira refresh

[openshift-ci-robot]

@malayparida2000: This pull request references [Jira Issue DFBUGS-1654](https://issues.redhat.com//browse/DFBUGS-1654), which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (odf-4.18) matches configured target version for branch (odf-4.18)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira ([email protected]), skipping review request.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[iamniting]

/override ci/prow/ci-bundle-ocs-operator-bundle
/override ci/prow/images
/override ci/prow/ocs-operator-bundle-e2e-aws

[openshift-ci]

@iamniting: Overrode contexts on behalf of iamniting: ci/prow/ci-bundle-ocs-operator-bundle, ci/prow/images, ci/prow/ocs-operator-bundle-e2e-aws

In response to this:

/override ci/prow/ci-bundle-ocs-operator-bundle
/override ci/prow/images
/override ci/prow/ocs-operator-bundle-e2e-aws

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci-robot]

@malayparida2000: [Jira Issue DFBUGS-1654](https://issues.redhat.com//browse/DFBUGS-1654): All pull requests linked via external trackers have merged:

• red-hat-storage/ocs-operator#3056

[Jira Issue DFBUGS-1654](https://issues.redhat.com//browse/DFBUGS-1654) has been moved to the MODIFIED state.

In response to this:

PrometheusDuplicateTimestamps alert is generated due to the presence of duplicate tolerations on the osd-prepare job pods & osd-key-rotation cronjob pods. Although the root of the issue has been fixed with another fix, the existing succeeded pods need to be cleaned up to stop the alert.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[malayparida2000]

/cherry-pick release-4.17

[openshift-cherrypick-robot]

@malayparida2000: #3056 failed to apply on top of branch "release-4.17":

Applying: Delete the succeeded pods with duplicate tolerations to avoid alert
Using index info to reconstruct a base tree...
M	controllers/storagecluster/cephcluster.go
M	controllers/util/k8sutil.go
A	metrics/vendor/github.com/red-hat-storage/ocs-operator/v4/controllers/util/k8sutil.go
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): metrics/vendor/github.com/red-hat-storage/ocs-operator/v4/controllers/util/k8sutil.go deleted in HEAD and modified in Delete the succeeded pods with duplicate tolerations to avoid alert. Version Delete the succeeded pods with duplicate tolerations to avoid alert of metrics/vendor/github.com/red-hat-storage/ocs-operator/v4/controllers/util/k8sutil.go left in tree.
Auto-merging controllers/util/k8sutil.go
Auto-merging controllers/storagecluster/cephcluster.go
CONFLICT (content): Merge conflict in controllers/storagecluster/cephcluster.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 Delete the succeeded pods with duplicate tolerations to avoid alert

In response to this:

/cherry-pick release-4.17

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.