Add OLM upgrade test

[bouskaJ]

Release Note

NONE

[bouskaJ]

@astefanutti Finally, the PR is there. Please review.

[bouskaJ]

@astefanutti the CI seems to be faulty. I see a lot of various errors that are not caused by my changes.

[astefanutti]

@bouskaJ awesome!

It's possible that CI is currently broken as the Camel / Camel Quarkus upgrade is being rolling out and needs #2197 to complete.

[nicolaferraro]

@bouskaJ awesome!

It's possible that CI is currently broken as the Camel / Camel Quarkus upgrade is being rolling out and needs #2197 to complete.

Yes, we may merge #2197 to have less errors, as the only issue seems in runtime: apache/camel-k-runtime#651. Wdyt?

[astefanutti]

@bouskaJ awesome!
It's possible that CI is currently broken as the Camel / Camel Quarkus upgrade is being rolling out and needs #2197 to complete.

Yes, we may merge #2197 to have less errors, as the only issue seems in runtime: apache/camel-k-runtime#651. Wdyt?

Good idea, +1 for me, then we can rebase the pending PRs.

[bouskaJ]

@astefanutti can you rerun the CI?

[astefanutti]

@bouskaJ could you please rebase to have the changes from #2197 taken into account?

[astefanutti]

@bouskaJ thanks again for the awesome work.

It seems the test sometimes fails with the following error:

    olm_upgrade_test.go:94: 
        Timed out after 300.006s.
        Expected
            <v1alpha1.ClusterServiceVersionPhase>: Succeeded
        to equal
            <v1alpha1.ClusterServiceVersionPhase>: Replacing
    --- FAIL: TestOLMAutomaticUpgrade/OLM_upgrade (300.01s)

I wonder if that's just a race issue, or that the phase is not as expected.

Also, I understand that the operator upgrade is triggered by updating the index image in the CatalogSource. However, the CSV generated for the new version does not have a replaces field. So I wonder, are we sure that this sequence mimics exactly a typical upgrade performed by OLM?

I'd like to be sure that what the test covers is an actual upgrade, rather than just a replace.

[bouskaJ]

Hi, @astefanutti yes, there is no replaces but the replace scenario is forced by the skipRange (see https://github.com/apache/camel-k/blob/master/.github/workflows/upgrade.yml#L138).
I was considering:

• replaces because it was close to the real upstream releases
• skipRange - used in the internal stream (at the day I was writing the test)

I decided on the skipRange because it is able to force an upgrade on the older releases as well so the test is not dependent on the very last release -> devel version. I think we can migrate to replaces since both streams use replaces at the moment in the real manifests.

[bouskaJ]

I am not sure why, it's failing. Maybe, the OLM needs more time to start the upgrade and we should use TestTimeoutLong https://github.com/apache/camel-k/blob/master/e2e/upgrade/olm_upgrade_test.go#L94

[bouskaJ]

Printing the patched CSV to the workflow log could help as well... (result of https://github.com/apache/camel-k/blob/master/.github/workflows/upgrade.yml#L143)