Come up with test plan for CodeFlare Operator

[anishasthana]

We currently don't have a complete test suite set up for CodeFlare. This resulted in our 0.0.2 release being a broken release. We should come up with a test plan for the CodeFlare operator, have it reviewed by the team, and create follow-on tasks for the implementation.

Unit tests -- testing specific functions within your code, use whatever you want
e2e tests -- Use either envtest (for a scaled down k8s cluster), OpenShift Local, Kind, or a fullblow OpenShift/k8s cluster, make sure your controller/operator performs correctly after a series of inputs.
Scorecard tests -- Deploy your operator via OLM, run through inbuilt tests + a number of custom test cases with inputs/expected outputs. We should investigate using kuttl for the custom test cases.
Integration tests -- Deploy your Operator on an actual OpenShift environment, Run through a number of test cases with inputs/expected outputs. We should think about wider integrations (such as that with ODH through the distributed workloads component) here too.

[tedhtchang]

The most basic form of operator e2e test that any contributor should run before pushing the bundle is described in testing-operator-deployment-on-openshift which is basically:

1. Create the bundle and push new bundle image to a registry.
2. Create a local file based catalog and push the image a registry. A free registry for temporarily store the bundle and catalog images.
3. Create og, catsrc, and sub on a local OCP cluster.
4. Create custom resources using CRs.

[astefanutti]

The test plan ADR proposal: project-codeflare/adr#6.

[astefanutti]

The most basic form of operator e2e test that any contributor should run before pushing the bundle is described in testing-operator-deployment-on-openshift which is basically:

1. Create the bundle and push new bundle image to a registry.
2. Create a local file based catalog and push the image a registry. A free registry for temporarily store the bundle and catalog images.
3. Create og, catsrc, and sub on a local OCP cluster.
4. Create custom resources using CRs.

@tedhtchang we are proposing to automate this, as described here: https://github.com/project-codeflare/adr/pull/6/files#diff-b40e444efa11cd7cea1dfc7e49effabd85bcbcb01cd3fd4f5269396e214935ecR248.

[anishasthana]

The ADR pr was merged in. Closing the issue.