feat: activation condition (#2105)

Signed-off-by: Attila Mészáros <[email protected]>

Author: csviri

docs/documentation/workflows.md

@@ -35,6 +35,14 @@ reconciliation process.
   proceeding until the condition checking whether the DR is ready holds true
 - **Delete postcondition** - is a condition on a given DR to check if the reconciliation of
   dependents can proceed after the DR is supposed to have been deleted
+- **Activation condition** - is a special condition meant to specify under which condition the DR is used in the
+  workflow. A typical use-case for this feature is to only activate some dependents depending on the presence of
+  optional resources / features on the target cluster. Without this activation condition, JOSDK would attempt to
+  register an informer for these optional resources, which would cause an error in the case where the resource is
+  missing. With this activation condition, you can now conditionally register informers depending on whether the
+  condition holds or not. This is a very useful feature when your operator needs to handle different flavors of the
+  platform (e.g. OpenShift vs plain Kubernetes) and/or change its behavior based on the availability of optional
+  resources / features (e.g. CertManager, a specific Ingress controller, etc.).
 
 ## Defining Workflows
 
@@ -66,6 +74,7 @@ will only consider the `ConfigMap` deleted until that post-condition becomes `tr
     @Dependent(type = ConfigMapDependentResource.class,
         reconcilePrecondition = ConfigMapReconcileCondition.class,
         deletePostcondition = ConfigMapDeletePostCondition.class,
+        activationCondition = ConfigMapActivationCondition.class,
         dependsOn = DEPLOYMENT_NAME)
 })
 public class SampleWorkflowReconciler implements Reconciler<WorkflowAllFeatureCustomResource>,
@@ -165,7 +174,7 @@ executed if a resource is marked for deletion.
 ## Common Principles
 
 - **As complete as possible execution** - when a workflow is reconciled, it tries to reconcile as
-  many resources as possible. Thus if an error happens or a ready condition is not met for a
+  many resources as possible. Thus, if an error happens or a ready condition is not met for a
   resources, all the other independent resources will be still reconciled. This is the opposite
   to a fail-fast approach. The assumption is that eventually in this way the overall state will
   converge faster towards the desired state than would be the case if the reconciliation was
@@ -186,13 +195,13 @@ demonstrated using examples:
    `depends-on` relations.
 2. Root nodes, i.e. nodes in the graph that do not depend on other nodes are reconciled first,
    in a parallel manner.
-2. A DR is reconciled if it does not depend on any other DRs, or *ALL* the DRs it depends on are
-   reconciled and ready. If a DR defines a reconcile pre-condition, then this condition must
-   become `true` before the DR is reconciled.
-2. A DR is considered *ready* if it got successfully reconciled and any ready post-condition it
+3. A DR is reconciled if it does not depend on any other DRs, or *ALL* the DRs it depends on are
+   reconciled and ready. If a DR defines a reconcile pre-condition and/or an activation condition, 
+   then these condition must become `true` before the DR is reconciled.
+4. A DR is considered *ready* if it got successfully reconciled and any ready post-condition it
    might define is `true`.
-3. If a DR's reconcile pre-condition is not met, this DR is deleted. All of the DRs that depend
-   on the dependent resource being considered are also recursively deleted. This implies that
+5. If a DR's reconcile pre-condition is not met, this DR is deleted. All the DRs that depend
+   on the dependent resource are also recursively deleted. This implies that
    DRs are deleted in reverse order compared the one in which they are reconciled. The reason
    for this behavior is (Will make a more detailed blog post about the design decision, much deeper
    than the reference documentation)
@@ -202,7 +211,10 @@ demonstrated using examples:
    idempotency (i.e. with the same input state, we should have the same output state), from this
    follows that if the condition doesn't hold `true` anymore, the associated resource needs to
    be deleted because the resource shouldn't exist/have been created.
-4. For a DR to be deleted by a workflow, it needs to implement the `Deleter` interface, in which
+6. If a DR's activation condition is not met, it won't be reconciled or deleted. If other DR's depend on it, those will
+   be recursively deleted in a way similar to reconcile pre-conditions. Event sources for a dependent resource with
+   activation condition are registered/de-registered dynamically, thus during the reconciliation.
+7. For a DR to be deleted by a workflow, it needs to implement the `Deleter` interface, in which
    case its `delete` method will be called, unless it also implements the `GarbageCollected`
    interface. If a DR doesn't implement `Deleter` it is considered as automatically deleted. If
    a delete post-condition exists for this DR, it needs to become `true` for the workflow to

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/BaseConfigurationService.java

@@ -230,6 +230,7 @@ private static List<DependentResourceSpec> dependentResources(
           Utils.instantiate(dependent.readyPostcondition(), Condition.class, context),
           Utils.instantiate(dependent.reconcilePrecondition(), Condition.class, context),
           Utils.instantiate(dependent.deletePostcondition(), Condition.class, context),
+          Utils.instantiate(dependent.activationCondition(), Condition.class, context),
           eventSourceName);
       specsMap.put(dependentName, spec);
     }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/dependent/DependentResourceSpec.java

@@ -22,18 +22,21 @@ public class DependentResourceSpec<R, P extends HasMetadata> {
 
   private final Condition<?, ?> deletePostCondition;
 
+  private final Condition<?, ?> activationCondition;
+
   private final String useEventSourceWithName;
 
   public DependentResourceSpec(Class<? extends DependentResource<R, P>> dependentResourceClass,
       String name, Set<String> dependsOn, Condition<?, ?> readyCondition,
       Condition<?, ?> reconcileCondition, Condition<?, ?> deletePostCondition,
-      String useEventSourceWithName) {
+      Condition<?, ?> activationCondition, String useEventSourceWithName) {
     this.dependentResourceClass = dependentResourceClass;
     this.name = name;
     this.dependsOn = dependsOn;
     this.readyCondition = readyCondition;
     this.reconcileCondition = reconcileCondition;
     this.deletePostCondition = deletePostCondition;
+    this.activationCondition = activationCondition;
     this.useEventSourceWithName = useEventSourceWithName;
   }
 
@@ -87,6 +90,11 @@ public Condition getDeletePostCondition() {
     return deletePostCondition;
   }
 
+  @SuppressWarnings("rawtypes")
+  public Condition getActivationCondition() {
+    return activationCondition;
+  }
+
   public Optional<String> getUseEventSourceWithName() {
     return Optional.ofNullable(useEventSourceWithName);
   }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/dependent/Dependent.java

@@ -50,6 +50,26 @@
    */
   Class<? extends Condition> deletePostcondition() default Condition.class;
 
+  /**
+   * <p>
+   * A condition that needs to become true for the dependent to even be considered as part of the
+   * workflow. This is useful for dependents that represent optional resources on the cluster and
+   * might not be present. In this case, a reconcile pre-condition is not enough because in that
+   * situation, the associated informer will still be registered. With this activation condition,
+   * the associated event source will only be registered if the condition is met. Otherwise, this
+   * behaves like a reconcile pre-condition in the sense that dependents, that depend on this one,
+   * will only get created if the condition is met and will get deleted if the condition becomes
+   * false.
+   * </p>
+   * <p>
+   * As other conditions, this gets evaluated at the beginning of every reconciliation, which means
+   * that it allows to react to optional resources becoming available on the cluster as the operator
+   * runs. More specifically, this means that the associated event source can get dynamically
+   * registered or de-registered during reconciliation.
+   * </p>
+   */
+  Class<? extends Condition> activationCondition() default Condition.class;
+
   /**
    * The list of named dependents that need to be reconciled before this one can be.
    *

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/Controller.java

@@ -76,6 +76,7 @@ public class Controller<P extends HasMetadata>
   private final GroupVersionKind associatedGVK;
   private final EventProcessor<P> eventProcessor;
   private final ControllerHealthInfo controllerHealthInfo;
+  private final EventSourceContext<P> eventSourceContext;
 
   public Controller(Reconciler<P> reconciler,
       ControllerConfiguration<P> configuration,
@@ -98,9 +99,9 @@ public Controller(Reconciler<P> reconciler,
     eventProcessor = new EventProcessor<>(eventSourceManager, configurationService);
     eventSourceManager.postProcessDefaultEventSourcesAfterProcessorInitializer();
     controllerHealthInfo = new ControllerHealthInfo(eventSourceManager);
-    final var context = new EventSourceContext<>(
+    eventSourceContext = new EventSourceContext<>(
         eventSourceManager.getControllerResourceEventSource(), configuration, kubernetesClient);
-    initAndRegisterEventSources(context);
+    initAndRegisterEventSources(eventSourceContext);
     configurationService.getMetrics().controllerRegistered(this);
   }
 
@@ -236,7 +237,8 @@ public void initAndRegisterEventSources(EventSourceContext<P> context) {
     }
 
     // register created event sources
-    final var dependentResourcesByName = managedWorkflow.getDependentResourcesByName();
+    final var dependentResourcesByName =
+        managedWorkflow.getDependentResourcesByNameWithoutActivationCondition();
     final var size = dependentResourcesByName.size();
     if (size > 0) {
       dependentResourcesByName.forEach((key, dependentResource) -> {
@@ -440,4 +442,8 @@ public EventProcessor<P> getEventProcessor() {
   public ExecutorServiceManager getExecutorServiceManager() {
     return getConfiguration().getConfigurationService().getExecutorServiceManager();
   }
+
+  public EventSourceContext<P> eventSourceContext() {
+    return eventSourceContext;
+  }
 }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/AbstractWorkflowExecutor.java

@@ -46,12 +46,15 @@ public AbstractWorkflowExecutor(Workflow<P> workflow, P primary, Context<P> cont
   protected synchronized void waitForScheduledExecutionsToRun() {
     while (true) {
       try {
-        this.wait();
+        // in case when workflow just contains non-activated dependents,
+        // it needs to be checked first if there are already no executions
+        // scheduled at the beginning.
         if (noMoreExecutionsScheduled()) {
           break;
         } else {
           logger().warn("Notified but still resources under execution. This should not happen.");
         }
+        this.wait();
       } catch (InterruptedException e) {
         if (noMoreExecutionsScheduled()) {
           logger().debug("interrupted, no more executions for: {}", primaryID);

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DefaultManagedWorkflow.java

@@ -81,6 +81,7 @@ public Workflow<P> resolve(KubernetesClient client,
           spec.getReconcileCondition(),
           spec.getDeletePostCondition(),
           spec.getReadyCondition(),
+          spec.getActivationCondition(),
           resolve(spec, client, configuration));
       alreadyResolved.put(node.getName(), node);
       spec.getDependsOn()

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DefaultWorkflow.java

@@ -147,4 +147,15 @@ public Map<String, DependentResource> getDependentResourcesByName() {
         .forEach((name, node) -> resources.put(name, node.getDependentResource()));
     return resources;
   }
+
+  public Map<String, DependentResource> getDependentResourcesByNameWithoutActivationCondition() {
+    final var resources = new HashMap<String, DependentResource>(dependentResourceNodes.size());
+    dependentResourceNodes
+        .forEach((name, node) -> {
+          if (node.getActivationCondition().isEmpty()) {
+            resources.put(name, node.getDependentResource());
+          }
+        });
+    return resources;
+  }
 }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/DependentResourceNode.java

@@ -16,19 +16,21 @@ public class DependentResourceNode<R, P extends HasMetadata> {
   private Condition<R, P> reconcilePrecondition;
   private Condition<R, P> deletePostcondition;
   private Condition<R, P> readyPostcondition;
+  private Condition<R, P> activationCondition;
   private final DependentResource<R, P> dependentResource;
 
   DependentResourceNode(DependentResource<R, P> dependentResource) {
-    this(getNameFor(dependentResource), null, null, null, dependentResource);
+    this(getNameFor(dependentResource), null, null, null, null, dependentResource);
   }
 
   public DependentResourceNode(String name, Condition<R, P> reconcilePrecondition,
       Condition<R, P> deletePostcondition, Condition<R, P> readyPostcondition,
-      DependentResource<R, P> dependentResource) {
+      Condition<R, P> activationCondition, DependentResource<R, P> dependentResource) {
     this.name = name;
     this.reconcilePrecondition = reconcilePrecondition;
     this.deletePostcondition = deletePostcondition;
     this.readyPostcondition = readyPostcondition;
+    this.activationCondition = activationCondition;
     this.dependentResource = dependentResource;
   }
 
@@ -63,6 +65,10 @@ public Optional<Condition<R, P>> getDeletePostcondition() {
     return Optional.ofNullable(deletePostcondition);
   }
 
+  public Optional<Condition<R, P>> getActivationCondition() {
+    return Optional.ofNullable(activationCondition);
+  }
+
   void setReconcilePrecondition(Condition<R, P> reconcilePrecondition) {
     this.reconcilePrecondition = reconcilePrecondition;
   }
@@ -71,6 +77,10 @@ void setDeletePostcondition(Condition<R, P> cleanupCondition) {
     this.deletePostcondition = cleanupCondition;
   }
 
+  void setActivationCondition(Condition<R, P> activationCondition) {
+    this.activationCondition = activationCondition;
+  }
+
   public Optional<Condition<R, P>> getReadyPostcondition() {
     return Optional.ofNullable(readyPostcondition);
   }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/Workflow.java

@@ -42,4 +42,9 @@ default boolean isEmpty() {
   default Map<String, DependentResource> getDependentResourcesByName() {
     return Collections.emptyMap();
   }
+
+  @SuppressWarnings("rawtypes")
+  default Map<String, DependentResource> getDependentResourcesByNameWithoutActivationCondition() {
+    return Collections.emptyMap();
+  }
 }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowBuilder.java

@@ -58,6 +58,11 @@ public WorkflowBuilder<P> withDeletePostcondition(Condition deletePostcondition)
     return this;
   }
 
+  public WorkflowBuilder<P> withActivationCondition(Condition activationCondition) {
+    currentNode.setActivationCondition(activationCondition);
+    return this;
+  }
+
   DependentResourceNode getNodeByDependentResource(DependentResource<?, ?> dependentResource) {
     // first check by name
     final var node =

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowCleanupExecutor.java

@@ -68,11 +68,20 @@ protected void doRun(DependentResourceNode<R, P> dependentResourceNode,
         DependentResource<R, P> dependentResource) {
       var deletePostCondition = dependentResourceNode.getDeletePostcondition();
 
-      if (dependentResource.isDeletable()) {
+      var active =
+          isConditionMet(dependentResourceNode.getActivationCondition(), dependentResource);
+
+      if (dependentResource.isDeletable() && active) {
         ((Deleter<P>) dependentResource).delete(primary, context);
         deleteCalled.add(dependentResourceNode);
       }
-      boolean deletePostConditionMet = isConditionMet(deletePostCondition, dependentResource);
+
+      boolean deletePostConditionMet;
+      if (active) {
+        deletePostConditionMet = isConditionMet(deletePostCondition, dependentResource);
+      } else {
+        deletePostConditionMet = true;
+      }
       if (deletePostConditionMet) {
         markAsVisited(dependentResourceNode);
         handleDependentCleaned(dependentResourceNode);