Workflows for Dependent Resource - Umbrella Issue

[csviri]

DependentResource serves as a building block of workflows of reconciling resources. Currently the managed dependent resources provides a limited way of expressing workflows - it's a list of resources what gets reconciled in defined order. Using standalone dependent resource in a reconciliation gives developers possibilities to use any java language constructs to describe arbitrary workflows (Think only conditionally create resource based in input resource values, or wait with API calls on a service until the deployment is not ready and things like that).

TODO:

• Workflow design and related implementation in: Depends On and Conditions to describe workflows Dependent Resources #850
• Define workflows with dependent resource annotations #1241
• Approach to exception handling in Dependent Resource reconcile()  #1126
• How fail fast / non fail fast approach should work with workflows, see also: Feature flag for fail fast behavior for Managed Dependent Resources #1107
• DependentResource and Workflow Design Details Questions #1263
• Documentation for Workflows #1262

[csviri]

to discuss: workflows should handle explicit delete if a condition not hold ?

This is actually a case to differentiate cleaner and deleter for the dependent resources, since it might make sense to explicitly delete a resource like for Kubernetes but not handle the cleanup explicitly. However this might be a special case, since only k8s handles gabrbage collection no other external resources.

cc @metacosm

[csviri]

Regarding workflows, I think this issue should fully cover it: #850

Will be simple enough, can be nicely describe by annotation,
will be familiar for people working with Terraform or Cloudformation.

[csviri]

Added the topic of error handling, there was a discussion if in dependent resources reconciliation (reconcile(...) method call) should throw Exception or in the resource there should be and Error status in the result enum. Probably this part deserves it's own issue (will create one).

[cmoulliard]

That should be great to have part of the description an example. If the idea is to express like a from-to-to route, the flow of the dependent resources to be processed and their order, then adopting a DSL language could be a better idea than using Java annotations

Example: https://camel.apache.org/manual/java-dsl.html

WDYT ?@csviri

[csviri]

That should be great to have part of the description an example. If the idea is to express like a from-to-to route, the flow of the dependent resources to be processed and their order, then adopting a DSL language could be a better idea than using Java annotations

Example: https://camel.apache.org/manual/java-dsl.html

WDYT ?@csviri

yes, that is a very good point, luckily we have actually both, see here in tests:
https://github.com/java-operator-sdk/java-operator-sdk/blob/4762be50af11fc0d085a3da635201b0074ea568f/operator-framework-core/src/test/java/io/javaoperatorsdk/operator/processing/dependent/workflow/WorkflowReconcileExecutorTest.java#L244-L252

or in a sample:
https://github.com/java-operator-sdk/java-operator-sdk/blob/4762be50af11fc0d085a3da635201b0074ea568f/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageDependentsWorkflowReconciler.java#L46-L51

(it's actually a builder both in this context those are very similar concepts)

so the annotation is just a wrapper on an inner java dsl / builder. In other words in the backround the annoations will be transformed to the target model using this DSL. But both can be used at the end in an operator.