Skip to content

more event filters #1309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 56 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
f07a014
feat: garbage collected interface (#1164)
csviri May 5, 2022
a84a1f1
fix: build
csviri May 9, 2022
e8ea4c9
Using Annotations to Identify primary for a secondary object if no ow…
csviri May 11, 2022
017058e
fix: build
csviri May 16, 2022
7da7c99
fix: format
csviri May 16, 2022
36983ef
fix: e2e test issue
csviri May 19, 2022
f2af7d9
fix: format
csviri May 19, 2022
49a1091
chore: change version to 3.1.0-SNAPSHOT (#1228)
csviri May 20, 2022
2f7f18a
Workflow engine implementation (#1153)
csviri May 25, 2022
80dc74e
refactor: rename JUnit extensions to be more explicit (#1254)
metacosm May 31, 2022
06434e2
fix: issues after rebase on main
csviri Jun 7, 2022
ca4eafd
feat: workflow Integration with API (dependent annotations, context) …
csviri Jun 9, 2022
8566672
fix: tests after rebase
csviri Jun 9, 2022
dea915b
fix: remove not used class
csviri Jun 9, 2022
487873a
fix: format
csviri Jun 9, 2022
1e4653b
feat: parallel start of additional event sources (#1284)
csviri Jun 15, 2022
fbc8c69
feat: retry and retry configuration decoupling (#1285)
csviri Jun 15, 2022
389ef97
fix: use synchronized instead reentrantlock in event processor (#1291)
csviri Jun 17, 2022
5822f72
feat: increase default thread number for executor service to 100 (#1293)
csviri Jun 17, 2022
a6676b4
fix: rebase on main
csviri Jun 17, 2022
d333758
feat: set default thread count to 10
csviri Jun 17, 2022
7d79cfc
fix: rebase on main
csviri Jun 27, 2022
64c7330
refactor: rename JUnit extensions to be more explicit (#1254)
metacosm May 31, 2022
43d846f
feat: workflow Integration with API (dependent annotations, context) …
csviri Jun 9, 2022
97fecff
feat: add filters to event sources
csviri Jun 20, 2022
b1ec502
predicates as filters
csviri Jun 20, 2022
5d563cf
wip
csviri Jun 20, 2022
413a21c
wip
csviri Jun 20, 2022
a00c111
wip
csviri Jun 21, 2022
dc97d5a
IT
csviri Jun 21, 2022
adb7888
unit test
csviri Jun 21, 2022
5252b12
wip
csviri Jun 21, 2022
9ebaea0
unit tests
csviri Jun 21, 2022
36c9692
wip
csviri Jun 21, 2022
0241210
filters for dependent resources
csviri Jun 21, 2022
bcf67ad
IT for dependent
csviri Jun 22, 2022
5ba87b6
javadocs
csviri Jun 22, 2022
c8b02cc
refactor: clean-up InformerEventSource constructors
metacosm Jun 23, 2022
c60d934
refactor: unify how filters are instantiated
metacosm Jun 24, 2022
193c6cb
docsfix
csviri Jun 24, 2022
876fabb
fix renaming caused problem
csviri Jun 24, 2022
124a15d
deprecated annotation
csviri Jun 24, 2022
afb0bb6
deprecations and remove unused class
csviri Jun 24, 2022
89c2c73
fix: generics for kube config
csviri Jun 24, 2022
5638d43
new internal event filters start
csviri Jun 24, 2022
ea095c8
removed old filters
csviri Jun 27, 2022
aba5d42
generic filter
csviri Jun 27, 2022
ba10b29
unit tests
csviri Jun 27, 2022
bd6e21c
controller unit test
csviri Jun 27, 2022
abb7332
tests
csviri Jun 27, 2022
b12b5ce
fix: problems with rebase
csviri Jun 27, 2022
4d71b66
refactor: rename more appropriately
metacosm Jun 30, 2022
7db8f8d
fix: incorrect error message
metacosm Jun 30, 2022
d615190
fix: incorrectly named parameter
metacosm Jun 30, 2022
e8d81e5
refactor: share filter handling (wip)
metacosm Jul 1, 2022
95ec9c1
refactor: simplify
metacosm Jul 1, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion micrometer-support/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>java-operator-sdk</artifactId>
<groupId>io.javaoperatorsdk</groupId>
<version>3.0.4-SNAPSHOT</version>
<version>3.1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
2 changes: 1 addition & 1 deletion operator-framework-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>io.javaoperatorsdk</groupId>
<artifactId>java-operator-sdk</artifactId>
<version>3.0.4-SNAPSHOT</version>
<version>3.1.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
package io.javaoperatorsdk.operator.api.config;

import java.lang.reflect.InvocationTargetException;
import java.time.Duration;
import java.util.Arrays;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.function.BiPredicate;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;

import io.fabric8.kubernetes.api.model.HasMetadata;
Expand All @@ -19,24 +22,30 @@
import io.javaoperatorsdk.operator.api.reconciler.Reconciler;
import io.javaoperatorsdk.operator.api.reconciler.dependent.Dependent;
import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResource;
import io.javaoperatorsdk.operator.api.reconciler.dependent.VoidCondition;
import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent;
import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource;
import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResourceConfig;
import io.javaoperatorsdk.operator.processing.dependent.workflow.Condition;
import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilter;
import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilters;
import io.javaoperatorsdk.operator.processing.event.source.filter.VoidGenericFilter;
import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnAddFilter;
import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnDeleteFilter;
import io.javaoperatorsdk.operator.processing.event.source.filter.VoidOnUpdateFilter;

import static io.javaoperatorsdk.operator.api.reconciler.Constants.DEFAULT_NAMESPACES_SET;

@SuppressWarnings("rawtypes")
public class AnnotationControllerConfiguration<R extends HasMetadata>
implements io.javaoperatorsdk.operator.api.config.ControllerConfiguration<R> {
public class AnnotationControllerConfiguration<P extends HasMetadata>
implements io.javaoperatorsdk.operator.api.config.ControllerConfiguration<P> {

protected final Reconciler<R> reconciler;
protected final Reconciler<P> reconciler;
private final ControllerConfiguration annotation;
private List<DependentResourceSpec> specs;
private Class<R> resourceClass;
private Class<P> resourceClass;

public AnnotationControllerConfiguration(Reconciler<R> reconciler) {
public AnnotationControllerConfiguration(Reconciler<P> reconciler) {
this.reconciler = reconciler;
this.annotation = reconciler.getClass().getAnnotation(ControllerConfiguration.class);
if (annotation == null) {
Expand Down Expand Up @@ -81,10 +90,10 @@ public Set<String> getNamespaces() {

@Override
@SuppressWarnings("unchecked")
public Class<R> getResourceClass() {
public Class<P> getResourceClass() {
if (resourceClass == null) {
resourceClass =
(Class<R>) Utils.getFirstTypeArgumentFromSuperClassOrInterface(reconciler.getClass(),
(Class<P>) Utils.getFirstTypeArgumentFromSuperClassOrInterface(reconciler.getClass(),
Reconciler.class);
}
return resourceClass;
Expand All @@ -102,16 +111,16 @@ public String getAssociatedReconcilerClassName() {

@SuppressWarnings("unchecked")
@Override
public ResourceEventFilter<R> getEventFilter() {
ResourceEventFilter<R> answer = null;
public ResourceEventFilter<P> getEventFilter() {
ResourceEventFilter<P> answer = null;

Class<ResourceEventFilter<R>>[] filterTypes =
(Class<ResourceEventFilter<R>>[]) valueOrDefault(annotation,
Class<ResourceEventFilter<P>>[] filterTypes =
(Class<ResourceEventFilter<P>>[]) valueOrDefault(annotation,
ControllerConfiguration::eventFilters, new Object[] {});
if (filterTypes.length > 0) {
for (var filterType : filterTypes) {
try {
ResourceEventFilter<R> filter = filterType.getConstructor().newInstance();
ResourceEventFilter<P> filter = filterType.getConstructor().newInstance();

if (answer == null) {
answer = filter;
Expand Down Expand Up @@ -141,17 +150,55 @@ public Optional<Duration> reconciliationMaxInterval() {
}
}

public static <T> T valueOrDefault(
ControllerConfiguration controllerConfiguration,
Function<ControllerConfiguration, T> mapper,
T defaultValue) {
if (controllerConfiguration == null) {
return defaultValue;
@Override
@SuppressWarnings("unchecked")
public Optional<Predicate<P>> onAddFilter() {
return (Optional<Predicate<P>>) createFilter(annotation.onAddFilter(), FilterType.onAdd,
annotation.getClass().getSimpleName());
}

private enum FilterType {
onAdd(VoidOnAddFilter.class), onUpdate(VoidOnUpdateFilter.class), onDelete(
VoidOnDeleteFilter.class), generic(VoidGenericFilter.class);

final Class<?> defaultValue;

FilterType(Class<?> defaultValue) {
this.defaultValue = defaultValue;
}
}

private <T> Optional<T> createFilter(Class<T> filter, FilterType filterType, String origin) {
if (filterType.defaultValue.equals(filter)) {
return Optional.empty();
} else {
return mapper.apply(controllerConfiguration);
try {
var instance = (T) filter.getDeclaredConstructor().newInstance();
return Optional.of(instance);
} catch (InstantiationException | IllegalAccessException | InvocationTargetException
| NoSuchMethodException e) {
throw new OperatorException(
"Couldn't create " + filterType + " filter from " + filter.getName() + " class in "
+ origin + " for reconciler " + getName(),
e);
}
}
}

@SuppressWarnings("unchecked")
@Override
public Optional<BiPredicate<P, P>> onUpdateFilter() {
return (Optional<BiPredicate<P, P>>) createFilter(annotation.onUpdateFilter(),
FilterType.onUpdate, annotation.getClass().getSimpleName());
}

@SuppressWarnings("unchecked")
@Override
public Optional<Predicate<P>> genericFilter() {
return (Optional<Predicate<P>>) createFilter(annotation.genericFilter(),
FilterType.generic, annotation.getClass().getSimpleName());
}

@SuppressWarnings({"rawtypes", "unchecked"})
@Override
public List<DependentResourceSpec> getDependentResources() {
Expand All @@ -172,19 +219,46 @@ public List<DependentResourceSpec> getDependentResources() {
}

final var name = getName(dependent, dependentType);
final var spec = specsMap.get(name);
var spec = specsMap.get(name);
if (spec != null) {
throw new IllegalArgumentException(
"A DependentResource named: " + name + " already exists: " + spec);
}
specsMap.put(name, new DependentResourceSpec(dependentType, config, name));
spec = new DependentResourceSpec(dependentType, config, name);
spec.setDependsOn(Set.of(dependent.dependsOn()));
addConditions(spec, dependent);
specsMap.put(name, spec);
}

specs = specsMap.values().stream().collect(Collectors.toUnmodifiableList());
}
return specs;
}

@SuppressWarnings("unchecked")
private void addConditions(DependentResourceSpec spec, Dependent dependent) {
if (dependent.deletePostcondition() != VoidCondition.class) {
spec.setDeletePostCondition(instantiateCondition(dependent.deletePostcondition()));
}
if (dependent.readyPostcondition() != VoidCondition.class) {
spec.setReadyPostcondition(instantiateCondition(dependent.readyPostcondition()));
}
if (dependent.reconcilePrecondition() != VoidCondition.class) {
spec.setReconcilePrecondition(instantiateCondition(dependent.reconcilePrecondition()));
}
}

private Condition<?, ?> instantiateCondition(Class<? extends Condition> condition) {
try {
return condition.getDeclaredConstructor().newInstance();
} catch (InstantiationException
| IllegalAccessException
| InvocationTargetException
| NoSuchMethodException e) {
throw new OperatorException(e);
}
}

private String getName(Dependent dependent, Class<? extends DependentResource> dependentType) {
var name = dependent.name();
if (name.isBlank()) {
Expand All @@ -193,26 +267,55 @@ private String getName(Dependent dependent, Class<? extends DependentResource> d
return name;
}

@SuppressWarnings("rawtypes")
private Object createKubernetesResourceConfig(Class<? extends DependentResource> dependentType) {

Object config;
final var kubeDependent = dependentType.getAnnotation(KubernetesDependent.class);

var namespaces = getNamespaces();
var configuredNS = false;
if (kubeDependent != null && !Arrays.equals(KubernetesDependent.DEFAULT_NAMESPACES,
kubeDependent.namespaces())) {
namespaces = Set.of(kubeDependent.namespaces());
configuredNS = true;
}

String labelSelector = null;
Predicate<? extends HasMetadata> onAddFilter = null;
BiPredicate<? extends HasMetadata, ? extends HasMetadata> onUpdateFilter = null;
BiPredicate<? extends HasMetadata, Boolean> onDeleteFilter = null;
if (kubeDependent != null) {
if (!Arrays.equals(KubernetesDependent.DEFAULT_NAMESPACES,
kubeDependent.namespaces())) {
namespaces = Set.of(kubeDependent.namespaces());
configuredNS = true;
}

final var fromAnnotation = kubeDependent.labelSelector();
labelSelector = Constants.NO_VALUE_SET.equals(fromAnnotation) ? null : fromAnnotation;

final var kubeDependentName = KubernetesDependent.class.getSimpleName();
onAddFilter = createFilter(kubeDependent.onAddFilter(), FilterType.onAdd, kubeDependentName)
.orElse(null);
onUpdateFilter =
createFilter(kubeDependent.onUpdateFilter(), FilterType.onUpdate, kubeDependentName)
.orElse(null);
onDeleteFilter =
createFilter(kubeDependent.onDeleteFilter(), FilterType.onDelete, kubeDependentName)
.orElse(null);
}

config =
new KubernetesDependentResourceConfig(namespaces, labelSelector, configuredNS);
new KubernetesDependentResourceConfig(namespaces, labelSelector, configuredNS, onAddFilter,
onUpdateFilter, onDeleteFilter);

return config;
}

public static <T> T valueOrDefault(
ControllerConfiguration controllerConfiguration,
Function<ControllerConfiguration, T> mapper,
T defaultValue) {
if (controllerConfiguration == null) {
return defaultValue;
} else {
return mapper.apply(controllerConfiguration);
}
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ default boolean checkCRDAndValidateLocalModel() {
return false;
}

int DEFAULT_RECONCILIATION_THREADS_NUMBER = 5;
int DEFAULT_RECONCILIATION_THREADS_NUMBER = 10;

/**
* Retrieves the maximum number of threads the operator can spin out to dispatch reconciliation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,8 @@
* to the ConfigurationService is via the reconciliation context.
*/
public class ConfigurationServiceProvider {
static final ConfigurationService DEFAULT =
new BaseConfigurationService(Utils.loadFromProperties());
private static ConfigurationService instance;
private static ConfigurationService defaultConfigurationService = DEFAULT;
private static ConfigurationService defaultConfigurationService = createDefault();
private static boolean alreadyConfigured = false;

private ConfigurationServiceProvider() {}
Expand Down Expand Up @@ -64,8 +62,12 @@ synchronized static ConfigurationService getDefault() {
}

public synchronized static void reset() {
defaultConfigurationService = DEFAULT;
defaultConfigurationService = createDefault();
instance = null;
alreadyConfigured = false;
}

static ConfigurationService createDefault() {
return new BaseConfigurationService(Utils.loadFromProperties());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
import io.javaoperatorsdk.operator.api.config.dependent.DependentResourceSpec;
import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilter;
import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilters;
import io.javaoperatorsdk.operator.processing.retry.GenericRetry;
import io.javaoperatorsdk.operator.processing.retry.Retry;

public interface ControllerConfiguration<R extends HasMetadata> extends ResourceConfiguration<R> {

Expand All @@ -27,6 +29,16 @@ default boolean isGenerationAware() {

String getAssociatedReconcilerClassName();

default Retry getRetry() {
return GenericRetry.fromConfiguration(getRetryConfiguration()); // NOSONAR
}

/**
* Use getRetry instead.
*
* @return configuration for retry.
*/
@Deprecated
default RetryConfiguration getRetryConfiguration() {
return RetryConfiguration.DEFAULT;
}
Expand Down
Loading