feat(docs): New Reference documentation section

deploy/helm/druid-operator/crds/crds.yaml

@@ -26,7 +26,7 @@ spec:
               description: A Druid cluster stacklet. This resource is managed by the Stackable operator for Apache Druid. Find more information on how to use it and the resources that the operator generates in the [operator documentation](https://docs.stackable.tech/home/nightly/druid/).
               properties:
                 brokers:
-                  description: Configuration of the broker role.
+                  description: This struct represents a role - e.g. HDFS datanodes or Trino workers. It has a [`HashMap`] containing all the roleGroups that are part of this role. Additionally, there is a `config`, which is configurable at the role *and* roleGroup level. Everything at roleGroup level is merged on top of what is configured on role level using the [`Merge`] trait. There is also a second form of config, which can only be configured at role level, the `roleConfig`.
                   properties:
                     cliOverrides:
                       additionalProperties:
@@ -8389,7 +8389,7 @@ spec:
                   default:
                     stopped: false
                     reconciliationPaused: false
-                  description: Cluster operations like pause reconciliation or cluster stop.
+                  description: '[Cluster operations](https://docs.stackable.tech/home/nightly/concepts/operations/cluster_operations) properties, allow stopping the product instance as well as pausing reconciliation.'
                   properties:
                     reconciliationPaused:
                       default: false
@@ -8401,7 +8401,7 @@ spec:
                       type: boolean
                   type: object
                 coordinators:
-                  description: Configuration of the coordinator role.
+                  description: This struct represents a role - e.g. HDFS datanodes or Trino workers. It has a [`HashMap`] containing all the roleGroups that are part of this role. Additionally, there is a `config`, which is configurable at the role *and* roleGroup level. Everything at roleGroup level is merged on top of what is configured on role level using the [`Merge`] trait. There is also a second form of config, which can only be configured at role level, the `roleConfig`.
                   properties:
                     cliOverrides:
                       additionalProperties:
@@ -15318,7 +15318,7 @@ spec:
                     - roleGroups
                   type: object
                 historicals:
-                  description: Configuration of the historical role.
+                  description: This struct represents a role - e.g. HDFS datanodes or Trino workers. It has a [`HashMap`] containing all the roleGroups that are part of this role. Additionally, there is a `config`, which is configurable at the role *and* roleGroup level. Everything at roleGroup level is merged on top of what is configured on role level using the [`Merge`] trait. There is also a second form of config, which can only be configured at role level, the `roleConfig`.
                   properties:
                     cliOverrides:
                       additionalProperties:
@@ -22303,7 +22303,10 @@ spec:
                         - productVersion
                     - required:
                         - productVersion
-                  description: Specify which image to use, the easiest way is to only configure the `productVersion`, it needs to be one of the [supported versions](https://docs.stackable.tech/home/nightly/druid/#_supported_versions). You can also configure a custom image registry to pull from, as well as completely custom images. Consult the [Product image selection documentation](https://docs.stackable.tech/home/nightly/concepts/product_image_selection) for details.
+                  description: |-
+                    Specify which image to use, the easiest way is to only configure the `productVersion`. You can also configure a custom image registry to pull from, as well as completely custom images.
+
+                    Consult the [Product image selection documentation](https://docs.stackable.tech/home/nightly/concepts/product_image_selection) for details.
                   properties:
                     custom:
                       description: Overwrite the docker image. Specify the full docker image name, e.g. `docker.stackable.tech/stackable/superset:1.4.1-stackable2.1.0`
@@ -22340,7 +22343,7 @@ spec:
                       type: string
                   type: object
                 middleManagers:
-                  description: Configuration of the middle managed role.
+                  description: This struct represents a role - e.g. HDFS datanodes or Trino workers. It has a [`HashMap`] containing all the roleGroups that are part of this role. Additionally, there is a `config`, which is configurable at the role *and* roleGroup level. Everything at roleGroup level is merged on top of what is configured on role level using the [`Merge`] trait. There is also a second form of config, which can only be configured at role level, the `roleConfig`.
                   properties:
                     cliOverrides:
                       additionalProperties:
@@ -29257,7 +29260,7 @@ spec:
                     - roleGroups
                   type: object
                 routers:
-                  description: Configuration of the router role.
+                  description: This struct represents a role - e.g. HDFS datanodes or Trino workers. It has a [`HashMap`] containing all the roleGroups that are part of this role. Additionally, there is a `config`, which is configurable at the role *and* roleGroup level. Everything at roleGroup level is merged on top of what is configured on role level using the [`Merge`] trait. There is also a second form of config, which can only be configured at role level, the `roleConfig`.
                   properties:
                     cliOverrides:
                       additionalProperties:

docs/modules/druid/pages/index.adoc

@@ -31,7 +31,7 @@ the Operator creates a **Service**.
 
 A **ConfigMap** is created for each RoleGroup containing 3 files: `jvm.config` and `runtime.properties` files generated
 from the DruidCluster configuration (See xref:usage-guide/index.adoc[] for more information), plus a `log4j2.properties`
-file used for xref:usage-guide/logging.adoc[]. For the whole DruidCluster a **xref:discovery.adoc[discovery ConfigMap]**
+file used for xref:usage-guide/logging.adoc[]. For the whole DruidCluster a **xref:reference/discovery.adoc[discovery ConfigMap]**
 is created which contains information on how to connect to the Druid cluster.
 
 == Dependencies and other Operators to connect to

docs/modules/druid/pages/commandline_args.adoc → docs/modules/druid/pages/reference/commandline-parameters.adoc

@@ -1,5 +1,8 @@
+= Command Line parameters
 
-=== product-config
+This operator accepts the following command line parameters:
+
+== product-config
 
 *Default value*: `/etc/stackable/druid-operator/config-spec/properties.yaml`
 
@@ -12,7 +15,7 @@
 stackable-druid-operator run --product-config /foo/bar/properties.yaml
 ----
 
-=== watch-namespace
+== watch-namespace
 
 *Default value*: All namespaces
 

docs/modules/druid/pages/reference/crds.adoc

@@ -0,0 +1,3 @@
+= CRD Reference
+
+Find all CRD reference for the Stackable Operator for Apache Druid at: {crd-docs-base-url}/druid-operator/{crd-docs-version}.

docs/modules/druid/pages/reference/discovery.adoc

@@ -0,0 +1,32 @@
+= Discovery ConfigMap
+:description: Reference documentation for the discovery ConfigMap generated by the Stackable Operator for Apache Druid.
+:keywords: reference, discovery, ConfigMap, connection
+:page-aliases: discovery.adoc
+:clusterName: simple-derby-druid
+:namespace: stackable
+:routerPort: 8888
+
+The Stackable Operator for Druid publishes a xref:concepts:service_discovery.adoc[] with the following properties,
+where `{clusterName}` represents the name and `{namespace}` the namespace of the cluster:
+
+`DRUID_AVATICA_JDBC`::
+====
+Contains the connection string for the https://calcite.apache.org/avatica/downloads/[Avatica] JDBC driver to connect to the router:
+[subs="normal"]
+  jdbc:avatica:remote:url=http://{clusterName}-router.{namespace}.svc.cluster.local:{routerPort}/druid/v2/sql/avatica/
+====
+
+`DRUID_SQLALCHEMY`::
+====
+Contains the connection string for the python SQLAlchemy toolkit:
+
+[subs="normal"]
+  druid://{clusterName}-router.{namespace}.svc.cluster.local:{routerPort}/druid/v2/sql
+====
+
+`DRUID_ROUTER`::
+====
+Contains the connection string for the Druid router:
+[subs="normal"]
+  {clusterName}-router.{namespace}.svc.cluster.local:{routerPort}
+====

docs/modules/druid/pages/env_var_args.adoc → docs/modules/druid/pages/reference/environment-variables.adoc

@@ -1,5 +1,8 @@
+= Environment variables
 
-=== PRODUCT_CONFIG
+This operator accepts the following environment variables:
+
+== PRODUCT_CONFIG
 
 *Default value*: `/etc/stackable/druid-operator/config-spec/properties.yaml`
 
@@ -25,7 +28,7 @@ docker run \
     docker.stackable.tech/stackable/druid-operator:latest
 ----
 
-=== WATCH_NAMESPACE
+== WATCH_NAMESPACE
 
 *Default value*: All namespaces
 
@@ -53,4 +56,3 @@ docker run \
 --mount type=bind,source="$HOME/.kube/config",target="/home/stackable/.kube/config" \
 docker.stackable.tech/stackable/druid-operator:latest
 ----
-

docs/modules/druid/pages/reference/index.adoc

@@ -0,0 +1,7 @@
+= Reference
+
+Consult the reference documentation section to find exhaustive information on:
+
+* Descriptions and default values of all properties in the CRDs used by this operator in the xref:reference/crds.adoc[].
+* The properties in the xref:reference/discovery.adoc[].
+* The xref:reference/commandline-parameters.adoc[] and xref:reference/environment-variables.adoc[] accepted by the operator.

docs/modules/druid/partials/nav.adoc

@@ -1,15 +1,13 @@
 * xref:druid:getting_started/index.adoc[]
 ** xref:druid:getting_started/installation.adoc[]
 ** xref:druid:getting_started/first_steps.adoc[]
-* xref:druid:configuration.adoc[]
 * xref:druid:required-external-components.adoc[]
 * xref:druid:usage-guide/index.adoc[]
 ** xref:druid:usage-guide/listenerclass.adoc[]
 ** xref:druid:usage-guide/ingestion.adoc[]
 ** xref:druid:usage-guide/deep-storage.adoc[]
 ** xref:druid:usage-guide/resources-and-storage.adoc[]
 ** xref:druid:usage-guide/security.adoc[]
-** xref:druid:usage-guide/discovery.adoc[]
 ** xref:druid:usage-guide/logging.adoc[]
 ** xref:druid:usage-guide/monitoring.adoc[]
 ** xref:druid:usage-guide/configuration-and-environment-overrides.adoc[]
@@ -18,3 +16,9 @@
 *** xref:druid:usage-guide/operations/pod-placement.adoc[]
 *** xref:druid:usage-guide/operations/pod-disruptions.adoc[]
 *** xref:druid:usage-guide/operations/graceful-shutdown.adoc[]
+* xref:druid:reference/index.adoc[]
+** xref:druid:reference/crds.adoc[]
+*** {crd-docs}/druid.stackable.tech/druidcluster/v1alpha1/[DruidCluster]
+** xref:druid:reference/discovery.adoc[]
+** xref:druid:reference/commandline-parameters.adoc[]
+** xref:druid:reference/environment-variables.adoc[]

rust/crd/src/lib.rs

@@ -189,26 +189,28 @@ pub enum Error {
 )]
 #[serde(rename_all = "camelCase")]
 pub struct DruidClusterSpec {
-    /// Specify which image to use, the easiest way is to only configure the `productVersion`,
-    /// it needs to be one of the [supported versions](DOCS_BASE_URL_PLACEHOLDER/druid/#_supported_versions).
-    /// You can also configure a custom image registry to pull from, as well as completely custom
-    /// images. Consult the
-    /// [Product image selection documentation](DOCS_BASE_URL_PLACEHOLDER/concepts/product_image_selection)
-    /// for details.
+    /// Common cluster wide configuration that can not differ or be overridden on a role or role group level.
+    pub cluster_config: DruidClusterConfig,
+
+    // no doc - docs provided by the struct.
     pub image: ProductImage,
-    /// Configuration of the broker role.
+
+    // no doc - docs provided by the struct.
     pub brokers: Role<BrokerConfigFragment>,
-    /// Configuration of the coordinator role.
+
+    // no doc - docs provided by the struct.
     pub coordinators: Role<CoordinatorConfigFragment>,
-    /// Configuration of the historical role.
+
+    // no doc - docs provided by the struct.
     pub historicals: Role<HistoricalConfigFragment>,
-    /// Configuration of the middle managed role.
+
+    // no doc - docs provided by the struct.
     pub middle_managers: Role<MiddleManagerConfigFragment>,
-    /// Configuration of the router role.
+
+    // no doc - docs provided by the struct.
     pub routers: Role<RouterConfigFragment>,
-    /// Common cluster wide configuration that can not differ or be overridden on a role or role group level.
-    pub cluster_config: DruidClusterConfig,
-    /// Cluster operations like pause reconciliation or cluster stop.
+
+    // no doc - docs provided by the struct.
     #[serde(default)]
     pub cluster_operation: ClusterOperation,
 }
@@ -245,39 +247,48 @@ pub struct DruidClusterConfig {
     /// as the SecretClass used for internal communication.
     #[serde(default)]
     pub authentication: Vec<DruidAuthentication>,
+
     /// Authorization settings for Druid like OPA
     #[serde(skip_serializing_if = "Option::is_none")]
     pub authorization: Option<DruidAuthorization>,
+
     /// [Druid deep storage configuration](DOCS_BASE_URL_PLACEHOLDER/druid/usage-guide/deep-storage).
     /// Only one backend can be used at a time. Either HDFS or S3 are supported.
     pub deep_storage: DeepStorageSpec,
+
     /// Configuration properties for data ingestion tasks.
     #[serde(skip_serializing_if = "Option::is_none")]
     pub ingestion: Option<IngestionSpec>,
+
     /// Druid requires an SQL database to store metadata into. Specify connection information here.
     pub metadata_storage_database: DatabaseConnectionSpec,
+
     /// TLS encryption settings for Druid, more information in the
     /// [security documentation](DOCS_BASE_URL_PLACEHOLDER/druid/usage-guide/security).
     /// This setting only affects server and internal communication.
     /// It does not affect client tls authentication, use `clusterConfig.authentication` instead.
     #[serde(default = "default_druid_tls", skip_serializing_if = "Option::is_none")]
     pub tls: Option<DruidTls>,
+
     /// Druid requires a ZooKeeper cluster connection to run.
     /// Provide the name of the ZooKeeper [discovery ConfigMap](DOCS_BASE_URL_PLACEHOLDER/concepts/service_discovery)
     /// here. When using the [Stackable operator for Apache ZooKeeper](DOCS_BASE_URL_PLACEHOLDER/zookeeper/)
     /// to deploy a ZooKeeper cluster, this will simply be the name of your ZookeeperCluster resource.
     pub zookeeper_config_map_name: String,
+
     /// Name of the Vector aggregator [discovery ConfigMap](DOCS_BASE_URL_PLACEHOLDER/concepts/service_discovery).
     /// It must contain the key `ADDRESS` with the address of the Vector aggregator.
     /// Follow the [logging tutorial](DOCS_BASE_URL_PLACEHOLDER/tutorials/logging-vector-aggregator)
     /// to learn how to configure log aggregation with Vector.
     #[serde(skip_serializing_if = "Option::is_none")]
     pub vector_aggregator_config_map_name: Option<String>,
+
     /// Extra volumes to mount into every container, this can be useful to for example make client
     /// certificates, keytabs or similar things available to processors
     /// These volumes will be mounted into all pods below `/stackable/userdata/{volumename}`
     #[serde(default, skip_serializing_if = "Vec::is_empty")]
     pub extra_volumes: Vec<Volume>,
+
     /// This field controls which type of Service the Operator creates for this DruidCluster:
     ///
     /// * `cluster-internal`: Use a ClusterIP service