CLOUDP-318518: Community private-preview docs (#102)

# Summary

This PR provides testable code snippets for deploying MongoDBCommunity
with MongoDBSearch.

Code snippets are created in the same way as the reference architecture
ones, but located under docs/community/quick-start. The difference is
that the article and snippets are assuming existence of the k8s cluster
and for e2e tests kind cluster is used as in other e2e tests.

**Important for code review**: for now, this PR contains two commits
that are to be included in search-main branch separately, but are
necessary for this work to pass the tests. To review, please view only
commits excluding two first commits ("Temporary private quay search
image", "Handle pull secret for community search private preview"):
[link to
diff](775908a)

## File structure

- `docs/community/quick-start` - main community search snippets
directory, intended for end-users to look into:
  With the following files: 
  - `code_snippets/` - shell scripts
  - `output/` - outputs gathered from e2e test run
  - `env_variables.sh` - env variables intended for end-users to adjust
- `env_variables_e2e_private.sh` - env variables adjusted for a
"e2e_private" flavor (running in e2e test so with additional helm chart
variables for specifying images built in pipeline).
  - `test.sh` - automated execution entry point defining all the steps
- `README.md` - rendered, final docs article intended for end-user
consumption. It will be immediately rendered on page upon entering
`docs/community/quick-start` folder in GH UI.
- `README.md.j2` - the source of the docs article, with jinja's include
directives to include snippets in markdown's code blocks. It is not
possible to include files directly in markdown files.
-
`scripts/code_snippets/kind_community_search_snippets_render_template.sh`
- render `README.md.j2` into `README.md`
- `scripts/code_snippets/task_kind_community_search_snippets_test.sh` -
entrypoint for running in e2e test. It's using `CODE_SNIPPETS_FLAVOR`
env var (defined in the context file) for choosing the flavor (sourcing
`env_variables${CODE_SNIPPETS_FLAVOR}.sh` file)

Code snippets are hooked into PR tests under
`private_kind_code_snippets` variant. "Public" variant will be added as
a followup. Snippets in e2e tests are executed by


## General changes to snippets and reference architectures

Additionally, some changes were made to GKE code snipets for Reference
Architectures:
* all e2e entrypoint scripts were renamed with a prefix `test_`, to
quickly identify which of those scripts are really entrypoints:
  * task_gke_multi_cluster_no_mesh_snippets_test.sh
  * task_gke_multi_cluster_snippets_test.sh
  * task_kind_community_search_snippets_test.sh 
* All snippets tasks in evergreen uses one `test_code_snippets`
evergreen function, that is executing entrypoint by naming convention
`./scripts/code_snippets/${task_name}_test.sh`.
* **Note:** using `CODE_SNIPPETS_FLAVOR` env var and defining flavor's
env vars directly in the snippets directory
(`env_variables${CODE_SNIPPETS_FLAVOR}.sh) makes a different mechanism
than what we have in reference architectures. We should decide which
mechanism is a better one and unify.

The documentation article is provided
[README.md](docs/community/quick-start/README.md) and is rendered from
the README.md.j2

## Proof of Work
* Green code snippets e2e tests (including GKE-based reference
architectures):
[link](https://spruce.mongodb.com/version/68210e1e8f1de0000798909a/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC)
* Rendered markdown file, visible when entering the folder in UI:
[link](https://github.com/mongodb/mongodb-kubernetes/tree/lsierant/search-docs/docs/community-search/quick-start)

## Checklist
- [x] Have you linked a jira ticket and/or is the ticket in the title?
- [x] Have you checked whether your jira ticket required DOCSP changes?
- [x] Have you checked for release_note changes?

---------

Co-authored-by: Yavor Georgiev <[email protected]>
Co-authored-by: Simon Bäumer <[email protected]>

Author: 3 people

.evergreen-functions.yml

@@ -333,6 +333,14 @@ functions:
           source .generated/context.export.env
           scripts/evergreen/e2e/setup_cloud_qa.py delete
 
+  dump_diagnostic_information_from_all_namespaces:
+    - command: subprocess.exec
+      params:
+        working_dir: src/github.com/mongodb/mongodb-kubernetes
+        add_to_path:
+          - bin
+        binary: scripts/evergreen/e2e/dump_diagnostic_information_from_all_namespaces.sh
+
   ### Publish and release image ###
 
   # Tags and pushes an image into an external Docker registry. The source image
@@ -733,25 +741,12 @@ functions:
       params:
         include_expansions_in_env:
           - GH_TOKEN
+          - code_snippets_commit_output
         working_dir: src/github.com/mongodb/mongodb-kubernetes
         binary: scripts/code_snippets/sample_commit_output.sh
 
-  gke_multi_cluster_snippets:
-    - *switch_context
-    - command: shell.exec
-      params:
-        shell: bash
-        working_dir: src/github.com/mongodb/mongodb-kubernetes
-        include_expansions_in_env:
-          - version_id
-          - code_snippets_teardown
-          - code_snippets_reset
-          - task_name
-        script: |
-          ./scripts/code_snippets/gke_multi_cluster_test.sh
-
-
-  gke_multi_cluster_no_mesh_snippets:
+  # it executes a script by convention: ./scripts/code_snippets/${task_name}_test.sh
+  test_code_snippets:
     - *switch_context
     - command: shell.exec
       params:
@@ -763,4 +758,4 @@ functions:
           - code_snippets_reset
           - task_name
         script: |
-          ./scripts/code_snippets/gke_multi_cluster_no_mesh_test.sh
+          ./scripts/code_snippets/${task_name}_test.sh

.evergreen-tasks.yml

@@ -70,16 +70,24 @@ tasks:
         vars:
           image_name: ops-manager
 
-  - name: gke_multi_cluster_snippets
+# Code snippets tasks
+# Each task is selected by convention by running scripts/code_snippets/${task_name}_test.sh
+  - name: task_gke_multi_cluster_snippets
     tags: [ "code_snippets" ]
     commands:
-      - func: gke_multi_cluster_snippets
+      - func: test_code_snippets
       - func: sample_commit_output
 
-  - name: gke_multi_cluster_no_mesh_snippets
+  - name: task_gke_multi_cluster_no_mesh_snippets
     tags: [ "code_snippets" ]
     commands:
-      - func: gke_multi_cluster_no_mesh_snippets
+      - func: test_code_snippets
+      - func: sample_commit_output
+
+  - name: task_kind_community_search_snippets
+    tags: [ "code_snippets", "patch-run" ]
+    commands:
+      - func: test_code_snippets
       - func: sample_commit_output
 
 ## Below are only e2e runs for .evergreen.yml ##

.evergreen.yml

@@ -86,6 +86,7 @@ variables:
       - func: build_multi_cluster_binary
 
   - &setup_and_teardown_group_gke_code_snippets
+    setup_task_can_fail_task: true
     setup_group:
       - func: clone
       - func: setup_gcloud_cli
@@ -95,6 +96,18 @@ variables:
     teardown_group:
       - func: upload_code_snippets_logs
 
+  - &setup_and_teardown_group_kind_code_snippets
+    setup_task_can_fail_task: true
+    setup_group:
+      - func: clone
+      - func: cleanup_exec_environment
+      - func: download_kube_tools
+      - func: configure_docker_auth
+      - func: setup_kubernetes_environment
+    teardown_task:
+      - func: upload_e2e_logs
+      - func: upload_code_snippets_logs
+
   - &setup_and_teardown_task_cloudqa
     setup_task_can_fail_task: true
     setup_task:
@@ -183,6 +196,10 @@ parameters:
     value: "false"
     description: set this to true if you would like to delete the resources created in the code snippet tests, but keep the clusters
 
+  - key: code_snippets_commit_output
+    value: "false"
+    description: set this to true if you would like the pipeline to automatically push a branch with updated snippets outputs
+
 
 # Triggered manually or by PCT.
 patch_aliases:
@@ -608,8 +625,14 @@ task_groups:
     <<: *setup_and_teardown_group_gke_code_snippets
     max_hosts: -1
     tasks:
-      - gke_multi_cluster_snippets
-      - gke_multi_cluster_no_mesh_snippets
+      - task_gke_multi_cluster_snippets
+      - task_gke_multi_cluster_no_mesh_snippets
+
+  - name: kind_code_snippets_task_group
+    <<: *setup_and_teardown_group_kind_code_snippets
+    max_hosts: -1
+    tasks:
+      - task_kind_community_search_snippets
 
   # Task group for deploying mongodbcommunity resources and testing the (former) MCO
   - name: e2e_mdb_community_task_group
@@ -1837,6 +1860,15 @@ buildvariants:
     tasks:
       - name: gke_code_snippets_task_group
 
+  - name: private_kind_code_snippets
+    display_name: private_kind_code_snippets
+    allowed_requesters: ["patch"]
+    run_on:
+      - ubuntu2204-large
+    <<: *base_om8_dependency
+    tasks:
+      - name: kind_code_snippets_task_group
+
   ### Build variants for manual patch only
 
   - name: publish_om60_images

.gitignore

@@ -91,3 +91,6 @@ docker/mongodb-kubernetes-tests/.test_identifiers*
 logs-debug/
 /ssdlc-report/*
 .gocache/
+
+docs/**/log/*
+docs/**/test.sh.run.log

RELEASE_NOTES.md

@@ -7,7 +7,7 @@
 
 * **MongoDBSearch (Community Private Preview)**: Added support for deploying MongoDB Search (Community Private Preview Edition) that enables full-text and vector search capabilities for MongoDBCommunity deployments.
   * Added new MongoDB CRD which is watched by default by the operator.
-    * For more information please see: `TBD LINK TO DOC MARKDOWN`
+    * For more information please see: [docs/community-search/quick-start/README.md](docs/community-search/quick-start/README.md)
   * Private Preview phase comes with some limitations:
     * minimum MongoDB Community version: 8.0.
     * TLS must be disabled in MongoDB (communication between mongot and mongod is in plaintext for now).