CLOUDP-318518: Community private-preview docs #102

lsierant · 2025-05-09T22:44:19Z

Summary

This PR provides testable code snippets for deploying MongoDBCommunity with MongoDBSearch.

Code snippets are created in the same way as the reference architecture ones, but located under docs/community/quick-start. The difference is that the article and snippets are assuming existence of the k8s cluster and for e2e tests kind cluster is used as in other e2e tests.

Important for code review: for now, this PR contains two commits that are to be included in search-main branch separately, but are necessary for this work to pass the tests. To review, please view only commits excluding two first commits ("Temporary private quay search image", "Handle pull secret for community search private preview"): link to diff

File structure

docs/community/quick-start - main community search snippets directory, intended for end-users to look into:
With the following files:
- code_snippets/ - shell scripts
- output/ - outputs gathered from e2e test run
- env_variables.sh - env variables intended for end-users to adjust
- env_variables_e2e_private.sh - env variables adjusted for a "e2e_private" flavor (running in e2e test so with additional helm chart variables for specifying images built in pipeline).
- test.sh - automated execution entry point defining all the steps
- README.md - rendered, final docs article intended for end-user consumption. It will be immediately rendered on page upon entering docs/community/quick-start folder in GH UI.
- README.md.j2 - the source of the docs article, with jinja's include directives to include snippets in markdown's code blocks. It is not possible to include files directly in markdown files.
scripts/code_snippets/kind_community_search_snippets_render_template.sh - render README.md.j2 into README.md
scripts/code_snippets/task_kind_community_search_snippets_test.sh - entrypoint for running in e2e test. It's using CODE_SNIPPETS_FLAVOR env var (defined in the context file) for choosing the flavor (sourcing env_variables${CODE_SNIPPETS_FLAVOR}.sh file)

Code snippets are hooked into PR tests under private_kind_code_snippets variant. "Public" variant will be added as a followup. Snippets in e2e tests are executed by

General changes to snippets and reference architectures

Additionally, some changes were made to GKE code snipets for Reference Architectures:

all e2e entrypoint scripts were renamed with a prefix test_, to quickly identify which of those scripts are really entrypoints:
- task_gke_multi_cluster_no_mesh_snippets_test.sh
- task_gke_multi_cluster_snippets_test.sh
- task_kind_community_search_snippets_test.sh
All snippets tasks in evergreen uses one test_code_snippets evergreen function, that is executing entrypoint by naming convention ./scripts/code_snippets/${task_name}_test.sh.
- Note: using CODE_SNIPPETS_FLAVOR env var and defining flavor's env vars directly in the snippets directory (`env_variables${CODE_SNIPPETS_FLAVOR}.sh) makes a different mechanism than what we have in reference architectures. We should decide which mechanism is a better one and unify.

The documentation article is provided README.md and is rendered from the README.md.j2

Proof of Work

Green code snippets e2e tests (including GKE-based reference architectures): link
Rendered markdown file, visible when entering the folder in UI: link

Checklist

Have you linked a jira ticket and/or is the ticket in the title?
Have you checked whether your jira ticket required DOCSP changes?
Have you checked for release_note changes?

fealebenpae

Looks excellent! I just noticed a couple of render errors in the preview markdown and suggested fixes for what looks like a few minor copy/paste naming errors.

docs/community-search/quick-start/README.md.j2

docs/community-search/quick-start/code_snippets/0315_wait_for_community_resource.sh

fealebenpae · 2025-05-12T08:58:46Z

docs/community-search/quick-start/README.md.j2

+* ```yaml
+requests:
+    cpu: 2
+    memory: 2G
+```


This doesn't render properly in the preview markdown. Perhaps the indentation is not right, or maybe code blocks can't be nested in lists?

fealebenpae · 2025-05-12T08:59:30Z

docs/community-search/quick-start/README.md.j2

+### 10. Wait for Search Resource to be Ready
+
+Similar to the MongoDB deployment, the Search deployment needs time to initialize. This step uses `kubectl wait` to pause until the `MongoDBSearch` resource `mdbc-rs` reports a `Running` status in its `.status.phase` field, indicating that the search nodes are operational and integrated.
+
+[code_snippets/0325_wait_for_search_resource.sh](code_snippets/0325_wait_for_search_resource.sh)
+```shell copy
+{% include "code_snippets/0325_wait_for_search_resource.sh" %}
+```
+This command polls the status of the `MongoDBSearch` resource `mdbc-rs`.


This also doesn't render correctly, perhaps because of the code block issue above.

fixed, yes it was because the above issue

docs/community-search/quick-start/README.md.j2

# Conflicts: # .evergreen-functions.yml # .evergreen.yml # scripts/evergreen/e2e/dump_diagnostic_information.sh # scripts/evergreen/e2e/dump_diagnostic_information_from_all_namespaces.sh

…community_resource.sh Co-authored-by: Yavor Georgiev <[email protected]>

Co-authored-by: Yavor Georgiev <[email protected]>

SimonBaeumer

LGTM. I really liked the PR description and how expressive the scripts are.

SimonBaeumer · 2025-05-12T19:23:16Z

.evergreen.yml

@@ -95,6 +96,18 @@ variables:
    teardown_group:
      - func: upload_code_snippets_logs

+  - &setup_and_teardown_group_kind_code_snippets
+    setup_task_can_fail_task: true


why is it okay to fail?

it's not OK to fail, but if the setup task fails (e.g. there was a problem creating kind cluster) and setup_task_can_fail_task is false then setup errors are ignored. So making it true means it's actually NOT OK to fail and we want the whole run to fail.

SimonBaeumer · 2025-05-12T19:23:57Z

.evergreen.yml

@@ -183,6 +196,10 @@ parameters:
    value: "false"
    description: set this to true if you would like to delete the resources created in the code snippet tests, but keep the clusters

+  - key: code_snippets_commit_output
+    value: "false"
+    description: set this to true if you would like the pipeline to automatically push a branch with updated snippets outputs


Is this push a prod push or development push?

what's prod and dev in this context? It's pushing the outputs and automatically creates a branch with diffs against master.

SimonBaeumer · 2025-05-12T19:25:44Z

docs/community-search/quick-start/README.md.j2

+- **kubectl**: The Kubernetes command-line tool, configured to communicate with your cluster.
+- **Helm**: The package manager for Kubernetes, used here to install the MongoDB Kubernetes Operator.
+- **jq**: A command-line JSON processor, used for parsing JSON output from kubectl commands.
+- **Bash 5.1+**: All shell commands in this guide are intended to be run in Bash. Ensure you are using at least version 5.


My mac has bash 3.2.57 and zsh 5.9, never updated it. I've never seen version requirements for bash before, do you use new features so it is necessary?

❯ bash --version GNU bash, version 3.2.57(1)-release (arm64-apple-darwin24) Copyright (C) 2007 Free Software Foundation, Inc. ❯ zsh --version zsh 5.9 (arm64-apple-darwin24.0)

Bash 3.2 vs 5.1 is a night and day comparison. But I'm not sure we do actually use any feature here. Problem is that in evergreen we have 5.1, most of us probably have 5.1 locally, so we are generally allowed to use more modern features of bash and we don't test if the scripts still work on 3.2

As for zsh - we never use it for our scripts. All our scripts have shebanged /usr/bin/env bash

Nevertheless it's a good point. From a quick poll, majority of the dev team have actually 3.2 locally...
I've created a ticket to triage and discuss to add mac tests that will ensure we can execute snippets on mac with a default bash version (3.2)

for now I'd like to keep the bash requirement just in case as the snippets were developed on mac with bash 5.1 and are tested automatically on linux with 5.1.

SimonBaeumer · 2025-05-12T19:30:47Z

docs/community-search/quick-start/code_snippets/0200_configure_community_search_pullsecret.sh

+if [[ "${pull_secrets}" ]]; then
+  kubectl patch --context "${K8S_CLUSTER_0_CONTEXT_NAME}" -n "${MDB_NAMESPACE}" \
+    sa mongodb-kubernetes-database-pods  \
+    --type=json -p='[{"op": "add", "path": "/imagePullSecrets/-", "value": {"name": "community-private-preview-pullsecret"}}]'


not blocking: Does Helm not overwrite the imagepullsecret patch of the SA after the next upgrade? Is this intentional?

Yes, another helm upgrade would probably remove that secret as there is no merge strategy on this field. But I think it's OK - it's only a temporary thing. Soon we'll have this image public so we'll remove all that pullsecret stuff.

docs/community-search/quick-start/README.md

SimonBaeumer · 2025-05-12T19:38:22Z

docs/community-search/quick-start/README.md

+
+Note: Private preview of MongoDB Community Search comes with some limitations, and it is not suitable for production use:
+* TLS cannot be enabled in MongoDB Community deployment (MongoD communicates with MongoT with plain text).
+* Only one node of search node is supported (load balancing not supported)


This sounds a bit weird, does this mean no replication support?

The search indexer is currently hardcoded to a single node. But the database it indexes can be multi-node (and the example here is just that), so replication is not impacted by Search.

SimonBaeumer · 2025-05-12T19:47:20Z

scripts/code_snippets/task_kind_community_search_snippets_test.sh

+set -eou pipefail
+source scripts/dev/set_env_context.sh
+
+set -x


nit: debug code intentional?

thanks, removed

scripts/evergreen/e2e/dump_diagnostic_information.sh

scripts/evergreen/e2e/dump_diagnostic_information_from_all_namespaces.sh

Co-authored-by: Simon Bäumer <[email protected]>

…espaces.sh Co-authored-by: Simon Bäumer <[email protected]>

Co-authored-by: Simon Bäumer <[email protected]>

lsierant changed the base branch from master to search-main May 9, 2025 22:44

lsierant force-pushed the lsierant/search-docs branch 3 times, most recently from 3f97137 to 35f5bb7 Compare May 11, 2025 20:52

lsierant changed the title ~~Lsierant/search docs~~ CLOUDP-318518: Community private-preview docs May 12, 2025

lsierant force-pushed the lsierant/search-docs branch from 63cc861 to 775908a Compare May 12, 2025 07:45

lsierant marked this pull request as ready for review May 12, 2025 07:48

lsierant requested a review from a team as a code owner May 12, 2025 07:48

lsierant requested review from m1kola, nammn, fealebenpae and lucian-tosa and removed request for a team May 12, 2025 07:48

fealebenpae approved these changes May 12, 2025

View reviewed changes

lsierant force-pushed the lsierant/search-docs branch from 775908a to 884dade Compare May 12, 2025 09:28

lsierant and others added 5 commits May 12, 2025 14:05

Search docs

57b9822

# Conflicts: # .evergreen-functions.yml # .evergreen.yml # scripts/evergreen/e2e/dump_diagnostic_information.sh # scripts/evergreen/e2e/dump_diagnostic_information_from_all_namespaces.sh

Update docs/community-search/quick-start/code_snippets/0315_wait_for_…

5bb8667

…community_resource.sh Co-authored-by: Yavor Georgiev <[email protected]>

Update docs/community-search/quick-start/README.md.j2

a3ab4bc

Co-authored-by: Yavor Georgiev <[email protected]>

Update docs/community-search/quick-start/README.md.j2

f58d156

Co-authored-by: Yavor Georgiev <[email protected]>

Review fixes

8f8deb7

fealebenpae changed the base branch from search-main to master May 12, 2025 13:17

fealebenpae requested review from dan-mckean and vinilage as code owners May 12, 2025 13:17

fealebenpae force-pushed the lsierant/search-docs branch from 5f883c9 to 8f8deb7 Compare May 12, 2025 13:17

SimonBaeumer approved these changes May 12, 2025

View reviewed changes

lsierant and others added 5 commits May 14, 2025 11:02

Update docs/community-search/quick-start/README.md

52e2653

Co-authored-by: Simon Bäumer <[email protected]>

Update scripts/evergreen/e2e/dump_diagnostic_information_from_all_nam…

7f1f1c8

…espaces.sh Co-authored-by: Simon Bäumer <[email protected]>

Update scripts/evergreen/e2e/dump_diagnostic_information.sh

c3bd5af

Co-authored-by: Simon Bäumer <[email protected]>

Merge branch 'master' into lsierant/search-docs

b36fbaf

Review fixes

e80393f

lsierant and others added 3 commits May 14, 2025 13:05

Review fixes

4f2c935

Update RELEASE_NOTES.md

54733be

Merge branch 'master' into lsierant/search-docs

39c4037

fealebenpae enabled auto-merge (squash) May 14, 2025 12:49

lint

1cec95f

fealebenpae merged commit 7fa6687 into master May 14, 2025
6 of 7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLOUDP-318518: Community private-preview docs #102

CLOUDP-318518: Community private-preview docs #102

lsierant commented May 9, 2025 •

edited

Loading

fealebenpae left a comment

fealebenpae May 12, 2025

lsierant May 12, 2025

fealebenpae May 12, 2025

lsierant May 12, 2025

SimonBaeumer left a comment

SimonBaeumer May 12, 2025

lsierant May 14, 2025 •

edited

Loading

SimonBaeumer May 12, 2025

lsierant May 14, 2025

SimonBaeumer May 12, 2025

lsierant May 14, 2025

lsierant May 14, 2025

lsierant May 14, 2025

SimonBaeumer May 12, 2025

lsierant May 14, 2025

SimonBaeumer May 12, 2025

fealebenpae May 14, 2025

SimonBaeumer May 12, 2025

lsierant May 14, 2025

CLOUDP-318518: Community private-preview docs #102

CLOUDP-318518: Community private-preview docs #102

Conversation

lsierant commented May 9, 2025 • edited Loading

Summary

File structure

General changes to snippets and reference architectures

Proof of Work

Checklist

fealebenpae left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimonBaeumer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lsierant May 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lsierant commented May 9, 2025 •

edited

Loading

lsierant May 14, 2025 •

edited

Loading