[WIP] Prototype/POC of remote server support + hosting on Databricks apps

[smurching]

Related Issues/PRs

#xxx

What changes are proposed in this pull request?

Just snapshotting / pushing some POC work towards enabling hosting the UC MCP server on Databricks apps. To get things to work, I:

• Created and deployed a Databricks apps
• Granted my app SP permissions (USE CATALOG, USE SCHEMA, Query on Genie spaces, SELECT on VS indexes, etc) on the necessary resources
• Generated an oauth token for use with API access against apps, by running databricks auth login --profile <your-profile-here> and then running the following script

from databricks import sdk
w = sdk.WorkspaceClient(profile="<your-profile-here>")
headers = w.config.authenticate()
print(headers)

• Used that token as the Authorization header in MCP inspector

I'm seeing some occasional server disconnects etc but I hope that'll go away once we can switch to streamable HTTP transport & clients support it. It seems client support is coming soon, with the first step of server support in modelcontextprotocol/python-sdk#553

How is this PR tested?

• Existing unit/integration tests
• New unit/integration tests
• Manual tests

Does this PR require documentation update?

• No. You can skip the rest of this section.
• Yes. I've updated the relevant server README.md

Release Notes

Is this a user-facing change?

• No. You can skip the rest of this section.
• Yes. Give a description of this change to be included in the release notes.

How should the PR be classified in the release notes? Choose one:

• rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
• rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
• rn/feature - A new user-facing feature worth mentioning in the release notes
• rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
• rn/documentation - A user-facing documentation change worth mentioning in the release notes