Skip to content

Fix: Fail early when database cluster does not respond #711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ Changes for crate
Unreleased
==========

- Changed connection behaviour to fail early if the database cluster
does not respond

2025/01/30 2.0.0
================

Expand Down
8 changes: 2 additions & 6 deletions docs/by-example/client.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,8 @@ respond, the request is automatically routed to the next server:
>>> connection = client.connect([invalid_host, crate_host])
>>> connection.close()

If no ``servers`` are given, the default one ``http://127.0.0.1:4200`` is used:

>>> connection = client.connect()
>>> connection.client._active_servers
['http://127.0.0.1:4200']
>>> connection.close()
If no ``servers`` are supplied to the ``connect`` method, the default address
``http://127.0.0.1:4200`` is used.
Comment on lines -38 to +39
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no server at http://127.0.0.1:4200. This test case just didn't fail because connect() did not raise an exception up until now.

Now, there is no longer a way to validate connecting to the default address per doctest file, because there is no server listening at http://127.0.0.1:4200. Because the core information is still viable, all what's left is pure prose, rephrased a bit.


If the option ``error_trace`` is set to ``True``, the client will print a whole
traceback if a server error occurs:
Expand Down
6 changes: 3 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,14 +49,15 @@ def read(path):
long_description=long_description,
long_description_content_type="text/x-rst",
platforms=["any"],
license="Apache License 2.0",
license="Apache-2.0",
license_files=["LICENSE"],
keywords="cratedb db api dbapi database sql http rdbms olap",
packages=find_namespace_packages("src"),
package_dir={"": "src"},
install_requires=[
"orjson<4",
"urllib3",
"verlib2",
"verlib2>=0.3",
],
extras_require={
"doc": [
Expand All @@ -82,7 +83,6 @@ def read(path):
classifiers=[
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
Expand Down
11 changes: 10 additions & 1 deletion src/crate/client/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,10 @@
# However, if you have executed another commercial license agreement
# with Crate these terms will supersede the license and you may use the
# software solely pursuant to the terms of the relevant commercial agreement.
import json

from verlib2 import Version
from verlib2.packaging.version import InvalidVersion

from .blob import BlobContainer
from .cursor import Cursor
Expand Down Expand Up @@ -197,14 +199,21 @@ def get_blob_container(self, container_name):

def _lowest_server_version(self):
lowest = None
server_count = len(self.client.active_servers)
connection_errors = []
for server in self.client.active_servers:
try:
_, _, version = self.client.server_infos(server)
version = Version(version)
except (ValueError, ConnectionError):
except ConnectionError as ex:
connection_errors.append(ex)
continue
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this continue let you try all active servers? Do you want to stop at the first exception?

Copy link
Member Author

@amotl amotl May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, like before, and no, like before. Do you think it should be done differently?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I probably take the change log entry too literally.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, so what's happening? does it retry all servers provided and then throw the last connection error? (shouldn't it throw the first connection error maybe?)

Copy link
Member

@seut seut May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it would throw the first error, the whole logic of supporting multiple servers would be invalidated I think.

Copy link
Member

@seut seut May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah sry got this wrong, not sure if throwing the first error would be better.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another option could be to put all connection errors into an exception if all servers raise connection errors.

Copy link
Member Author

@amotl amotl May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it would throw the first error, the whole logic of supporting multiple servers would be invalidated I think.

I share the same opinion that it would defeat the whole purpose of relevant routines, right.

Copy link
Member Author

@amotl amotl May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another option could be to put all connection errors into [a single] exception if all servers raise connection errors.

0046f03 implements this suggestion, thanks!

except (ValueError, InvalidVersion):
continue
if not lowest or version < lowest:
lowest = version
if connection_errors and len(connection_errors) == server_count:
raise ConnectionError(json.dumps(list(map(str, connection_errors))))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just raise ConnectionError(error_list)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've elaborated about it here, but I am also not sure, that's why I am also asking about your opinion.

return lowest or Version("0.0.0")

def __repr__(self):
Expand Down
25 changes: 16 additions & 9 deletions tests/client/test_connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

from urllib3 import Timeout

import crate.client.exceptions
from crate.client import connect
from crate.client.connection import Connection
from crate.client.http import Client
Expand All @@ -21,21 +22,27 @@ def test_connection_mock(self):
"""

class MyConnectionClient:
active_servers = ["localhost:4200"]
active_servers = [crate_host]

def __init__(self):
pass

def server_infos(self, server):
return ("localhost:4200", "my server", "0.42.0")
return (crate_host, "my server", "0.42.0")

connection = connect([crate_host], client=MyConnectionClient())
self.assertIsInstance(connection, Connection)
self.assertEqual(
connection.client.server_infos("foo"),
("localhost:4200", "my server", "0.42.0"),
(crate_host, "my server", "0.42.0"),
)

def test_invalid_server_address(self):
client = Client(servers="localhost:4202")
with self.assertRaises(crate.client.exceptions.ConnectionError) as ex:
connect(client=client)
self.assertIn("Server not available", ex.exception.message)

def test_lowest_server_version(self):
infos = [
(None, None, "0.42.3"),
Expand All @@ -50,14 +57,14 @@ def test_lowest_server_version(self):
connection.close()

def test_invalid_server_version(self):
client = Client(servers="localhost:4200")
client = Client(servers=crate_host)
client.server_infos = lambda server: (None, None, "No version")
connection = connect(client=client)
self.assertEqual((0, 0, 0), connection.lowest_server_version.version)
connection.close()

def test_context_manager(self):
with connect("localhost:4200") as conn:
with connect(crate_host) as conn:
pass
self.assertEqual(conn._closed, True)

Expand All @@ -70,7 +77,7 @@ def test_with_timezone(self):
"""

tz_mst = datetime.timezone(datetime.timedelta(hours=7), name="MST")
connection = connect("localhost:4200", time_zone=tz_mst)
connection = connect(crate_host, time_zone=tz_mst)
cursor = connection.cursor()
self.assertEqual(cursor.time_zone.tzname(None), "MST")
self.assertEqual(
Expand All @@ -88,20 +95,20 @@ def test_timeout_float(self):
"""
Verify setting the timeout value as a scalar (float) works.
"""
with connect("localhost:4200", timeout=2.42) as conn:
with connect(crate_host, timeout=2.42) as conn:
self.assertEqual(conn.client._pool_kw["timeout"], 2.42)

def test_timeout_string(self):
"""
Verify setting the timeout value as a scalar (string) works.
"""
with connect("localhost:4200", timeout="2.42") as conn:
with connect(crate_host, timeout="2.42") as conn:
self.assertEqual(conn.client._pool_kw["timeout"], 2.42)

def test_timeout_object(self):
"""
Verify setting the timeout value as a Timeout object works.
"""
timeout = Timeout(connect=2.42, read=0.01)
with connect("localhost:4200", timeout=timeout) as conn:
with connect(crate_host, timeout=timeout) as conn:
self.assertEqual(conn.client._pool_kw["timeout"], timeout)
5 changes: 4 additions & 1 deletion tests/client/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ def test_suite():
suite.addTest(makeSuite(KeepAliveClientTest))
suite.addTest(makeSuite(ThreadSafeHttpClientTest))
suite.addTest(makeSuite(ParamsTest))
suite.addTest(makeSuite(ConnectionTest))
suite.addTest(makeSuite(RetryOnTimeoutServerTest))
suite.addTest(makeSuite(RequestsCaBundleTest))
suite.addTest(makeSuite(TestUsernameSentAsHeader))
Expand Down Expand Up @@ -65,6 +64,10 @@ def test_suite():
# Integration tests.
layer = ensure_cratedb_layer()

s = makeSuite(ConnectionTest)
s.layer = layer
suite.addTest(s)

s = doctest.DocFileSuite(
"docs/by-example/http.rst",
"docs/by-example/client.rst",
Expand Down