Skip to content

iproto: document box.iproto module #3766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions doc/dev_guide/internals/box_protocol.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@ The binary protocol provides complete access to Tarantool functionality, includi
asynchronously via the same connection
* response format that supports zero-copy writes

.. note::

Since version :doc:`2.11.0 </release/2.11.0>`, you can use the :ref:`box.iproto <box_iproto>` submodule to access
IPROTO constants and features from Lua. The submodule enables to :ref:`send arbitrary IPROTO packets <reference_lua-box_iproto_send>`
over the session's socket and :ref:`override the behavior <reference_lua-box_iproto_override>` for all IPROTO
request types. Also, :ref:`IPROTO_UNKNOWN <box_iproto-unknown>` constant is introduced. The constant is used for the
:ref:`box.iproto.override() <reference_lua-box_iproto_override>` API, which allows setting a handler for incoming requests with an unknown type.

.. toctree::
:maxdepth: 1

Expand Down
14 changes: 8 additions & 6 deletions doc/dev_guide/internals/iproto/format.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ The header is an MP_MAP. It may contain, in any order:
.. raw:: html
:file: images/header.svg

* Both the request and response make use of the :ref:`IPROTO_REQUEST_TYPE <internals-iproto-keys-request_type>` key.
* Both the request and response use the :ref:`IPROTO_REQUEST_TYPE <internals-iproto-keys-request_type>` key.
It denotes the type of the packet.

* The request and the matching response have the same sync number (:ref:`IPROTO_SYNC <internals-iproto-keys-sync>`).
Expand All @@ -61,22 +61,24 @@ To see how Tarantool decodes the header, have a look at file
function ``netbox_decode_data``.

For example, in a successful response to ``box.space:select()``,
the IPROTO_REQUEST_TYPE value will be 0 = ``IPROTO_OK`` and the
array will have all the tuples of the result.
the IPROTO_REQUEST_TYPE value is 0 = ``IPROTO_OK`` and the
array has all the tuples of the result.

Read the source code file `net_box.c <https://github.com/tarantool/tarantool/blob/master/src/box/lua/net_box.c>`__
where the function ``decode_metadata_optional`` is an example of how Tarantool
itself decodes extra items.

.. _box_protocol-body:

Body
----

The body is an MP_MAP. Maximal iproto package body length is 2 GiB.

The body has the details of the request or response. In a request, it can also
be absent or be an empty map. Both these states will be interpreted equally.
Responses will contain the body anyway even for an
:ref:`IPROTO_PING <box_protocol-ping>` request, where it will be an empty MP_MAP.
be absent or be an empty map. Both these states are interpreted equally.
Responses contain the body anyway even for an
:ref:`IPROTO_PING <box_protocol-ping>` request, where it is an empty MP_MAP.

A lot of responses contain the IPROTO_DATA map:

Expand Down
25 changes: 16 additions & 9 deletions doc/dev_guide/internals/iproto/keys.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ The keys are Tarantool constants that are either defined or mentioned in the

While the keys themselves are unsigned 8-bit integers, their values can have different types.

.. _internals-iproto-keys-basic:

Basic description
-----------------

Expand Down Expand Up @@ -132,6 +134,8 @@ General
- If ``IPROTO_FETCH_POSITION`` is **true**, returns a base64-encoded string representing
the position of the last selected tuple

.. _internals-iproto-keys-streams:

Streams
~~~~~~~

Expand Down Expand Up @@ -192,8 +196,7 @@ General replication

* - IPROTO_REPLICASET_UUID
- 0x25 |br| MP_STR
- UUID of the replica set.
Prior to Tarantool version 2.11, IPROTO_REPLICASET_UUID was called IPROTO_CLUSTER_UUID.
- Before Tarantool version 2.11, IPROTO_REPLICASET_UUID was called IPROTO_CLUSTER_UUID.

* - IPROTO_LSN
- 0x03 |br| MP_UINT
Expand Down Expand Up @@ -271,7 +274,7 @@ General replication
followed by an array of ids of instances whose rows won't be relayed to the replica.
Since v. :doc:`2.10.0 </release/2.10.0>`

.. _internals-iproto-keys-syncro-replication:
.. _internals-iproto-keys-synchro-replication:

Synchronous replication
~~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -319,6 +322,8 @@ Synchronous replication

All ``IPROTO_RAFT_*`` keys are used only in ``IPROTO_RAFT*`` requests.

.. _internals-iproto-keys-events:

Events and subscriptions
~~~~~~~~~~~~~~~~~~~~~~~~

Expand All @@ -342,6 +347,8 @@ Events and subscriptions

:ref:`Learn more about events and subscriptions in iproto <box-protocol-watchers>`.

.. _internals-iproto-keys-sql-specific:

SQL-specific
~~~~~~~~~~~~

Expand Down Expand Up @@ -460,7 +467,7 @@ Available IPROTO_FEATURES are the following:
in the request header). Learn more about :ref:`sending transaction commands <box_protocol-stream_transactions>`.

- ``IPROTO_FEATURE_ERROR_EXTENSION = 2`` -- :ref:`MP_ERROR <msgpack_ext-error>`
MsgPack extension support. Clients that don't support this feature will receive
MsgPack extension support. Clients that don't support this feature receive
error responses for :ref:`IPROTO_EVAL <box_protocol-eval>` and
:ref:`IPROTO_CALL <box_protocol-call>` encoded to string error messages.

Expand Down Expand Up @@ -606,7 +613,7 @@ IPROTO_ERROR_24

Code: 0x31.

IPROTO_ERROR_24 is used in Tarantool versions prior to :doc:`2.4.1 </release/2.4.1>`.
IPROTO_ERROR_24 is used in Tarantool versions before :doc:`2.4.1 </release/2.4.1>`.
The key contains the error in the string format.

Since :doc:`Tarantool 2.4.1 </release/2.4.1>`,
Expand Down Expand Up @@ -654,10 +661,10 @@ Code: 0x09.
When it comes to replicating synchronous transactions, the IPROTO_FLAGS key is included in the header.
The key contains an MP_UINT value of one or more bits:

* IPROTO_FLAG_COMMIT (0x01) will be set if this is the last message for a transaction.
* IPROTO_FLAG_WAIT_SYNC (0x02) will be set if this is the last message
* IPROTO_FLAG_COMMIT (0x01) is set if this is the last message for a transaction.
* IPROTO_FLAG_WAIT_SYNC (0x02) is set if this is the last message
for a transaction which cannot be completed immediately.
* IPROTO_FLAG_WAIT_ACK (0x04) will be set if this is the last message for a synchronous transaction.
* IPROTO_FLAG_WAIT_ACK (0x04) is set if this is the last message for a synchronous transaction.

Example:

Expand Down Expand Up @@ -739,7 +746,7 @@ at least IPROTO_FIELD_NAME (0x00) and MP_STR, and IPROTO_FIELD_TYPE (0x01) and M

Additionally, if ``sql_full_metadata`` in the
:ref:`_session_settings <box_space-session_settings>` system space
is TRUE, then the array will have these additional column maps
is TRUE, then the array has these additional column maps
which correspond to components described in the :ref:`box.execute() <box-sql_if_full_metadata>` section.

.. _internals-iproto-keys-sql_bind:
Expand Down
4 changes: 3 additions & 1 deletion doc/dev_guide/internals/iproto/replication.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ These values and the corresponding packet body structures are considered below.

Connectors and clients do not need to send replication packets.

.. _box_protocol-general:

General
-------

Expand Down Expand Up @@ -198,7 +200,7 @@ Synchronous
- 0x28
- Confirm that the RAFT transactions have achieved quorum and can be committed

* - :ref:`IPROTO_RAFT_ROLLBACK <box_protocol-raft_confirm>`
* - :ref:`IPROTO_RAFT_ROLLBACK <box_protocol-raft_rollback>`
- 0x29
- Roll back the RAFT transactions because they haven't achieved quorum

Expand Down
16 changes: 16 additions & 0 deletions doc/dev_guide/internals/iproto/requests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ Overview
- 0x8XXX |br| MP_INT
- Error response

* - :ref:`IPROTO_UNKNOWN <internals-iproto-unknown>`
- -1 |br| MP_UINT
- An unknown request type

* - :ref:`IPROTO_SELECT <box_protocol-select>`
- 0x01
- :ref:`Select <box_space-select>` request
Expand Down Expand Up @@ -126,6 +130,18 @@ constants for return codes.
To learn more about error responses,
check the section :ref:`Request and response format <box_protocol-responses_error>`.

.. _internals-iproto-unknown:

IPROTO_UNKNOWN
~~~~~~~~~~~~~~

Since :doc:`2.11.0 </release/2.11.0>`.

Code: -1.

An unknown request type. The constant is used to override the handler of unknown IPROTO request types.
Learn more: :ref:`box.iproto.override() <reference_lua-box_iproto_override>` and :ref:`box_iproto_override <box_box_iproto_override>`.

.. _box_protocol-select:

IPROTO_SELECT
Expand Down
Loading