Skip to content

Commit feaf36e

Browse files
patiencedaurpatiencedaur
committed
Outline initial structure
* Split the Binary protocol page into several sections * Move examples to How-to
1 parent d8519f0 commit feaf36e

File tree

13 files changed

+2065
-1785
lines changed

13 files changed

+2065
-1785
lines changed

‎doc/dev_guide/internals/box_protocol.rst

Lines changed: 69 additions & 829 deletions
Large diffs are not rendered by default.

‎doc/dev_guide/internals/iproto/events.rst

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
.. _internals-events:
2+
3+
Events and subscriptions
4+
========================
5+
6+
.. _box-protocol-watchers:
7+
8+
Watchers
9+
--------
10+
11+
The commands below support asynchronous server-client notifications signalled
12+
with :ref:`box.broadcast() <box-broadcast>`.
13+
Servers that support the new feature set the ``IPROTO_FEATURE_WATCHERS`` feature in reply to the ``IPROTO_ID`` command.
14+
When the connection is closed, all watchers registered for it are unregistered.
15+
16+
The remote :ref:`watcher <box-watchers>` protocol works in the following way:
17+
18+
#. The client sends an ``IPROTO_WATCH`` packet to subscribe to the updates of a specified key defined on the server.
19+
20+
#. The server sends an ``IPROTO_EVENT`` packet to the subscribed client after registration.
21+
The packet contains the key name and its current value.
22+
After that, the packet is sent every time the key value is updated with
23+
``box.broadcast()``, provided that the last notification was acknowledged (see below).
24+
25+
#. After receiving the notification, the client sends an ``IPROTO_WATCH`` packet to acknowledge the notification.
26+
27+
#. If the client doesn't want to receive any more notifications, it unsubscribes by sending
28+
an ``IPROTO_UNWATCH`` packet.
29+
30+
All the three request types are asynchronous -- the receiving end doesn't send a packet in reply to any of them.
31+
Therefore, neither of them has a sync number.
32+
33+
.. _box_protocol-watch:
34+
35+
IPROTO_WATCH = 0x4a
36+
~~~~~~~~~~~~~~~~~~~
37+
38+
Registers a new watcher for the given notification key or confirms a notification if the watcher is
39+
already subscribed.
40+
The watcher is notified after registration.
41+
After that, the notification is sent every time the key is updated.
42+
The server doesn't reply to the request unless it fails to parse the packet.
43+
44+
The body is a 2-item map:
45+
46+
.. cssclass:: highlight
47+
.. parsed-literal::
48+
49+
# <size>
50+
msgpack(:samp:`{{MP_UINT unsigned integer = size(<header>) + size(<body>)}}`)
51+
# <header>
52+
msgpack({
53+
IPROTO_REQUEST_TYPE: IPROTO_WATCH
54+
})
55+
# <body>
56+
msgpack({
57+
IPROTO_EVENT_KEY: :samp:`{{MP_STR string}}}`
58+
})
59+
60+
``IPROTO_EVENT_KEY`` (code 0x56) contains the key name.
61+
62+
.. _box_protocol-unwatch:
63+
64+
IPROTO_UNWATCH = 0x4b
65+
~~~~~~~~~~~~~~~~~~~~~
66+
67+
Unregisters a watcher subscribed to the given notification key.
68+
The server doesn't reply to the request unless it fails to parse the packet.
69+
70+
The body is a 2-item map:
71+
72+
.. cssclass:: highlight
73+
.. parsed-literal::
74+
75+
# <size>
76+
msgpack(:samp:`{{MP_UINT unsigned integer = size(<header>) + size(<body>)}}`)
77+
# <header>
78+
msgpack({
79+
IPROTO_REQUEST_TYPE: IPROTO_UNWATCH
80+
})
81+
# <body>
82+
msgpack({
83+
IPROTO_EVENT_KEY: :samp:`{{MP_STR string}}}`
84+
})
85+
86+
``IPROTO_EVENT_KEY`` (code 0x56) contains a key name.
87+
88+
.. _box_protocol-event:
89+
90+
IPROTO_EVENT = 0x4c
91+
~~~~~~~~~~~~~~~~~~~
92+
93+
Sent by the server to notify a client about an update of a key.
94+
95+
The body is a 2-item map:
96+
97+
.. cssclass:: highlight
98+
.. parsed-literal::
99+
100+
# <size>
101+
msgpack(:samp:`{{MP_UINT unsigned integer = size(<header>) + size(<body>)}}`)
102+
# <header>
103+
msgpack({
104+
IPROTO_REQUEST_TYPE: IPROTO_EVENT
105+
})
106+
# <body>
107+
msgpack({
108+
IPROTO_EVENT_KEY: :samp:`{{MP_STR string}}}`,
109+
IPROTO_EVENT_DATA: :samp:`{{MP_OBJECT value}}}`
110+
})
111+
112+
``IPROTO_EVENT_KEY`` (code 0x56) contains the key name.
113+
114+
``IPROTO_EVENT_DATA`` (code 0x57) contains data sent to a remote watcher.
115+
The parameter is optional, the default value is ``nil``.

‎doc/dev_guide/internals/iproto/format.rst

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
.. _internals-iproto-format:
2+
3+
Request and response format
4+
===========================
5+
6+
* We use `MessagePack <http://MessagePack.org>`_ and refer to MessagePack types in this document.
7+
* Request size is passed before the request
8+
* Request and response have the same sync number (IPROTO_SYNC)
9+
* It is legal to put more than one request in a packet.
10+
11+
.. _internals-unified_packet_structure:
12+
13+
Requests and responses usually contain three sections: size, header, and body.
14+
15+
Size
16+
~~~~
17+
18+
The size is an MP_UINT -- unsigned integer, usually 32-bit.
19+
The header and body are maps (MP_MAP).
20+
21+
.. _box_protocol-header:
22+
23+
Header
24+
~~~~~~
25+
26+
.. cssclass:: highlight
27+
.. parsed-literal::
28+
29+
# <size>
30+
:samp:`{{MP_UINT unsigned integer}}`
31+
# <header>
32+
:samp:`{{MP_MAP with <header> map-items}}`
33+
# <body>
34+
:samp:`{{MP_MAP with <body> map-items}}`
35+
36+
``<size>`` is the size of the header plus the size of the body.
37+
It may be useful to compare it with the number of bytes remaining in the packet.
38+
39+
``<header>`` may contain, in any order:
40+
41+
.. cssclass:: highlight
42+
.. parsed-literal::
43+
44+
msgpack({
45+
<request type>: :samp:`{{MP_UINT unsigned integer}}`,
46+
IPROTO_SYNC: :samp:`{{MP_UINT unsigned integer}}`,
47+
IPROTO_SCHEMA_VERSION: :samp:`{{MP_UINT unsigned integer}}`
48+
IPROTO_STREAM_ID: :samp:`{{MP_UINT unsigned integer}}`
49+
})
50+
51+
IPROTO_SYNC
52+
^^^^^^^^^^^
53+
54+
Binary code: 0x01.
55+
56+
This is an unsigned integer that should be incremented so that it is unique in every
57+
request. This integer is also returned from :doc:`community:reference/reference_lua/box_session/sync`.
58+
59+
The IPROTO_SYNC value of a response should be the same as
60+
the IPROTO_SYNC value of a request.
61+
62+
IPROTO_SCHEMA_VERSION
63+
^^^^^^^^^^^^^^^^^^^^^
64+
65+
Binary code: 0x05.
66+
67+
An unsigned number, sometimes called SCHEMA_ID, that goes up when there is a
68+
major change.
69+
70+
In a request header, IPROTO_SCHEMA_VERSION is optional, so the version will not
71+
be checked if it is absent.
72+
73+
In a response header, IPROTO_SCHEMA_VERSION is always present, and it is up to
74+
the client to check if it has changed.
75+
76+
.. _box_protocol-iproto_stream_id:
77+
78+
IPROTO_STREAM_ID
79+
^^^^^^^^^^^^^^^^
80+
81+
Binary code: 0x0a.
82+
83+
Only used in :ref:`streams <txn_mode_stream-interactive-transactions>`.
84+
This is an unsigned number that should be unique in every stream.
85+
86+
In requests, IPROTO_STREAM_ID is useful for two things:
87+
ensuring that requests within transactions are done in separate groups,
88+
and ensuring strictly consistent execution of requests (whether or not they are within transactions).
89+
90+
In responses, IPROTO_STREAM_ID does not appear.
91+
92+
See :ref:`Binary protocol -- streams <box_protocol-streams>`.
93+
94+
95+
Encoding and decoding
96+
^^^^^^^^^^^^^^^^^^^^^
97+
98+
To see how Tarantool encodes the header, have a look at file
99+
`xrow.c <https://github.com/tarantool/tarantool/blob/master/src/box/xrow.c>`_,
100+
function ``xrow_header_encode``.
101+
102+
To see how Tarantool decodes the header, have a look at file ``net_box.c``,
103+
function ``netbox_decode_data``.
104+
105+
For example, in a successful response to ``box.space:select()``,
106+
the Response-Code-Indicator value will be 0 = ``IPROTO_OK`` and the
107+
array will have all the tuples of the result.
108+
109+
Body
110+
~~~~
111+
112+
The ``<body>`` has the details of the request or response. In a request, it can also
113+
be absent or be an empty map. Both these states will be interpreted equally.
114+
Responses will contain the ``<body>`` anyway even for an
115+
:ref:`IPROTO_PING <box_protocol-ping>` request.
116+

‎doc/dev_guide/internals/iproto/keys.rst

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
.. _box_protocol-key_list:
2+
.. _internals-iproto-keys:
3+
4+
List of IPROTO keys
5+
===================
6+
7+
This section describes all the iproto keys used to pass request arguments.
8+
Все типы ключей, используемых для передачи аргументов, с их типами и кратким описанием
9+
(например, `IPROTO_SPACE_ID` - идентификато таблицы, unsigned integer).
10+
(всё, что встречается внутри запросов)
11+
12+
.. container:: table
13+
14+
.. list-table::
15+
:header-rows: 1
16+
:widths: 17 16 17 50
17+
18+
* - Name
19+
- Binary code
20+
- Type
21+
- Description
22+
* - IPROTO_SPACE_ID
23+
- 0x10
24+
- unsigned
25+
- Space identifier
26+
* - IPROTO_INDEX_ID
27+
- 0x11
28+
- unsigned
29+
- Index identifier
30+
* - IPROTO_REQUEST_TYPE
31+
- 0x00
32+
- unsigned
33+
- Request type
34+
* -
35+
used with SQL within IPROTO_EXECUTE:
36+
IPROTO_OPTIONS=0x2b
37+
IPROTO_METADATA=0x32
38+
IPROTO_BIND_METADATA=0x33
39+
IPROTO_BIND_COUNT=0x34
40+
IPROTO_SQL_TEXT=0x40
41+
IPROTO_SQL_BIND=0x41
42+
IPROTO_SQL_INFO=0x42
43+
IPROTO_STMT_ID=0x43
44+
45+
"No error" constant
46+
IPROTO_OK=0x00
47+
48+
box.session.sync(), like a clock, must be same for response and request
49+
IPROTO_SYNC=0x01
50+
51+
Replication
52+
IPROTO_REPLICA_ID=0x02
53+
IPROTO_INSTANCE_UUID=0x24
54+
IPROTO_RAFT_TERM=0x00
55+
IPROTO_RAFT_VOTE=0x01
56+
IPROTO_RAFT_STATE=0x02
57+
IPROTO_RAFT_VCLOCK=0x03
58+
IPROTO_RAFT_LEADER_ID=0x04
59+
IPROTO_RAFT_IS_LEADER_SEEN=0x05
60+
IPROTO_VCLOCK=0x26
61+
IPROTO_CLUSTER_UUID=0x25
62+
IPROTO_BALLOT=0x29
63+
IPROTO_BALLOT_IS_RO_CFG=0x01
64+
IPROTO_BALLOT_VCLOCK=0x02
65+
IPROTO_BALLOT_GC_VCLOCK=0x03
66+
IPROTO_BALLOT_IS_RO=0x04
67+
IPROTO_BALLOT_IS_ANON=0x05
68+
IPROTO_BALLOT_IS_BOOTED=0x06
69+
IPROTO_BALLOT_CAN_LEAD=0x07
70+
IPROTO_REQUEST_TYPE
71+
IPROTO_KEY=0x20
72+
Maximum number of tuples in the space
73+
IPROTO_LIMIT=0x12
74+
Number of tuples to skip in the select
75+
IPROTO_OFFSET=0x13
76+
iterator type (gt, lt, eq...)
77+
IPROTO_ITERATOR=0x14
78+
what is the first field number (like 1 or 0)
79+
IPROTO_INDEX_BASE=0x15
80+
IPROTO_TUPLE=0x21
81+
IPROTO_TUPLE_META=0x2a
82+
IPROTO_LSN=0x03
83+
IPROTO_TIMESTAMP=0x04 Float 64 MP_DOUBLE 8-byte timestamp
84+
IPROTO_SCHEMA_VERSION=0x05
85+
Stream transactions
86+
IPROTO_STREAM_ID=0x0a
87+
IPROTO_TXN_ISOLATION=0x59
88+
IPROTO_FUNCTION_NAME=0x22
89+
user name
90+
IPROTO_USER_NAME=0x23
91+
IPROTO_FIELD_NAME=0x00
92+
IPROTO_FIELD_TYPE=0x01
93+
IPROTO_FIELD_COLL=0x02
94+
IPROTO_FIELD_IS_NULLABLE=0x03
95+
IPROTO_FIELD_IS_AUTOINCREMENT=0x04
96+
IPROTO_FIELD_SPAN=0x05
97+
array of operations
98+
IPROTO_OPS=0x28
99+
OPERATOR
100+
Command argument (passed within IPROTO_EVAL)
101+
IPROTO_EXPR=0x27
102+
used by all requests, contains response data
103+
IPROTO_DATA=0x30
104+
IPROTO_ERROR=0x52
105+
IPROTO_ERROR_24=0x31
106+
107+
IPROTO_CHUNK=0x80
108+
109+
IPROTO_TIMEOUT=0x56
110+
111+
Synchronous replication
112+
IPROTO_FLAGS=0x09
113+
IPROTO_FLAG_COMMIT=0x01
114+
IPROTO_FLAG_WAIT_SYNC=0x02
115+
IPROTO_FLAG_WAIT_ACK=0x04
116+
-- parts of IPROTO_ID
117+
IPROTO_VERSION=0x54
118+
IPROTO_FEATURES=0x55
119+
120+
IPROTO_EVENT_KEY=0x57
121+
IPROTO_EVENT_DATA=0x58

0 commit comments

Comments
 (0)