Transport packets

← Back to Home

The Plabble Transport Protocol (PTP) uses a special transport codec for the delivery of the packets between clients and servers. There are two types of transport packets: Request Packets and Response Packets. The packets are sized dynamically using flags and length attributes to reduce the size of the packets to a minimum.

Bucket storage

A Plabble server stores the data in buckets. You can see a bucket as a key-value slot on a server with an authorization system to prevent access to users that do not own the secret key. Every bucket consists of a list with a maximum of 65,536 slots. You can add, edit or remove slots from/to a bucket. Thanks to a permission system it is possible to add access restrictions to a bucket, so you can make buckets immutable or unreadable for users without the authorization key. The bucket storage system can optionally be used for multiple purposes on top of multiple protocols because the server acts as a key-value storage system.

Request packet

Transport Packet

Figure A: Plabble Request Packet codec

Type-flag bitmap

Figure B: Type/Flag bitmap. First 4 bits are the type, the last 4 bits are the flags

In Figure A the byte map of the Plabble Request Packet is shown. The packets consists of the following parts:

Response packet

Transport Packet

Figure C: Plabble Response Packet codec

Type-flag bitmap

Figure D: Type/Flag bitmap. First 4 bits are the type, the last 4 bits are the flags

In Figure C the byte map of the Plabble Response Packet is shown. The packets consists of the following parts:

Packet types

There are several types of packets specified in the protocol, with also a bunch reserved for future use. The packets are used to create, read, update and delete storage buckets on a server. All packets types are wrapped in a Request packet or a Response packet.

Name Value Description
CONNECT 0 Start session on a server
CREATE 1 Create a new bucket
PUT 2 Put one or multiple values to a bucket on a specific index
APPEND 3 Append one or multiple values to the first free slots in the bucket
WIPE 4 Wipe one or more values from the bucket, or delete entire bucket
REQUEST 5 Read one or more values from the bucket
SUBSCRIBE 6 Subscribe to bucket updates
UNSUBSCRIBE 7 Unsubscribe from bucket updates
reserved 8 - 14 Reserved types for future use
ERROR 15 Error type, response only

Request flags

In a Plabble Request Packet there are 4 bits in the type/flags field that are used to set some properties on a packet. The first bit (flag #5) is global and is the same for every packet type: when this flag is set, a MAC is included in the request. The other flags differ per type:

Packet type Flag #6 Flag #7 Flag #8
CONNECT Upgrade to encrypted connection Send certificate in response reserved
CREATE Subscribe to the created bucket with an optional range Do not persist bucket (Create RAM bucket) reserved
PUT reserved reserved reserved
APPEND reserved reserved _reserved
WIPE Delete entire bucket reserved reserved
REQUEST Also subscribe to the bucket or to the selected range reserved reserved
SUBSCRIBE reserved reserved reserved
UNSUBSCRIBE reserved reserved reserved

Response flags

In the Plabble Response Packet there are also 4 bits in the type/flags field. Also the first bit, flag #5, is reserved for indicating of a MAC is present. But there are no other flags yet in use and all are reserved for later usage.

Packet encryption and authentication

Encryption of packets in PTP is optional, because Plabble also allows the usage of TLS. However, Plabble still wants to maintain authenticity on protocol-level because the user has to prove it has access to a bucket. To solve this, Plabble has basically two modes:

  1. Not encrypted: The packets include a MAC to ensure authenticity.
  2. Encrypted: Plabble uses an Authenticated Encryption with Associated Data (AEAD) algorithm to ensure both confidentiality and authenticity.

Authentication

Authentication data byte map

Figure E: Authentication data bytemap

The data hashed in the MAC (Message Authentication Code) or AD (Associated Data) contains the following fields (see Figure E):

If a packet is encrypted, the header is encrypted with the ChaCha20 algorithm while the body is encrypted with ChaCha20-Poly1305 AEAD algorithm like shown in Figure A and Figure C. The authentication data poly hash is thus passed to the body while the header does not contain authentication data. Because the header data is included in the body authentication hash it can still be verified safely. No MAC is included.

If no encryption is used, the a 16-byte Message Authentication Code (MAC) is appended to the packet. For the generation of the MAC Poly1305 is used to generate a 16-byte authentication hash. When a MAC is included a flag must be set, for the request packet flag #5 in the type/flag field will be set and for a response the first bit of the status field.

When sending a CONNECT packet, encryption is not used and no MAC is included because the keys are not exchanged yet.

MAC and Encryption keys

The keys used to encrypt the packet are generated using a Key Derivation Function (KDF). We use HKDF-SHA256 to derive keys from a shared secret generated in the CONNECT step, referred to as the session key. For generating encryption or authorization keys we use this session key. We generate 2 keys every request:

These two keys are generated because it is not secure to reuse keys with the same nonce when using chacha20 (we use an empty byte array of 12 bytes as nonce at this point). When encrypting the packet we use Key 0 to encrypt the header and Key 1 to encrypt the body. If no encryption is used, Key 0 is used to generate the MAC. Because the counter differs every time, different encryption keys are used for every packet.

Counters

The client and the server both keep two counters which are stored in a 2-byte unsigned integer (uint-16) each, a client counter and a server counter. These counters are used as a nonce in the Key Derivation Function (KDF) to generate a shared secret that differs every time and to keep track on requests/responses. To encrypt the data or create a MAC the counters are used to generate a key. A client or server always uses his own counter when sending a message to the other party. Because they both keep the two counters, the counter of the other party is also known. After each message sent to the other party own counter is incremented. After each message that is received from the other party, the other counter is incremented, so a counter increments after the processing of a message.

Counter overflow

If a counter reaches the maximum value 65535 (because it is an u16), the connection must be closed because we don’t want the risk of reusing encryption keys. The client needs to reconnect to the server.

Packet data

Dynamically sized length

Dynlen integer bitmap

Figure F: Dynamic length bitmap

Plabble uses dynamically sized length unsigned integers (see Figure F). Plabble uses this system to avoid unnecessary bytes to be send, what makes Plabble faster and more efficient in its transport system. The length can have 1 to 4 bytes, and every last bit in every byte is used to indicate if the next byte is present. So if you have the first byte and the last bit equals 1, than a second byte is present. If on the second byte the last bit is set, a third byte is present etc. So the range of this integer is between 0 and 268435455.

Below is a code snippet that indicates how the Plabble dynamic length integer could be implemented (Go):

func encode(num int) (result []byte) {
	result = make([]byte, 0, 4)
	for num > 0 { // while loop
        // in dynamic typed languages, remember to Floor the divisions!
		encodedByte := byte(num % 128)
		num /= 128
		if num > 0 {
			encodedByte |= 128
		}
		result = append(result, encodedByte)
	}
	return //result
}

func decode(arr []byte) (num int) {
	multiplier := 1
	for _, encodedByte := range arr {
		num += (int(encodedByte) & 127) * multiplier
		multiplier *= 128
	}
	return //num
}

Endianness

Everywhere throughout the protocol big endian encoding is used for consistency when encoding numbers. The big endian encoding is easier to understand and performance-wise it doesn’t matter on the computers today.

← Back to Home - Next: CONNECT packet →