Comms CCF
This is a simple communication layer based on COBS, CBOR and FNV-1A
Loading...
Searching...
No Matches
cbor.hpp File Reference

Encoding for structured data (numbers, strings, bytes, arrays and maps). Also supports custom tags. More...

#include "types.hpp"
#include <float.h>
#include <math.h>
#include <stddef.h>
#include <stdint.h>
#include <string.h>
#include <algorithm>
#include <array>
#include <bit>
#include <concepts>
#include <optional>
#include <span>
#include <string_view>
#include <tuple>
#include <type_traits>
#include <utility>

Go to the source code of this file.

Classes

struct  Cbor::Undefined
struct  Cbor::WrapVoid< T, Placeholder >
 Allows materialising void values using a placeholder. More...
struct  Cbor::WrapVoid< void, Placeholder >
struct  Cbor::WrapVoid< U, Placeholder >
struct  Cbor::Item
struct  Cbor::Cbor< T >
 Parametrized CBOR interface. More...
struct  Cbor::Cbor< I >
struct  Cbor::Cbor< F >
struct  Cbor::Cbor< bool >
struct  Cbor::Cbor< T * >
struct  Cbor::Cbor< void >
struct  Cbor::Cbor< std::string_view >
struct  Cbor::Cbor< const char * >
struct  Cbor::Cbor< const char(&)[size]>
struct  Cbor::Cbor< std::tuple< Item... > >
struct  Cbor::Cbor< std::span< uint8_t, Size > >
struct  Cbor::Cbor< std::array< T, Size > >
class  Cbor::Sequence< major >

Typedefs

template<typename T, typename Placeholder>
using Cbor::WrapVoidT = WrapVoid<T, Placeholder>::T

Enumerations

enum class  Cbor::Major : uint8_t {
  U64 = 0 , Neg64 = 1 , Bytes = 2 , Utf8 = 3 ,
  Array = 4 , Map = 5 , Tagged = 6 , Simple = 7 ,
  Float = 7
}
 Upper three bits of the initial/header byte. More...
enum class  Cbor::Minor : uint8_t {
  EmbeddedFirst = 0 , EmbeddedLast = 23 , OneByteFollows = 24 , TwoByteFollows = 25 ,
  FourByteFollows = 26 , EightByteFollows = 27 , Indefinite = 31
}
enum class  Cbor::SimpleValues : uint8_t {
  Unassigned0First = 0 , Unassigned0Last = 19 , False = 20 , True = 21 ,
  Null = 22 , Undefined = 23 , ReservedFirst = 24 , ReservedLast = 31 ,
  Unassigned1First = 32 , Unassigned1Last = 255
}
 Possible values for Minor when Major is Simple/Float.

Functions

constexpr Major Cbor::getMajor (uint8_t initial)
constexpr uint8_t Cbor::setMajor (uint8_t initial, Major major)
constexpr Minor Cbor::getMinor (uint8_t initial)
constexpr uint8_t Cbor::setMinor (uint8_t initial, Minor minor)
constexpr uint8_t Cbor::initialByte (Major major, Minor minor)
consteval Minor Cbor::minorFromSizeOf (const size_t bytes)
bool Cbor::packEmbedded (Major major, uint8_t value, std::span< uint8_t > &buf)
 Pack a value [0, 23] embedded into the first header byte.
template<std::unsigned_integral Int>
bool Cbor::pack (Major major, Int value, std::span< uint8_t > &buf)
template<>
bool Cbor::pack< unsigned char > (Major major, unsigned char value, std::span< uint8_t > &buf)
template<>
bool Cbor::pack< unsigned short > (Major major, unsigned short value, std::span< uint8_t > &buf)
template<>
bool Cbor::pack< unsigned int > (Major major, unsigned int value, std::span< uint8_t > &buf)
template<>
bool Cbor::pack< unsigned long > (Major major, unsigned long value, std::span< uint8_t > &buf)
template<>
bool Cbor::pack< unsigned long long > (Major major, unsigned long long value, std::span< uint8_t > &buf)
std::optional< Item > Cbor::unpack (std::span< uint8_t > &buf)
 Unpack one item.
template<std::unsigned_integral Int>
bool Cbor::encode (Major major, Int value, std::span< uint8_t > &buf)
template<>
bool Cbor::encode< unsigned char > (Major major, unsigned char value, std::span< uint8_t > &buf)
template<>
bool Cbor::encode< unsigned short > (Major major, unsigned short value, std::span< uint8_t > &buf)
template<>
bool Cbor::encode< unsigned int > (Major major, unsigned int value, std::span< uint8_t > &buf)
template<>
bool Cbor::encode< unsigned long > (Major major, unsigned long value, std::span< uint8_t > &buf)
template<>
bool Cbor::encode< unsigned long long > (Major major, unsigned long long value, std::span< uint8_t > &buf)

Variables

constexpr uint8_t Cbor::EMBEDDED_MIN = static_cast<uint8_t>(Minor::EmbeddedFirst)
constexpr uint8_t Cbor::EMBEDDED_MAX = static_cast<uint8_t>(Minor::EmbeddedLast)

Detailed Description

Encoding for structured data (numbers, strings, bytes, arrays and maps). Also supports custom tags.

Concise Binary Object Representation (CBOR)

We need a simple, standardised (makes implementing in other languages easier) way to serialise and deserialise data. There's a couple of approaches:

  1. Define a protocol at the byte level, works well but manual.
  2. Define a protocol in a DSL/schema and let that define the byte level, which works well but can have issues with backwards compatibility when the protocol is extended. Examples of this are Cap'n Proto, ProtoBuf, ASN.1.
  3. Text-based generic/schemaless encodings such as JSON and XML, unnecessary overhead for embedded devices.
  4. Byte-based variants of JSON/XML, which is what we are going for. Some alternatives are:
    1. BSON, a binary variant of JSON, which is designed for being modifiable rather than compact.
    2. MessagePack which seems good, but compared to CBOR is limited (doesn't support tagging). Though it would probably be sufficient here, I have gone for CBOR in case it comes in useful in the future.
    3. CBOR, which is what I have gone for. This is inspired by MessagePack, and has a description which is reasonably simple to understand/implement.

High-level overview

The CBOR format has a consistent encoding across the different data-types, which makes implementation simple. The initial byte of a CBOR value is composed of: 

[](u3 major, u5 minor) { return ((major << 5) | minor); }

The major types are (from RFC8949 table 1):

Major Type Meaning Content
0 unsigned integer N -
1 negative integer -1-N -
2 byte string N bytes
3 text string N bytes (UTF-8 text)
4 array N data items (elements)
5 map 2N data items (key/value pairs)
6 tag of number N 1 data item
7 simple/float -

And the value has the following forms (adapted from RFC8949 table 3):

5-Bit Minor Semantics
0..23/0x17 Simple value (value 0..23)
24/0x18 Simple value (value 32..255 in following byte)
25/0x19 A 16 bit value
26/0x1A A 32 bit value
27/0x1B A 64 bit value
28-30/C/D/E Reserved, not well-formed in the present document
31/0x1F Start/stop indefinite length values

Floating-point values are either 16/32/64 bit values, the major 7 minor values 0-255 are simple values. Indefinite length values are strings/arrays/maps without a length known ahead of time. They are started by e.g. an array with minor value 31, and are ended with a simple/float of minor value 31. In the case of strings, it is a concatenation of definite length strings

The table of simple values from RFC8949 table 4

Value Semantics
0..19 (unassigned)
20 false
21 true
22 null
23 undefined
24..31 (reserved)
32..255 (unassigned)

Examples

I will be using the byte-string syntax b"\1\x0A" to denote a sequence of byte values in hexadecimal, so this is bytes 1, 10. The syntax N(X) denotes an item X that has an integer tag N.

Some examples to aid, from RFC8949 Appendix A (apologies about the wide table, the test/cbor.cpp code parses it so for simplicity I didn't split the rows up, but if this is rendered as HTML that doesn't matter):

Marker for test/cbor.cpp*

Decoded Encoded (initial byte, hexdump) Type
0 0:0 \ilinebr </td> <td class="markdownTableBodyNone"> uint64_t \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> 1 \ilinebr </td> <td class="markdownTableBodyNone"> `0:1` uint64_t
10 0:10 \ilinebr </td> <td class="markdownTableBodyNone"> uint64_t \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> 23 \ilinebr </td> <td class="markdownTableBodyNone"> `0:23` uint64_t
24 0:24 18 uint64_t
25 0:24 19 uint64_t
100 0:24 64 uint64_t
1000 0:25 03E8 uint64_t
1000000 0:26 000F4240 uint64_t
1000000000000 0:27 000000E8D4A51000 uint64_t
18446744073709551615 0:27 FFFFFFFFFFFFFFFF uint64_t
18446744073709551616 6:2 49010000000000000000 Skip (value too large for 64_t)
-18446744073709551616 1:27 FFFFFFFFFFFFFFFF Skip (value too large for 64_t)
-18446744073709551617 6:3 49010000000000000000 Skip (value too large for 64_t)
-1 1:0 \ilinebr </td> <td class="markdownTableBodyNone"> int64_t \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> -10 \ilinebr </td> <td class="markdownTableBodyNone"> `1:9` int64_t
-100 1:24 63 int64_t
-1000 1:25 03E7 int64_t
0.0 7:25 0000 _Float16
-0.0 7:25 8000 _Float16
1.0 7:25 3C00 _Float16
1.1 7:27 3FF199999999999A double
1.5 7:25 3E00 _Float16
65504.0 7:25 7BFF _Float16
100000.0 7:26 47C35000 float
3.4028234663852886e+38 7:26 7F7FFFFF float
1.0e+300 7:27 7E37E43C8800759C double
5.960464477539063e-8 7:25 0001 _Float16
0.00006103515625 7:25 0400 _Float16
-4.0 7:25 C400 _Float16
-4.1 7:27 C010666666666666 double
Infinity 7:25 7C00 _Float16
NaN 7:25 7E00 _Float16
-Infinity 7:25 FC00 _Float16
Infinity 7:25 7C00 float
NaN 7:25 7E00 float
-Infinity 7:25 FC00 float
Infinity 7:25 7C00 double
NaN 7:25 7E00 double
-Infinity 7:25 FC00 double
false 7:20 \ilinebr </td> <td class="markdownTableBodyNone"> bool \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> true \ilinebr </td> <td class="markdownTableBodyNone"> `7:21` bool
null 7:22 \ilinebr </td> <td class="markdownTableBodyNone"> void * \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> undefined \ilinebr </td> <td class="markdownTableBodyNone"> `7:23` N/A
simple(16) 7:16 \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> simple(255) \ilinebr </td> <td class="markdownTableBodyNone"> `7:24` `FF` \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> 0("2013-03-21T20:04:00Z") \ilinebr </td> <td class="markdownTableBodyNone"> `6:0` `74323031332D30332D32315432303A 30343A30305A` \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> 1(1363896240) \ilinebr </td> <td class="markdownTableBodyNone"> `6:1` `1A514B67B0` \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> 1(1363896240.5) \ilinebr </td> <td class="markdownTableBodyNone"> `6:1` `FB41D452D9EC200000` \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> 23(h'01020304') \ilinebr </td> <td class="markdownTableBodyNone"> `6:23` `4401020304` \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> 24(h'6449455446') \ilinebr </td> <td class="markdownTableBodyNone"> `6:24` `18456449455446` \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> 32("http://www.example.com") \ilinebr </td> <td class="markdownTableBodyNone"> `6:24` `2076687474703A2F2F7777772E6578 616D706C652E636F6D` \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> h'' \ilinebr </td> <td class="markdownTableBodyNone"> `2:0` std::span<uint8_t>
h'01020304' 2:4 01020304 std::span<uint8_t>
h'31323334' 2:4 31323334 std::span<uint8_t>
"" 3:0 \ilinebr </td> <td class="markdownTableBodyNone"> std::string_view \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> "a" \ilinebr </td> <td class="markdownTableBodyNone"> `3:1` `61` \ilinebr </td> <td class="markdownTableBodyNone"> std::string_view \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> "IETF" \ilinebr </td> <td class="markdownTableBodyNone"> `3:4` `49455446` \ilinebr </td> <td class="markdownTableBodyNone"> std::string_view \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> "\"" \ilinebr </td> <td class="markdownTableBodyNone"> `3:2` `225C` \ilinebr </td> <td class="markdownTableBodyNone"> Skip (not parsing escapes) \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> "\u00fc" \ilinebr </td> <td class="markdownTableBodyNone"> `3:2` `C3BC` \ilinebr </td> <td class="markdownTableBodyNone"> Skip (not parsing escapes) \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> "\u6c34" \ilinebr </td> <td class="markdownTableBodyNone"> `3:3` `E6B0B4` \ilinebr </td> <td class="markdownTableBodyNone"> Skip (not parsing escapes) \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> "\ud800\udd51" \ilinebr </td> <td class="markdownTableBodyNone"> `3:4` `F0908591` \ilinebr </td> <td class="markdownTableBodyNone"> Skip (not parsing escapes) \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> [] \ilinebr </td> <td class="markdownTableBodyNone"> `4:0` std::span<int>
[1, 2, 3] 4:3 010203 std::span<int>
[1, [2, 3], [4, 5]] 4:3 01820203820405 tuple<int, span<int>, span<int>>
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25] 4:24 190102030405060708090A0B0C0D0E 0F101112131415161718181819 std::span<int>
{} 5:0 ` \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> {1: 2, 3: 4} \ilinebr </td> <td class="markdownTableBodyNone"> 5:2 01020304 \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> {"a": 1, "b": [2, 3]} \ilinebr </td> <td class="markdownTableBodyNone"> 5:2 6161016162820203 \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowEven"> <td class="markdownTableBodyNone"> ["a", {"b": "c"}] \ilinebr </td> <td class="markdownTableBodyNone"> 4:2 6161A161626163 \ilinebr </td> <td class="markdownTableBodyNone"> N/A \ilinebr </td> </tr> <tr class="markdownTableRowOdd"> <td class="markdownTableBodyNone"> {"a": "A", "b": "B", "c": "C", "d": "D", "e": "E"} \ilinebr </td> <td class="markdownTableBodyNone"> 5:5 616161416162614261636143616461 4461656145‘ N/A
(_ h'0102’, h'030405') 2:31 42010243030405FF N/A
(_ "strea", "ming") 3:31 657374726561646D696E67FF N/A
[_ ] 4:31 FF N/A
[_ 1, [2, 3], [_ 4, 5]] 4:31 018202039F0405FFFF N/A
[_ 1, [2, 3], [4, 5]] 4:31 01820203820405FF N/A
[1, [2, 3], [_ 4, 5]] 4:3 018202039F0405FF N/A
[1, [_ 2, 3], [4, 5]] 4:3 019F0203FF820405 N/A
[_ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25] 4:31 0102030405060708090A0B0C0D0E0F 101112131415161718181819FF N/A
{_ "a": 1, "b": [_ 2, 3]} 5:31 61610161629F0203FFFF N/A
["a", {_ "b": "c"}] 4:2 6161BF61626163FF N/A
{_ "Fun": true, "Amt": -2} 5:31 6346756EF563416D7421FF N/A

Enumeration Type Documentation

◆ Major

enum class Cbor::Major : uint8_t
strong

Upper three bits of the initial/header byte.

Enumerator
U64 

Value is N the positive integer.

Neg64 

Value is -N-1 (note this is ~N in 2s complement)

Bytes 

Value is the number of bytes that follow, unescaped. Or the indefinite sequence of zero or more byte chunks (bytes with specified length) concatenated together.

Utf8 

Value is the number of bytes (not code-points) that follow, unescaped. Or the indefinite sequence of zero or more UTF-8 chunks (bytes with specified length) concatenated together

Array 

Value is number of items that follow, or indefinite sequence of items.

Map 

Value is number of key-value pairs that follow, or indefinite sequence of items.

Tagged 

Value is the tag, followed by a single item.

Simple 

Simple value (embedded or 8b), float (16b, 32b, 64b), or indefinite sequence.

◆ Minor

enum class Cbor::Minor : uint8_t
strong
Enumerator
EmbeddedFirst 

Embedded means no next byte, value is contained within.

OneByteFollows 

These contain the next value in the following byte(s).

Indefinite 

Bytes/Utf8/Array/Map+Indefinite starts an indefinite-length item. Simple/Float+Indefinite is a terminator for indefinite-length items.

Function Documentation

◆ encode()

template<std::unsigned_integral Int>
bool Cbor::encode ( Major major,
Int value,
std::span< uint8_t > & buf )

Encodes a value that can be up to N bits. Unlike

See also
pack<T>, it uses the smallest encoding.

◆ pack()

template<std::unsigned_integral Int>
bool Cbor::pack ( Major major,
Int value,
std::span< uint8_t > & buf )

Pack an N bit value (usually if the value is [0, 23] you want packEmbedded). Ends up as an initial header byte and ceil(N/8) value bytes