The core Trillian service provides a Merkle tree store (and primitives to interact with it), but a full transparency application requires additional functionality on top of that base, which needs to be provided by a separate personality.
The following sections describe the conceptual responsibilities of a Trillian personality, followed by the practical responsibilities that are likely to be needed for real-world use. Overall design considerations for transparent Log applications are also discussed in a separate document.
The primary purpose of a personality is to implement admission criteria for the store, so that only particular types of data are added to the store. For example, a Certificate Transparency Log only accepts data items that are valid certificates; a “CT Log” personality would police this, so that the Trillian service can process all incoming data blindly.
It is common for these admission criteria to be broken into two distinct but related areas:
A personality may also perform canonicalization on incoming data, to convert equivalent formulations of the same underlying data to a single canonical format, avoiding needless duplication. For example, keys in JSON dictionaries could be sorted, or Unicode string data could be normalised.
One particularly important form of canonicalization is de-duplication of timestamped data: if the Merkle leaf contents includes a timestamp indicating when the leaf was added, subsequent submissions of the same data with a new timestamp should not be treated as a new leaf.
This use case is supported by the Trillian API, which includes the concept of an identity hash associated with a data item: items with the same identity hash are assumed equal, even if the full Merkle leaf structure differs (e.g. because it has a different timestamp attached).
This means that a personality that wants to prevent duplicate entries can
do so by setting the LeafIdentityHash
value on new Merkle tree leaves
appropriately. The personality code also needs to cope with duplicates: the
Merkle tree leaf in a successful QueueLeaves
response may not be the same
as the tree leaf in the corresponding QueueLeaves
request (it will have the
same LeafIdentityHash
but may differ in fields that are not covered by this
hash, e.g. it may have an earlier timestamp).
In many environments the personality provides an externally-visible API. The design of such an API depends on the application, but key points to consider include:
curl
and wget
. (The Certificate Transparency
API described in RFC 6962 takes
this approach.)A Trillian personality may need an additional storage mechanism over and above the Merkle tree storage provided by Trillian. For example, if Trillian is run by a separate operator, then audit verification data may need to be stored.
However, note that the Trillian APIs allow a personality to store additional
data that is associated with a leaf in the underlying Merkle tree, in the
ExtraData
field. This extra data is immutable but is not covered by the
Merkle tree hashing algorithms, and so cannot be used for
verifiable/cryptographic purposes (as it could be modified without affecting
tree hash values).
The core Trillian service can be configured to provide traffic control limits using its quota system; out of the box, this can be governed by:
A particular application may also want to provide other kinds of limits. This
is supported by the per-user quota limits, where the user is entirely
specified by the Trillian personality, using the ChargeTo
field in each
Trillian API request.
A typical use case for this is to add a ChargeTo.User
string that identifies
an external requestor, so traffic from a single source can be limited without
affecting other users.
However, the user quota system can also be used for more flexible limits – for example, by applying limits to particular authentication keys.
A personality that is in real-world use should probably be monitored, for performance, alerting and debugging. Trillian uses a metrics interface to support this, which can also be re-used in an associated personality.
This section describes two Trillian personalities, together with their choices for the various design decisions described above.
Certificate Transparency (CT) is the original transparency application, and has been successful in improving the security of the web PKI ecosystem. It is also the most mature and widely deployed Trillian use case, built on the CTFE personality.
The conceptual responsibilities of the CTFE personality are given by:
LeafIdentityHash
which covers the
leaf certificate, but also have a timestamp generated by the CTFE. A
re-submission of a previously-seen certificate will return the original
leaf with its earlier timestamp.In terms of practical responsibilities:
ExtraData
associated with a Merkle
tree leaf.Users of individual transparent Logs are theoretically susceptible to a split-view attack, where a Log prevents a different view of its Merkle tree to different users – each user’s view of the tree is internally consistent, but differs between users.
A gossip protocol is a key mechanism to prevent this: different observers of a transparent Log share their views of the Log, allowing cross-comparison and eventual detection of split views.
The Gossip Hub is an experimental implementation of a repository for information about a Log’s Merkle tree, in the form of (another) transparent Log. This allows cross-comparison of the source Log’s Merkle tree: different observers can submit their view of the source Log, and retrieve other observers’ views of the same source Log.
(Aside: in practice, split-view attacks are less of a concern for Certificate Transparency and the web PKI because browser policies typically require individual certificates to be logged in multiple independent CT Logs, limiting the scope of any individual Log’s split-view attack.)
The conceptual responsibilities of the Gossip Hub personality are given by:
LeafIdentityHash
which covers the
submitted data and signature, but also have a timestamp generated by the
Hub. A re-submission of a previously-seen entry will return the original
leaf with its earlier timestamp.In terms of practical responsibilities:
ExtraData
field is not (currently) used.