trillian

Trillian Personalities

Overview

The core Trillian service provides a Merkle tree store (and primitives to interact with it), but a full transparency application requires additional functionality on top of that base, which needs to be provided by a separate personality.

The following sections describe the conceptual responsibilities of a Trillian personality, followed by the practical responsibilities that are likely to be needed for real-world use. Overall design considerations for transparent Log applications are also discussed in a separate document.

Conceptual Responsibilities

Admission Control

The primary purpose of a personality is to implement admission criteria for the store, so that only particular types of data are added to the store. For example, a Certificate Transparency Log only accepts data items that are valid certificates; a “CT Log” personality would police this, so that the Trillian service can process all incoming data blindly.

It is common for these admission criteria to be broken into two distinct but related areas:

Canonicalization

A personality may also perform canonicalization on incoming data, to convert equivalent formulations of the same underlying data to a single canonical format, avoiding needless duplication. For example, keys in JSON dictionaries could be sorted, or Unicode string data could be normalised.

One particularly important form of canonicalization is de-duplication of timestamped data: if the Merkle leaf contents includes a timestamp indicating when the leaf was added, subsequent submissions of the same data with a new timestamp should not be treated as a new leaf.

This use case is supported by the Trillian API, which includes the concept of an identity hash associated with a data item: items with the same identity hash are assumed equal, even if the full Merkle leaf structure differs (e.g. because it has a different timestamp attached).

This means that a personality that wants to prevent duplicate entries can do so by setting the LeafIdentityHash value on new Merkle tree leaves appropriately. The personality code also needs to cope with duplicates: the Merkle tree leaf in a successful QueueLeaves response may not be the same as the tree leaf in the corresponding QueueLeaves request (it will have the same LeafIdentityHash but may differ in fields that are not covered by this hash, e.g. it may have an earlier timestamp).

Practical Responsibilities

External API

In many environments the personality provides an externally-visible API. The design of such an API depends on the application, but key points to consider include:

Storage

A Trillian personality may need an additional storage mechanism over and above the Merkle tree storage provided by Trillian. For example, if Trillian is run by a separate operator, then audit verification data may need to be stored.

However, note that the Trillian APIs allow a personality to store additional data that is associated with a leaf in the underlying Merkle tree, in the ExtraData field. This extra data is immutable but is not covered by the Merkle tree hashing algorithms, and so cannot be used for verifiable/cryptographic purposes (as it could be modified without affecting tree hash values).

Traffic Control

The core Trillian service can be configured to provide traffic control limits using its quota system; out of the box, this can be governed by:

A particular application may also want to provide other kinds of limits. This is supported by the per-user quota limits, where the user is entirely specified by the Trillian personality, using the ChargeTo field in each Trillian API request.

A typical use case for this is to add a ChargeTo.User string that identifies an external requestor, so traffic from a single source can be limited without affecting other users.

However, the user quota system can also be used for more flexible limits – for example, by applying limits to particular authentication keys.

Monitoring

A personality that is in real-world use should probably be monitored, for performance, alerting and debugging. Trillian uses a metrics interface to support this, which can also be re-used in an associated personality.

Examples

This section describes two Trillian personalities, together with their choices for the various design decisions described above.

Certificate Transparency

Certificate Transparency (CT) is the original transparency application, and has been successful in improving the security of the web PKI ecosystem. It is also the most mature and widely deployed Trillian use case, built on the CTFE personality.

The conceptual responsibilities of the CTFE personality are given by:

In terms of practical responsibilities:

Gossip Hub

Users of individual transparent Logs are theoretically susceptible to a split-view attack, where a Log prevents a different view of its Merkle tree to different users – each user’s view of the tree is internally consistent, but differs between users.

A gossip protocol is a key mechanism to prevent this: different observers of a transparent Log share their views of the Log, allowing cross-comparison and eventual detection of split views.

The Gossip Hub is an experimental implementation of a repository for information about a Log’s Merkle tree, in the form of (another) transparent Log. This allows cross-comparison of the source Log’s Merkle tree: different observers can submit their view of the source Log, and retrieve other observers’ views of the same source Log.

(Aside: in practice, split-view attacks are less of a concern for Certificate Transparency and the web PKI because browser policies typically require individual certificates to be logged in multiple independent CT Logs, limiting the scope of any individual Log’s split-view attack.)

The conceptual responsibilities of the Gossip Hub personality are given by:

In terms of practical responsibilities: