Near Real Time Mirroring (NRTM) version 4

3.1. Key Configuration

When enabling NRTMv4 publication for an IRR Database, the operator MUST generate and configure a private Elliptic Curve JSON Web Key [RFC7517] The operator then provides this public key, the name of the IRR Database, and publication URL of the Update Notification File to any operators of mirror clients. The published public key MUST be encoded in PEM. The process for providing this is not in scope of this protocol, but a typical case is publication on the operator's known website. Key rotation is described in Section 8.4.¶

It is RECOMMENDED that implementations provide easily accessible tools for operators to generate new signing keys to enter into their configuration and assist with key rotation. All configuration options SHOULD be clearly named to indicate that they are private keys.¶

3.2. Snapshot Initialization

A mirror server MUST follow the initialization steps upon the first export for an IRR Database by that mirror server, or if the server lost history and can not reliably produce a continuous set of deltas from a previous state.¶

In other words, either the mirror server guarantees that clients following the deltas have a correct and complete view, or MUST reinitialize, which will force clients to reinitialize as well.¶

Initialization consists of these actions:¶

• The mirror server MUST generate a new session ID. This MUST be a random v4 UUID [RFC4122] and MUST be the same across all client sessions. The session ID is unique to the IRR Database, so an instance that serves multiple IRR Databases, will create a separate session ID for each.¶
• The server MUST generate a snapshot for version number one. This may contain an empty array of objects if the IRR Database is currently empty.¶
• The server MUST generate a new Update Notification File with the new session ID, a reference to the new snapshot, and no deltas.¶

Note that a publication, and its associated session IDs and versions, always relates to a single specific IRR Database, even if multiple databases are published from one instance. For example, a mirror server publishing NRTMv4 for RIPE and RIPE-NONAUTH, will generate two Update Notification Files, referring two Snapshot Files, and two sets of Delta Files each with contiguous version numbers - all completely independent to each other, with different session IDs, potentially at different times. This applies even if the same IRR server instance produces both.¶

3.3. Publishing updates

4.1. Client Configuration

Mirror clients are configured with the name of the IRR Database, the URL of the Update Notification File, and the public key currently used for signing the Update Notification File. Key rotation is described in Section 8.4.¶

4.2. Initialization from snapshot

Clients MUST initialize from a Snapshot File when initially configured or if they are not able to update their local data from the provided Delta Files:¶

• The client MUST retrieve the Update Notification File.¶
• The client MUST verify that the source attribute in the Update Notification File matches the configured IRR Database name.¶
• The client MUST retrieve the Snapshot File and load the objects into its local storage.¶
• The mirror client MUST verify that the hash of the Snapshot File matches the hash in the Update Notification File that referenced it. If the Snapshot File was compressed with GZIP, the hash MUST match the compressed data. In case of a mismatch of this hash, the file MUST be rejected.¶
• The client MUST record the session_id and version of the loaded Snapshot File.¶

4.3. Processing Delta Files

If a mirror client has previously initialized from a snapshot:¶

• The client MUST retrieve the Update Notification File.¶
• The client MUST verify that the session ID matches the previously known session ID. If this does not match, the client MUST reinitialize from the snapshot.¶
• The client MUST verify that the Update Notification File version is the same or higher than the client's current most recent version, to the latest version in the Update Notification File. If not, the Update Notification File MUST be rejected. It is RECOMMENDED for the client to distinguish between an Update Notification File that is a single version older, and a much older version, in any status messages. The former can occur from time to time in synchronisation issues, the latter is more likely a faulty implementation.¶
• The client MUST verify that the Update Notification File contains one contiguous set of Delta File versions after the client's current most recent version up to the latest version in the Update Notification File. If the Delta File versions are not contiguous, the Update Notification File MUST be rejected. If the available Delta File versions do not range from the client's most recent version plus one, the client MUST reinitialize from the snapshot.¶
• The mirror client MUST verify that the hashes of each Delta and Snapshot File have not changed compared to previous entries seen for the same file type and version. If a newer Update Notification File contains a different hash for a specific file, this indicates a misconfiguration in the server and the client MUST reject the Update Notification File. The client can do this by recording the files referenced by the previous valid Update Notification File and comparing the overlapping entries with the retrieved Update Notification File.¶
• The client MUST retrieve all Delta Files for versions since the client's last known version, if there are any.¶
• The mirror client MUST verify that the hash of each newly downloaded Delta File matches the hash in the Update Notification File that referenced it. If the Delta File was compressed with GZIP, the hash MUST match the compressed file. In case of a mismatch of this hash, the Delta File MUST be rejected.¶
• The client MUST process all changes in the Delta Files in order: lowest Delta File version number first, and in the order of the changes list in the Delta File.¶
• The client MUST update its record of the most recent version to the version of the Update Notification File.¶

If the Update Notification File or one of the Delta Files is rejected, the mirror client MUST NOT process any newer Deltas than those that are valid and have been successfully verified. If some Delta Files are rejected, it MAY process the valid Delta Files, but MUST NOT skip over any rejected Delta Files while doing so.¶

4.4. Signature and Staleness Verification

Every time a mirror client retrieves a new version of the Update Notification File, it MUST verify the included signature. The signature MUST be valid for the configured public key for the contents of the Update Notification File. If the signature does not match, the mirror client MUST reject the Update Notification File, unless a key rotation is in progress as described in Section 8.4.¶

A mirror client can use the generation timestamp in the Update Notification File to check whether the file is stale, as the mirror server must update this file at least every 24 hours. If the generation timestamp is more than 24 hours ago, the file is stale and the mirror client SHOULD warn the operator in log messages or other alerting, but MAY continue to process it otherwise.¶

4.5. Policy Restrictions

A mirror client MAY have a policy that restricts the processing of objects to certain object classes, or other limitations on which objects it processes.¶

If a mirror client has a policy that restricts object processing, this MUST be applied consistently to Snapshot Files and Delta Files from the moment the policy is enacted or modified.¶

5.1. Purpose

The Update Notification File is generated by the mirror server and used by mirror clients to discover whether any changes exist between the state of the IRR mirror server and of the mirror client. It also describes the location of the Snapshot File and incremental Delta Files. Finally, the generation timestamp can be used to detect whether the file is stale.¶

The mirror server MUST generate a new Update Notification File every time there are new deltas or snapshots and, even if there have been no changes, at least every 24 hours.¶

5.2. Cache concerns

A mirror server may use caching infrastructure to cache the Update Notification File and reduce the load of HTTPS requests.¶

However, since this file is used by mirror clients to determine whether any updates are available, the mirror server SHOULD ensure that this file is not cached for longer than one minute. An exception to this rule is that it is better to serve a stale Update Notification File rather than no Update Notification File.¶

5.3. Payload format and validation

Example payload of an Update Notification File:¶

{
  "nrtm_version": 4,
  "timestamp": "2022-01-01T15:00:00Z",
  "type": "notification",
  "next_signing_key": "bnJ0..bXY0",
  "source": "EXAMPLE",
  "session_id": "ca128382-78d9-41d1-8927-1ecef15275be",
  "version": 4,
  "snapshot": {
    "version": 3,
    "url": "ca128382-78d9-41d1-8927-1ecef15275be/nrtm-snapshot.2.047595d0fae972fbed0c51b4a41c7a349e0c47bb.json.gz",
    "hash": "9a..86"
  },
  "deltas": [
    {
      "version": 2,
      "url": "ca128382-78d9-41d1-8927-1ecef15275be/nrtm-delta.1.784a2a65aba22e001fd25a1b9e8544e058fbc703.json",
      "hash": "62..a2"
    },
    {
      "version": 3,
      "url": "ca128382-78d9-41d1-8927-1ecef15275be/nrtm-delta.2.0f681f07cfab5611f3681bf030ec9f6fa3442fb0.json",
      "hash": "25..9a"
    },
    {
      "version": 4,
      "url": "ca128382-78d9-41d1-8927-1ecef15275be/nrtm-delta.3.d9c194acbb2cb0d4088c9d8a25d5871cdd802c79.json",
      "hash": "b4..13"
    }
  ],
  "metadata": {}
}

Note: hash and key values in this example are shortened because of formatting.¶

The following validation rules MUST be observed when creating or parsing Update Notification Files:¶

• The nrtm_version MUST be 4.¶
• The timestamp MUST be an [RFC3339] timestamp.¶
• The type MUST be "notification".¶
• The optional field next_signing_key is used for in-band key rotation. If present, it MUST be an Elliptic Curve JWK [RFC7517] public key encoded in PEM, which matches the private key the mirror server will start using to sign the Update Notification File in the near future. Key rotation is described in Section 8.4. If there is no next signing key, this key MUST be omitted.¶
• The source MUST be a valid IRR object name [RFC2622].¶
• The session_id attribute MUST be a random v4 UUID [RFC4122] unique to this session for this source.¶
• The version MUST be an unsigned positive integer and be equal to the highest version of the deltas and snapshot.¶
• The file MUST contain exactly one snapshot.¶
• The file MAY contain one or more deltas.¶
• The deltas MUST have a sequential contiguous set of version numbers.¶
• Each snapshot and delta element MUST have a version, URL and hash attribute. The URL must be relative to the path of the Update Notification File. For example, if the Update Notification File example above is published on https://example.com/nrtm/update-notification-file.json, the full URL for the referred snapshot is https://example.com/nrtm/ca128382-78d9-41d1-8927-1ecef15275be/nrtm-snapshot.2.047595d0fae972fbed0c51b4a41c7a349e0c47bb.json.gz. If the snapshot or delta file was compressed with GZIP, the filename MUST end in ".gz". and the hash MUST match the compressed data.¶
• The hash attribute in snapshot and delta elements MUST be the hexadecimal encoding of the SHA-256 hash [SHS] of the referenced file. The mirror client MUST verify this hash when the file is retrieved and reject the file if the hash does not match.¶
• The metadata key MAY be present, used for metadata produced by the server to aid in tracing and debugging. This can contain information like the name of the host on which the file was generated or the name and version of the software used. Each mirror server may choose which fields to include, or choose to not include any metadata. The mirror server SHOULD not cause excessive size increases by adding extensive metadata in the Update Notification File, as it is the most frequently retrieved file.¶

5.4. Encoding and signature

• The actual Update Notification File contents MUST be a JSON Web Signature [RFC7515] and MUST use JWS Compact Serialization.¶
• The JWS Payload MUST be the JavaScript Object Notation (JSON) [RFC8259] serialization of the structure described in the previous section.¶
• The filename of the serialized data MUST be "update-notification-file.jose".¶
• The algorithm MUST NOT be Deprecated, and it is RECOMMENDED to use Recommended or Recommended+ algorithms, as defined in JSON Web Algorithms [RFC7518]¶

6.1. Purpose

The Snapshot File reflects the complete and current contents of all IRR objects in an IRR Database. Mirror clients MUST use this to initialize their local copy of the IRR Database.¶

6.2. Cache Concerns

A snapshot reflects the content of the IRR Database at a specific point in time; for that reason, it can be considered immutable data. Snapshot Files MUST be published at a URL that is unique to the specific session and version. The URL MUST also contain a random value that can not be predicted before publication, to counter negative caching issues.¶

Because these files never change, they MAY be cached indefinitely. However, as snapshots are large and old snapshots will no longer be referred by newer Update Notification Files, it is RECOMMENDED that a limited interval is used in the order of hours or days.¶

To avoid race conditions where a mirror client retrieves an Update Notification File moments before it's updated, mirror servers SHOULD retain old Snapshot Files for at least 5 minutes after a new Update Notification File is published.¶

6.3. File format and validation

Example Snapshot File:¶

␞{
  "nrtm_version": 4,
  "type": "snapshot",
  "source": "EXAMPLE",
  "session_id": "ca128382-78d9-41d1-8927-1ecef15275be",
  "version": 3
}
␞{"object": "route: 192.0.2.0/24\norigin: AS65530\nsource: EXAMPLE"}
␞{"object": "route: 2001:db8::/32\norigin: AS65530\nsource: EXAMPLE"}

Note: IRR object texts in this example are shortened because of formatting.¶

The file is in JSON Text Sequences [RFC7464] format, and MUST contain one or more records (it must contain at least the header). The first record is the file header, and the following validation rules MUST be observed when creating or parsing a Snapshot File header:¶

• The nrtm_version MUST be 4.¶
• The type MUST be "snapshot".¶
• The source MUST match the source in the Update Notification File.¶
• The session_id attribute MUST match the session_id in the Update Notification File.¶
• The version MUST be an unsigned positive integer, matching the Update Notification File entry for this snapshot.¶

The remaining records (zero or more) MUST each contain a string representation of an IRR object. The source attribute in the IRR object texts MUST match the source attribute of the Snapshot File.¶

7.1. Purpose

A Delta File contains all changes for exactly one incremental update of the IRR Database. It may include new, modified and deleted objects. Delta Files can contain multiple alterations to multiple objects.¶

7.2. Cache Concerns

Deltas reflect the difference in content of the IRR Database from one version to another; for that reason, it can be considered immutable data. Delta Files MUST be published at a URL that is unique to the specific session and version. The URL MUST also contain a random value that can not be predicted before publication, to counter negative caching issues.¶

To avoid race conditions where a mirror client retrieves an Update Notification File moments before it's updated, mirror servers SHOULD retain old Delta Files for at least 5 minutes after a new Update Notification File is published that no longer contains these Delta Files.¶

7.3. File format and validation

Example Delta File:¶

␞{
  "nrtm_version": 4,
  "type": "delta",
  "source": "EXAMPLE",
  "session_id": "ca128382-78d9-41d1-8927-1ecef15275be",
  "version": 3
}
␞{
  "action": "delete",
  "object_class": "person",
  "primary_key": "PRSN1-EXAMPLE"
}
␞{
  "action": "delete",
  "object_class": "route",
  "primary_key": "192.0.2.0/24AS65530"
}
␞{
  "action": "add_modify",
  "object": "route: 2001:db8::/32\norigin: AS65530\nsource: EXAMPLE"
}

Note: IRR object texts in this example are shortened because of formatting.¶

The file is in JSON Text Sequences [RFC7464] format, and MUST contain two or more records (at least the header and one change). The first record is the file header, and the following validation rules MUST be observed when creating or parsing a Delta File header:¶

• The nrtm_version MUST be 4.¶
• The type MUST be "delta".¶
• The source MUST match the source in the Update Notification File.¶
• The session_id attribute MUST match the session_id in the Update Notification File.¶
• The version MUST be an unsigned positive integer, matching the Update Notification File entry for this delta.¶

The remaining records (one or more) MUST each contain a JSON object representing a change, which MUST meet the following rules:¶

• An action attribute, which is either "delete" for object deletions, or "add_modify" for additions or modifications.¶
• If action is "delete": an object_class attribute with the RPSL object class name, and a primary_key attribute with the primary key, of the deleted object. For objects that are listed in [RFC2622] and [RFC4012] the primary key is the value of the RPSL field defined as "class key". For object classes that define a pair of attributes as class key, e.g. route, the values of the individual attributes are appended together without separators. For any other objects, the primary key is the value of the RPSL field with the same name as the object class name. The primary key and object class name are not case sensitive and therefore mirror clients MUST use case insensitive matching against their local database.¶
• If action is "add_modify": an object attribute with the RPSL text of the new version of the object.¶

8.1. IRR object Validation

Throughout the years, various implementations of IRR servers have taken liberties with the various RFCs regarding RPSL. Implementations have introduced different new object classes, attributes and validation rules. Current IRR Databases also contain legacy objects which were created under different validation rules. In practice, there is no uniformly implemented standard for RPSL, but merely rough outlines partially documented in different places.¶

This has the potential to create interoperability issues. Some are addressed by NRTMv4, like having a consistent character set when mirroring data between implementations. However, some issues can not be addressed in this way, such as one implementation introducing a new object class that is entirely unknown to another implementation.¶

A mirror client SHOULD be able to handle unknown object classes and objects that are invalid according to its own validation rules, which may mean simply discarding them, without rejecting remaining objects or preventing future updates.¶

It is RECOMMENDED for mirror clients to log these cases, particularly those where an object was discarded due to violating validation rules. These cases create an inconsistency between the IRR objects of the server and client, and logs facilitate later analysis.¶

It is RECOMMENDED for mirror clients to be flexible where possible and reasonable when applying their own validation rules to IRR objects retrieved from mirror servers. For example, a route object with an origin attribute that is not a valid AS number can't be usefully interpreted. There is no way for an IRR server to correctly parse and index such an object. However, a route-set object whose name does not start with "RS-" [RFC2622], or an inetnum with an unknown extra "org" attribute, still allows the mirror client to interpret it unambiguously even if it does not meet the mirror client's own validation rules for authoritative records.¶

8.2. Intermediate mirror instances

An IRR Database generally has a single authoritative source. In some cases, an instance run by a third party will function as a kind of intermediate: both being a mirror client, mirroring IRR objects from the authoritative source, and simultaneously function as a mirror server to yet another mirror client.¶

There are various operational reasons for such a setup, such as the intermediate filtering certain records. Regardless of the reason, the mirror client and server function of an IRR server must be treated as separate processes. In particular, this means they MUST have separate session IDs. The intermediate server MUST NOT republish the same files it retrieved from the authoritative source with the same session ID.¶

8.3. Reading from local files

In the typical use case for NRTMv4, a mirror client retrieves files from an HTTPS endpoint. However, implementations MAY also support reading from files on the local filesystem instead, for when operators want to use a different method to retrieve or distribute the files. When reading from local files, mirror clients SHOULD still follow all validation rules, including the validation of the signature and hashes. Mirror clients MUST NOT allow a mix of reading from local files and HTTPS for a specific IRR Database.¶