Understanding Reticulum

-

This chapter will briefly describe the overall purpose and operating principles of Reticulum. -It should give you an overview of how the stack works, and an understanding of how to -develop networked applications using Reticulum.

-

This chapter is not an exhaustive source of information on Reticulum, at least not yet. Currently, -the only complete repository, and final authority on how Reticulum actually functions, is the Python -reference implementation and API reference. That being said, this chapter is an essential resource in -understanding how Reticulum works from a high-level perspective, along with the general principles of -Reticulum, and how to apply them when creating your own networks or software.

-

After reading this chapter, you should be well-equipped to understand how a Reticulum network -operates, what it can achieve, and how you can use it yourself. This chapter also seeks to provide an overview of the -sentiments and the philosophy behind Reticulum, what problems it seeks to solve, and how it -approaches those solutions.

+

This chapter will briefly describe the overall purpose and operating principles of Reticulum. It should give you an overview of how the stack works, and an understanding of how to develop networked applications using Reticulum.

+

This chapter is not an exhaustive source of information on Reticulum, at least not yet. Currently, the only complete repository, and final authority on how Reticulum actually functions, is the Python reference implementation and API reference. That being said, this chapter is an essential resource in understanding how Reticulum works from a high-level perspective, along with the general principles of Reticulum, and how to apply them when creating your own networks or software.

+

After reading this chapter, you should be well-equipped to understand how a Reticulum network operates, what it can achieve, and how you can use it yourself. This chapter also seeks to provide an overview of the sentiments and the philosophy behind Reticulum, what problems it seeks to solve, and how it approaches those solutions.

Motivation

-

The primary motivation for designing and implementing Reticulum has been the current lack of -reliable, functional and secure minimal-infrastructure modes of digital communication. It is my -belief that it is highly desirable to create a reliable and efficient way to set up long-range digital -communication networks that can securely allow exchange of information between people and -machines, with no central point of authority, control, censorship or barrier to entry.

-

Almost all of the various networking systems in use today share a common limitation: They -require large amounts of coordination and centralised trust and power to function. To join such networks, you need approval -of gatekeepers in control. This need for coordination and trust inevitably leads to an environment of -central control, where it’s very easy for infrastructure operators or governments to control or alter -traffic, and censor or persecute unwanted actors. It also makes it completely impossible to freely deploy -and use networks at will, like one would use other common tools that enhance individual agency and freedom.

-

Reticulum aims to require as little coordination and trust as possible. It aims to make secure, -anonymous and permissionless networking and information exchange a tool that anyone can just pick up and use.

-

Since Reticulum is completely medium agnostic, it can be used to build networks on whatever is best -suited to the situation, or whatever you have available. In some cases, this might be packet radio -links over VHF frequencies, in other cases it might be a 2.4 GHz -network using off-the-shelf radios, or it might be using common LoRa development boards.

-

At the time of release of this document, the fastest and easiest setup for development and testing is using -LoRa radio modules with an open source firmware (see the section Reference Setup), -connected to any kind of computer or mobile device that Reticulum can run on.

-

The ultimate aim of Reticulum is to allow anyone to be their own network operator, and to make it -cheap and easy to cover vast areas with a myriad of independent, interconnectable and autonomous networks. -Reticulum is not one network, it is a tool to build thousands of networks. Networks without -kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate -with each other, and require no central oversight. Networks for human beings. Networks for the people.

+

The primary motivation for designing and implementing Reticulum has been the current lack of reliable, functional and secure minimal-infrastructure modes of digital communication. It is my belief that it is highly desirable to create a reliable and efficient way to set up long-range digital communication networks that can securely allow exchange of information between people and machines, with no central point of authority, control, censorship or barrier to entry.

+

Almost all of the various networking systems in use today share a common limitation: They require large amounts of coordination and centralised trust and power to function. To join such networks, you need approval of gatekeepers in control. This need for coordination and trust inevitably leads to an environment of central control, where it’s very easy for infrastructure operators or governments to control or alter traffic, and censor or persecute unwanted actors. It also makes it completely impossible to freely deploy and use networks at will, like one would use other common tools that enhance individual agency and freedom.

+

Reticulum aims to require as little coordination and trust as possible. It aims to make secure, anonymous and permissionless networking and information exchange a tool that anyone can just pick up and use.

+

Since Reticulum is completely medium agnostic, it can be used to build networks on whatever is best suited to the situation, or whatever you have available. In some cases, this might be packet radio links over VHF frequencies, in other cases it might be a 2.4 GHz network using off-the-shelf radios, or it might be using common LoRa development boards.

+

At the time of release of this document, the fastest and easiest setup for development and testing is using LoRa radio modules with an open source firmware (see the section Reference Setup), connected to any kind of computer or mobile device that Reticulum can run on.

+

The ultimate aim of Reticulum is to allow anyone to be their own network operator, and to make it cheap and easy to cover vast areas with a myriad of independent, interconnectable and autonomous networks. Reticulum is not one network, it is a tool to build thousands of networks. Networks without kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate with each other, and require no central oversight. Networks for human beings. Networks for the people.

Goals

-

To be as widely usable and efficient to deploy as possible, the following goals have been used to -guide the design of Reticulum:

+

To be as widely usable and efficient to deploy as possible, the following goals have been used to guide the design of Reticulum:

  • Fully useable as open source software stack

    Reticulum must be implemented with, and be able to run using only open source software. This is @@ -344,8 +315,8 @@ information about oneself.

  • Unlicensed use

    Reticulum shall be functional over physical communication mediums that do not require any form of license to use. Reticulum must be designed in a way, so it is usable over ISM radio -frequency bands, and can provide functional long distance links in such conditions, for example -by connecting a modem to a PMR or CB radio, or by using LoRa or WiFi modules.

    +frequency bands, and can provide functional long distance links in such conditions, for +example by connecting a modem to a PMR or CB radio, or by using LoRa or WiFi modules.

    • @@ -353,15 +324,15 @@ by connecting a modem to a PMR or CB radio, or by using LoRa or WiFi modules.

    Supplied software

    In addition to the core networking stack and API, that allows a developer to build applications with Reticulum, a basic set of Reticulum-based communication tools must be implemented and released along with Reticulum itself. These shall serve both as a -functional, basic communication suite, and as an example and learning resource to others wishing -to build applications with Reticulum.

    +functional, basic communication suite, and as an example and learning resource to others +wishing to build applications with Reticulum.

  • -
    Ease of use

    The reference implementation of Reticulum is written in Python, to make it easy to use -and understand. A programmer with only basic experience should be able to use -Reticulum to write networked applications.

    +
    Ease of use

    The reference implementation of Reticulum is written in Python, to make it easy to use and +understand. A programmer with only basic experience should be able to use Reticulum to write +networked applications.

    • @@ -377,43 +348,16 @@ needs to be purchased.

Introduction & Basic Functionality

-

Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at its -core a message oriented system. It is suited for both local point-to-point or point-to-multipoint -scenarios where all nodes are within range of each other, as well as scenarios where packets need -to be transported over multiple hops in a complex network to reach the recipient.

-

Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead -Reticulum uses the singular concept of destinations. Any application using Reticulum as its -networking stack will need to create one or more destinations to receive data, and know the -destinations it needs to send data to.

-

All destinations in Reticulum are represented as a 16 byte hash. This hash is derived from truncating a full -SHA-256 hash of identifying characteristics of the destination. To users, the destination addresses -will be displayed as 16 hexadecimal bytes, like this example: <13425ec15b621c1d928589718000d814>.

-

The truncation size of 16 bytes (128 bits) for destinations has been chosen as a reasonable trade-off -between address space -and packet overhead. The address space accommodated by this size can support many billions of -simultaneously active devices on the same network, while keeping packet overhead low, which is -essential on low-bandwidth networks. In the very unlikely case that this address space nears -congestion, a one-line code change can upgrade the Reticulum address space all the way up to 256 -bits, ensuring the Reticulum address space could potentially support galactic-scale networks. -This is obviously complete and ridiculous over-allocation, and as such, the current 128 bits should -be sufficient, even far into the future.

-

By default Reticulum encrypts all data using elliptic curve cryptography and AES. Any packet sent to a -destination is encrypted with a per-packet derived key. Reticulum can also set up an encrypted -channel to a destination, called a Link. Both data sent over Links and single packets offer -Initiator Anonymity. Links additionally offer Forward Secrecy by default, employing an Elliptic Curve -Diffie Hellman key exchange on Curve25519 to derive per-link ephemeral keys. Asymmetric, link-less -packet communication can also provide forward secrecy, with automatic key ratcheting, by enabling -ratchets on a per-destination basis. The multi-hop transport, coordination, verification and reliability -layers are fully autonomous and also based on elliptic curve cryptography.

-

Reticulum also offers symmetric key encryption for group-oriented communications, as well as -unencrypted packets (for local broadcast purposes only).

-

Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports, -and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or -private IP networks.

+

Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at its core a message oriented system. It is suited for both local point-to-point or point-to-multipoint scenarios where all nodes are within range of each other, as well as scenarios where packets need to be transported over multiple hops in a complex network to reach the recipient.

+

Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead Reticulum uses the singular concept of destinations. Any application using Reticulum as its networking stack will need to create one or more destinations to receive data, and know the destinations it needs to send data to.

+

All destinations in Reticulum are represented as a 16 byte hash. This hash is derived from truncating a full SHA-256 hash of identifying characteristics of the destination. To users, the destination addresses will be displayed as 16 hexadecimal bytes, like this example: <13425ec15b621c1d928589718000d814>.

+

The truncation size of 16 bytes (128 bits) for destinations has been chosen as a reasonable trade-off between address space and packet overhead. The address space accommodated by this size can support many billions of simultaneously active devices on the same network, while keeping packet overhead low, which is essential on low-bandwidth networks. In the very unlikely case that this address space nears congestion, a one-line code change can upgrade the Reticulum address space all the way up to 256 bits, ensuring the Reticulum address space could potentially support galactic-scale networks. This is obviously complete and ridiculous over-allocation, and as such, the current 128 bits should be sufficient, even far into the future.

+

By default Reticulum encrypts all data using elliptic curve cryptography and AES. Any packet sent to a destination is encrypted with a per-packet derived key. Reticulum can also set up an encrypted channel to a destination, called a Link. Both data sent over Links and single packets offer Initiator Anonymity. Links additionally offer Forward Secrecy by default, employing an Elliptic Curve Diffie Hellman key exchange on Curve25519 to derive per-link ephemeral keys. Asymmetric, link-less packet communication can also provide forward secrecy, with automatic key ratcheting, by enabling ratchets on a per-destination basis. The multi-hop transport, coordination, verification and reliability layers are fully autonomous and also based on elliptic curve cryptography.

+

Reticulum also offers symmetric key encryption for group-oriented communications, as well as unencrypted packets (for local broadcast purposes only).

+

Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports, and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or private IP networks.

Destinations

-

To receive and send data with the Reticulum stack, an application needs to create one or more -destinations. Reticulum uses three different basic destination types, and one special:

+

To receive and send data with the Reticulum stack, an application needs to create one or more destinations. Reticulum uses three different basic destination types, and one special:

  • Single

    The single destination type is the most common type in Reticulum, and should be used for @@ -428,9 +372,9 @@ only be readable by the creator of the destination, who holds the corresponding number of users, or should be readable by anyone. Traffic to a plain destination is not encrypted. Generally, plain destinations can be used for broadcast information intended to be public. Plain destinations are only reachable directly, and packets addressed to plain destinations are -never transported over multiple hops in the network. To be transportable over multiple hops in Reticulum, information -must be encrypted, since Reticulum uses the per-packet encryption to verify routing paths and -keep them alive.

    +never transported over multiple hops in the network. To be transportable over multiple hops in +Reticulum, information must be encrypted, since Reticulum uses the per-packet encryption to verify +routing paths and keep them alive.

    • @@ -455,14 +399,9 @@ out requests and responses, large data transfers and more.

Destination Naming

-

Destinations are created and named in an easy to understand dotted notation of aspects, and -represented on the network as a hash of this value. The hash is a SHA-256 truncated to 128 bits. The -top level aspect should always be a unique identifier for the application using the destination. -The next levels of aspects can be defined in any way by the creator of the application.

-

Aspects can be as long and as plentiful as required, and a resulting long destination name will not -impact efficiency, as names are always represented as truncated SHA-256 hashes on the network.

-

As an example, a destination for a environmental monitoring application could be made up of the -application name, a device type and measurement type, like this:

+

Destinations are created and named in an easy to understand dotted notation of aspects, and represented on the network as a hash of this value. The hash is a SHA-256 truncated to 128 bits. The top level aspect should always be a unique identifier for the application using the destination. The next levels of aspects can be defined in any way by the creator of the application.

+

Aspects can be as long and as plentiful as required, and a resulting long destination name will not impact efficiency, as names are always represented as truncated SHA-256 hashes on the network.

+

As an example, a destination for a environmental monitoring application could be made up of the application name, a device type and measurement type, like this:

app name  : environmentlogger
 aspects   : remotesensor, temperature
 
@@ -470,26 +409,15 @@ full name : environmentlogger.remotesensor.temperature
 hash      : 4faf1b2e0a077e6a9d92fa051f256038
-

For the single destination, Reticulum will automatically append the associated public key as a -destination aspect before hashing. This is done to ensure only the correct destination is reached, -since anyone can listen to any destination name. Appending the public key ensures that a given -packet is only directed at the destination that holds the corresponding private key to decrypt the -packet.

+

For the single destination, Reticulum will automatically append the associated public key as a destination aspect before hashing. This is done to ensure only the correct destination is reached, since anyone can listen to any destination name. Appending the public key ensures that a given packet is only directed at the destination that holds the corresponding private key to decrypt the packet.

Take note! There is a very important concept to understand here:

  • Anyone can use the destination name environmentlogger.remotesensor.temperature

  • Each destination that does so will still have a unique destination hash, and thus be uniquely addressable, because their public keys will differ.

-

In actual use of single destination naming, it is advisable not to use any uniquely identifying -features in aspect naming. Aspect names should be general terms describing what kind of destination -is represented. The uniquely identifying aspect is always achieved by appending the public key, -which expands the destination into a uniquely identifiable one. Reticulum does this automatically.

-

Any destination on a Reticulum network can be addressed and reached simply by knowing its -destination hash (and public key, but if the public key is not known, it can be requested from the -network simply by knowing the destination hash). The use of app names and aspects makes it easy to -structure Reticulum programs and makes it possible to filter what information and data your program -receives.

+

In actual use of single destination naming, it is advisable not to use any uniquely identifying features in aspect naming. Aspect names should be general terms describing what kind of destination is represented. The uniquely identifying aspect is always achieved by appending the public key, which expands the destination into a uniquely identifiable one. Reticulum does this automatically.

+

Any destination on a Reticulum network can be addressed and reached simply by knowing its destination hash (and public key, but if the public key is not known, it can be requested from the network simply by knowing the destination hash). The use of app names and aspects makes it easy to structure Reticulum programs and makes it possible to filter what information and data your program receives.

To recap, the different destination types should be used in the following situations:

  • @@ -509,25 +437,13 @@ indirectly, but must first be established through a single destination.
-

To communicate with a single destination, you need to know its public key. Any method for -obtaining the public key is valid, but Reticulum includes a simple mechanism for making other -nodes aware of your destinations public key, called the announce. It is also possible to request -an unknown public key from the network, as all transport instances serve as a distributed ledger -of public keys.

-

Note that public key information can be shared and verified in other ways than using the -built-in announce functionality, and that it is therefore not required to use the announce and path request -functionality to obtain public keys. It is by far the easiest though, and should definitely be used -if there is not a very good reason for doing it differently.

+

To communicate with a single destination, you need to know its public key. Any method for obtaining the public key is valid, but Reticulum includes a simple mechanism for making other nodes aware of your destinations public key, called the announce. It is also possible to request an unknown public key from the network, as all transport instances serve as a distributed ledger of public keys.

+

Note that public key information can be shared and verified in other ways than using the built-in announce functionality, and that it is therefore not required to use the announce and path request functionality to obtain public keys. It is by far the easiest though, and should definitely be used if there is not a very good reason for doing it differently.

Public Key Announcements

-

An announce will send a special packet over any relevant interfaces, containing all needed -information about the destination hash and public key, and can also contain some additional, -application specific data. The entire packet is signed by the sender to ensure authenticity. It is not -required to use the announce functionality, but in many cases it will be the simplest way to share -public keys on the network. The announce mechanism also serves to establish end-to-end connectivity -to the announced destination, as the announce propagates through the network.

+

An announce will send a special packet over any relevant interfaces, containing all needed information about the destination hash and public key, and can also contain some additional, application specific data. The entire packet is signed by the sender to ensure authenticity. It is not required to use the announce functionality, but in many cases it will be the simplest way to share public keys on the network. The announce mechanism also serves to establish end-to-end connectivity to the announced destination, as the announce propagates through the network.

As an example, an announce in a simple messenger application might contain the following information:

  • The announcers destination hash

    • @@ -536,78 +452,39 @@ to the announced destination, as the announce propagates through the network.

    A random blob, making each new announce unique

  • An Ed25519 signature of the above information, verifying authenticity

-

With this information, any Reticulum node that receives it will be able to reconstruct an outgoing -destination to securely communicate with that destination. You might have noticed that there is one -piece of information lacking to reconstruct full knowledge of the announced destination, and that is -the aspect names of the destination. These are intentionally left out to save bandwidth, since they -will be implicit in almost all cases. The receiving application will already know them. If a destination -name is not entirely implicit, information can be included in the application specific data part that -will allow the receiver to infer the naming.

+

With this information, any Reticulum node that receives it will be able to reconstruct an outgoing destination to securely communicate with that destination. You might have noticed that there is one piece of information lacking to reconstruct full knowledge of the announced destination, and that is the aspect names of the destination. These are intentionally left out to save bandwidth, since they will be implicit in almost all cases. The receiving application will already know them. If a destination name is not entirely implicit, information can be included in the application specific data part that will allow the receiver to infer the naming.

It is important to note that announces will be forwarded throughout the network according to a certain pattern. This will be detailed in the section The Announce Mechanism in Detail.

-

In Reticulum, destinations are allowed to move around the network at will. This is very different from -protocols such as IP, where an address is always expected to stay within the network segment it was assigned in. -This limitation does not exist in Reticulum, and any destination is completely portable over the entire topography -of the network, and can even be moved to other Reticulum networks than the one it was created in, and -still become reachable. To update its reachability, a destination simply needs to send an announce on any -networks it is part of. After a short while, it will be globally reachable in the network.

+

In Reticulum, destinations are allowed to move around the network at will. This is very different from protocols such as IP, where an address is always expected to stay within the network segment it was assigned in. This limitation does not exist in Reticulum, and any destination is completely portable over the entire topography of the network, and can even be moved to other Reticulum networks than the one it was created in, and still become reachable. To update its reachability, a destination simply needs to send an announce on any networks it is part of. After a short while, it will be globally reachable in the network.

Seeing how single destinations are always tied to a private/public key pair leads us to the next topic.

Identities

-

In Reticulum, an identity does not necessarily represent a personal identity, but is an abstraction that -can represent any kind of verifiable entity. This could very well be a person, but it could also be the -control interface of a machine, a program, robot, computer, sensor or something else entirely. In -general, any kind of agent that can act, or be acted upon, or store or manipulate information, can be -represented as an identity. An identity can be used to create any number of destinations.

-

A single destination will always have an identity tied to it, but not plain or group -destinations. Destinations and identities share a multilateral connection. You can create a -destination, and if it is not connected to an identity upon creation, it will just create a new one to use -automatically. This may be desirable in some situations, but often you will probably want to create -the identity first, and then use it to create new destinations.

-

As an example, we could use an identity to represent the user of a messaging application. -Destinations can then be created by this identity to allow communication to reach the user. -In all cases it is of great importance to store the private keys associated with any -Reticulum Identity securely and privately, since obtaining access to the identity keys equals -obtaining access and controlling reachability to any destinations created by that identity.

+

In Reticulum, an identity does not necessarily represent a personal identity, but is an abstraction that can represent any kind of verifiable entity. This could very well be a person, but it could also be the control interface of a machine, a program, robot, computer, sensor or something else entirely. In general, any kind of agent that can act, or be acted upon, or store or manipulate information, can be represented as an identity. An identity can be used to create any number of destinations.

+

A single destination will always have an identity tied to it, but not plain or group destinations. Destinations and identities share a multilateral connection. You can create a destination, and if it is not connected to an identity upon creation, it will just create a new one to use automatically. This may be desirable in some situations, but often you will probably want to create the identity first, and then use it to create new destinations.

+

As an example, we could use an identity to represent the user of a messaging application. Destinations can then be created by this identity to allow communication to reach the user. In all cases it is of great importance to store the private keys associated with any Reticulum Identity securely and privately, since obtaining access to the identity keys equals obtaining access and controlling reachability to any destinations created by that identity.

Getting Further

-

The above functions and principles form the core of Reticulum, and would suffice to create -functional networked applications in local clusters, for example over radio links where all interested -nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple -hops in the network.

+

The above functions and principles form the core of Reticulum, and would suffice to create functional networked applications in local clusters, for example over radio links where all interested nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple hops in the network.

In the following sections, two concepts that allow this will be introduced, paths and links.

Reticulum Transport

-

The methods of routing used in traditional networks are fundamentally incompatible with the physical medium -types and circumstances that Reticulum was designed to handle. These mechanisms mostly assume trust at the physical layer, -and often needs a lot more bandwidth than Reticulum can assume is available. Since Reticulum is designed to -survive running over open radio spectrum, no such trust can be assumed, and bandwidth is often very limited.

-

To overcome such challenges, Reticulum’s Transport system uses asymmetric elliptic curve cryptography to -implement the concept of paths that allow discovery of how to get information closer to a certain -destination. It is important to note that no single node in a Reticulum network knows the complete -path to a destination. Every Transport node participating in a Reticulum network will only -know the most direct way to get a packet one hop closer to it’s destination.

+

The methods of routing used in traditional networks are fundamentally incompatible with the physical medium types and circumstances that Reticulum was designed to handle. These mechanisms mostly assume trust at the physical layer, and often needs a lot more bandwidth than Reticulum can assume is available. Since Reticulum is designed to survive running over open radio spectrum, no such trust can be assumed, and bandwidth is often very limited.

+

To overcome such challenges, Reticulum’s Transport system uses asymmetric elliptic curve cryptography to implement the concept of paths that allow discovery of how to get information closer to a certain destination. It is important to note that no single node in a Reticulum network knows the complete path to a destination. Every Transport node participating in a Reticulum network will only know the most direct way to get a packet one hop closer to it’s destination.

Node Types

-

Currently, Reticulum distinguishes between two types of network nodes. All nodes on a Reticulum network -are Reticulum Instances, and some are also Transport Nodes. If a system running Reticulum is fixed in -one place, and is intended to be kept available most of the time, it is a good contender to be a Transport Node.

-

Any Reticulum Instance can become a Transport Node by enabling it in the configuration. -This distinction is made by the user configuring the node, and is used to determine what nodes on the -network will help forward traffic, and what nodes rely on other nodes for wider connectivity.

-

If a node is an Instance it should be given the configuration directive enable_transport = No, which -is the default setting.

+

Currently, Reticulum distinguishes between two types of network nodes. All nodes on a Reticulum network are Reticulum Instances, and some are also Transport Nodes. If a system running Reticulum is fixed in one place, and is intended to be kept available most of the time, it is a good contender to be a Transport Node.

+

Any Reticulum Instance can become a Transport Node by enabling it in the configuration. This distinction is made by the user configuring the node, and is used to determine what nodes on the network will help forward traffic, and what nodes rely on other nodes for wider connectivity.

+

If a node is an Instance it should be given the configuration directive enable_transport = No, which is the default setting.

If it is a Transport Node, it should be given the configuration directive enable_transport = Yes.

The Announce Mechanism in Detail

-

When an announce for a destination is transmitted by a Reticulum instance, it will be forwarded by -any transport node receiving it, but according to some specific rules:

+

When an announce for a destination is transmitted by a Reticulum instance, it will be forwarded by any transport node receiving it, but according to some specific rules:

  • If this exact announce has already been received before, ignore it.
    @@ -654,16 +531,8 @@ application specific data, it will replace the old announce.
-

Once an announce has reached a transport node in the network, any other node in direct contact with that -transport node will be able to reach the destination the announce originated from, simply by sending a packet -addressed to that destination. Any transport node with knowledge of the announce will be able to direct the -packet towards the destination by looking up the most efficient next node to the destination.

-

According to these rules, an announce will propagate throughout the network in a predictable way, -and make the announced destination reachable in a short amount of time. Fast networks that have the -capacity to process many announces can reach full convergence very quickly, even when constantly adding -new destinations. Slower segments of such networks might take a bit longer to gain full knowledge about -the wide and fast networks they are connected to, but can still do so over time, while prioritising full -and quickly converging end-to-end connectivity for their local, slower segments.

+

Once an announce has reached a transport node in the network, any other node in direct contact with that transport node will be able to reach the destination the announce originated from, simply by sending a packet addressed to that destination. Any transport node with knowledge of the announce will be able to direct the packet towards the destination by looking up the most efficient next node to the destination.

+

According to these rules, an announce will propagate throughout the network in a predictable way, and make the announced destination reachable in a short amount of time. Fast networks that have the capacity to process many announces can reach full convergence very quickly, even when constantly adding new destinations. Slower segments of such networks might take a bit longer to gain full knowledge about the wide and fast networks they are connected to, but can still do so over time, while prioritising full and quickly converging end-to-end connectivity for their local, slower segments.

Tip

Even very slow networks, that simply don’t have the capacity to ever reach full convergence @@ -674,16 +543,11 @@ actually requested by nodes on these.

knowledge, since paths can always be recursively resolved from other segments that do have knowledge about them.

-

In general, even extremely complex networks, that utilize the maximum 128 hops will converge to full -end-to-end connectivity in about one minute, given there is enough bandwidth available to process -the required amount of announces.

+

In general, even extremely complex networks, that utilize the maximum 128 hops will converge to full end-to-end connectivity in about one minute, given there is enough bandwidth available to process the required amount of announces.

Reaching the Destination

-

In networks with changing topology and trustless connectivity, nodes need a way to establish -verified connectivity with each other. Since the underlying network mediums are assumed to be trustless, Reticulum -must provide a way to guarantee that the peer you are communicating with is actually who you -expect. Reticulum offers two ways to do this.

+

In networks with changing topology and trustless connectivity, nodes need a way to establish verified connectivity with each other. Since the underlying network mediums are assumed to be trustless, Reticulum must provide a way to guarantee that the peer you are communicating with is actually who you expect. Reticulum offers two ways to do this.

For exchanges of small amounts of information, Reticulum offers the Packet API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet:

  • @@ -769,30 +633,12 @@ link, and is only revealed to the verified destination, and no intermediaries.
-

In a moment, we will discuss the details of how this methodology is -implemented, but let’s first recap what purposes this methodology serves. We -first ensure that the node answering our request is actually the one we want to -communicate with, and not a malicious actor pretending to be so. At the same -time we establish an efficient encrypted channel. The setup of this is -relatively cheap in terms of bandwidth, so it can be used just for a short -exchange, and then recreated as needed, which will also rotate encryption keys. -The link can also be kept alive for longer periods of time, if this is more -suitable to the application. The procedure also inserts the link id , a hash -calculated from the link request packet, into the memory of forwarding nodes, -which means that the communicating nodes can thereafter reach each other simply -by referring to this link id.

-

The combined bandwidth cost of setting up a link is 3 packets totalling 297 bytes (more info in the -Binary Packet Format section). The amount of bandwidth used on keeping -a link open is practically negligible, at 0.45 bits per second. Even on a slow 1200 bits per second packet -radio channel, 100 concurrent links will still leave 96% channel capacity for actual data.

+

In a moment, we will discuss the details of how this methodology is implemented, but let’s first recap what purposes this methodology serves. We first ensure that the node answering our request is actually the one we want to communicate with, and not a malicious actor pretending to be so. At the same time we establish an efficient encrypted channel. The setup of this is relatively cheap in terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is more suitable to the application. The procedure also inserts the link id , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this link id.

+

The combined bandwidth cost of setting up a link is 3 packets totalling 297 bytes (more info in the Binary Packet Format section). The amount of bandwidth used on keeping a link open is practically negligible, at 0.45 bits per second. Even on a slow 1200 bits per second packet radio channel, 100 concurrent links will still leave 96% channel capacity for actual data.

Resources

-

For exchanging small amounts of data over a Reticulum network, the Packet interface -is sufficient, but for exchanging data that would require many packets, an efficient way to coordinate -the transfer is needed.

-

This is the purpose of the Reticulum Resource. A Resource can automatically -handle the reliable transfer of an arbitrary amount of data over an established Link. -Resources can auto-compress data, will handle breaking the data into individual packets, sequencing -the transfer, integrity verification and reassembling the data on the other end.

-

Resources are programmatically very simple to use, and only requires a few lines -of codes to reliably transfer any amount of data. They can be used to transfer data stored in memory, -or stream data directly from files.

+

For exchanging small amounts of data over a Reticulum network, the Packet interface is sufficient, but for exchanging data that would require many packets, an efficient way to coordinate the transfer is needed.

+

This is the purpose of the Reticulum Resource. A Resource can automatically handle the reliable transfer of an arbitrary amount of data over an established Link. Resources can auto-compress data, will handle breaking the data into individual packets, sequencing the transfer, integrity verification and reassembling the data on the other end.

+

Resources are programmatically very simple to use, and only requires a few lines of codes to reliably transfer any amount of data. They can be used to transfer data stored in memory, or stream data directly from files.

@@ -939,17 +777,9 @@ In the [reticulum]<

Reference Setup

-

This section will detail a recommended Reference Setup for Reticulum. It is important to -note that Reticulum is designed to be usable on more or less any computing device, and over more -or less any medium that allows you to send and receive data, which satisfies some very low -minimum requirements.

-

The communication channel must support at least half-duplex operation, and provide an average -throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The -Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x -runtime environment.

-

That being said, this reference setup has been outlined to provide a common platform for anyone -who wants to help in the development of Reticulum, and for everyone who wants to know a -recommended setup to get started experimenting. A reference system consists of three parts:

+

This section will detail a recommended Reference Setup for Reticulum. It is important to note that Reticulum is designed to be usable on more or less any computing device, and over more or less any medium that allows you to send and receive data, which satisfies some very low minimum requirements.

+

The communication channel must support at least half-duplex operation, and provide an average throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x runtime environment.

+

That being said, this reference setup has been outlined to provide a common platform for anyone who wants to help in the development of Reticulum, and for everyone who wants to know a recommended setup to get started experimenting. A reference system consists of three parts:

  • An Interface Device

    Which provides access to the physical medium whereupon the communication @@ -970,11 +800,7 @@ interface device, and provide user interaction.

-

The reference setup can be considered a relatively stable platform to develop on, and also to start -building networks or applications on. While details of the implementation might change at the current stage of -development, it is the goal to maintain hardware compatibility for as long as entirely possible, and -the current reference setup has been determined to provide a functional platform for many years -into the future. The current Reference System Setup is as follows:

+

The reference setup can be considered a relatively stable platform to develop on, and also to start building networks or applications on. While details of the implementation might change at the current stage of development, it is the goal to maintain hardware compatibility for as long as entirely possible, and the current reference setup has been determined to provide a functional platform for many years into the future. The current Reference System Setup is as follows:

  • Interface Device

    A data radio consisting of a LoRa radio module, and a microcontroller with open source @@ -1003,36 +829,21 @@ use the LoRaWAN standard, but uses a custom MAC layer on top of the plain LoRa m need a plain LoRa radio module connected to a controller with the correct firmware. Full details on how to get or make such a device is available on the RNode Page.

    -

    With the current reference setup, it should be possible to get on a Reticulum network for around 100$ -even if you have none of the hardware already, and need to purchase everything.

    -

    This reference setup is of course just a recommendation for getting started easily, and you should -tailor it to your own specific needs, or whatever hardware you have available.

    +

    With the current reference setup, it should be possible to get on a Reticulum network for around 100$ even if you have none of the hardware already, and need to purchase everything.

    +

    This reference setup is of course just a recommendation for getting started easily, and you should tailor it to your own specific needs, or whatever hardware you have available.

Protocol Specifics

-

This chapter will detail protocol specific information that is essential to the implementation of -Reticulum, but non-critical in understanding how the protocol works on a general level. It should be -treated more as a reference than as essential reading.

+

This chapter will detail protocol specific information that is essential to the implementation of Reticulum, but non-critical in understanding how the protocol works on a general level. It should be treated more as a reference than as essential reading.

Packet Prioritisation

-

Currently, Reticulum is completely priority-agnostic regarding general traffic. All traffic is handled -on a first-come, first-serve basis. Announce re-transmission and other maintenance traffic is handled -according to the re-transmission times and priorities described earlier in this chapter.

+

Currently, Reticulum is completely priority-agnostic regarding general traffic. All traffic is handled on a first-come, first-serve basis. Announce re-transmission and other maintenance traffic is handled according to the re-transmission times and priorities described earlier in this chapter.

Interface Access Codes

-

Reticulum can create named virtual networks, and networks that are only accessible by knowing a preshared -passphrase. The configuration of this is detailed in the Common Interface Options -section. To implement this feature, Reticulum uses the concept of Interface Access Codes, that are calculated -and verified per-packet.

-

An interface with a named virtual network or passphrase authentication enabled will derive a shared Ed25519 -signing identity, and for every outbound packet generate a signature of the entire packet. This signature is -then inserted into the packet as an Interface Access Code before transmission. Depending on the speed and -capabilities of the interface, the IFAC can be the full 512-bit Ed25519 signature, or a truncated version. -Configured IFAC length can be inspected for all interfaces with the rnstatus utility.

-

Upon receipt, the interface will check that the signature matches the expected value, and drop the packet if it -does not. This ensures that only packets sent with the correct naming and/or passphrase parameters are allowed to -pass onto the network.

+

Reticulum can create named virtual networks, and networks that are only accessible by knowing a preshared passphrase. The configuration of this is detailed in the Common Interface Options section. To implement this feature, Reticulum uses the concept of Interface Access Codes, that are calculated and verified per-packet.

+

An interface with a named virtual network or passphrase authentication enabled will derive a shared Ed25519 signing identity, and for every outbound packet generate a signature of the entire packet. This signature is then inserted into the packet as an Interface Access Code before transmission. Depending on the speed and capabilities of the interface, the IFAC can be the full 512-bit Ed25519 signature, or a truncated version. Configured IFAC length can be inspected for all interfaces with the rnstatus utility.

+

Upon receipt, the interface will check that the signature matches the expected value, and drop the packet if it does not. This ensures that only packets sent with the correct naming and/or passphrase parameters are allowed to pass onto the network.

Wire Format

@@ -1168,24 +979,15 @@ but excluding any interface access codes.

Announce Propagation Rules

-

The following table illustrates the rules for automatically propagating announces -from one interface type to another, for all possible combinations. For the purpose -of announce propagation, the Full and Gateway modes are identical.

+

The following table illustrates the rules for automatically propagating announces from one interface type to another, for all possible combinations. For the purpose of announce propagation, the Full and Gateway modes are identical.

_images/if_mode_graph_b.png -

See the Interface Modes section for a conceptual overview -of the different interface modes, and how they are configured.

+

See the Interface Modes section for a conceptual overview of the different interface modes, and how they are configured.

Cryptographic Primitives

-

Reticulum uses a simple suite of efficient, strong and well-tested cryptographic -primitives, with widely available implementations that can be used both on -general-purpose CPUs and on microcontrollers.

-

One of the primary considerations for choosing this particular set of primitives is -that they can be implemented safely with relatively few pitfalls, on practically -all current computing platforms.

-

The primitives listed here are authoritative. Anything claiming to be Reticulum, -but not using these exact primitives is not Reticulum, and possibly an -intentionally compromised or weakened clone. The utilised primitives are:

+

Reticulum uses a simple suite of efficient, strong and well-tested cryptographic primitives, with widely available implementations that can be used both on general-purpose CPUs and on microcontrollers.

+

One of the primary considerations for choosing this particular set of primitives is that they can be implemented safely with relatively few pitfalls, on practically all current computing platforms.

+

The primitives listed here are authoritative. Anything claiming to be Reticulum, but not using these exact primitives is not Reticulum, and possibly an intentionally compromised or weakened clone. The utilised primitives are:

  • Ed25519 for signatures

  • X25519 for ECDH key exchanges

    • @@ -1202,28 +1004,15 @@ intentionally compromised or weakened clone. The utilised primitives are:

  • SHA-256

  • SHA-512

-

In the default installation configuration, the X25519, Ed25519 and AES-256-CBC -primitives are provided by OpenSSL (via the PyCA/cryptography -package). The hashing functions SHA-256 and SHA-512 are provided by the standard -Python hashlib. The HKDF, HMAC, -Token primitives, and the PKCS7 padding function are always provided by the -following internal implementations:

+

In the default installation configuration, the X25519, Ed25519 and AES-256-CBC primitives are provided by OpenSSL (via the PyCA/cryptography package). The hashing functions SHA-256 and SHA-512 are provided by the standard Python hashlib. The HKDF, HMAC, Token primitives, and the PKCS7 padding function are always provided by the following internal implementations:

  • RNS/Cryptography/HKDF.py

  • RNS/Cryptography/HMAC.py

  • RNS/Cryptography/Token.py

  • RNS/Cryptography/PKCS7.py

-

Reticulum also includes a complete implementation of all necessary primitives in pure Python. -If OpenSSL & PyCA are not available on the system when Reticulum is started, Reticulum will -instead use the internal pure-python primitives. A trivial consequence of this is performance, -with the OpenSSL backend being much faster. The most important consequence however, is the -potential loss of security by using primitives that has not seen the same amount of scrutiny, -testing and review as those from OpenSSL.

-

Using the normal RNS installation procedures, it is not possible to install Reticulum on a -system without the required OpenSSL primitives being available, and if they are not, they will -be resolved and installed as a dependency. It is only possible to use the pure-python primitives -by manually specifying this, for example by using the rnspure package.

+

Reticulum also includes a complete implementation of all necessary primitives in pure Python. If OpenSSL & PyCA are not available on the system when Reticulum is started, Reticulum will instead use the internal pure-python primitives. A trivial consequence of this is performance, with the OpenSSL backend being much faster. The most important consequence however, is the potential loss of security by using primitives that has not seen the same amount of scrutiny, testing and review as those from OpenSSL.

+

Using the normal RNS installation procedures, it is not possible to install Reticulum on a system without the required OpenSSL primitives being available, and if they are not, they will be resolved and installed as a dependency. It is only possible to use the pure-python primitives by manually specifying this, for example by using the rnspure package.

Warning

If you want to use the internal pure-python primitives, it is highly advisable that you diff --git a/docs/markdown/interfaces.md b/docs/markdown/interfaces.md index 8b8a335e..6e0ed974 100644 --- a/docs/markdown/interfaces.md +++ b/docs/markdown/interfaces.md @@ -901,10 +901,10 @@ When `discoverable` is enabled, a variety of additional options become available `reachable_on` : Specifies the address that remote peers should use to connect to this interface. -
+ * For TCP and Backbone interfaces, this is typically the public IP address or hostname. Do not include the port, this is fetched automatically from the interface. * For I2P interfaces, this is usually the I2P `b32` address. This value is fetched automatically from the `I2PInterface` once it is up and connected to the I2P network, so you should not set this manually, unless you absolutely know what you’re doing. -
+ **Dynamic Resolution:** This option also accepts a path to an external executable script or binary. If a path is provided, Reticulum will execute the script and use its `stdout` as the reachability address. This is useful for devices behind dynamic DNS, NATs, or complex cloud environments where the external IP is not known locally. The script must simply print the address to stdout and exit. #### NOTE @@ -1064,10 +1064,10 @@ These can be used to control various aspects of interface behaviour. > to bring up the interface. Defaults to `False`. For any > interface to be brought up, the `enabled` option > must be set to `True` or `Yes`. ->
+> > * The `mode` option allows selecting the high-level behaviour > of the interface from a number of options. ->
+> > > - The default value is `full`. In this mode, all discovery, > > meshing and transport functionality is available. > > - In the `access_point` (or shorthand `ap`) mode, the @@ -1083,15 +1083,15 @@ These can be used to control various aspects of interface behaviour. > * The `outgoing` option sets whether an interface is allowed > to transmit. Defaults to `True`. If set to `False` or `No` > the interface will only receive data, and never transmit. ->
+> > * The `network_name` option sets the virtual network name for > the interface. This allows multiple separate network segments > to exist on the same physical channel or medium. ->
+> > * The `passphrase` option sets an authentication passphrase on > the interface. This option can be used in conjunction with the > `network_name` option, or be used alone. ->
+> > * The `ifac_size` option allows customising the length of the > Interface Authentication Codes carried by each packet on named > and/or authenticated network segments. It is set by default to @@ -1099,13 +1099,13 @@ These can be used to control various aspects of interface behaviour. > to a custom size between 8 and 512 bits by using this option. > In normal usage, this option should not be changed from the > default. ->
+> > * The `announce_cap` option lets you configure the maximum > bandwidth to allocate, at any given time, to propagating > announces and other network upkeep traffic. It is configured at > 2% by default, and should normally not need to be changed. Can > be set to any value between `1` and `100`. ->
+> > > *If an interface exceeds its announce cap, it will queue announces > > for later transmission. Reticulum will always prioritise propagating > > announces from nearby nodes first. This ensures that the local @@ -1126,7 +1126,7 @@ These can be used to control various aspects of interface behaviour. > any. In most cases, the automatically found rate should be > sufficient, but it can be configured by using the `bitrate` > option, to set the interface speed in *bits per second*. ->
+> > * The `bootstrap_only` option designates an interface as a temporary > bridge for initial connectivity. If this option is enabled, the > interface will be monitored and automatically detached once the @@ -1135,7 +1135,7 @@ These can be used to control various aspects of interface behaviour. > for using a slow or expensive connection (such as a single LoRa > link or a remote TCP tunnel) solely to discover better local > infrastructure, which then supersedes the bootstrap interface. ->
+> ## Interface Modes @@ -1152,7 +1152,7 @@ the default mode. > * The default mode is `full`. In this mode, all discovery, > meshing and transport functionality is activated. ->
+> > * The `gateway` mode (or shorthand `gw`) also has all > discovery, meshing and transport functionality available, > but will additionally try to discover unknown paths on @@ -1162,18 +1162,18 @@ the default mode. > will try to discover this path via all other active interfaces, > and forward the discovered path to the requestor if one is > found. ->
+> > If you want to allow other nodes to widely resolve paths or connect > to a network via an interface, it might be useful to put it in this > mode. By creating a chain of `gateway` interfaces, other > nodes will be able to immediately discover paths to any > destination along the chain. ->
+> > *Please note!* It is the interface *facing the clients* that > must be put into `gateway` mode for this to work, not > the interface facing the wider network (for this, the `boundary` > mode can be useful, though). ->
+> > * In the `access_point` (or shorthand `ap`) mode, the > interface will operate as a network access point. In this > mode, announces will not be automatically broadcasted on @@ -1181,13 +1181,13 @@ the default mode. > will have a much shorter expiry time. In addition, path > requests from clients on the access point interface will > be handled in the same way as the `gateway` interface. ->
+> > This mode is useful for creating interfaces that remain > quiet, until someone actually starts using them. An example > of this could be a radio interface serving a wide area, > where users are expected to connect momentarily, use the > network, and then disappear again. ->
+> > * The `roaming` mode should be used on interfaces that are > roaming (physically mobile), seen from the perspective of > other nodes in the network. As an example, if a vehicle is @@ -1202,7 +1202,7 @@ the default mode. > side of the network will also be able to reach devices > internal to the vehicle, when it is in range. Paths via > `roaming` interfaces also expire faster. ->
+> > * The purpose of the `boundary` mode is to specify interfaces > that establish connectivity with network segments that are > significantly different than the one this node exists on. @@ -1210,7 +1210,7 @@ the default mode. > network, but also has a high-speed connection to a > public Transport Node available on the Internet, the interface > connecting over the Internet should be set to `boundary` mode. ->
+> For a table describing the impact of all modes on announce propagation, please see the [Announce Propagation Rules](understanding.md#understanding-announcepropagation) section. @@ -1233,16 +1233,16 @@ re-broadcasted to other interfaces*. > announces *received* on this interface will only be re-transmitted and > propagated to other interfaces once every hour, no matter how often they > are received. ->
+> > * The optional `announce_rate_grace` defines the number of times a destination > can violate the announce rate before the target rate is enforced. ->
+> > * The optional `announce_rate_penalty` configures an extra amount of > time that is added to the normal rate target. As an example, if a penalty > of `7200` seconds is defined, once the rate target is enforced, the > destination in question will only have its announces propagated every > 3 hours, until it lowers its actual announce rate to within the target. ->
+> You can also configure default announce rate parameters for all interfaces that do not have these parameters set explicitly by setting the `default_ar_target` @@ -1309,41 +1309,41 @@ but all the parameters are exposed for configuration if needed. > * The `ingress_control` option tells Reticulum whether or not > to enable ingress control on the interface. Defaults to `True`. ->
+> > * The `ic_new_time` option configures how long (in seconds) an > interface is considered newly spawned. Defaults to `2*60*60` seconds. This > option is useful on publicly accessible interfaces that spawn new > sub-interfaces when a new client connects. ->
+> > * The `ic_burst_freq_new` option sets the maximum announce ingress > frequency for newly spawned interfaces. Defaults to `3.5` > announces per second. ->
+> > * The `ic_burst_freq` option sets the maximum announce ingress > frequency for other interfaces. Defaults to `12` announces > per second. ->
+> > > *If an interface exceeds its burst frequency, incoming announces > > for unknown destinations will be temporarily held in a queue, and > > not processed until later.* > * The `ic_max_held_announces` option sets the maximum amount of > unique announces that will be held in the queue. Any additional > unique announces will be dropped. Defaults to `256` announces. ->
+> > * The `ic_burst_hold` option sets how much time (in seconds) must > pass after the burst frequency drops below its threshold, for the > announce burst to be considered cleared. Defaults to `60` > seconds. ->
+> > * The `ic_burst_penalty` option sets how much time (in seconds) must > pass after the burst is considered cleared, before held announces can > start being released from the queue. Defaults to `5*60` > seconds. ->
+> > * The `ic_held_release_interval` option sets how much time (in seconds) > must pass between releasing each held announce from the queue. Defaults > to `30` seconds. ->
+> All of the above settings can be configured both as instance-wide defaults under the `[reticulum]` section of the configuration file, or on a per- @@ -1373,28 +1373,28 @@ but all the parameters are exposed for configuration if needed. > * The `ingress_control` option tells Reticulum whether or not > to enable ingress control on the interface. Defaults to `True`. ->
+> > * The `ic_new_time` option configures how long (in seconds) an > interface is considered newly spawned. Defaults to `2*60*60` seconds. This > option is useful on publicly accessible interfaces that spawn new > sub-interfaces when a new client connects. ->
+> > * The `ic_pr_burst_freq_new` option sets the maximum path request > ingress frequency for newly spawned interfaces. Defaults to `3` > path requests per second. ->
+> > * The `ic_pr_burst_freq` option sets the maximum path request > ingress frequency for other interfaces. Defaults to `8` path requests > per second. ->
+> > > *If an interface exceeds its burst frequency, incoming path requests > > from that system will not traverse the network further.* > * The `egress_control` option enables hard-limiting path request egress > control per-interface. Defaults to `False` ->
+> > * The `ec_pr_freq` option sets the hard limit for outbound path requests > per second on a given interface. ->
+> All of the above settings can be configured both as instance-wide defaults under the `[reticulum]` section of the configuration file, or on a per- diff --git a/docs/markdown/networks.md b/docs/markdown/networks.md index a2ef9e39..147b2e33 100644 --- a/docs/markdown/networks.md +++ b/docs/markdown/networks.md @@ -46,11 +46,11 @@ with Reticulum: > (called *destinations* in Reticulum terminology) as it needs, which become > globally reachable to the rest of the network. There is no central point of > control over the address space. ->
+> > * Reticulum was designed to handle both very small, and very large networks. > While the address space can support billions of endpoints, Reticulum is > also very useful when just a few devices needs to communicate. ->
+> > * Low-bandwidth networks, like LoRa and packet radio, can interoperate and > interconnect with much larger and higher bandwidth networks without issue. > Reticulum automatically manages the flow of information to and from various @@ -60,27 +60,27 @@ with Reticulum: > it will try as best as possible to comply with this, while still respecting > bandwidth limits, but you *will* waste a lot of precious bandwidth and airtime, > and your LoRa network will not work very well. ->
+> > * Reticulum provides sender/initiator anonymity by default. There is no way > to filter traffic or discriminate it based on the source of the traffic. ->
+> > * All traffic is encrypted using ephemeral keys generated by an Elliptic Curve > Diffie-Hellman key exchange on Curve25519. There is no way to inspect traffic > contents, and no way to prioritise or throttle certain kinds of traffic. > All transport and routing layers are thus completely agnostic to traffic type, > and will pass all traffic equally. ->
+> > * Reticulum can function both with and without infrastructure. When *transport > nodes* are available, they can route traffic over multiple hops for other > nodes, and will function as a distributed cryptographic keystore. When there > is no transport nodes available, all nodes that are within communication range > can still communicate. ->
+> > * Every node can become a transport node, simply by enabling it in it’s > configuration, but there is no need for every node on the network to be a > transport node. Letting every node be a transport node will in most cases > degrade the performance and reliability of the network. ->
+> > > *In general terms, if a node is stationary, well-connected and kept running > > most of the time, it is a good candidate to be a transport node. For optimal > > performance, a network should contain the amount of transport nodes that @@ -92,14 +92,14 @@ with Reticulum: > entirely new, and so far, mostly unexplored class of networked applications, > where networks, and the information flow within them can form and dissolve > organically. ->
+> > * You can just as easily create closed networks, since Reticulum allows you to > add authentication to any interface. This means you can restrict access on > any interface type, even when using legacy devices, such as modems. You can > also mix authenticated and open interfaces on the same system. See the > [Common Interface Options](interfaces.md#interfaces-options) section of the [Interfaces](interfaces.md#interfaces-main) > chapter of this manual for information on how to set up interface authentication. ->
+> Reticulum allows you to mix very different kinds of networking mediums into a unified mesh, or to keep everything within one medium. You could build a “virtual diff --git a/docs/markdown/understanding.md b/docs/markdown/understanding.md index 2540d980..1bca0822 100644 --- a/docs/markdown/understanding.md +++ b/docs/markdown/understanding.md @@ -1,57 +1,28 @@ # Understanding Reticulum -This chapter will briefly describe the overall purpose and operating principles of Reticulum. -It should give you an overview of how the stack works, and an understanding of how to -develop networked applications using Reticulum. +This chapter will briefly describe the overall purpose and operating principles of Reticulum. It should give you an overview of how the stack works, and an understanding of how to develop networked applications using Reticulum. -This chapter is not an exhaustive source of information on Reticulum, at least not yet. Currently, -the only complete repository, and final authority on how Reticulum actually functions, is the Python -reference implementation and API reference. That being said, this chapter is an essential resource in -understanding how Reticulum works from a high-level perspective, along with the general principles of -Reticulum, and how to apply them when creating your own networks or software. +This chapter is not an exhaustive source of information on Reticulum, at least not yet. Currently, the only complete repository, and final authority on how Reticulum actually functions, is the Python reference implementation and API reference. That being said, this chapter is an essential resource in understanding how Reticulum works from a high-level perspective, along with the general principles of Reticulum, and how to apply them when creating your own networks or software. -After reading this chapter, you should be well-equipped to understand how a Reticulum network -operates, what it can achieve, and how you can use it yourself. This chapter also seeks to provide an overview of the -sentiments and the philosophy behind Reticulum, what problems it seeks to solve, and how it -approaches those solutions. +After reading this chapter, you should be well-equipped to understand how a Reticulum network operates, what it can achieve, and how you can use it yourself. This chapter also seeks to provide an overview of the sentiments and the philosophy behind Reticulum, what problems it seeks to solve, and how it approaches those solutions. ## Motivation -The primary motivation for designing and implementing Reticulum has been the current lack of -reliable, functional and secure minimal-infrastructure modes of digital communication. It is my -belief that it is highly desirable to create a reliable and efficient way to set up long-range digital -communication networks that can securely allow exchange of information between people and -machines, with no central point of authority, control, censorship or barrier to entry. +The primary motivation for designing and implementing Reticulum has been the current lack of reliable, functional and secure minimal-infrastructure modes of digital communication. It is my belief that it is highly desirable to create a reliable and efficient way to set up long-range digital communication networks that can securely allow exchange of information between people and machines, with no central point of authority, control, censorship or barrier to entry. -Almost all of the various networking systems in use today share a common limitation: They -require large amounts of coordination and centralised trust and power to function. To join such networks, you need approval -of gatekeepers in control. This need for coordination and trust inevitably leads to an environment of -central control, where it’s very easy for infrastructure operators or governments to control or alter -traffic, and censor or persecute unwanted actors. It also makes it completely impossible to freely deploy -and use networks at will, like one would use other common tools that enhance individual agency and freedom. +Almost all of the various networking systems in use today share a common limitation: They require large amounts of coordination and centralised trust and power to function. To join such networks, you need approval of gatekeepers in control. This need for coordination and trust inevitably leads to an environment of central control, where it’s very easy for infrastructure operators or governments to control or alter traffic, and censor or persecute unwanted actors. It also makes it completely impossible to freely deploy and use networks at will, like one would use other common tools that enhance individual agency and freedom. -Reticulum aims to require as little coordination and trust as possible. It aims to make secure, -anonymous and permissionless networking and information exchange a tool that anyone can just pick up and use. +Reticulum aims to require as little coordination and trust as possible. It aims to make secure, anonymous and permissionless networking and information exchange a tool that anyone can just pick up and use. -Since Reticulum is completely medium agnostic, it can be used to build networks on whatever is best -suited to the situation, or whatever you have available. In some cases, this might be packet radio -links over VHF frequencies, in other cases it might be a 2.4 GHz -network using off-the-shelf radios, or it might be using common LoRa development boards. +Since Reticulum is completely medium agnostic, it can be used to build networks on whatever is best suited to the situation, or whatever you have available. In some cases, this might be packet radio links over VHF frequencies, in other cases it might be a 2.4 GHz network using off-the-shelf radios, or it might be using common LoRa development boards. -At the time of release of this document, the fastest and easiest setup for development and testing is using -LoRa radio modules with an open source firmware (see the section [Reference Setup](#understanding-referencesystem)), -connected to any kind of computer or mobile device that Reticulum can run on. +At the time of release of this document, the fastest and easiest setup for development and testing is using LoRa radio modules with an open source firmware (see the section [Reference Setup](#understanding-referencesystem)), connected to any kind of computer or mobile device that Reticulum can run on. -The ultimate aim of Reticulum is to allow anyone to be their own network operator, and to make it -cheap and easy to cover vast areas with a myriad of independent, interconnectable and autonomous networks. -Reticulum **is not** *one network*, it **is a tool** to build *thousands of networks*. Networks without -kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate -with each other, and require no central oversight. Networks for human beings. *Networks for the people*. +The ultimate aim of Reticulum is to allow anyone to be their own network operator, and to make it cheap and easy to cover vast areas with a myriad of independent, interconnectable and autonomous networks. Reticulum **is not** *one network*, it **is a tool** to build *thousands of networks*. Networks without kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate with each other, and require no central oversight. Networks for human beings. *Networks for the people*. ## Goals -To be as widely usable and efficient to deploy as possible, the following goals have been used to -guide the design of Reticulum: +To be as widely usable and efficient to deploy as possible, the following goals have been used to guide the design of Reticulum: * **Fully useable as open source software stack** : Reticulum must be implemented with, and be able to run using only open source software. This is @@ -73,18 +44,18 @@ guide the design of Reticulum: * **Unlicensed use** : Reticulum shall be functional over physical communication mediums that do not require any form of license to use. Reticulum must be designed in a way, so it is usable over ISM radio - frequency bands, and can provide functional long distance links in such conditions, for example - by connecting a modem to a PMR or CB radio, or by using LoRa or WiFi modules. + frequency bands, and can provide functional long distance links in such conditions, for + example by connecting a modem to a PMR or CB radio, or by using LoRa or WiFi modules. * **Supplied software** : In addition to the core networking stack and API, that allows a developer to build applications with Reticulum, a basic set of Reticulum-based communication tools must be implemented and released along with Reticulum itself. These shall serve both as a - functional, basic communication suite, and as an example and learning resource to others wishing - to build applications with Reticulum. + functional, basic communication suite, and as an example and learning resource to others + wishing to build applications with Reticulum. * **Ease of use** - : The reference implementation of Reticulum is written in Python, to make it easy to use - and understand. A programmer with only basic experience should be able to use - Reticulum to write networked applications. + : The reference implementation of Reticulum is written in Python, to make it easy to use and + understand. A programmer with only basic experience should be able to use Reticulum to write + networked applications. * **Low cost** : It shall be as cheap as possible to deploy a communication system based on Reticulum. This should be achieved by using cheap off-the-shelf hardware that potential users might already @@ -93,50 +64,23 @@ guide the design of Reticulum: ## Introduction & Basic Functionality -Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at its -core a *message oriented* system. It is suited for both local point-to-point or point-to-multipoint -scenarios where all nodes are within range of each other, as well as scenarios where packets need -to be transported over multiple hops in a complex network to reach the recipient. +Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at its core a *message oriented* system. It is suited for both local point-to-point or point-to-multipoint scenarios where all nodes are within range of each other, as well as scenarios where packets need to be transported over multiple hops in a complex network to reach the recipient. -Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead -Reticulum uses the singular concept of *destinations*. Any application using Reticulum as its -networking stack will need to create one or more destinations to receive data, and know the -destinations it needs to send data to. +Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead Reticulum uses the singular concept of *destinations*. Any application using Reticulum as its networking stack will need to create one or more destinations to receive data, and know the destinations it needs to send data to. -All destinations in Reticulum are *represented* as a 16 byte hash. This hash is derived from truncating a full -SHA-256 hash of identifying characteristics of the destination. To users, the destination addresses -will be displayed as 16 hexadecimal bytes, like this example: `<13425ec15b621c1d928589718000d814>`. +All destinations in Reticulum are *represented* as a 16 byte hash. This hash is derived from truncating a full SHA-256 hash of identifying characteristics of the destination. To users, the destination addresses will be displayed as 16 hexadecimal bytes, like this example: `<13425ec15b621c1d928589718000d814>`. -The truncation size of 16 bytes (128 bits) for destinations has been chosen as a reasonable trade-off -between address space -and packet overhead. The address space accommodated by this size can support many billions of -simultaneously active devices on the same network, while keeping packet overhead low, which is -essential on low-bandwidth networks. In the very unlikely case that this address space nears -congestion, a one-line code change can upgrade the Reticulum address space all the way up to 256 -bits, ensuring the Reticulum address space could potentially support galactic-scale networks. -This is obviously complete and ridiculous over-allocation, and as such, the current 128 bits should -be sufficient, even far into the future. +The truncation size of 16 bytes (128 bits) for destinations has been chosen as a reasonable trade-off between address space and packet overhead. The address space accommodated by this size can support many billions of simultaneously active devices on the same network, while keeping packet overhead low, which is essential on low-bandwidth networks. In the very unlikely case that this address space nears congestion, a one-line code change can upgrade the Reticulum address space all the way up to 256 bits, ensuring the Reticulum address space could potentially support galactic-scale networks. This is obviously complete and ridiculous over-allocation, and as such, the current 128 bits should be sufficient, even far into the future. -By default Reticulum encrypts all data using elliptic curve cryptography and AES. Any packet sent to a -destination is encrypted with a per-packet derived key. Reticulum can also set up an encrypted -channel to a destination, called a *Link*. Both data sent over Links and single packets offer -*Initiator Anonymity*. Links additionally offer *Forward Secrecy* by default, employing an Elliptic Curve -Diffie Hellman key exchange on Curve25519 to derive per-link ephemeral keys. Asymmetric, link-less -packet communication can also provide forward secrecy, with automatic key ratcheting, by enabling -ratchets on a per-destination basis. The multi-hop transport, coordination, verification and reliability -layers are fully autonomous and also based on elliptic curve cryptography. +By default Reticulum encrypts all data using elliptic curve cryptography and AES. Any packet sent to a destination is encrypted with a per-packet derived key. Reticulum can also set up an encrypted channel to a destination, called a *Link*. Both data sent over Links and single packets offer *Initiator Anonymity*. Links additionally offer *Forward Secrecy* by default, employing an Elliptic Curve Diffie Hellman key exchange on Curve25519 to derive per-link ephemeral keys. Asymmetric, link-less packet communication can also provide forward secrecy, with automatic key ratcheting, by enabling ratchets on a per-destination basis. The multi-hop transport, coordination, verification and reliability layers are fully autonomous and also based on elliptic curve cryptography. -Reticulum also offers symmetric key encryption for group-oriented communications, as well as -unencrypted packets (for local broadcast purposes **only**). +Reticulum also offers symmetric key encryption for group-oriented communications, as well as unencrypted packets (for local broadcast purposes **only**). -Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports, -and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or -private IP networks. +Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports, and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or private IP networks. ### Destinations -To receive and send data with the Reticulum stack, an application needs to create one or more -destinations. Reticulum uses three different basic destination types, and one special: +To receive and send data with the Reticulum stack, an application needs to create one or more destinations. Reticulum uses three different basic destination types, and one special: * **Single** : The *single* destination type is the most common type in Reticulum, and should be used for @@ -148,9 +92,9 @@ destinations. Reticulum uses three different basic destination types, and one sp number of users, or should be readable by anyone. Traffic to a *plain* destination is not encrypted. Generally, *plain* destinations can be used for broadcast information intended to be public. Plain destinations are only reachable directly, and packets addressed to plain destinations are - never transported over multiple hops in the network. To be transportable over multiple hops in Reticulum, information - *must* be encrypted, since Reticulum uses the per-packet encryption to verify routing paths and - keep them alive. + never transported over multiple hops in the network. To be transportable over multiple hops in + Reticulum, information *must* be encrypted, since Reticulum uses the per-packet encryption to verify + routing paths and keep them alive. * **Group** : The *group* special destination type, that defines a symmetrically encrypted virtual destination. Data sent to this destination will be encrypted with a symmetric key, and will be readable by @@ -166,16 +110,11 @@ destinations. Reticulum uses three different basic destination types, and one sp #### Destination Naming -Destinations are created and named in an easy to understand dotted notation of *aspects*, and -represented on the network as a hash of this value. The hash is a SHA-256 truncated to 128 bits. The -top level aspect should always be a unique identifier for the application using the destination. -The next levels of aspects can be defined in any way by the creator of the application. +Destinations are created and named in an easy to understand dotted notation of *aspects*, and represented on the network as a hash of this value. The hash is a SHA-256 truncated to 128 bits. The top level aspect should always be a unique identifier for the application using the destination. The next levels of aspects can be defined in any way by the creator of the application. -Aspects can be as long and as plentiful as required, and a resulting long destination name will not -impact efficiency, as names are always represented as truncated SHA-256 hashes on the network. +Aspects can be as long and as plentiful as required, and a resulting long destination name will not impact efficiency, as names are always represented as truncated SHA-256 hashes on the network. -As an example, a destination for a environmental monitoring application could be made up of the -application name, a device type and measurement type, like this: +As an example, a destination for a environmental monitoring application could be made up of the application name, a device type and measurement type, like this: ```text app name : environmentlogger @@ -185,11 +124,7 @@ full name : environmentlogger.remotesensor.temperature hash : 4faf1b2e0a077e6a9d92fa051f256038 ``` -For the *single* destination, Reticulum will automatically append the associated public key as a -destination aspect before hashing. This is done to ensure only the correct destination is reached, -since anyone can listen to any destination name. Appending the public key ensures that a given -packet is only directed at the destination that holds the corresponding private key to decrypt the -packet. +For the *single* destination, Reticulum will automatically append the associated public key as a destination aspect before hashing. This is done to ensure only the correct destination is reached, since anyone can listen to any destination name. Appending the public key ensures that a given packet is only directed at the destination that holds the corresponding private key to decrypt the packet. **Take note!** There is a very important concept to understand here: @@ -197,16 +132,9 @@ packet. * Each destination that does so will still have a unique destination hash, and thus be uniquely addressable, because their public keys will differ. -In actual use of *single* destination naming, it is advisable not to use any uniquely identifying -features in aspect naming. Aspect names should be general terms describing what kind of destination -is represented. The uniquely identifying aspect is always achieved by appending the public key, -which expands the destination into a uniquely identifiable one. Reticulum does this automatically. +In actual use of *single* destination naming, it is advisable not to use any uniquely identifying features in aspect naming. Aspect names should be general terms describing what kind of destination is represented. The uniquely identifying aspect is always achieved by appending the public key, which expands the destination into a uniquely identifiable one. Reticulum does this automatically. -Any destination on a Reticulum network can be addressed and reached simply by knowing its -destination hash (and public key, but if the public key is not known, it can be requested from the -network simply by knowing the destination hash). The use of app names and aspects makes it easy to -structure Reticulum programs and makes it possible to filter what information and data your program -receives. +Any destination on a Reticulum network can be addressed and reached simply by knowing its destination hash (and public key, but if the public key is not known, it can be requested from the network simply by knowing the destination hash). The use of app names and aspects makes it easy to structure Reticulum programs and makes it possible to filter what information and data your program receives. To recap, the different destination types should be used in the following situations: @@ -218,25 +146,13 @@ To recap, the different destination types should be used in the following situat * **Plain** : When plain-text communication is desirable, for example when broadcasting information, or for local discovery purposes. -To communicate with a *single* destination, you need to know its public key. Any method for -obtaining the public key is valid, but Reticulum includes a simple mechanism for making other -nodes aware of your destinations public key, called the *announce*. It is also possible to request -an unknown public key from the network, as all transport instances serve as a distributed ledger -of public keys. +To communicate with a *single* destination, you need to know its public key. Any method for obtaining the public key is valid, but Reticulum includes a simple mechanism for making other nodes aware of your destinations public key, called the *announce*. It is also possible to request an unknown public key from the network, as all transport instances serve as a distributed ledger of public keys. -Note that public key information can be shared and verified in other ways than using the -built-in *announce* functionality, and that it is therefore not required to use the *announce* and *path request* -functionality to obtain public keys. It is by far the easiest though, and should definitely be used -if there is not a very good reason for doing it differently. +Note that public key information can be shared and verified in other ways than using the built-in *announce* functionality, and that it is therefore not required to use the *announce* and *path request* functionality to obtain public keys. It is by far the easiest though, and should definitely be used if there is not a very good reason for doing it differently. ### Public Key Announcements -An *announce* will send a special packet over any relevant interfaces, containing all needed -information about the destination hash and public key, and can also contain some additional, -application specific data. The entire packet is signed by the sender to ensure authenticity. It is not -required to use the announce functionality, but in many cases it will be the simplest way to share -public keys on the network. The announce mechanism also serves to establish end-to-end connectivity -to the announced destination, as the announce propagates through the network. +An *announce* will send a special packet over any relevant interfaces, containing all needed information about the destination hash and public key, and can also contain some additional, application specific data. The entire packet is signed by the sender to ensure authenticity. It is not required to use the announce functionality, but in many cases it will be the simplest way to share public keys on the network. The announce mechanism also serves to establish end-to-end connectivity to the announced destination, as the announce propagates through the network. As an example, an announce in a simple messenger application might contain the following information: @@ -246,259 +162,189 @@ As an example, an announce in a simple messenger application might contain the f * A random blob, making each new announce unique * An Ed25519 signature of the above information, verifying authenticity -With this information, any Reticulum node that receives it will be able to reconstruct an outgoing -destination to securely communicate with that destination. You might have noticed that there is one -piece of information lacking to reconstruct full knowledge of the announced destination, and that is -the aspect names of the destination. These are intentionally left out to save bandwidth, since they -will be implicit in almost all cases. The receiving application will already know them. If a destination -name is not entirely implicit, information can be included in the application specific data part that -will allow the receiver to infer the naming. +With this information, any Reticulum node that receives it will be able to reconstruct an outgoing destination to securely communicate with that destination. You might have noticed that there is one piece of information lacking to reconstruct full knowledge of the announced destination, and that is the aspect names of the destination. These are intentionally left out to save bandwidth, since they will be implicit in almost all cases. The receiving application will already know them. If a destination name is not entirely implicit, information can be included in the application specific data part that will allow the receiver to infer the naming. It is important to note that announces will be forwarded throughout the network according to a certain pattern. This will be detailed in the section [The Announce Mechanism in Detail](#understanding-announce). -In Reticulum, destinations are allowed to move around the network at will. This is very different from -protocols such as IP, where an address is always expected to stay within the network segment it was assigned in. -This limitation does not exist in Reticulum, and any destination is *completely portable* over the entire topography -of the network, and *can even be moved to other Reticulum networks* than the one it was created in, and -still become reachable. To update its reachability, a destination simply needs to send an announce on any -networks it is part of. After a short while, it will be globally reachable in the network. +In Reticulum, destinations are allowed to move around the network at will. This is very different from protocols such as IP, where an address is always expected to stay within the network segment it was assigned in. This limitation does not exist in Reticulum, and any destination is *completely portable* over the entire topography of the network, and *can even be moved to other Reticulum networks* than the one it was created in, and still become reachable. To update its reachability, a destination simply needs to send an announce on any networks it is part of. After a short while, it will be globally reachable in the network. Seeing how *single* destinations are always tied to a private/public key pair leads us to the next topic. ### Identities -In Reticulum, an *identity* does not necessarily represent a personal identity, but is an abstraction that -can represent any kind of *verifiable entity*. This could very well be a person, but it could also be the -control interface of a machine, a program, robot, computer, sensor or something else entirely. In -general, any kind of agent that can act, or be acted upon, or store or manipulate information, can be -represented as an identity. An *identity* can be used to create any number of destinations. +In Reticulum, an *identity* does not necessarily represent a personal identity, but is an abstraction that can represent any kind of *verifiable entity*. This could very well be a person, but it could also be the control interface of a machine, a program, robot, computer, sensor or something else entirely. In general, any kind of agent that can act, or be acted upon, or store or manipulate information, can be represented as an identity. An *identity* can be used to create any number of destinations. -A *single* destination will always have an *identity* tied to it, but not *plain* or *group* -destinations. Destinations and identities share a multilateral connection. You can create a -destination, and if it is not connected to an identity upon creation, it will just create a new one to use -automatically. This may be desirable in some situations, but often you will probably want to create -the identity first, and then use it to create new destinations. +A *single* destination will always have an *identity* tied to it, but not *plain* or *group* destinations. Destinations and identities share a multilateral connection. You can create a destination, and if it is not connected to an identity upon creation, it will just create a new one to use automatically. This may be desirable in some situations, but often you will probably want to create the identity first, and then use it to create new destinations. -As an example, we could use an identity to represent the user of a messaging application. -Destinations can then be created by this identity to allow communication to reach the user. -In all cases it is of great importance to store the private keys associated with any -Reticulum Identity securely and privately, since obtaining access to the identity keys equals -obtaining access and controlling reachability to any destinations created by that identity. +As an example, we could use an identity to represent the user of a messaging application. Destinations can then be created by this identity to allow communication to reach the user. In all cases it is of great importance to store the private keys associated with any Reticulum Identity securely and privately, since obtaining access to the identity keys equals obtaining access and controlling reachability to any destinations created by that identity. ### Getting Further -The above functions and principles form the core of Reticulum, and would suffice to create -functional networked applications in local clusters, for example over radio links where all interested -nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple -hops in the network. +The above functions and principles form the core of Reticulum, and would suffice to create functional networked applications in local clusters, for example over radio links where all interested nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple hops in the network. In the following sections, two concepts that allow this will be introduced, *paths* and *links*. ## Reticulum Transport -The methods of routing used in traditional networks are fundamentally incompatible with the physical medium -types and circumstances that Reticulum was designed to handle. These mechanisms mostly assume trust at the physical layer, -and often needs a lot more bandwidth than Reticulum can assume is available. Since Reticulum is designed to -survive running over open radio spectrum, no such trust can be assumed, and bandwidth is often very limited. +The methods of routing used in traditional networks are fundamentally incompatible with the physical medium types and circumstances that Reticulum was designed to handle. These mechanisms mostly assume trust at the physical layer, and often needs a lot more bandwidth than Reticulum can assume is available. Since Reticulum is designed to survive running over open radio spectrum, no such trust can be assumed, and bandwidth is often very limited. -To overcome such challenges, Reticulum’s *Transport* system uses asymmetric elliptic curve cryptography to -implement the concept of *paths* that allow discovery of how to get information closer to a certain -destination. It is important to note that no single node in a Reticulum network knows the complete -path to a destination. Every Transport node participating in a Reticulum network will only -know the most direct way to get a packet one hop closer to it’s destination. +To overcome such challenges, Reticulum’s *Transport* system uses asymmetric elliptic curve cryptography to implement the concept of *paths* that allow discovery of how to get information closer to a certain destination. It is important to note that no single node in a Reticulum network knows the complete path to a destination. Every Transport node participating in a Reticulum network will only know the most direct way to get a packet one hop closer to it’s destination. ### Node Types -Currently, Reticulum distinguishes between two types of network nodes. All nodes on a Reticulum network -are *Reticulum Instances*, and some are also *Transport Nodes*. If a system running Reticulum is fixed in -one place, and is intended to be kept available most of the time, it is a good contender to be a *Transport Node*. +Currently, Reticulum distinguishes between two types of network nodes. All nodes on a Reticulum network are *Reticulum Instances*, and some are also *Transport Nodes*. If a system running Reticulum is fixed in one place, and is intended to be kept available most of the time, it is a good contender to be a *Transport Node*. -Any Reticulum Instance can become a Transport Node by enabling it in the configuration. -This distinction is made by the user configuring the node, and is used to determine what nodes on the -network will help forward traffic, and what nodes rely on other nodes for wider connectivity. +Any Reticulum Instance can become a Transport Node by enabling it in the configuration. This distinction is made by the user configuring the node, and is used to determine what nodes on the network will help forward traffic, and what nodes rely on other nodes for wider connectivity. -If a node is an *Instance* it should be given the configuration directive `enable_transport = No`, which -is the default setting. +If a node is an *Instance* it should be given the configuration directive `enable_transport = No`, which is the default setting. If it is a *Transport Node*, it should be given the configuration directive `enable_transport = Yes`. ### The Announce Mechanism in Detail -When an *announce* for a destination is transmitted by a Reticulum instance, it will be forwarded by -any transport node receiving it, but according to some specific rules: +When an *announce* for a destination is transmitted by a Reticulum instance, it will be forwarded by any transport node receiving it, but according to some specific rules: * If this exact announce has already been received before, ignore it. -
+ * If not, record into a table which Transport Node the announce was received from, and how many times in total it has been retransmitted to get here. -
+ * If the announce has been retransmitted *m+1* times, it will not be forwarded any more. By default, *m* is set to 128. -
+ * After a randomised delay, the announce will be retransmitted on all interfaces that have bandwidth available for processing announces. By default, the maximum bandwidth allocation for processing announces is set at 2%, but can be configured on a per-interface basis. -
+ * If any given interface does not have enough bandwidth available for retransmitting the announce, the announce will be assigned a priority inversely proportional to its hop count, and be inserted into a queue managed by the interface. -
+ * When the interface has bandwidth available for processing an announce, it will prioritise announces for destinations that are closest in terms of hops, thus prioritising reachability and connectivity of local nodes, even on slow networks that connect to wider and faster networks. -
+ * After the announce has been re-transmitted, and if no other nodes are heard retransmitting the announce with a greater hop count than when it left this node, transmitting it will be retried *r* times. By default, *r* is set to 1. -
+ * If a newer announce from the same destination arrives, while an identical one is already waiting to be transmitted, the newest announce is discarded. If the newest announce contains different application specific data, it will replace the old announce. -
+ -Once an announce has reached a transport node in the network, any other node in direct contact with that -transport node will be able to reach the destination the announce originated from, simply by sending a packet -addressed to that destination. Any transport node with knowledge of the announce will be able to direct the -packet towards the destination by looking up the most efficient next node to the destination. +Once an announce has reached a transport node in the network, any other node in direct contact with that transport node will be able to reach the destination the announce originated from, simply by sending a packet addressed to that destination. Any transport node with knowledge of the announce will be able to direct the packet towards the destination by looking up the most efficient next node to the destination. -According to these rules, an announce will propagate throughout the network in a predictable way, -and make the announced destination reachable in a short amount of time. Fast networks that have the -capacity to process many announces can reach full convergence very quickly, even when constantly adding -new destinations. Slower segments of such networks might take a bit longer to gain full knowledge about -the wide and fast networks they are connected to, but can still do so over time, while prioritising full -and quickly converging end-to-end connectivity for their local, slower segments. +According to these rules, an announce will propagate throughout the network in a predictable way, and make the announced destination reachable in a short amount of time. Fast networks that have the capacity to process many announces can reach full convergence very quickly, even when constantly adding new destinations. Slower segments of such networks might take a bit longer to gain full knowledge about the wide and fast networks they are connected to, but can still do so over time, while prioritising full and quickly converging end-to-end connectivity for their local, slower segments. -In general, even extremely complex networks, that utilize the maximum 128 hops will converge to full -end-to-end connectivity in about one minute, given there is enough bandwidth available to process -the required amount of announces. +In general, even extremely complex networks, that utilize the maximum 128 hops will converge to full end-to-end connectivity in about one minute, given there is enough bandwidth available to process the required amount of announces. ### Reaching the Destination -In networks with changing topology and trustless connectivity, nodes need a way to establish -*verified connectivity* with each other. Since the underlying network mediums are assumed to be trustless, Reticulum -must provide a way to guarantee that the peer you are communicating with is actually who you -expect. Reticulum offers two ways to do this. +In networks with changing topology and trustless connectivity, nodes need a way to establish *verified connectivity* with each other. Since the underlying network mediums are assumed to be trustless, Reticulum must provide a way to guarantee that the peer you are communicating with is actually who you expect. Reticulum offers two ways to do this. For exchanges of small amounts of information, Reticulum offers the *Packet* API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet: * A packet is always created with an associated destination and some payload data. When the packet is sent to a *single* destination type, Reticulum will automatically create an ephemeral encryption key, perform an ECDH key exchange with the destination’s public key (or ratchet key, if available), and encrypt the information. -
+ * It is important to note that this key exchange does not require any network traffic. The sender already knows the public key of the destination from an earlier received announce, and can thus perform the ECDH key exchange locally, before sending the packet. -
+ * The public part of the newly generated ephemeral key-pair is included with the encrypted token, and sent along with the encrypted payload data in the packet. -
+ * When the destination receives the packet, it can itself perform an ECDH key exchange and decrypt the packet. -
+ * A new ephemeral key is used for every packet sent in this way. -
+ * Once the packet has been received and decrypted by the addressed destination, that destination can opt to *prove* its receipt of the packet. It does this by calculating the SHA-256 hash of the received packet, and signing this hash with its Ed25519 signing key. Transport nodes in the network can then direct this *proof* back to the packets origin, where the signature can be verified against the destination’s known public signing key. -
+ * In case the packet is addressed to a *group* destination type, the packet will be encrypted with the pre-shared AES-256 key associated with the destination. In case the packet is addressed to a *plain* destination type, the payload data will not be encrypted. Neither of these two destination types can offer forward secrecy. In general, it is recommended to always use the *single* destination type, unless it is strictly necessary to use one of the others. -
+ For exchanges of larger amounts of data, or when longer sessions of bidirectional communication is desired, Reticulum offers the *Link* API. To establish a *link*, the following process is employed: * First, the node that wishes to establish a link will send out a *link request* packet, that traverses the network and locates the desired destination. Along the way, the Transport Nodes that forward the packet will take note of this *link request*, and mark it as pending. -
+ * Second, if the destination accepts the *link request* , it will send back a packet that proves the authenticity of its identity (and the receipt of the link request) to the initiating node. All nodes that initially forwarded the packet will also be able to verify this proof, and thus accept the validity of the *link* throughout the network. The link is now marked as *established*. -
+ * When the validity of the *link* has been accepted by forwarding nodes, these nodes will remember the *link* , and it can subsequently be used by referring to a hash representing it. -
+ * As a part of the *link request*, an Elliptic Curve Diffie-Hellman key exchange takes place, that sets up an efficiently encrypted tunnel between the two nodes. As such, this mode of communication is preferred, even for situations when nodes can directly communicate, when the amount of data to be exchanged numbers in the tens of packets, or whenever the use of the more advanced API functions is desired. -
+ * When a *link* has been set up, it automatically provides message receipt functionality, through the same *proof* mechanism discussed before, so the sending node can obtain verified confirmation that the information reached the intended recipient. -
+ * Once the *link* has been set up, the initiator can remain anonymous, or choose to authenticate towards the destination using a Reticulum Identity. This authentication is happening inside the encrypted link, and is only revealed to the verified destination, and no intermediaries. -
+ -In a moment, we will discuss the details of how this methodology is -implemented, but let’s first recap what purposes this methodology serves. We -first ensure that the node answering our request is actually the one we want to -communicate with, and not a malicious actor pretending to be so. At the same -time we establish an efficient encrypted channel. The setup of this is -relatively cheap in terms of bandwidth, so it can be used just for a short -exchange, and then recreated as needed, which will also rotate encryption keys. -The link can also be kept alive for longer periods of time, if this is more -suitable to the application. The procedure also inserts the *link id* , a hash -calculated from the link request packet, into the memory of forwarding nodes, -which means that the communicating nodes can thereafter reach each other simply -by referring to this *link id*. +In a moment, we will discuss the details of how this methodology is implemented, but let’s first recap what purposes this methodology serves. We first ensure that the node answering our request is actually the one we want to communicate with, and not a malicious actor pretending to be so. At the same time we establish an efficient encrypted channel. The setup of this is relatively cheap in terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is more suitable to the application. The procedure also inserts the *link id* , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this *link id*. -The combined bandwidth cost of setting up a link is 3 packets totalling 297 bytes (more info in the -[Binary Packet Format](#understanding-packetformat) section). The amount of bandwidth used on keeping -a link open is practically negligible, at 0.45 bits per second. Even on a slow 1200 bits per second packet -radio channel, 100 concurrent links will still leave 96% channel capacity for actual data. +The combined bandwidth cost of setting up a link is 3 packets totalling 297 bytes (more info in the [Binary Packet Format](#understanding-packetformat) section). The amount of bandwidth used on keeping a link open is practically negligible, at 0.45 bits per second. Even on a slow 1200 bits per second packet radio channel, 100 concurrent links will still leave 96% channel capacity for actual data. #### Link Establishment in Detail -After exploring the basics of the announce mechanism, finding a path through the network, and an overview -of the link establishment procedure, this section will go into greater detail about the Reticulum link -establishment process. +After exploring the basics of the announce mechanism, finding a path through the network, and an overview of the link establishment procedure, this section will go into greater detail about the Reticulum link establishment process. -The *link* in Reticulum terminology should not be viewed as a direct node-to-node link on the -physical layer, but as an abstract channel, that can be open for any amount of time, and can span -an arbitrary number of hops, where information will be exchanged between two nodes. +The *link* in Reticulum terminology should not be viewed as a direct node-to-node link on the physical layer, but as an abstract channel, that can be open for any amount of time, and can span an arbitrary number of hops, where information will be exchanged between two nodes. * When a node in the network wants to establish verified connectivity with another node, it will randomly generate a new X25519 private/public key pair. It then creates a *link request* packet, and broadcast it. -
-
+ + *It should be noted that the X25519 public/private keypair mentioned above is two separate keypairs: An encryption key pair, used for derivation of a shared symmetric key, and a signing key pair, used for signing and verifying messages on the link. They are sent together over the wire, and can be considered as single public key for simplicity in this explanation.* -
+ * The *link request* is addressed to the destination hash of the desired destination, and contains the following data: The newly generated X25519 public key *LKi*. -
+ * The broadcasted packet will be directed through the network according to the rules laid out previously. -
+ * Any node that forwards the link request will store a *link id* in it’s *link table* , along with the amount of hops the packet had taken when received. The link id is a hash of the entire link request packet. If the link request packet is not *proven* by the addressed destination within some set amount of time, the entry will be dropped from the *link table* again. -
+ * When the destination receives the link request packet, it will decide whether to accept the request. If it is accepted, the destination will also generate a new X25519 private/public key pair, and perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used to encrypt the channel, once it has been established. -
+ * A *link proof* packet is now constructed and transmitted over the network. This packet is addressed to the *link id* of the *link*. It contains the following data: The newly generated X25519 public key *LKr* and an Ed25519 signature of the *link id* and *LKr* made by the *original signing key* of the addressed destination. -
+ * By verifying this *link proof* packet, all nodes that originally transported the *link request* packet to the destination from the originator can now verify that the intended destination received the request and accepted it, and that the path they chose for forwarding the request was valid. @@ -506,34 +352,26 @@ an arbitrary number of hops, where information will be exchanged between two nod An abstract bi-directional communication channel has now been established along a path in the network. Packets can now be exchanged bi-directionally from either end of the link simply by adressing the packets to the *link id* of the link. -
+ * When the source receives the *proof* , it will know unequivocally that a verified path has been established to the destination. It can now also use the X25519 public key contained in the *link proof* to perform it’s own Diffie Hellman Key Exchange and derive the symmetric key that is used to encrypt the channel. Information can now be exchanged reliably and securely. -
+ #### NOTE It’s important to note that this methodology ensures that the source of the request does not need to reveal any identifying information about itself. **The link initiator remains completely anonymous**. -When using *links*, Reticulum will automatically verify all data sent over the link, and can also -automate retransmissions if *Resources* are used. +When using *links*, Reticulum will automatically verify all data sent over the link, and can also automate retransmissions if *Resources* are used. ### Resources -For exchanging small amounts of data over a Reticulum network, the [Packet](reference.md#api-packet) interface -is sufficient, but for exchanging data that would require many packets, an efficient way to coordinate -the transfer is needed. +For exchanging small amounts of data over a Reticulum network, the [Packet](reference.md#api-packet) interface is sufficient, but for exchanging data that would require many packets, an efficient way to coordinate the transfer is needed. -This is the purpose of the Reticulum [Resource](reference.md#api-resource). A *Resource* can automatically -handle the reliable transfer of an arbitrary amount of data over an established [Link](reference.md#api-link). -Resources can auto-compress data, will handle breaking the data into individual packets, sequencing -the transfer, integrity verification and reassembling the data on the other end. +This is the purpose of the Reticulum [Resource](reference.md#api-resource). A *Resource* can automatically handle the reliable transfer of an arbitrary amount of data over an established [Link](reference.md#api-link). Resources can auto-compress data, will handle breaking the data into individual packets, sequencing the transfer, integrity verification and reassembling the data on the other end. -[Resources](reference.md#api-resource) are programmatically very simple to use, and only requires a few lines -of codes to reliably transfer any amount of data. They can be used to transfer data stored in memory, -or stream data directly from files. +[Resources](reference.md#api-resource) are programmatically very simple to use, and only requires a few lines of codes to reliably transfer any amount of data. They can be used to transfer data stored in memory, or stream data directly from files. ## Network Identities @@ -598,19 +436,11 @@ Once configured, your instances will automatically utilize this identity for sig ## Reference Setup -This section will detail a recommended *Reference Setup* for Reticulum. It is important to -note that Reticulum is designed to be usable on more or less any computing device, and over more -or less any medium that allows you to send and receive data, which satisfies some very low -minimum requirements. +This section will detail a recommended *Reference Setup* for Reticulum. It is important to note that Reticulum is designed to be usable on more or less any computing device, and over more or less any medium that allows you to send and receive data, which satisfies some very low minimum requirements. -The communication channel must support at least half-duplex operation, and provide an average -throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The -Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x -runtime environment. +The communication channel must support at least half-duplex operation, and provide an average throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x runtime environment. -That being said, this reference setup has been outlined to provide a common platform for anyone -who wants to help in the development of Reticulum, and for everyone who wants to know a -recommended setup to get started experimenting. A reference system consists of three parts: +That being said, this reference setup has been outlined to provide a common platform for anyone who wants to help in the development of Reticulum, and for everyone who wants to know a recommended setup to get started experimenting. A reference system consists of three parts: * **An Interface Device** : Which provides access to the physical medium whereupon the communication @@ -622,11 +452,7 @@ recommended setup to get started experimenting. A reference system consists of t * **A Software Stack** : The software implementing the Reticulum protocol and applications using it. -The reference setup can be considered a relatively stable platform to develop on, and also to start -building networks or applications on. While details of the implementation might change at the current stage of -development, it is the goal to maintain hardware compatibility for as long as entirely possible, and -the current reference setup has been determined to provide a functional platform for many years -into the future. The current Reference System Setup is as follows: +The reference setup can be considered a relatively stable platform to develop on, and also to start building networks or applications on. While details of the implementation might change at the current stage of development, it is the goal to maintain hardware compatibility for as long as entirely possible, and the current reference setup has been determined to provide a functional platform for many years into the future. The current Reference System Setup is as follows: * **Interface Device** : A data radio consisting of a LoRa radio module, and a microcontroller with open source @@ -645,40 +471,25 @@ use the LoRaWAN standard, but uses a custom MAC layer on top of the plain LoRa m need a plain LoRa radio module connected to a controller with the correct firmware. Full details on how to get or make such a device is available on the [RNode Page](https://github.com/markqvist/rnode_firmware). -With the current reference setup, it should be possible to get on a Reticulum network for around 100$ -even if you have none of the hardware already, and need to purchase everything. +With the current reference setup, it should be possible to get on a Reticulum network for around 100$ even if you have none of the hardware already, and need to purchase everything. -This reference setup is of course just a recommendation for getting started easily, and you should -tailor it to your own specific needs, or whatever hardware you have available. +This reference setup is of course just a recommendation for getting started easily, and you should tailor it to your own specific needs, or whatever hardware you have available. ## Protocol Specifics -This chapter will detail protocol specific information that is essential to the implementation of -Reticulum, but non-critical in understanding how the protocol works on a general level. It should be -treated more as a reference than as essential reading. +This chapter will detail protocol specific information that is essential to the implementation of Reticulum, but non-critical in understanding how the protocol works on a general level. It should be treated more as a reference than as essential reading. ### Packet Prioritisation -Currently, Reticulum is completely priority-agnostic regarding *general* traffic. All traffic is handled -on a first-come, first-serve basis. Announce re-transmission and other maintenance traffic is handled -according to the re-transmission times and priorities described earlier in this chapter. +Currently, Reticulum is completely priority-agnostic regarding *general* traffic. All traffic is handled on a first-come, first-serve basis. Announce re-transmission and other maintenance traffic is handled according to the re-transmission times and priorities described earlier in this chapter. ### Interface Access Codes -Reticulum can create named virtual networks, and networks that are only accessible by knowing a preshared -passphrase. The configuration of this is detailed in the [Common Interface Options](interfaces.md#interfaces-options) -section. To implement this feature, Reticulum uses the concept of Interface Access Codes, that are calculated -and verified per-packet. +Reticulum can create named virtual networks, and networks that are only accessible by knowing a preshared passphrase. The configuration of this is detailed in the [Common Interface Options](interfaces.md#interfaces-options) section. To implement this feature, Reticulum uses the concept of Interface Access Codes, that are calculated and verified per-packet. -An interface with a named virtual network or passphrase authentication enabled will derive a shared Ed25519 -signing identity, and for every outbound packet generate a signature of the entire packet. This signature is -then inserted into the packet as an Interface Access Code before transmission. Depending on the speed and -capabilities of the interface, the IFAC can be the full 512-bit Ed25519 signature, or a truncated version. -Configured IFAC length can be inspected for all interfaces with the `rnstatus` utility. +An interface with a named virtual network or passphrase authentication enabled will derive a shared Ed25519 signing identity, and for every outbound packet generate a signature of the entire packet. This signature is then inserted into the packet as an Interface Access Code before transmission. Depending on the speed and capabilities of the interface, the IFAC can be the full 512-bit Ed25519 signature, or a truncated version. Configured IFAC length can be inspected for all interfaces with the `rnstatus` utility. -Upon receipt, the interface will check that the signature matches the expected value, and drop the packet if it -does not. This ensures that only packets sent with the correct naming and/or passphrase parameters are allowed to -pass onto the network. +Upon receipt, the interface will check that the signature matches the expected value, and drop the packet if it does not. This ensures that only packets sent with the correct naming and/or passphrase parameters are allowed to pass onto the network. ### Wire Format @@ -814,14 +625,11 @@ but excluding any interface access codes. ### Announce Propagation Rules -The following table illustrates the rules for automatically propagating announces -from one interface type to another, for all possible combinations. For the purpose -of announce propagation, the *Full* and *Gateway* modes are identical. +The following table illustrates the rules for automatically propagating announces from one interface type to another, for all possible combinations. For the purpose of announce propagation, the *Full* and *Gateway* modes are identical. ![image](graphics/if_mode_graph_b.png) -See the [Interface Modes](interfaces.md#interfaces-modes) section for a conceptual overview -of the different interface modes, and how they are configured. +See the [Interface Modes](interfaces.md#interfaces-modes) section for a conceptual overview of the different interface modes, and how they are configured. ### Cryptographic Primitives -Reticulum uses a simple suite of efficient, strong and well-tested cryptographic -primitives, with widely available implementations that can be used both on -general-purpose CPUs and on microcontrollers. +Reticulum uses a simple suite of efficient, strong and well-tested cryptographic primitives, with widely available implementations that can be used both on general-purpose CPUs and on microcontrollers. -One of the primary considerations for choosing this particular set of primitives is -that they can be implemented *safely* with relatively few pitfalls, on practically -all current computing platforms. +One of the primary considerations for choosing this particular set of primitives is that they can be implemented *safely* with relatively few pitfalls, on practically all current computing platforms. -The primitives listed here **are authoritative**. Anything claiming to be Reticulum, -but not using these exact primitives **is not** Reticulum, and possibly an -intentionally compromised or weakened clone. The utilised primitives are: +The primitives listed here **are authoritative**. Anything claiming to be Reticulum, but not using these exact primitives **is not** Reticulum, and possibly an intentionally compromised or weakened clone. The utilised primitives are: * Ed25519 for signatures * X25519 for ECDH key exchanges @@ -870,29 +672,16 @@ intentionally compromised or weakened clone. The utilised primitives are: * SHA-256 * SHA-512 -In the default installation configuration, the `X25519`, `Ed25519` and `AES-256-CBC` -primitives are provided by [OpenSSL](https://www.openssl.org/) (via the [PyCA/cryptography](https://github.com/pyca/cryptography) -package). The hashing functions `SHA-256` and `SHA-512` are provided by the standard -Python [hashlib](https://docs.python.org/3/library/hashlib.html). The `HKDF`, `HMAC`, -`Token` primitives, and the `PKCS7` padding function are always provided by the -following internal implementations: +In the default installation configuration, the `X25519`, `Ed25519` and `AES-256-CBC` primitives are provided by [OpenSSL](https://www.openssl.org/) (via the [PyCA/cryptography](https://github.com/pyca/cryptography) package). The hashing functions `SHA-256` and `SHA-512` are provided by the standard Python [hashlib](https://docs.python.org/3/library/hashlib.html). The `HKDF`, `HMAC`, `Token` primitives, and the `PKCS7` padding function are always provided by the following internal implementations: - `RNS/Cryptography/HKDF.py` - `RNS/Cryptography/HMAC.py` - `RNS/Cryptography/Token.py` - `RNS/Cryptography/PKCS7.py` -Reticulum also includes a complete implementation of all necessary primitives in pure Python. -If OpenSSL & PyCA are not available on the system when Reticulum is started, Reticulum will -instead use the internal pure-python primitives. A trivial consequence of this is performance, -with the OpenSSL backend being *much* faster. The most important consequence however, is the -potential loss of security by using primitives that has not seen the same amount of scrutiny, -testing and review as those from OpenSSL. +Reticulum also includes a complete implementation of all necessary primitives in pure Python. If OpenSSL & PyCA are not available on the system when Reticulum is started, Reticulum will instead use the internal pure-python primitives. A trivial consequence of this is performance, with the OpenSSL backend being *much* faster. The most important consequence however, is the potential loss of security by using primitives that has not seen the same amount of scrutiny, testing and review as those from OpenSSL. -Using the normal RNS installation procedures, it is not possible to install Reticulum on a -system without the required OpenSSL primitives being available, and if they are not, they will -be resolved and installed as a dependency. It is only possible to use the pure-python primitives -by manually specifying this, for example by using the `rnspure` package. +Using the normal RNS installation procedures, it is not possible to install Reticulum on a system without the required OpenSSL primitives being available, and if they are not, they will be resolved and installed as a dependency. It is only possible to use the pure-python primitives by manually specifying this, for example by using the `rnspure` package. #### WARNING If you want to use the internal pure-python primitives, it is **highly advisable** that you diff --git a/docs/source/understanding.rst b/docs/source/understanding.rst index b1e45d28..f12974ec 100644 --- a/docs/source/understanding.rst +++ b/docs/source/understanding.rst @@ -3,64 +3,35 @@ *********************** Understanding Reticulum *********************** -This chapter will briefly describe the overall purpose and operating principles of Reticulum. -It should give you an overview of how the stack works, and an understanding of how to -develop networked applications using Reticulum. +This chapter will briefly describe the overall purpose and operating principles of Reticulum. It should give you an overview of how the stack works, and an understanding of how to develop networked applications using Reticulum. -This chapter is not an exhaustive source of information on Reticulum, at least not yet. Currently, -the only complete repository, and final authority on how Reticulum actually functions, is the Python -reference implementation and API reference. That being said, this chapter is an essential resource in -understanding how Reticulum works from a high-level perspective, along with the general principles of -Reticulum, and how to apply them when creating your own networks or software. +This chapter is not an exhaustive source of information on Reticulum, at least not yet. Currently, the only complete repository, and final authority on how Reticulum actually functions, is the Python reference implementation and API reference. That being said, this chapter is an essential resource in understanding how Reticulum works from a high-level perspective, along with the general principles of Reticulum, and how to apply them when creating your own networks or software. -After reading this chapter, you should be well-equipped to understand how a Reticulum network -operates, what it can achieve, and how you can use it yourself. This chapter also seeks to provide an overview of the -sentiments and the philosophy behind Reticulum, what problems it seeks to solve, and how it -approaches those solutions. +After reading this chapter, you should be well-equipped to understand how a Reticulum network operates, what it can achieve, and how you can use it yourself. This chapter also seeks to provide an overview of the sentiments and the philosophy behind Reticulum, what problems it seeks to solve, and how it approaches those solutions. .. _understanding-motivation: Motivation ========== -The primary motivation for designing and implementing Reticulum has been the current lack of -reliable, functional and secure minimal-infrastructure modes of digital communication. It is my -belief that it is highly desirable to create a reliable and efficient way to set up long-range digital -communication networks that can securely allow exchange of information between people and -machines, with no central point of authority, control, censorship or barrier to entry. +The primary motivation for designing and implementing Reticulum has been the current lack of reliable, functional and secure minimal-infrastructure modes of digital communication. It is my belief that it is highly desirable to create a reliable and efficient way to set up long-range digital communication networks that can securely allow exchange of information between people and machines, with no central point of authority, control, censorship or barrier to entry. -Almost all of the various networking systems in use today share a common limitation: They -require large amounts of coordination and centralised trust and power to function. To join such networks, you need approval -of gatekeepers in control. This need for coordination and trust inevitably leads to an environment of -central control, where it's very easy for infrastructure operators or governments to control or alter -traffic, and censor or persecute unwanted actors. It also makes it completely impossible to freely deploy -and use networks at will, like one would use other common tools that enhance individual agency and freedom. +Almost all of the various networking systems in use today share a common limitation: They require large amounts of coordination and centralised trust and power to function. To join such networks, you need approval of gatekeepers in control. This need for coordination and trust inevitably leads to an environment of central control, where it's very easy for infrastructure operators or governments to control or alter traffic, and censor or persecute unwanted actors. It also makes it completely impossible to freely deploy and use networks at will, like one would use other common tools that enhance individual agency and freedom. -Reticulum aims to require as little coordination and trust as possible. It aims to make secure, -anonymous and permissionless networking and information exchange a tool that anyone can just pick up and use. +Reticulum aims to require as little coordination and trust as possible. It aims to make secure, anonymous and permissionless networking and information exchange a tool that anyone can just pick up and use. -Since Reticulum is completely medium agnostic, it can be used to build networks on whatever is best -suited to the situation, or whatever you have available. In some cases, this might be packet radio -links over VHF frequencies, in other cases it might be a 2.4 GHz -network using off-the-shelf radios, or it might be using common LoRa development boards. +Since Reticulum is completely medium agnostic, it can be used to build networks on whatever is best suited to the situation, or whatever you have available. In some cases, this might be packet radio links over VHF frequencies, in other cases it might be a 2.4 GHz network using off-the-shelf radios, or it might be using common LoRa development boards. -At the time of release of this document, the fastest and easiest setup for development and testing is using -LoRa radio modules with an open source firmware (see the section :ref:`Reference Setup`), -connected to any kind of computer or mobile device that Reticulum can run on. +At the time of release of this document, the fastest and easiest setup for development and testing is using LoRa radio modules with an open source firmware (see the section :ref:`Reference Setup`), connected to any kind of computer or mobile device that Reticulum can run on. -The ultimate aim of Reticulum is to allow anyone to be their own network operator, and to make it -cheap and easy to cover vast areas with a myriad of independent, interconnectable and autonomous networks. -Reticulum **is not** *one network*, it **is a tool** to build *thousands of networks*. Networks without -kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate -with each other, and require no central oversight. Networks for human beings. *Networks for the people*. +The ultimate aim of Reticulum is to allow anyone to be their own network operator, and to make it cheap and easy to cover vast areas with a myriad of independent, interconnectable and autonomous networks. Reticulum **is not** *one network*, it **is a tool** to build *thousands of networks*. Networks without kill-switches, surveillance, censorship and control. Networks that can freely interoperate, associate and disassociate with each other, and require no central oversight. Networks for human beings. *Networks for the people*. .. _understanding-goals: Goals ===== -To be as widely usable and efficient to deploy as possible, the following goals have been used to -guide the design of Reticulum: +To be as widely usable and efficient to deploy as possible, the following goals have been used to guide the design of Reticulum: * **Fully useable as open source software stack** @@ -83,18 +54,18 @@ guide the design of Reticulum: * **Unlicensed use** Reticulum shall be functional over physical communication mediums that do not require any form of license to use. Reticulum must be designed in a way, so it is usable over ISM radio - frequency bands, and can provide functional long distance links in such conditions, for example - by connecting a modem to a PMR or CB radio, or by using LoRa or WiFi modules. + frequency bands, and can provide functional long distance links in such conditions, for + example by connecting a modem to a PMR or CB radio, or by using LoRa or WiFi modules. * **Supplied software** In addition to the core networking stack and API, that allows a developer to build applications with Reticulum, a basic set of Reticulum-based communication tools must be implemented and released along with Reticulum itself. These shall serve both as a - functional, basic communication suite, and as an example and learning resource to others wishing - to build applications with Reticulum. + functional, basic communication suite, and as an example and learning resource to others + wishing to build applications with Reticulum. * **Ease of use** - The reference implementation of Reticulum is written in Python, to make it easy to use - and understand. A programmer with only basic experience should be able to use - Reticulum to write networked applications. + The reference implementation of Reticulum is written in Python, to make it easy to use and + understand. A programmer with only basic experience should be able to use Reticulum to write + networked applications. * **Low cost** It shall be as cheap as possible to deploy a communication system based on Reticulum. This should be achieved by using cheap off-the-shelf hardware that potential users might already @@ -106,53 +77,26 @@ guide the design of Reticulum: Introduction & Basic Functionality ================================== -Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at its -core a *message oriented* system. It is suited for both local point-to-point or point-to-multipoint -scenarios where all nodes are within range of each other, as well as scenarios where packets need -to be transported over multiple hops in a complex network to reach the recipient. +Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at its core a *message oriented* system. It is suited for both local point-to-point or point-to-multipoint scenarios where all nodes are within range of each other, as well as scenarios where packets need to be transported over multiple hops in a complex network to reach the recipient. -Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead -Reticulum uses the singular concept of *destinations*. Any application using Reticulum as its -networking stack will need to create one or more destinations to receive data, and know the -destinations it needs to send data to. +Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead Reticulum uses the singular concept of *destinations*. Any application using Reticulum as its networking stack will need to create one or more destinations to receive data, and know the destinations it needs to send data to. -All destinations in Reticulum are *represented* as a 16 byte hash. This hash is derived from truncating a full -SHA-256 hash of identifying characteristics of the destination. To users, the destination addresses -will be displayed as 16 hexadecimal bytes, like this example: ``<13425ec15b621c1d928589718000d814>``. +All destinations in Reticulum are *represented* as a 16 byte hash. This hash is derived from truncating a full SHA-256 hash of identifying characteristics of the destination. To users, the destination addresses will be displayed as 16 hexadecimal bytes, like this example: ``<13425ec15b621c1d928589718000d814>``. -The truncation size of 16 bytes (128 bits) for destinations has been chosen as a reasonable trade-off -between address space -and packet overhead. The address space accommodated by this size can support many billions of -simultaneously active devices on the same network, while keeping packet overhead low, which is -essential on low-bandwidth networks. In the very unlikely case that this address space nears -congestion, a one-line code change can upgrade the Reticulum address space all the way up to 256 -bits, ensuring the Reticulum address space could potentially support galactic-scale networks. -This is obviously complete and ridiculous over-allocation, and as such, the current 128 bits should -be sufficient, even far into the future. +The truncation size of 16 bytes (128 bits) for destinations has been chosen as a reasonable trade-off between address space and packet overhead. The address space accommodated by this size can support many billions of simultaneously active devices on the same network, while keeping packet overhead low, which is essential on low-bandwidth networks. In the very unlikely case that this address space nears congestion, a one-line code change can upgrade the Reticulum address space all the way up to 256 bits, ensuring the Reticulum address space could potentially support galactic-scale networks. This is obviously complete and ridiculous over-allocation, and as such, the current 128 bits should be sufficient, even far into the future. -By default Reticulum encrypts all data using elliptic curve cryptography and AES. Any packet sent to a -destination is encrypted with a per-packet derived key. Reticulum can also set up an encrypted -channel to a destination, called a *Link*. Both data sent over Links and single packets offer -*Initiator Anonymity*. Links additionally offer *Forward Secrecy* by default, employing an Elliptic Curve -Diffie Hellman key exchange on Curve25519 to derive per-link ephemeral keys. Asymmetric, link-less -packet communication can also provide forward secrecy, with automatic key ratcheting, by enabling -ratchets on a per-destination basis. The multi-hop transport, coordination, verification and reliability -layers are fully autonomous and also based on elliptic curve cryptography. +By default Reticulum encrypts all data using elliptic curve cryptography and AES. Any packet sent to a destination is encrypted with a per-packet derived key. Reticulum can also set up an encrypted channel to a destination, called a *Link*. Both data sent over Links and single packets offer *Initiator Anonymity*. Links additionally offer *Forward Secrecy* by default, employing an Elliptic Curve Diffie Hellman key exchange on Curve25519 to derive per-link ephemeral keys. Asymmetric, link-less packet communication can also provide forward secrecy, with automatic key ratcheting, by enabling ratchets on a per-destination basis. The multi-hop transport, coordination, verification and reliability layers are fully autonomous and also based on elliptic curve cryptography. -Reticulum also offers symmetric key encryption for group-oriented communications, as well as -unencrypted packets (for local broadcast purposes **only**). +Reticulum also offers symmetric key encryption for group-oriented communications, as well as unencrypted packets (for local broadcast purposes **only**). -Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports, -and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or -private IP networks. +Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports, and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or private IP networks. .. _understanding-destinations: Destinations ------------ -To receive and send data with the Reticulum stack, an application needs to create one or more -destinations. Reticulum uses three different basic destination types, and one special: +To receive and send data with the Reticulum stack, an application needs to create one or more destinations. Reticulum uses three different basic destination types, and one special: * **Single** @@ -165,9 +109,9 @@ destinations. Reticulum uses three different basic destination types, and one sp number of users, or should be readable by anyone. Traffic to a *plain* destination is not encrypted. Generally, *plain* destinations can be used for broadcast information intended to be public. Plain destinations are only reachable directly, and packets addressed to plain destinations are - never transported over multiple hops in the network. To be transportable over multiple hops in Reticulum, information - *must* be encrypted, since Reticulum uses the per-packet encryption to verify routing paths and - keep them alive. + never transported over multiple hops in the network. To be transportable over multiple hops in + Reticulum, information *must* be encrypted, since Reticulum uses the per-packet encryption to verify + routing paths and keep them alive. * **Group** The *group* special destination type, that defines a symmetrically encrypted virtual destination. Data sent to this destination will be encrypted with a symmetric key, and will be readable by @@ -186,16 +130,11 @@ destinations. Reticulum uses three different basic destination types, and one sp Destination Naming ^^^^^^^^^^^^^^^^^^ -Destinations are created and named in an easy to understand dotted notation of *aspects*, and -represented on the network as a hash of this value. The hash is a SHA-256 truncated to 128 bits. The -top level aspect should always be a unique identifier for the application using the destination. -The next levels of aspects can be defined in any way by the creator of the application. +Destinations are created and named in an easy to understand dotted notation of *aspects*, and represented on the network as a hash of this value. The hash is a SHA-256 truncated to 128 bits. The top level aspect should always be a unique identifier for the application using the destination. The next levels of aspects can be defined in any way by the creator of the application. -Aspects can be as long and as plentiful as required, and a resulting long destination name will not -impact efficiency, as names are always represented as truncated SHA-256 hashes on the network. +Aspects can be as long and as plentiful as required, and a resulting long destination name will not impact efficiency, as names are always represented as truncated SHA-256 hashes on the network. -As an example, a destination for a environmental monitoring application could be made up of the -application name, a device type and measurement type, like this: +As an example, a destination for a environmental monitoring application could be made up of the application name, a device type and measurement type, like this: .. code-block:: text @@ -205,11 +144,7 @@ application name, a device type and measurement type, like this: full name : environmentlogger.remotesensor.temperature hash : 4faf1b2e0a077e6a9d92fa051f256038 -For the *single* destination, Reticulum will automatically append the associated public key as a -destination aspect before hashing. This is done to ensure only the correct destination is reached, -since anyone can listen to any destination name. Appending the public key ensures that a given -packet is only directed at the destination that holds the corresponding private key to decrypt the -packet. +For the *single* destination, Reticulum will automatically append the associated public key as a destination aspect before hashing. This is done to ensure only the correct destination is reached, since anyone can listen to any destination name. Appending the public key ensures that a given packet is only directed at the destination that holds the corresponding private key to decrypt the packet. **Take note!** There is a very important concept to understand here: @@ -218,16 +153,9 @@ packet. * Each destination that does so will still have a unique destination hash, and thus be uniquely addressable, because their public keys will differ. -In actual use of *single* destination naming, it is advisable not to use any uniquely identifying -features in aspect naming. Aspect names should be general terms describing what kind of destination -is represented. The uniquely identifying aspect is always achieved by appending the public key, -which expands the destination into a uniquely identifiable one. Reticulum does this automatically. +In actual use of *single* destination naming, it is advisable not to use any uniquely identifying features in aspect naming. Aspect names should be general terms describing what kind of destination is represented. The uniquely identifying aspect is always achieved by appending the public key, which expands the destination into a uniquely identifiable one. Reticulum does this automatically. -Any destination on a Reticulum network can be addressed and reached simply by knowing its -destination hash (and public key, but if the public key is not known, it can be requested from the -network simply by knowing the destination hash). The use of app names and aspects makes it easy to -structure Reticulum programs and makes it possible to filter what information and data your program -receives. +Any destination on a Reticulum network can be addressed and reached simply by knowing its destination hash (and public key, but if the public key is not known, it can be requested from the network simply by knowing the destination hash). The use of app names and aspects makes it easy to structure Reticulum programs and makes it possible to filter what information and data your program receives. To recap, the different destination types should be used in the following situations: @@ -239,56 +167,32 @@ To recap, the different destination types should be used in the following situat * **Plain** When plain-text communication is desirable, for example when broadcasting information, or for local discovery purposes. -To communicate with a *single* destination, you need to know its public key. Any method for -obtaining the public key is valid, but Reticulum includes a simple mechanism for making other -nodes aware of your destinations public key, called the *announce*. It is also possible to request -an unknown public key from the network, as all transport instances serve as a distributed ledger -of public keys. +To communicate with a *single* destination, you need to know its public key. Any method for obtaining the public key is valid, but Reticulum includes a simple mechanism for making other nodes aware of your destinations public key, called the *announce*. It is also possible to request an unknown public key from the network, as all transport instances serve as a distributed ledger of public keys. -Note that public key information can be shared and verified in other ways than using the -built-in *announce* functionality, and that it is therefore not required to use the *announce* and *path request* -functionality to obtain public keys. It is by far the easiest though, and should definitely be used -if there is not a very good reason for doing it differently. +Note that public key information can be shared and verified in other ways than using the built-in *announce* functionality, and that it is therefore not required to use the *announce* and *path request* functionality to obtain public keys. It is by far the easiest though, and should definitely be used if there is not a very good reason for doing it differently. .. _understanding-keyannouncements: Public Key Announcements ------------------------ -An *announce* will send a special packet over any relevant interfaces, containing all needed -information about the destination hash and public key, and can also contain some additional, -application specific data. The entire packet is signed by the sender to ensure authenticity. It is not -required to use the announce functionality, but in many cases it will be the simplest way to share -public keys on the network. The announce mechanism also serves to establish end-to-end connectivity -to the announced destination, as the announce propagates through the network. +An *announce* will send a special packet over any relevant interfaces, containing all needed information about the destination hash and public key, and can also contain some additional, application specific data. The entire packet is signed by the sender to ensure authenticity. It is not required to use the announce functionality, but in many cases it will be the simplest way to share public keys on the network. The announce mechanism also serves to establish end-to-end connectivity to the announced destination, as the announce propagates through the network. As an example, an announce in a simple messenger application might contain the following information: - * The announcers destination hash * The announcers public key * Application specific data, in this case the users nickname and availability status * A random blob, making each new announce unique * An Ed25519 signature of the above information, verifying authenticity -With this information, any Reticulum node that receives it will be able to reconstruct an outgoing -destination to securely communicate with that destination. You might have noticed that there is one -piece of information lacking to reconstruct full knowledge of the announced destination, and that is -the aspect names of the destination. These are intentionally left out to save bandwidth, since they -will be implicit in almost all cases. The receiving application will already know them. If a destination -name is not entirely implicit, information can be included in the application specific data part that -will allow the receiver to infer the naming. +With this information, any Reticulum node that receives it will be able to reconstruct an outgoing destination to securely communicate with that destination. You might have noticed that there is one piece of information lacking to reconstruct full knowledge of the announced destination, and that is the aspect names of the destination. These are intentionally left out to save bandwidth, since they will be implicit in almost all cases. The receiving application will already know them. If a destination name is not entirely implicit, information can be included in the application specific data part that will allow the receiver to infer the naming. It is important to note that announces will be forwarded throughout the network according to a certain pattern. This will be detailed in the section :ref:`The Announce Mechanism in Detail`. -In Reticulum, destinations are allowed to move around the network at will. This is very different from -protocols such as IP, where an address is always expected to stay within the network segment it was assigned in. -This limitation does not exist in Reticulum, and any destination is *completely portable* over the entire topography -of the network, and *can even be moved to other Reticulum networks* than the one it was created in, and -still become reachable. To update its reachability, a destination simply needs to send an announce on any -networks it is part of. After a short while, it will be globally reachable in the network. +In Reticulum, destinations are allowed to move around the network at will. This is very different from protocols such as IP, where an address is always expected to stay within the network segment it was assigned in. This limitation does not exist in Reticulum, and any destination is *completely portable* over the entire topography of the network, and *can even be moved to other Reticulum networks* than the one it was created in, and still become reachable. To update its reachability, a destination simply needs to send an announce on any networks it is part of. After a short while, it will be globally reachable in the network. Seeing how *single* destinations are always tied to a private/public key pair leads us to the next topic. @@ -297,33 +201,18 @@ Seeing how *single* destinations are always tied to a private/public key pair le Identities ---------- -In Reticulum, an *identity* does not necessarily represent a personal identity, but is an abstraction that -can represent any kind of *verifiable entity*. This could very well be a person, but it could also be the -control interface of a machine, a program, robot, computer, sensor or something else entirely. In -general, any kind of agent that can act, or be acted upon, or store or manipulate information, can be -represented as an identity. An *identity* can be used to create any number of destinations. +In Reticulum, an *identity* does not necessarily represent a personal identity, but is an abstraction that can represent any kind of *verifiable entity*. This could very well be a person, but it could also be the control interface of a machine, a program, robot, computer, sensor or something else entirely. In general, any kind of agent that can act, or be acted upon, or store or manipulate information, can be represented as an identity. An *identity* can be used to create any number of destinations. -A *single* destination will always have an *identity* tied to it, but not *plain* or *group* -destinations. Destinations and identities share a multilateral connection. You can create a -destination, and if it is not connected to an identity upon creation, it will just create a new one to use -automatically. This may be desirable in some situations, but often you will probably want to create -the identity first, and then use it to create new destinations. +A *single* destination will always have an *identity* tied to it, but not *plain* or *group* destinations. Destinations and identities share a multilateral connection. You can create a destination, and if it is not connected to an identity upon creation, it will just create a new one to use automatically. This may be desirable in some situations, but often you will probably want to create the identity first, and then use it to create new destinations. -As an example, we could use an identity to represent the user of a messaging application. -Destinations can then be created by this identity to allow communication to reach the user. -In all cases it is of great importance to store the private keys associated with any -Reticulum Identity securely and privately, since obtaining access to the identity keys equals -obtaining access and controlling reachability to any destinations created by that identity. +As an example, we could use an identity to represent the user of a messaging application. Destinations can then be created by this identity to allow communication to reach the user. In all cases it is of great importance to store the private keys associated with any Reticulum Identity securely and privately, since obtaining access to the identity keys equals obtaining access and controlling reachability to any destinations created by that identity. .. _understanding-gettingfurther: Getting Further --------------- -The above functions and principles form the core of Reticulum, and would suffice to create -functional networked applications in local clusters, for example over radio links where all interested -nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple -hops in the network. +The above functions and principles form the core of Reticulum, and would suffice to create functional networked applications in local clusters, for example over radio links where all interested nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple hops in the network. In the following sections, two concepts that allow this will be introduced, *paths* and *links*. @@ -332,16 +221,9 @@ In the following sections, two concepts that allow this will be introduced, *pat Reticulum Transport =================== -The methods of routing used in traditional networks are fundamentally incompatible with the physical medium -types and circumstances that Reticulum was designed to handle. These mechanisms mostly assume trust at the physical layer, -and often needs a lot more bandwidth than Reticulum can assume is available. Since Reticulum is designed to -survive running over open radio spectrum, no such trust can be assumed, and bandwidth is often very limited. +The methods of routing used in traditional networks are fundamentally incompatible with the physical medium types and circumstances that Reticulum was designed to handle. These mechanisms mostly assume trust at the physical layer, and often needs a lot more bandwidth than Reticulum can assume is available. Since Reticulum is designed to survive running over open radio spectrum, no such trust can be assumed, and bandwidth is often very limited. -To overcome such challenges, Reticulum’s *Transport* system uses asymmetric elliptic curve cryptography to -implement the concept of *paths* that allow discovery of how to get information closer to a certain -destination. It is important to note that no single node in a Reticulum network knows the complete -path to a destination. Every Transport node participating in a Reticulum network will only -know the most direct way to get a packet one hop closer to it's destination. +To overcome such challenges, Reticulum’s *Transport* system uses asymmetric elliptic curve cryptography to implement the concept of *paths* that allow discovery of how to get information closer to a certain destination. It is important to note that no single node in a Reticulum network knows the complete path to a destination. Every Transport node participating in a Reticulum network will only know the most direct way to get a packet one hop closer to it's destination. .. _understanding-nodetypes: @@ -349,16 +231,11 @@ know the most direct way to get a packet one hop closer to it's destination. Node Types ---------- -Currently, Reticulum distinguishes between two types of network nodes. All nodes on a Reticulum network -are *Reticulum Instances*, and some are also *Transport Nodes*. If a system running Reticulum is fixed in -one place, and is intended to be kept available most of the time, it is a good contender to be a *Transport Node*. +Currently, Reticulum distinguishes between two types of network nodes. All nodes on a Reticulum network are *Reticulum Instances*, and some are also *Transport Nodes*. If a system running Reticulum is fixed in one place, and is intended to be kept available most of the time, it is a good contender to be a *Transport Node*. -Any Reticulum Instance can become a Transport Node by enabling it in the configuration. -This distinction is made by the user configuring the node, and is used to determine what nodes on the -network will help forward traffic, and what nodes rely on other nodes for wider connectivity. +Any Reticulum Instance can become a Transport Node by enabling it in the configuration. This distinction is made by the user configuring the node, and is used to determine what nodes on the network will help forward traffic, and what nodes rely on other nodes for wider connectivity. -If a node is an *Instance* it should be given the configuration directive ``enable_transport = No``, which -is the default setting. +If a node is an *Instance* it should be given the configuration directive ``enable_transport = No``, which is the default setting. If it is a *Transport Node*, it should be given the configuration directive ``enable_transport = Yes``. @@ -368,8 +245,7 @@ If it is a *Transport Node*, it should be given the configuration directive ``en The Announce Mechanism in Detail -------------------------------- -When an *announce* for a destination is transmitted by a Reticulum instance, it will be forwarded by -any transport node receiving it, but according to some specific rules: +When an *announce* for a destination is transmitted by a Reticulum instance, it will be forwarded by any transport node receiving it, but according to some specific rules: * | If this exact announce has already been received before, ignore it. @@ -400,17 +276,9 @@ any transport node receiving it, but according to some specific rules: to be transmitted, the newest announce is discarded. If the newest announce contains different application specific data, it will replace the old announce. -Once an announce has reached a transport node in the network, any other node in direct contact with that -transport node will be able to reach the destination the announce originated from, simply by sending a packet -addressed to that destination. Any transport node with knowledge of the announce will be able to direct the -packet towards the destination by looking up the most efficient next node to the destination. +Once an announce has reached a transport node in the network, any other node in direct contact with that transport node will be able to reach the destination the announce originated from, simply by sending a packet addressed to that destination. Any transport node with knowledge of the announce will be able to direct the packet towards the destination by looking up the most efficient next node to the destination. -According to these rules, an announce will propagate throughout the network in a predictable way, -and make the announced destination reachable in a short amount of time. Fast networks that have the -capacity to process many announces can reach full convergence very quickly, even when constantly adding -new destinations. Slower segments of such networks might take a bit longer to gain full knowledge about -the wide and fast networks they are connected to, but can still do so over time, while prioritising full -and quickly converging end-to-end connectivity for their local, slower segments. +According to these rules, an announce will propagate throughout the network in a predictable way, and make the announced destination reachable in a short amount of time. Fast networks that have the capacity to process many announces can reach full convergence very quickly, even when constantly adding new destinations. Slower segments of such networks might take a bit longer to gain full knowledge about the wide and fast networks they are connected to, but can still do so over time, while prioritising full and quickly converging end-to-end connectivity for their local, slower segments. .. tip:: @@ -423,19 +291,14 @@ and quickly converging end-to-end connectivity for their local, slower segments. knowledge, since paths can always be recursively resolved from other segments that do have knowledge about them. -In general, even extremely complex networks, that utilize the maximum 128 hops will converge to full -end-to-end connectivity in about one minute, given there is enough bandwidth available to process -the required amount of announces. +In general, even extremely complex networks, that utilize the maximum 128 hops will converge to full end-to-end connectivity in about one minute, given there is enough bandwidth available to process the required amount of announces. .. _understanding-paths: Reaching the Destination ------------------------ -In networks with changing topology and trustless connectivity, nodes need a way to establish -*verified connectivity* with each other. Since the underlying network mediums are assumed to be trustless, Reticulum -must provide a way to guarantee that the peer you are communicating with is actually who you -expect. Reticulum offers two ways to do this. +In networks with changing topology and trustless connectivity, nodes need a way to establish *verified connectivity* with each other. Since the underlying network mediums are assumed to be trustless, Reticulum must provide a way to guarantee that the peer you are communicating with is actually who you expect. Reticulum offers two ways to do this. For exchanges of small amounts of information, Reticulum offers the *Packet* API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet: @@ -495,35 +358,17 @@ For exchanges of larger amounts of data, or when longer sessions of bidirectiona the destination using a Reticulum Identity. This authentication is happening inside the encrypted link, and is only revealed to the verified destination, and no intermediaries. -In a moment, we will discuss the details of how this methodology is -implemented, but let’s first recap what purposes this methodology serves. We -first ensure that the node answering our request is actually the one we want to -communicate with, and not a malicious actor pretending to be so. At the same -time we establish an efficient encrypted channel. The setup of this is -relatively cheap in terms of bandwidth, so it can be used just for a short -exchange, and then recreated as needed, which will also rotate encryption keys. -The link can also be kept alive for longer periods of time, if this is more -suitable to the application. The procedure also inserts the *link id* , a hash -calculated from the link request packet, into the memory of forwarding nodes, -which means that the communicating nodes can thereafter reach each other simply -by referring to this *link id*. +In a moment, we will discuss the details of how this methodology is implemented, but let’s first recap what purposes this methodology serves. We first ensure that the node answering our request is actually the one we want to communicate with, and not a malicious actor pretending to be so. At the same time we establish an efficient encrypted channel. The setup of this is relatively cheap in terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is more suitable to the application. The procedure also inserts the *link id* , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this *link id*. -The combined bandwidth cost of setting up a link is 3 packets totalling 297 bytes (more info in the -:ref:`Binary Packet Format` section). The amount of bandwidth used on keeping -a link open is practically negligible, at 0.45 bits per second. Even on a slow 1200 bits per second packet -radio channel, 100 concurrent links will still leave 96% channel capacity for actual data. +The combined bandwidth cost of setting up a link is 3 packets totalling 297 bytes (more info in the :ref:`Binary Packet Format` section). The amount of bandwidth used on keeping a link open is practically negligible, at 0.45 bits per second. Even on a slow 1200 bits per second packet radio channel, 100 concurrent links will still leave 96% channel capacity for actual data. Link Establishment in Detail ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -After exploring the basics of the announce mechanism, finding a path through the network, and an overview -of the link establishment procedure, this section will go into greater detail about the Reticulum link -establishment process. +After exploring the basics of the announce mechanism, finding a path through the network, and an overview of the link establishment procedure, this section will go into greater detail about the Reticulum link establishment process. -The *link* in Reticulum terminology should not be viewed as a direct node-to-node link on the -physical layer, but as an abstract channel, that can be open for any amount of time, and can span -an arbitrary number of hops, where information will be exchanged between two nodes. +The *link* in Reticulum terminology should not be viewed as a direct node-to-node link on the physical layer, but as an abstract channel, that can be open for any amount of time, and can span an arbitrary number of hops, where information will be exchanged between two nodes. * | When a node in the network wants to establish verified connectivity with another node, it @@ -574,26 +419,18 @@ an arbitrary number of hops, where information will be exchanged between two nod It’s important to note that this methodology ensures that the source of the request does not need to reveal any identifying information about itself. **The link initiator remains completely anonymous**. -When using *links*, Reticulum will automatically verify all data sent over the link, and can also -automate retransmissions if *Resources* are used. +When using *links*, Reticulum will automatically verify all data sent over the link, and can also automate retransmissions if *Resources* are used. .. _understanding-resources: Resources --------- -For exchanging small amounts of data over a Reticulum network, the :ref:`Packet` interface -is sufficient, but for exchanging data that would require many packets, an efficient way to coordinate -the transfer is needed. +For exchanging small amounts of data over a Reticulum network, the :ref:`Packet` interface is sufficient, but for exchanging data that would require many packets, an efficient way to coordinate the transfer is needed. -This is the purpose of the Reticulum :ref:`Resource`. A *Resource* can automatically -handle the reliable transfer of an arbitrary amount of data over an established :ref:`Link`. -Resources can auto-compress data, will handle breaking the data into individual packets, sequencing -the transfer, integrity verification and reassembling the data on the other end. +This is the purpose of the Reticulum :ref:`Resource`. A *Resource* can automatically handle the reliable transfer of an arbitrary amount of data over an established :ref:`Link`. Resources can auto-compress data, will handle breaking the data into individual packets, sequencing the transfer, integrity verification and reassembling the data on the other end. -:ref:`Resources` are programmatically very simple to use, and only requires a few lines -of codes to reliably transfer any amount of data. They can be used to transfer data stored in memory, -or stream data directly from files. +:ref:`Resources` are programmatically very simple to use, and only requires a few lines of codes to reliably transfer any amount of data. They can be used to transfer data stored in memory, or stream data directly from files. .. _understanding-network_identities: @@ -676,19 +513,11 @@ Once configured, your instances will automatically utilize this identity for sig Reference Setup ====================== -This section will detail a recommended *Reference Setup* for Reticulum. It is important to -note that Reticulum is designed to be usable on more or less any computing device, and over more -or less any medium that allows you to send and receive data, which satisfies some very low -minimum requirements. +This section will detail a recommended *Reference Setup* for Reticulum. It is important to note that Reticulum is designed to be usable on more or less any computing device, and over more or less any medium that allows you to send and receive data, which satisfies some very low minimum requirements. -The communication channel must support at least half-duplex operation, and provide an average -throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The -Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x -runtime environment. +The communication channel must support at least half-duplex operation, and provide an average throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x runtime environment. -That being said, this reference setup has been outlined to provide a common platform for anyone -who wants to help in the development of Reticulum, and for everyone who wants to know a -recommended setup to get started experimenting. A reference system consists of three parts: +That being said, this reference setup has been outlined to provide a common platform for anyone who wants to help in the development of Reticulum, and for everyone who wants to know a recommended setup to get started experimenting. A reference system consists of three parts: * **An Interface Device** Which provides access to the physical medium whereupon the communication @@ -700,11 +529,7 @@ recommended setup to get started experimenting. A reference system consists of t * **A Software Stack** The software implementing the Reticulum protocol and applications using it. -The reference setup can be considered a relatively stable platform to develop on, and also to start -building networks or applications on. While details of the implementation might change at the current stage of -development, it is the goal to maintain hardware compatibility for as long as entirely possible, and -the current reference setup has been determined to provide a functional platform for many years -into the future. The current Reference System Setup is as follows: +The reference setup can be considered a relatively stable platform to develop on, and also to start building networks or applications on. While details of the implementation might change at the current stage of development, it is the goal to maintain hardware compatibility for as long as entirely possible, and the current reference setup has been determined to provide a functional platform for many years into the future. The current Reference System Setup is as follows: * **Interface Device** @@ -725,47 +550,32 @@ into the future. The current Reference System Setup is as follows: need a plain LoRa radio module connected to a controller with the correct firmware. Full details on how to get or make such a device is available on the `RNode Page `_. -With the current reference setup, it should be possible to get on a Reticulum network for around 100$ -even if you have none of the hardware already, and need to purchase everything. +With the current reference setup, it should be possible to get on a Reticulum network for around 100$ even if you have none of the hardware already, and need to purchase everything. -This reference setup is of course just a recommendation for getting started easily, and you should -tailor it to your own specific needs, or whatever hardware you have available. +This reference setup is of course just a recommendation for getting started easily, and you should tailor it to your own specific needs, or whatever hardware you have available. .. _understanding-protocolspecifics: Protocol Specifics ================== -This chapter will detail protocol specific information that is essential to the implementation of -Reticulum, but non-critical in understanding how the protocol works on a general level. It should be -treated more as a reference than as essential reading. +This chapter will detail protocol specific information that is essential to the implementation of Reticulum, but non-critical in understanding how the protocol works on a general level. It should be treated more as a reference than as essential reading. Packet Prioritisation --------------------- -Currently, Reticulum is completely priority-agnostic regarding *general* traffic. All traffic is handled -on a first-come, first-serve basis. Announce re-transmission and other maintenance traffic is handled -according to the re-transmission times and priorities described earlier in this chapter. +Currently, Reticulum is completely priority-agnostic regarding *general* traffic. All traffic is handled on a first-come, first-serve basis. Announce re-transmission and other maintenance traffic is handled according to the re-transmission times and priorities described earlier in this chapter. Interface Access Codes ---------------------- -Reticulum can create named virtual networks, and networks that are only accessible by knowing a preshared -passphrase. The configuration of this is detailed in the :ref:`Common Interface Options` -section. To implement this feature, Reticulum uses the concept of Interface Access Codes, that are calculated -and verified per-packet. +Reticulum can create named virtual networks, and networks that are only accessible by knowing a preshared passphrase. The configuration of this is detailed in the :ref:`Common Interface Options` section. To implement this feature, Reticulum uses the concept of Interface Access Codes, that are calculated and verified per-packet. -An interface with a named virtual network or passphrase authentication enabled will derive a shared Ed25519 -signing identity, and for every outbound packet generate a signature of the entire packet. This signature is -then inserted into the packet as an Interface Access Code before transmission. Depending on the speed and -capabilities of the interface, the IFAC can be the full 512-bit Ed25519 signature, or a truncated version. -Configured IFAC length can be inspected for all interfaces with the ``rnstatus`` utility. +An interface with a named virtual network or passphrase authentication enabled will derive a shared Ed25519 signing identity, and for every outbound packet generate a signature of the entire packet. This signature is then inserted into the packet as an Interface Access Code before transmission. Depending on the speed and capabilities of the interface, the IFAC can be the full 512-bit Ed25519 signature, or a truncated version. Configured IFAC length can be inspected for all interfaces with the ``rnstatus`` utility. -Upon receipt, the interface will check that the signature matches the expected value, and drop the packet if it -does not. This ensures that only packets sent with the correct naming and/or passphrase parameters are allowed to -pass onto the network. +Upon receipt, the interface will check that the signature matches the expected value, and drop the packet if it does not. This ensures that only packets sent with the correct naming and/or passphrase parameters are allowed to pass onto the network. .. _understanding-packetformat: @@ -909,14 +719,11 @@ Wire Format Announce Propagation Rules -------------------------- -The following table illustrates the rules for automatically propagating announces -from one interface type to another, for all possible combinations. For the purpose -of announce propagation, the *Full* and *Gateway* modes are identical. +The following table illustrates the rules for automatically propagating announces from one interface type to another, for all possible combinations. For the purpose of announce propagation, the *Full* and *Gateway* modes are identical. .. image:: graphics/if_mode_graph_b.png -See the :ref:`Interface Modes` section for a conceptual overview -of the different interface modes, and how they are configured. +See the :ref:`Interface Modes` section for a conceptual overview of the different interface modes, and how they are configured. .. (.. code-block:: text) @@ -946,17 +753,11 @@ of the different interface modes, and how they are configured. Cryptographic Primitives ------------------------ -Reticulum uses a simple suite of efficient, strong and well-tested cryptographic -primitives, with widely available implementations that can be used both on -general-purpose CPUs and on microcontrollers. +Reticulum uses a simple suite of efficient, strong and well-tested cryptographic primitives, with widely available implementations that can be used both on general-purpose CPUs and on microcontrollers. -One of the primary considerations for choosing this particular set of primitives is -that they can be implemented *safely* with relatively few pitfalls, on practically -all current computing platforms. +One of the primary considerations for choosing this particular set of primitives is that they can be implemented *safely* with relatively few pitfalls, on practically all current computing platforms. -The primitives listed here **are authoritative**. Anything claiming to be Reticulum, -but not using these exact primitives **is not** Reticulum, and possibly an -intentionally compromised or weakened clone. The utilised primitives are: +The primitives listed here **are authoritative**. Anything claiming to be Reticulum, but not using these exact primitives **is not** Reticulum, and possibly an intentionally compromised or weakened clone. The utilised primitives are: * Ed25519 for signatures @@ -980,12 +781,7 @@ intentionally compromised or weakened clone. The utilised primitives are: * SHA-512 -In the default installation configuration, the ``X25519``, ``Ed25519`` and ``AES-256-CBC`` -primitives are provided by `OpenSSL `_ (via the `PyCA/cryptography `_ -package). The hashing functions ``SHA-256`` and ``SHA-512`` are provided by the standard -Python `hashlib `_. The ``HKDF``, ``HMAC``, -``Token`` primitives, and the ``PKCS7`` padding function are always provided by the -following internal implementations: +In the default installation configuration, the ``X25519``, ``Ed25519`` and ``AES-256-CBC`` primitives are provided by `OpenSSL `_ (via the `PyCA/cryptography `_ package). The hashing functions ``SHA-256`` and ``SHA-512`` are provided by the standard Python `hashlib `_. The ``HKDF``, ``HMAC``, ``Token`` primitives, and the ``PKCS7`` padding function are always provided by the following internal implementations: - ``RNS/Cryptography/HKDF.py`` - ``RNS/Cryptography/HMAC.py`` @@ -993,17 +789,9 @@ following internal implementations: - ``RNS/Cryptography/PKCS7.py`` -Reticulum also includes a complete implementation of all necessary primitives in pure Python. -If OpenSSL & PyCA are not available on the system when Reticulum is started, Reticulum will -instead use the internal pure-python primitives. A trivial consequence of this is performance, -with the OpenSSL backend being *much* faster. The most important consequence however, is the -potential loss of security by using primitives that has not seen the same amount of scrutiny, -testing and review as those from OpenSSL. +Reticulum also includes a complete implementation of all necessary primitives in pure Python. If OpenSSL & PyCA are not available on the system when Reticulum is started, Reticulum will instead use the internal pure-python primitives. A trivial consequence of this is performance, with the OpenSSL backend being *much* faster. The most important consequence however, is the potential loss of security by using primitives that has not seen the same amount of scrutiny, testing and review as those from OpenSSL. -Using the normal RNS installation procedures, it is not possible to install Reticulum on a -system without the required OpenSSL primitives being available, and if they are not, they will -be resolved and installed as a dependency. It is only possible to use the pure-python primitives -by manually specifying this, for example by using the ``rnspure`` package. +Using the normal RNS installation procedures, it is not possible to install Reticulum on a system without the required OpenSSL primitives being available, and if they are not, they will be resolved and installed as a dependency. It is only possible to use the pure-python primitives by manually specifying this, for example by using the ``rnspure`` package. .. warning:: If you want to use the internal pure-python primitives, it is **highly advisable** that you