Fix destination hash construction and random blob extraction

Huy proofread

.gitignore

@@ -3,6 +3,12 @@
 testutils
 TODO
 Examples/RNS
+RNS/Utilities/RNS
 build
 dist
-rns*.egg-info
+docs/build
+rns*.egg-info
+profile.data
+tests/rnsconfig/storage
+*.data
+*.result

Documentation/Reticulum Overview.md

@@ -1,492 +0,0 @@
-# Reticulum Overview
-
-This paper will briefly describe the overall purpose and operating principles of Reticulum, a
-networking stack designed for reliable and secure communication over high-latency, low-bandwidth
-links. It should give you an overview of how the stack works, and an understanding of how to
-develop networked applications using Reticulum.
-
-This document is not an exhaustive source of information on Reticulum, at least not yet. Currently,
-the best place to go for such information is the Python reference implementation of Reticulum. Both
-the reference implementation and this document may (and will) change rapidly in the current phase
-of development, but historical versions will always be available in the Git repositories.
-
-After reading this document, you should be well-equipped to understand how a Reticulum network
-operates, what it can achieve, and how you can use it yourself. If you want to help out with the
-development, this is also the place to start, since it will also provide a pretty clear overview of the
-sentiments and the philosophy behind Reticulum.
-
-## Motivation
-
-The primary motivation for designing and implementing Reticulum has been the current lack of
-reliable, functional and secure minimal-infrastructure modes of digital communication. It is my
-belief that it is highly desirable to create a cheap and reliable way to set up a wide-range digital
-communication network that can securely allow exchange of information between people and
-machines, with no central point of authority, control, censorship or barrier to entry.
-
-Almost all of the various networking stacks in wide use today share a common limitation, namely
-that they require large amounts of coordination to work. You can’t just plug in a bunch of ethernet
-cables to the same switch, or turn on a number of WiFi radios, and expect such a setup to provide a
-reliable platform for communication.
-
-The designers of the Internet Protocol had the foresight to create a protocol that powers the modern
-Internet, and works brilliantly in world very different from when it was conceived. But networks
-using the traditional IP stack needs large amounts of coordination from the people involved, and
-without central actors in ultimate control of network segments, it is very easy for a single person to
-render the platform unusable for everyone else. These limitations are inherent to the design
-principles of IP, and during the design of IP, this was a very reasonable tradeoff indeed.
-
-Reticulum aims to require as little coordination and trust as possible. In fact, the only
-“coordination” required is to know how to get connected to a Reticulum network. Since Reticulum
-is medium agnostic, this could be whatever is best suited to the situation. In some cases, this might
-be 1200 baud packet radio links over VHF frequencies, in other cases it might be a microwave
-network using off-the-shelf radios. At the time of release of this document, the recommended setup
-is using cheap LoRa radio modules with an open source firmware (see the chapter _Reference System
-Setup_ ), connected to a small computer like a Raspberry Pi. As an example, the default reference
-setup provides a channel capacity of 5.4 Kbps, and a usable direct node-to-node range of around 15
-kilometers (indefinitely extendable by using multiple hops).
-
-
-## Goals
-
-To be as widely usable and easy to implement as possible, the following goals have been used to
-guide the design of Reticulum:
-
-- **Fully useable as open source software stack**
-    Reticulum must be implemented, and be able to run using only open source software. This is
-    critical to ensuring availability, security and transparency of the system.
-- **Hardware layer agnosticism**
-    Reticulum shall be fully hardware agnostic, and should be useable over a wide range
-    physical networking layers, such as data radios, serial lines, modems, handheld transceivers,
-    wired ethernet, wifi, or anything else that can carry a digital data stream. Hardware made for
-    dedicated Reticulum use shall be as cheap as possible and use off-the-shelf components, so
-    it can be easily replicated.
-- **Very low bandwidth requirements**
-    Reticulum should be able to function reliably over links with a data capacity as low as _1,_
-    _bps_.
-- **Encryption by default**
-    Reticulum must use encryption by default where possible and applicable.
-- **Unlicensed use**
-    Reticulum shall be functional over physical communication mediums that do not require any
-    form of license to use. Reticulum must be designed in a way, so it is usable over ISM radio
-    frequency bands, and can provide functional long distance links in such conditions.
-- **Supplied software**
-    Apart from the core networking stack and API, that allows any developer to build
-    applications with Reticulum, a basic communication suite using Reticulum must be
-    implemented and released at the same time as Reticulum itself. This shall serve both as a
-    functional communication suite, and as an example and learning resource to others wishing
-    to build applications with Reticulum.
-- **Ease of use**
-    The reference implementation of Reticulum is written in Python, to make it very easy to use
-    and understand. Any programmer with only basic experience should be able to use
-    Reticulum in their own applications.
-- **Low cost**
-    It shall be as cheap as possible to deploy a communication system based on Reticulum. This
-    should be achieved by using cheap off-the-shelf hardware that potential users might already
-    own. The cost of setting up a functioning node should be less than $100 even if all parts
-    needs to be purchased.
-
-
-# Introduction & Basic Functionality
-
-Reticulum is a networking stack suited for high-latency, low-bandwidth links. Reticulum is at it’s
-core _message oriented_ , but can provide connection oriented sessions. It is suited for both local
-point-to-point or point-to-multipoint scenarios where alle nodes are within range of each other, as
-well as scenarios where packets need to be transported over multiple hops to reach the recipient.
-
-Reticulum does away with the idea of addresses and ports known from IP, TCP and UDP. Instead
-Reticulum uses the singular concept of _destinations_. Any application using Reticulum as it’s
-networking stack will need to create one or more destinations to receive data, and know the
-destinations it needs to send data to.
-
-Reticulum encrypts all data by default using public-key cryptography. Any message sent to a
-destination is encrypted with that destinations public key. Reticulum also offers symmetric key
-encryption for group-oriented communications, as well as unencrypted packets for broadcast
-purposes, or situations where you need the communication to be in plain text. The multi-hop
-transport, coordination, verification and reliability layers are fully autonomous and based on public
-key cryptography.
-
-Reticulum can connect to a variety of interfaces such as radio modems, data radios and serial ports,
-and offers the possibility to easily tunnel Reticulum traffic over IP links such as the Internet or
-private IP networks.
-
-## Destinations
-
-To receive and send data with the Reticulum stack, an application needs to create one or more
-destinations. Reticulum uses three different basic destination types, and one special:
-
-- **Single**
-    The _single_ destination type defines a public-key encrypted destination. Any data sent to this
-    destination will be encrypted with the destination’s public key, and will only be readable by
-    the creator of the destination.
-- **Group**
-    The _group_ destination type defines a symmetrically encrypted destination. Data sent to this
-    destination will be encrypted with a symmetric key, and will be readable by anyone in
-    possession of the key. The _group_ destination can be used just as well by only two peers, as it
-    can by many.
-- **Plain**
-    A _plain_ destination type is unencrypted, and suited for traffic that should be broadcast to a
-    number of users, or should be readable by anyone.
-- **Link**
-    A _link_ is a special destination type, that serves as an abstract channel between two _single_
-    destinations, directly connected or over multiple hops. The _link_ also offers reliability and
-    more efficient encryption, and as such is useful even when nodes are directly connected.
-
-
-## Destination Naming
-
-Destinations are created and named in an easy to understand dotted notation of _aspects_ , and
-represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The
-top level aspect should always be the a unique identifier for the application using the destination.
-The next levels of aspects can be defined in any way by the creator of the application. For example,
-a destination for a messaging application could be made up of the application name and a username,
-and look like this:
-
-```
-name: simplemessenger.someuser hash: 2a7ddfab5213f916dea
-```
-For the _single_ destination, Reticulum will automatically append the associated public key as a
-destination aspect before hashing. This is done to ensure only the correct destination is reached,
-since anyone can listen to any destination name. Appending the public key ensures that a given
-packet is only directed at the destination that holds the corresponding private key to decrypt the
-packet. It is important to understand that anyone can use the destination name
-_simplemessenger.myusername_ , but each person that does so will still have a different destination
-hash, because their public keys will differ. In actual use of _single_ destination naming, it is advisable
-not to use any uniquely identifying features in aspect naming, though. In the simple messenger
-example, when using _single_ destinations, we would instead use a destination naming scheme such
-as _simplemessenger.user_ where appending the public key expands the destination into a uniquely
-identifying one.
-
-To recap, the destination types should be used in the following situations:
-
-- **Single**
-    When private communication between two endpoints is needed. Supports routing.
-- **Group**
-    When private communication between two or more endpoints is needed. More efficient in
-    data usage than _single_ destinations. Supports routing indirectly, but must first be established
-    through a _single_ destination.
-- **Plain**
-    When plain-text communication is desirable, for example when broadcasting information.
-
-To communicate with a _single_ destination, you need to know it’s public key. Any method for
-obtaining the public key is valid, but Reticulum includes a simple mechanism for making other
-nodes aware of your destinations public key, called the _announce_.
-
-Note that this information could be shared and verified in many other ways, and that it is therefore
-not required to use the announce functionality, although it is by far the easiest, and should probably
-be used if you are not confident in how to verify public keys and signatures manually.
-
-
-## Public key announcements
-
-An _announce_ will send a special packet over any configured interfaces, containing all needed
-information about the destination hash and public key, and can also contain some additional,
-application specific data. The entire packet is signed by the sender to ensure authenticity. It is not
-required to use the announce functionality, but in many cases it will be the simplest way to share
-public keys on the network. As an example, an announce in a simple messenger application might
-contain the following information:
-
-- The announcers destination hash
-- The announcers public key
-- Application specific data, in this case the users nickname and availability status
-- A random blob, making each new announce unique
-- A signature of the above information, verifying authenticity
-
-With this information, any Reticulum node that receives it will be able to reconstruct an outgoing
-destination to securely communicate with that destination. You might have noticed that there is one
-piece of information lacking to reconstruct full knowledge of the announced destination, and that is
-the aspect names of the destination. These are intentionally left out to save bandwidth, since they
-will be implicit in almost all cases. If a destination name is not entirely implicit, information can be
-included in the application specific data part that will allow the receiver to infer the naming.
-
-It is important to note that announcements will be forwarded throughout the network according to a
-certain pattern. This will be detailed later. Seeing how _single_ destinations are always tied to a
-private/public key pair leads us to the next topic.
-
-
-## Identities
-
-In Reticulum, an _identity_ does not necessarily represent a personal identity, but is an abstraction that
-can represent any kind of _verified entity_. This could very well be a person, but it could also be the
-control interface of a machine, a program, robot, computer, sensor or something else entirely. In
-general, any kind of agent that can act, or be acted upon, or store or manipulate information, can be
-represented as an identity.
-
-As we have seen, a _single_ destination will always have an _identity_ tied to it, but not _plain_ or _group_
-destinations. Destinations and identities share a multilateral connection. You can create a
-destination, and if it is not connected to an identity upon creation, it will just create a new one to use
-automatically. This may be desirable in some situations, but often you will probably want to create
-the identity first, and then link it to created destinations.
-
-Building upon the simple messenger example, we could use an identity to represent the user of the
-application. Destinations created will then be linked to this identity to allow communication to
-reach the user. In such a case it is of great importance to store the user’s identity securely and
-privately.
-
-## Getting Further
-
-The above functions and principles form the core of Reticulum, and would suffice to create
-functional networked applications in local clusters, for example over radio links where all interested
-nodes can hear each other. But to be truly useful, we need a way to go further. In the next chapter,
-two concepts that allow this will be introduced, _paths_ and _resources_.
-
-
-# Transport
-
-I have purposefully avoided the term routing until now, and will continue to do so, because the
-current methods of routing used in IP based networks are fundamentally incompatible for the link
-types that Reticulum was designed to handle. These routing methodologies assume trust at the
-physical layer. Since Reticulum is designed to run over open radio spectrum, no such trust exists.
-Furthermore, existing routing protocols like BGP or OSPF carry too much overhead to be
-practically useable over bandwidth-limited, high-latency links.
-
-To overcome such challenges, Reticulum’s _Transport_ system uses public-key cryptography to
-implement the concept of _paths_ that allow discovery of how to get information to a certain
-destination, and _resources_ that help alleviate congestion and make reliable communication more
-efficient and less bandwidth-hungry.
-
-## Threading a Path
-
-In networks with changing topology and trustless connectivity, nodes need a way to establish
-_verified connectivity_ with each other. To do this, the following process is employed:
-
-- First, the node that wishes to establish connectivity will send out a special packet, that
-    traverses the network and locates the desired destination. Along the way, the nodes that
-    forward the packet will take note of this _link request_.
-- Second, if the destination accepts the _link request_ , it will send back a packet that proves the
-    authenticity of it’s identity (and the receipt of the link request) to the initiating node. All
-    nodes that initially forwarded the packet will also be able to verify this proof, and thus
-    accept the validity of the _link_ throughout the network.
-- When the validity of the _link_ has been accepted by forwarding nodes, these nodes will
-    remember the _link_ , and it can subsequently be used by referring to a hash representing it.
-- As a part of the _link request_ , a Diffie-Hellman key exchange takes place, that sets up an
-    efficient symmetrically encrypted tunnel between the two nodes, using elliptic curve
-    cryptography. As such, this mode of communication is preferred, even for situations when
-    nodes can directly communicate, when the amount of data to be exchanged numbers in the
-    tens of packets.
-- When a _link_ has been set up, it automatically provides message receipt functionality, so the
-    sending node can obtain verified confirmation that the information reached the intended
-    recipient.
-
-In a moment, we will discuss the specifics of how this methodology is implemented, but let’s first
-recap what purposes this serves. We first ensure that the node answering our request is actually the
-one we want to communicate with, and not a malicious actor pretending to be so. At the same time
-we establish an efficient encrypted channel. The setup of this is relatively cheap in terms of
-bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will also
-
-
-rotate encryption keys (keys can also be rotated over an existing path), but the link can also be kept
-alive for longer periods of time, if this is more suitable to the application. The amount of bandwidth
-used on keeping a link open is practically negligible. The procedure also inserts the _link id_ , a hash
-calculated from the link request packet, into the memory of forwarding nodes, which means that the
-communicating nodes can thereafter reach each other simply by referring to this _link id_.
-
-**Step 1, pathfinding**
-
-The pathfinding method builds on the _announce_ functionality discussed earlier. When an announce
-is sent out by a node, it will be forwarded by any node receiving it, but according to some specific
-rules:
-
-- If this announce has already been received before, ignore it.
-- Record into a table which node the announce was received from, and how many times in
-    total it has been retransmitted to get here.
-- If the announce has been retransmitted _m+1_ times, it will not be forwarded. By default, _m_ is
-    set to 18.
-- The announce will be assigned a delay _d_ = _ch_ seconds, where _c_ is a decay constant, by
-    default 2, and _h_ is the amount of times this packet has already been forwarded.
-- The packet will be given a priority _p = 1/d_.
-- If at least _d_ seconds has passed since the announce was received, and no other packets with a
-    priority higher than _p_ are waiting in the queue (see Packet Prioritisation), and the channel is
-    not utilized by other traffic, the announce will be forwarded.
-- If no other nodes are heard retransmitting the announce with a greater hop count than when
-    it left this node, transmitting it will be retried _r_ times. By default, _r_ is set to 2. Retries follow
-    same rules as above, with the exception that it must wait for at least _d = ch+1 + t_ seconds, ie.,
-    the amount of time it would take the next node to retransmit the packet. By default, _t_ is set to
-    10.
-- If a newer announce from the same destination arrives, while an identical one is already in
-    the queue, the newest announce is discarded. If the newest announce contains different
-    application specific data, it will replace the old announce, but will use _d_ and _p_ of the old
-    announce.
-
-Once an announce has reached a node in the network, any other node in direct contact with that
-node will be able to reach the destination the announce originated from, simply by sending a packet
-addressed to that destination. Any node with knowledge of the announce will be able to direct the
-packet towards the destination by looking up the next node with the shortest amount of hops to the
-destination. The specifics of this process is detailed in _Path Calculation_.
-
-According to these rules and default constants, an announce will propagate throughout the network
-in a predictable way. In an example network utilising the default constants, and with an average link
-
-
-distance of _Lavg =_ 15 kilometers, an announce will be able to propagate outwards to a radius of 180
-kilometers in 34 minutes, and a _maximum announce radius_ of 270 kilometers in approximately 3
-days. Methods for overcoming the distance limitation of _m * Lavg_ will be introduced later in this
-chapter.
-
-**Step 2, link establishment**
-
-After seeing how the conditions for finding a path through the network are created, we will now
-explore how two nodes can establish reliable communications over multiple hops. The _link_ in
-Reticulum terminology should not be viewed as a direct node-to-node link on the physical layer, but
-as an abstract channel, that can be open for any amount of time, and can span an arbitrary number
-of hops, where information will be exchanged between two nodes.
-
-- When a node in the network wants to establish verified connectivity with another node, it
-    will create a _link request_ packet, and broadcast it.
-- The _link request_ packet contains the destination hash _Hd_ , and an asymmetrically encrypted
-    part containing the following data: The source hash _Hs_ , a symmetric key _Lk_ , a truncated
-    hash of a random number _Hr_ , and a signature _S_ of the plaintext values of _Hd_ , _Hs_ , _Lk_ and _Hr_.
-- The broadcasted packet will be directed through the network according to the rules laid out
-    previously.
-- Any node that forwards the link request will store a _link id_ in it’s _link table_ , along with the
-    amount of hops the packet had taken when received. The link id is a hash of the entire link
-    request packet. If the path is not _proven_ within some set amount of time, the entry will be
-    dropped from the table again.
-- When the destination receives the link request packet, it will decide whether to accept the
-    request. If it is accepted, it will create a special packet called a _proof_. A _proof_ is a simple
-    construct, consisting of a truncated hash of the message that needs to be proven, and a
-    signature (made by the destination’s private key) of this hash. This _proof_ effectively verifies
-    that the intended recipient got the packet, and also serves to verify the discovered path
-    through the network. Since the _proof_ hash matches the _path id_ in the intermediary nodes’
-    _path tables_ , the intermediary nodes can forward the proof all the way back to the source.
-- When the source receives the _proof_ , it will know unequivocally that a verified path has been
-    established to the destination, and that information can now be exchanged reliably and
-    securely.
-
-It’s important to note that this methodology ensures that the source of the request does not need to
-reveal any identifying information. Only the intended destination will know “who called”, so to
-speak. This is a huge improvement to protocols like IP, where by design, you have to reveal your
-own address to communicate with anyone, unless you jump through a lot of hoops to hide it.
-Reticulum offers initiator anonymity by design.
-
-
-When using _links_ , Reticulum will automatically verify anything sent over the link, and also
-automates retransmissions if parts of a message was lost along the way. Due to the caching features
-of Reticulum, such a retransmission does not need to travel the entire length of an established path.
-If a packet is lost on the 8th hop of a 12 hop path, it can be fetched from the last hop that received it
-reliably.
-
-## Crossing Continents
-
-When a packet needs to travel farther than local network topology knowledge stretches, a system of
-geographical or topological hinting is used to direct the packet towards a network segment with
-direct knowledge of the intended destination. This functionality is currently left out of the protocol
-for simplicity of testing other parts, but will be activated in a future release. For more information
-on when, refer to the roadmap on the website.
-
-## Resourceful Memory
-
-In traditional networks, large amounts of data is rapidly exchanged with very low latency. Links of
-several thousand kilometers will often only have round-trip latency in the tens of milliseconds, and
-as such, traditional protocols are often designed to not store any transmitted data at intermediary
-hops. If a transmission error occurs, the sending node will simply notice the lack of a packet
-acknowledgement, and retransmit the packet all the way, until it hears back from the receiver that it
-got the intended data.
-
-In bandwidth-limited and high-latency conditions, such behaviour quickly causes congestion on the
-network, and communications that span many hops become exceedingly expensive in terms of
-bandwidth usage, due to the higher risk of some packets failing.
-
-Reticulum alleviates this in part with it’s _path_ discovery methodology, and in part by implementing
-_resource_ caching at all nodes that can support it. Network operation can be made much more
-efficient by caching everything for a period of time, and given the availability of cheap memory and
-storage, this is a very welcome tradeoff. A gigabyte of memory can store millions of Reticulum
-packets, and since everything is encrypted by default, the storing poses very little privacy risk.
-
-In a Reticulum network, any node that is able to do so, should cache as many packets as it’s
-memory will allow for. When a packet is received, a timestamp and a hash of the packet is stored
-along with the full packet itself, and it will be kept in storage until the allocated cache storage is
-full, whereupon the packet that was last accessed in the cache will be deleted. If a packet is accessed
-from the cache, it’s timestamp will be updated to the current time, to ensure that packets that are
-used stay in the cache, and packets that are not used are dropped from memory.
-
-Some packet types are stored in separate caching tables, that allow easier lookup for other nodes.
-For example, an announce is stored in a way, that allows other nodes to request the public key for a
-certain destination, and as such the network as a whole operates as a distributed key ledger.
-
-For more details on how the caching works and is used, see the reference implementation source
-code.
-
-
-# Reference System Setup
-
-This section will detail the recommended _Reference System Setup_ for Reticulum. It is important to
-note that Reticulum is designed to be usable over more or less any medium that allows you to send
-and receive data in a digital form, and satisfies some very low minimum requirements. The
-communication channel must support at least half-duplex operation, and provide an average
-throughput of around 1000 bits per second, and supports a physical layer MTU of 500 bytes. The
-Reticulum software should be able to run on more or less any hardware that can provide a Python
-runtime environment.
-
-That being said, the reference setup has been outlined to provide a common platform for anyone
-who wants to help in the development of Reticulum, and for everyone who wants to know a
-recommended setup to get started. A reference system consists of three parts:
-
-- **A channel access device**
-    Or _CAD_ , in short, provides access to the physical medium whereupon the communication
-    takes place, for example a radio with an integrated modem. A setup with a separate modem
-    connected to a radio would also be termed a “channel access device”.
-- **A host device**
-    Some sort of computing device that can run the necessary software, communicates with the
-    channel access device, and provides user interaction.
-- **A software stack**
-    The software implementing the Reticulum protocol and applications using it.
-
-The reference setup can be considered a relatively stable platform to develop on, and also to start
-building networks on. While details of the implementation might change at the current stage of
-development, it is the goal to maintain hardware compatibility for as long as entirely possible, and
-the current reference setup has been determined to provide a functional platform for many years
-into the future. The current Reference System Setup is as follows:
-
-- **Channel Access Device**
-    A data radio consisting of a LoRa radio module, and a microcontroller with open source
-    firmware, that can connect to host devices via USB. It operates in either the 430, 868 or 900
-    MHz frequency bands. More details on the exact parts and how to get/make one can be
-    found on the website.
-- **Host device**
-    Any computer device running Linux and Python. A Raspberry Pi with Raspbian is
-    recommended.
-- **Software stack**
-    The current Reference Implementation Release of Reticulum, running on a Debian based
-    operating system.
-
-
-It is very important to note, that the reference channel access device **does not** use the LoRaWAN
-standard, but uses a custom MAC layer on top of the plain LoRa modulation! As such, you will
-need a plain LoRa radio module connected to an MCU with the correct Reticulum firmware. Full
-details on how to get or make such a device is available on the website.
-
-With the current reference setup, it should be possible to get on a Reticulum network for around 70$
-even if you have none of the hardware already.
-
-
-# Protocol Specifics
-
-This chapter will detail protocol specific information that is essential to the implementation of
-Reticulum, but non critical in understanding how the protocol works on a general level. It should be
-treated more as a reference than as essential reading.
-
-## Node Types
-
-Currently Reticulum defines two node types, the _Station_ and the _Peer_. A node is a _station_ if it fixed
-in one place, and if it is intended to be kept online at all times. Otherwise the node is a _peer_. This
-distinction is made by the user configuring the node, and is used to determine what nodes on the
-network will help forward traffic, and what nodes rely on other nodes for connectivity.
-
-## Packet Prioritisation
-
-_The packet prioritisation algorithms are subject to rapid change at the moment, and for now, they
-are not documented here. See the reference implementation for more info on how this functionality
-works._
-
-## Path Calculation
-
-_The path calculation algorithms are subject to rapid change at the moment, and for now, they are
-not documented here. See the reference implementation for more info on how this functionality
-works._
-
-## Binary Packet Format
-
-_The binary packet format is subject to rapid change at the moment, and for now, it is not
-documented here. See the reference implementation for the specific details on this topic._
-
-

Examples/Announce.py

@@ -0,0 +1,170 @@
+##########################################################
+# This RNS example demonstrates setting up announce      #
+# callbacks, which will let an application receive a     #
+# notification when an announce relevant for it arrives  #
+##########################################################
+
+import argparse
+import random
+import RNS
+
+# Let's define an app name. We'll use this for all
+# destinations we create. Since this basic example
+# is part of a range of example utilities, we'll put
+# them all within the app namespace "example_utilities"
+APP_NAME = "example_utilities"
+
+# We initialise two lists of strings to use as app_data
+fruits = ["Peach", "Quince", "Date", "Tangerine", "Pomelo", "Carambola", "Grape"]
+noble_gases = ["Helium", "Neon", "Argon", "Krypton", "Xenon", "Radon", "Oganesson"]
+
+# This initialisation is executed when the program is started
+def program_setup(configpath):
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # Randomly create a new identity for our example
+    identity = RNS.Identity()
+
+    # Using the identity we just created, we create two destinations
+    # in the "example_utilities.announcesample" application space.
+    #
+    # Destinations are endpoints in Reticulum, that can be addressed
+    # and communicated with. Destinations can also announce their
+    # existence, which will let the network know they are reachable
+    # and autoomatically create paths to them, from anywhere else
+    # in the network.
+    destination_1 = RNS.Destination(
+        identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "announcesample",
+        "fruits"
+    )
+
+    destination_2 = RNS.Destination(
+        identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "announcesample",
+        "noble_gases"
+    )
+
+    # We configure the destinations to automatically prove all
+    # packets adressed to it. By doing this, RNS will automatically
+    # generate a proof for each incoming packet and transmit it
+    # back to the sender of that packet. This will let anyone that
+    # tries to communicate with the destination know whether their
+    # communication was received correctly.
+    destination_1.set_proof_strategy(RNS.Destination.PROVE_ALL)
+    destination_2.set_proof_strategy(RNS.Destination.PROVE_ALL)
+
+    # We create an announce handler and configure it to only ask for
+    # announces from "example_utilities.announcesample.fruits".
+    # Try changing the filter and see what happens.
+    announce_handler = ExampleAnnounceHandler(
+        aspect_filter="example_utilities.announcesample.fruits"
+    )
+
+    # We register the announce handler with Reticulum
+    RNS.Transport.register_announce_handler(announce_handler)
+    
+    # Everything's ready!
+    # Let's hand over control to the announce loop
+    announceLoop(destination_1, destination_2)
+
+
+def announceLoop(destination_1, destination_2):
+    # Let the user know that everything is ready
+    RNS.log("Announce example running, hit enter to manually send an announce (Ctrl-C to quit)")
+
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will announce our server
+    # destination on the network, which will let clients
+    # know how to create messages directed towards it.
+    while True:
+        entered = input()
+        
+        # Randomly select a fruit
+        fruit = fruits[random.randint(0,len(fruits)-1)]
+
+        # Send the announce including the app data
+        destination_1.announce(app_data=fruit.encode("utf-8"))
+        RNS.log(
+            "Sent announce from "+
+            RNS.prettyhexrep(destination_1.hash)+
+            " ("+destination_1.name+")"
+        )
+
+        # Randomly select a noble gas
+        noble_gas = noble_gases[random.randint(0,len(noble_gases)-1)]
+
+        # Send the announce including the app data
+        destination_2.announce(app_data=noble_gas.encode("utf-8"))
+        RNS.log(
+            "Sent announce from "+
+            RNS.prettyhexrep(destination_2.hash)+
+            " ("+destination_2.name+")"
+        )
+
+# We will need to define an announce handler class that
+# Reticulum can message when an announce arrives.
+class ExampleAnnounceHandler:
+    # The initialisation method takes the optional
+    # aspect_filter argument. If aspect_filter is set to
+    # None, all announces will be passed to the instance.
+    # If only some announces are wanted, it can be set to
+    # an aspect string.
+    def __init__(self, aspect_filter=None):
+        self.aspect_filter = aspect_filter
+
+    # This method will be called by Reticulums Transport
+    # system when an announce arrives that matches the
+    # configured aspect filter. Filters must be specific,
+    # and cannot use wildcards.
+    def received_announce(self, destination_hash, announced_identity, app_data):
+        RNS.log(
+            "Received an announce from "+
+            RNS.prettyhexrep(destination_hash)
+        )
+
+        RNS.log(
+            "The announce contained the following app data: "+
+            app_data.decode("utf-8")
+        )
+
+##########################################################
+#### Program Startup #####################################
+##########################################################
+
+# This part of the program gets run at startup,
+# and parses input from the user, and then starts
+# the desired program mode.
+if __name__ == "__main__":
+    try:
+        parser = argparse.ArgumentParser(
+            description="Reticulum example that demonstrates announces and announce handlers"
+        )
+
+        parser.add_argument(
+            "--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
+
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        program_setup(configarg)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()

Examples/Broadcast.py

@@ -11,56 +11,67 @@ import RNS
 # destinations we create. Since this basic example
 # is part of a range of example utilities, we'll put
 # them all within the app namespace "example_utilities"
-APP_NAME = "example_utilitites"
+APP_NAME = "example_utilities"
 
 # This initialisation is executed when the program is started
 def program_setup(configpath, channel=None):
-	# We must first initialise Reticulum
-	reticulum = RNS.Reticulum(configpath)
-	
-	# If the user did not select a "channel" we use
-	# a default one called "public_information".
-	# This "channel" is added to the destination name-
-	# space, so the user can select different broadcast
-	# channels.
-	if channel == None:
-		channel = "public_information"
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # If the user did not select a "channel" we use
+    # a default one called "public_information".
+    # This "channel" is added to the destination name-
+    # space, so the user can select different broadcast
+    # channels.
+    if channel == None:
+        channel = "public_information"
 
-	# We create a PLAIN destination. This is an uncencrypted endpoint
-	# that anyone can listen to and send information to.
-	broadcast_destination = RNS.Destination(None, RNS.Destination.IN, RNS.Destination.PLAIN, APP_NAME, "broadcast", channel)
+    # We create a PLAIN destination. This is an uncencrypted endpoint
+    # that anyone can listen to and send information to.
+    broadcast_destination = RNS.Destination(
+        None,
+        RNS.Destination.IN,
+        RNS.Destination.PLAIN,
+        APP_NAME,
+        "broadcast",
+        channel
+    )
 
-	# We specify a callback that will get called every time
-	# the destination receives data.
-	broadcast_destination.packet_callback(packet_callback)
-	
-	# Everything's ready!
-	# Let's hand over control to the main loop
-	broadcastLoop(broadcast_destination)
+    # We specify a callback that will get called every time
+    # the destination receives data.
+    broadcast_destination.set_packet_callback(packet_callback)
+    
+    # Everything's ready!
+    # Let's hand over control to the main loop
+    broadcastLoop(broadcast_destination)
 
 def packet_callback(data, packet):
-	# Simply print out the received data
-	print("")
-	print("Received data: "+data.decode("utf-8")+"\r\n> ", end="")
-	sys.stdout.flush()
+    # Simply print out the received data
+    print("")
+    print("Received data: "+data.decode("utf-8")+"\r\n> ", end="")
+    sys.stdout.flush()
 
 def broadcastLoop(destination):
-	# Let the user know that everything is ready
-	RNS.log("Broadcast example "+RNS.prettyhexrep(destination.hash)+" running, enter text and hit enter to broadcast (Ctrl-C to quit)")
+    # Let the user know that everything is ready
+    RNS.log(
+        "Broadcast example "+
+        RNS.prettyhexrep(destination.hash)+
+        " running, enter text and hit enter to broadcast (Ctrl-C to quit)"
+    )
 
-	# We enter a loop that runs until the users exits.
-	# If the user hits enter, we will send the information
-	# that the user entered into the prompt.
-	while True:
-		print("> ", end="")
-		entered = input()
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will send the information
+    # that the user entered into the prompt.
+    while True:
+        print("> ", end="")
+        entered = input()
+
+        if entered != "":
+            data    = entered.encode("utf-8")
+            packet  = RNS.Packet(destination, data)
+            packet.send()
 
-		if entered != "":
-			data    = entered.encode("utf-8")
-			packet  = RNS.Packet(destination, data)
-			packet.send()
 
-		
 
 ##########################################################
 #### Program Startup #####################################
@@ -70,24 +81,41 @@ def broadcastLoop(destination):
 # and parses input from the user, and then starts
 # the program.
 if __name__ == "__main__":
-	try:
-		parser = argparse.ArgumentParser(description="Reticulum example that demonstrates sending and receiving unencrypted broadcasts")
-		parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
-		parser.add_argument("--channel", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
-		args = parser.parse_args()
+    try:
+        parser = argparse.ArgumentParser(
+            description="Reticulum example demonstrating sending and receiving broadcasts"
+        )
 
-		if args.config:
-			configarg = args.config
-		else:
-			configarg = None
+        parser.add_argument(
+            "--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
 
-		if args.channel:
-			channelarg = args.channel
-		else:
-			channelarg = None
+        parser.add_argument(
+            "--channel",
+            action="store",
+            default=None,
+            help="broadcast channel name",
+            type=str
+        )
 
-		program_setup(configarg, channelarg)
+        args = parser.parse_args()
 
-	except KeyboardInterrupt:
-		print("")
-		exit()
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        if args.channel:
+            channelarg = args.channel
+        else:
+            channelarg = None
+
+        program_setup(configarg, channelarg)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()

Examples/Echo.py

@@ -12,7 +12,7 @@ import RNS
 # destinations we create. Since this echo example
 # is part of a range of example utilities, we'll put
 # them all within the app namespace "example_utilities"
-APP_NAME = "example_utilitites"
+APP_NAME = "example_utilities"
 
 
 ##########################################################
@@ -22,56 +22,90 @@ APP_NAME = "example_utilitites"
 # This initialisation is executed when the users chooses
 # to run as a server
 def server(configpath):
-	# We must first initialise Reticulum
-	reticulum = RNS.Reticulum(configpath)
-	
-	# Randomly create a new identity for our echo server
-	server_identity = RNS.Identity()
+    global reticulum
 
-	# We create a destination that clients can query. We want
-	# to be able to verify echo replies to our clients, so we
-	# create a "single" destination that can receive encrypted
-	# messages. This way the client can send a request and be
-	# certain that no-one else than this destination was able
-	# to read it. 
-	echo_destination = RNS.Destination(server_identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, "echo", "request")
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # Randomly create a new identity for our echo server
+    server_identity = RNS.Identity()
 
-	# We configure the destination to automatically prove all
-	# packets adressed to it. By doing this, RNS will automatically
-	# generate a proof for each incoming packet and transmit it
-	# back to the sender of that packet.
-	echo_destination.set_proof_strategy(RNS.Destination.PROVE_ALL)
-	
-	# Tell the destination which function in our program to
-	# run when a packet is received. We do this so we can
-	# print a log message when the server receives a request
-	echo_destination.packet_callback(server_callback)
+    # We create a destination that clients can query. We want
+    # to be able to verify echo replies to our clients, so we
+    # create a "single" destination that can receive encrypted
+    # messages. This way the client can send a request and be
+    # certain that no-one else than this destination was able
+    # to read it. 
+    echo_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "echo",
+        "request"
+    )
 
-	# Everything's ready!
-	# Let's Wait for client requests or user input
-	announceLoop(echo_destination)
+    # We configure the destination to automatically prove all
+    # packets adressed to it. By doing this, RNS will automatically
+    # generate a proof for each incoming packet and transmit it
+    # back to the sender of that packet.
+    echo_destination.set_proof_strategy(RNS.Destination.PROVE_ALL)
+    
+    # Tell the destination which function in our program to
+    # run when a packet is received. We do this so we can
+    # print a log message when the server receives a request
+    echo_destination.set_packet_callback(server_callback)
+
+    # Everything's ready!
+    # Let's Wait for client requests or user input
+    announceLoop(echo_destination)
 
 
 def announceLoop(destination):
-	# Let the user know that everything is ready
-	RNS.log("Echo server "+RNS.prettyhexrep(destination.hash)+" running, hit enter to manually send an announce (Ctrl-C to quit)")
+    # Let the user know that everything is ready
+    RNS.log(
+        "Echo server "+
+        RNS.prettyhexrep(destination.hash)+
+        " running, hit enter to manually send an announce (Ctrl-C to quit)"
+    )
 
-	# We enter a loop that runs until the users exits.
-	# If the user hits enter, we will announce our server
-	# destination on the network, which will let clients
-	# know how to create messages directed towards it.
-	while True:
-		entered = input()
-		destination.announce()
-		RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will announce our server
+    # destination on the network, which will let clients
+    # know how to create messages directed towards it.
+    while True:
+        entered = input()
+        destination.announce()
+        RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
 
 
 def server_callback(message, packet):
-	# Tell the user that we received an echo request, and
-	# that we are going to send a reply to the requester.
-	# Sending the proof is handled automatically, since we
-	# set up the destination to prove all incoming packets.
-	RNS.log("Received packet from echo client, proof sent")
+    global reticulum
+    
+    # Tell the user that we received an echo request, and
+    # that we are going to send a reply to the requester.
+    # Sending the proof is handled automatically, since we
+    # set up the destination to prove all incoming packets.
+
+    reception_stats = ""
+    if reticulum.is_connected_to_shared_instance:
+        reception_rssi = reticulum.get_packet_rssi(packet.packet_hash)
+        reception_snr  = reticulum.get_packet_snr(packet.packet_hash)
+
+        if reception_rssi != None:
+            reception_stats += " [RSSI "+str(reception_rssi)+" dBm]"
+        
+        if reception_snr != None:
+            reception_stats += " [SNR "+str(reception_snr)+" dBm]"
+
+    else:
+        if packet.rssi != None:
+            reception_stats += " [RSSI "+str(packet.rssi)+" dBm]"
+        
+        if packet.snr != None:
+            reception_stats += " [SNR "+str(packet.snr)+" dB]"
+
+    RNS.log("Received packet from echo client, proof sent"+reception_stats)
 
 
 ##########################################################
@@ -81,103 +115,147 @@ def server_callback(message, packet):
 # This initialisation is executed when the users chooses
 # to run as a client
 def client(destination_hexhash, configpath, timeout=None):
-	# We need a binary representation of the destination
-	# hash that was entered on the command line
-	try:
-		if len(destination_hexhash) != 20:
-			raise ValueError("Destination length is invalid, must be 20 hexadecimal characters (10 bytes)")
-		destination_hash = bytes.fromhex(destination_hexhash)
-	except:
-		RNS.log("Invalid destination entered. Check your input!\n")
-		exit()
+    global reticulum
+    
+    # We need a binary representation of the destination
+    # hash that was entered on the command line
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination_hexhash) != dest_len:
+            raise ValueError(
+                "Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2)
+            )
 
-	# We must first initialise Reticulum
-	reticulum = RNS.Reticulum(configpath)
+        destination_hash = bytes.fromhex(destination_hexhash)
+    except Exception as e:
+        RNS.log("Invalid destination entered. Check your input!")
+        RNS.log(str(e)+"\n")
+        exit()
 
-	# We override the loglevel to provide feedback when
-	# an announce is received
-	if RNS.loglevel < RNS.LOG_INFO:
-		RNS.loglevel = RNS.LOG_INFO
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
 
-	# Tell the user that the client is ready!
-	RNS.log("Echo client ready, hit enter to send echo request to "+destination_hexhash+" (Ctrl-C to quit)")
+    # We override the loglevel to provide feedback when
+    # an announce is received
+    if RNS.loglevel < RNS.LOG_INFO:
+        RNS.loglevel = RNS.LOG_INFO
 
-	# We enter a loop that runs until the user exits.
-	# If the user hits enter, we will try to send an
-	# echo request to the destination specified on the
-	# command line.
-	while True:
-		input()
-		
-		# Let's first check if RNS knows a path to the destination.
-		# If it does, we'll load the server identity and create a packet
-		if RNS.Transport.hasPath(destination_hash):
+    # Tell the user that the client is ready!
+    RNS.log(
+        "Echo client ready, hit enter to send echo request to "+
+        destination_hexhash+
+        " (Ctrl-C to quit)"
+    )
 
-			# To address the server, we need to know it's public
-			# key, so we check if Reticulum knows this destination.
-			# This is done by calling the "recall" method of the
-			# Identity module. If the destination is known, it will
-			# return an Identity instance that can be used in
-			# outgoing destinations.
-			server_identity = RNS.Identity.recall(destination_hash)
+    # We enter a loop that runs until the user exits.
+    # If the user hits enter, we will try to send an
+    # echo request to the destination specified on the
+    # command line.
+    while True:
+        input()
+        
+        # Let's first check if RNS knows a path to the destination.
+        # If it does, we'll load the server identity and create a packet
+        if RNS.Transport.has_path(destination_hash):
 
-			# We got the correct identity instance from the
-			# recall method, so let's create an outgoing
-			# destination. We use the naming convention:
-			# example_utilities.echo.request
-			# This matches the naming we specified in the
-			# server part of the code.
-			request_destination = RNS.Destination(server_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "echo", "request")
+            # To address the server, we need to know it's public
+            # key, so we check if Reticulum knows this destination.
+            # This is done by calling the "recall" method of the
+            # Identity module. If the destination is known, it will
+            # return an Identity instance that can be used in
+            # outgoing destinations.
+            server_identity = RNS.Identity.recall(destination_hash)
 
-			# The destination is ready, so let's create a packet.
-			# We set the destination to the request_destination
-			# that was just created, and the only data we add
-			# is a random hash.
-			echo_request = RNS.Packet(request_destination, RNS.Identity.getRandomHash())
+            # We got the correct identity instance from the
+            # recall method, so let's create an outgoing
+            # destination. We use the naming convention:
+            # example_utilities.echo.request
+            # This matches the naming we specified in the
+            # server part of the code.
+            request_destination = RNS.Destination(
+                server_identity,
+                RNS.Destination.OUT,
+                RNS.Destination.SINGLE,
+                APP_NAME,
+                "echo",
+                "request"
+            )
 
-			# Send the packet! If the packet is successfully
-			# sent, it will return a PacketReceipt instance.
-			packet_receipt = echo_request.send()
+            # The destination is ready, so let's create a packet.
+            # We set the destination to the request_destination
+            # that was just created, and the only data we add
+            # is a random hash.
+            echo_request = RNS.Packet(request_destination, RNS.Identity.get_random_hash())
 
-			# If the user specified a timeout, we set this
-			# timeout on the packet receipt, and configure
-			# a callback function, that will get called if
-			# the packet times out.
-			if timeout != None:
-				packet_receipt.set_timeout(timeout)
-				packet_receipt.timeout_callback(packet_timed_out)
+            # Send the packet! If the packet is successfully
+            # sent, it will return a PacketReceipt instance.
+            packet_receipt = echo_request.send()
 
-			# We can then set a delivery callback on the receipt.
-			# This will get automatically called when a proof for
-			# this specific packet is received from the destination.
-			packet_receipt.delivery_callback(packet_delivered)
+            # If the user specified a timeout, we set this
+            # timeout on the packet receipt, and configure
+            # a callback function, that will get called if
+            # the packet times out.
+            if timeout != None:
+                packet_receipt.set_timeout(timeout)
+                packet_receipt.set_timeout_callback(packet_timed_out)
 
-			# Tell the user that the echo request was sent
-			RNS.log("Sent echo request to "+RNS.prettyhexrep(request_destination.hash))
-		else:
-			# If we do not know this destination, tell the
-			# user to wait for an announce to arrive.
-			RNS.log("Destination is not yet known. Requesting path...")
-			RNS.Transport.requestPath(destination_hash)
+            # We can then set a delivery callback on the receipt.
+            # This will get automatically called when a proof for
+            # this specific packet is received from the destination.
+            packet_receipt.set_delivery_callback(packet_delivered)
+
+            # Tell the user that the echo request was sent
+            RNS.log("Sent echo request to "+RNS.prettyhexrep(request_destination.hash))
+        else:
+            # If we do not know this destination, tell the
+            # user to wait for an announce to arrive.
+            RNS.log("Destination is not yet known. Requesting path...")
+            RNS.Transport.request_path(destination_hash)
 
 # This function is called when our reply destination
 # receives a proof packet.
 def packet_delivered(receipt):
-	if receipt.status == RNS.PacketReceipt.DELIVERED:
-		rtt = receipt.rtt()
-		if (rtt >= 1):
-			rtt = round(rtt, 3)
-			rttstring = str(rtt)+" seconds"
-		else:
-			rtt = round(rtt*1000, 3)
-			rttstring = str(rtt)+" milliseconds"
+    global reticulum
 
-		RNS.log("Valid reply received from "+RNS.prettyhexrep(receipt.destination.hash)+", round-trip time is "+rttstring)
+    if receipt.status == RNS.PacketReceipt.DELIVERED:
+        rtt = receipt.get_rtt()
+        if (rtt >= 1):
+            rtt = round(rtt, 3)
+            rttstring = str(rtt)+" seconds"
+        else:
+            rtt = round(rtt*1000, 3)
+            rttstring = str(rtt)+" milliseconds"
+
+        reception_stats = ""
+        if reticulum.is_connected_to_shared_instance:
+            reception_rssi = reticulum.get_packet_rssi(receipt.proof_packet.packet_hash)
+            reception_snr  = reticulum.get_packet_snr(receipt.proof_packet.packet_hash)
+
+            if reception_rssi != None:
+                reception_stats += " [RSSI "+str(reception_rssi)+" dBm]"
+            
+            if reception_snr != None:
+                reception_stats += " [SNR "+str(reception_snr)+" dB]"
+
+        else:
+            if receipt.proof_packet != None:
+                if receipt.proof_packet.rssi != None:
+                    reception_stats += " [RSSI "+str(receipt.proof_packet.rssi)+" dBm]"
+                
+                if receipt.proof_packet.snr != None:
+                    reception_stats += " [SNR "+str(receipt.proof_packet.snr)+" dB]"
+
+        RNS.log(
+            "Valid reply received from "+
+            RNS.prettyhexrep(receipt.destination.hash)+
+            ", round-trip time is "+rttstring+
+            reception_stats
+        )
 
 # This function is called if a packet times out.
 def packet_timed_out(receipt):
-	if receipt.status == RNS.PacketReceipt.FAILED:
-		RNS.log("Packet "+RNS.prettyhexrep(receipt.hash)+" timed out")
+    if receipt.status == RNS.PacketReceipt.FAILED:
+        RNS.log("Packet "+RNS.prettyhexrep(receipt.hash)+" timed out")
 
 
 ##########################################################
@@ -188,36 +266,65 @@ def packet_timed_out(receipt):
 # and parses input from the user, and then starts
 # the desired program mode.
 if __name__ == "__main__":
-	try:
-		parser = argparse.ArgumentParser(description="Simple echo server and client utility")
-		parser.add_argument("-s", "--server", action="store_true", help="wait for incoming packets from clients")
-		parser.add_argument("-t", "--timeout", action="store", metavar="s", default=None, help="set a reply timeout in seconds", type=float)
-		parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
-		parser.add_argument("destination", nargs="?", default=None, help="hexadecimal hash of the server destination", type=str)
-		args = parser.parse_args()
+    try:
+        parser = argparse.ArgumentParser(description="Simple echo server and client utility")
 
-		if args.server:
-			configarg=None
-			if args.config:
-				configarg = args.config
-			server(configarg)
-		else:
-			if args.config:
-				configarg = args.config
-			else:
-				configarg = None
+        parser.add_argument(
+            "-s",
+            "--server",
+            action="store_true",
+            help="wait for incoming packets from clients"
+        )
 
-			if args.timeout:
-				timeoutarg = float(args.timeout)
-			else:
-				timeoutarg = None
+        parser.add_argument(
+            "-t",
+            "--timeout",
+            action="store",
+            metavar="s",
+            default=None,
+            help="set a reply timeout in seconds",
+            type=float
+        )
 
-			if (args.destination == None):
-				print("")
-				parser.print_help()
-				print("")
-			else:
-				client(args.destination, configarg, timeout=timeoutarg)
-	except KeyboardInterrupt:
-		print("")
-		exit()
+        parser.add_argument("--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
+
+        parser.add_argument(
+            "destination",
+            nargs="?",
+            default=None,
+            help="hexadecimal hash of the server destination",
+            type=str
+        )
+
+        args = parser.parse_args()
+
+        if args.server:
+            configarg=None
+            if args.config:
+                configarg = args.config
+            server(configarg)
+        else:
+            if args.config:
+                configarg = args.config
+            else:
+                configarg = None
+
+            if args.timeout:
+                timeoutarg = float(args.timeout)
+            else:
+                timeoutarg = None
+
+            if (args.destination == None):
+                print("")
+                parser.print_help()
+                print("")
+            else:
+                client(args.destination, configarg, timeout=timeoutarg)
+    except KeyboardInterrupt:
+        print("")
+        exit()

Examples/Filetransfer.py

@@ -3,6 +3,17 @@
 # server and client program. The server will serve a     #
 # directory of files, and the clients can list and       #
 # download files from the server.                        #
+#                                                        #
+# Please note that using RNS Resources for large file    #
+# transfers is not recommended, since compression,       #
+# encryption and hashmap sequencing can take a long time #
+# on systems with slow CPUs, which will probably result  #
+# in the client timing out before the resource sender    #
+# can complete preparing the resource.                   #
+#                                                        #
+# If you need to transfer large files, use the Bundle    #
+# class instead, which will automatically slice the data #
+# into chunks suitable for packing as a Resource.        #
 ##########################################################
 
 import os
@@ -17,10 +28,10 @@ import RNS.vendor.umsgpack as umsgpack
 # destinations we create. Since this echo example
 # is part of a range of example utilities, we'll put
 # them all within the app namespace "example_utilities"
-APP_NAME = "example_utilitites"
+APP_NAME = "example_utilities"
 
 # We'll also define a default timeout, in seconds
-APP_TIMEOUT = 15.0
+APP_TIMEOUT = 45.0
 
 ##########################################################
 #### Server Part #########################################
@@ -31,260 +42,297 @@ serve_path = None
 # This initialisation is executed when the users chooses
 # to run as a server
 def server(configpath, path):
-	# We must first initialise Reticulum
-	reticulum = RNS.Reticulum(configpath)
-	
-	# Randomly create a new identity for our file server
-	server_identity = RNS.Identity()
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # Randomly create a new identity for our file server
+    server_identity = RNS.Identity()
 
-	global serve_path
-	serve_path = path
+    global serve_path
+    serve_path = path
 
-	# We create a destination that clients can connect to. We
-	# want clients to create links to this destination, so we
-	# need to create a "single" destination type.
-	server_destination = RNS.Destination(server_identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, "filetransfer", "server")
+    # We create a destination that clients can connect to. We
+    # want clients to create links to this destination, so we
+    # need to create a "single" destination type.
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "filetransfer",
+        "server"
+    )
 
-	# We configure a function that will get called every time
-	# a new client creates a link to this destination.
-	server_destination.link_established_callback(client_connected)
+    # We configure a function that will get called every time
+    # a new client creates a link to this destination.
+    server_destination.set_link_established_callback(client_connected)
 
-	# Everything's ready!
-	# Let's Wait for client requests or user input
-	announceLoop(server_destination)
+    # Everything's ready!
+    # Let's Wait for client requests or user input
+    announceLoop(server_destination)
 
 def announceLoop(destination):
-	# Let the user know that everything is ready
-	RNS.log("File server "+RNS.prettyhexrep(destination.hash)+" running")
-	RNS.log("Hit enter to manually send an announce (Ctrl-C to quit)")
+    # Let the user know that everything is ready
+    RNS.log("File server "+RNS.prettyhexrep(destination.hash)+" running")
+    RNS.log("Hit enter to manually send an announce (Ctrl-C to quit)")
 
-	# We enter a loop that runs until the users exits.
-	# If the user hits enter, we will announce our server
-	# destination on the network, which will let clients
-	# know how to create messages directed towards it.
-	while True:
-		entered = input()
-		destination.announce()
-		RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will announce our server
+    # destination on the network, which will let clients
+    # know how to create messages directed towards it.
+    while True:
+        entered = input()
+        destination.announce()
+        RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
 
 # Here's a convenience function for listing all files
 # in our served directory
 def list_files():
-	# We add all entries from the directory that are
-	# actual files, and does not start with "."
-	global serve_path
-	return [file for file in os.listdir(serve_path) if os.path.isfile(os.path.join(serve_path, file)) and file[:1] != "."]
+    # We add all entries from the directory that are
+    # actual files, and does not start with "."
+    global serve_path
+    return [file for file in os.listdir(serve_path) if os.path.isfile(os.path.join(serve_path, file)) and file[:1] != "."]
 
 # When a client establishes a link to our server
 # destination, this function will be called with
 # a reference to the link. We then send the client
 # a list of files hosted on the server.
 def client_connected(link):
-	# Check if the served directory still exists
-	if os.path.isdir(serve_path):
-		RNS.log("Client connected, sending file list...")
+    # Check if the served directory still exists
+    if os.path.isdir(serve_path):
+        RNS.log("Client connected, sending file list...")
 
-		link.link_closed_callback(client_disconnected)
+        link.set_link_closed_callback(client_disconnected)
 
-		# We pack a list of files for sending in a packet
-		data = umsgpack.packb(list_files())
+        # We pack a list of files for sending in a packet
+        data = umsgpack.packb(list_files())
 
-		# Check the size of the packed data
-		if len(data) <= RNS.Link.MDU:
-			# If it fits in one packet, we will just
-			# send it as a single packet over the link.
-			list_packet = RNS.Packet(link, data)
-			list_receipt = list_packet.send()
-			list_receipt.set_timeout(APP_TIMEOUT)
-			list_receipt.delivery_callback(list_delivered)
-			list_receipt.timeout_callback(list_timeout)
-		else:
-			RNS.log("Too many files in served directory!", RNS.LOG_ERROR)
-			RNS.log("You should implement a function to split the filelist over multiple packets.", RNS.LOG_ERROR)
-			RNS.log("Hint: The client already supports it :)", RNS.LOG_ERROR)
-			
-		# After this, we're just going to keep the link
-		# open until the client requests a file. We'll
-		# configure a function that get's called when
-		# the client sends a packet with a file request.
-		link.packet_callback(client_request)
-	else:
-		RNS.log("Client connected, but served path no longer exists!", RNS.LOG_ERROR)
-		link.teardown()
+        # Check the size of the packed data
+        if len(data) <= RNS.Link.MDU:
+            # If it fits in one packet, we will just
+            # send it as a single packet over the link.
+            list_packet = RNS.Packet(link, data)
+            list_receipt = list_packet.send()
+            list_receipt.set_timeout(APP_TIMEOUT)
+            list_receipt.set_delivery_callback(list_delivered)
+            list_receipt.set_timeout_callback(list_timeout)
+        else:
+            RNS.log("Too many files in served directory!", RNS.LOG_ERROR)
+            RNS.log("You should implement a function to split the filelist over multiple packets.", RNS.LOG_ERROR)
+            RNS.log("Hint: The client already supports it :)", RNS.LOG_ERROR)
+            
+        # After this, we're just going to keep the link
+        # open until the client requests a file. We'll
+        # configure a function that get's called when
+        # the client sends a packet with a file request.
+        link.set_packet_callback(client_request)
+    else:
+        RNS.log("Client connected, but served path no longer exists!", RNS.LOG_ERROR)
+        link.teardown()
 
 def client_disconnected(link):
-	RNS.log("Client disconnected")
+    RNS.log("Client disconnected")
 
 def client_request(message, packet):
-	global serve_path
-	filename = message.decode("utf-8")
-	if filename in list_files():
-		try:
-			# If we have the requested file, we'll
-			# read it and pack it as a resource
-			RNS.log("Client requested \""+filename+"\"")
-			file = open(os.path.join(serve_path, filename), "rb")
-			file_data = file.read()
-			file.close()
+    global serve_path
 
-			file_resource = RNS.Resource(file_data, packet.link, callback=resource_sending_concluded)
-			file_resource.filename = filename
-		except:
-			# If somethign went wrong, we close
-			# the link
-			RNS.log("Error while reading file \""+filename+"\"", RNS.LOG_ERROR)
-			packet.link.teardown()
-	else:
-		# If we don't have it, we close the link
-		RNS.log("Client requested an unknown file")
-		packet.link.teardown()
+    try:
+        filename = message.decode("utf-8")
+    except Exception as e:
+        filename = None
+
+    if filename in list_files():
+        try:
+            # If we have the requested file, we'll
+            # read it and pack it as a resource
+            RNS.log("Client requested \""+filename+"\"")
+            file = open(os.path.join(serve_path, filename), "rb")
+            
+            file_resource = RNS.Resource(
+                file,
+                packet.link,
+                callback=resource_sending_concluded
+            )
+
+            file_resource.filename = filename
+        except Exception as e:
+            # If somethign went wrong, we close
+            # the link
+            RNS.log("Error while reading file \""+filename+"\"", RNS.LOG_ERROR)
+            packet.link.teardown()
+            raise e
+    else:
+        # If we don't have it, we close the link
+        RNS.log("Client requested an unknown file")
+        packet.link.teardown()
 
 # This function is called on the server when a
 # resource transfer concludes.
 def resource_sending_concluded(resource):
-	if hasattr(resource, "filename"):
-		name = resource.filename
-	else:
-		name = "resource"
+    if hasattr(resource, "filename"):
+        name = resource.filename
+    else:
+        name = "resource"
 
-	if resource.status == RNS.Resource.COMPLETE:
-		RNS.log("Done sending \""+name+"\" to client")
-	elif resource.status == RNS.Resource.FAILED:
-		RNS.log("Sending \""+name+"\" to client failed")
+    if resource.status == RNS.Resource.COMPLETE:
+        RNS.log("Done sending \""+name+"\" to client")
+    elif resource.status == RNS.Resource.FAILED:
+        RNS.log("Sending \""+name+"\" to client failed")
 
 def list_delivered(receipt):
-	RNS.log("The file list was received by the client")
+    RNS.log("The file list was received by the client")
 
 def list_timeout(receipt):
-	RNS.log("Sending list to client timed out, closing this link")
-	link = receipt.destination
-	link.teardown()
+    RNS.log("Sending list to client timed out, closing this link")
+    link = receipt.destination
+    link.teardown()
 
 ##########################################################
 #### Client Part #########################################
 ##########################################################
 
 # We store a global list of files available on the server
-server_files = []
+server_files      = []
 
 # A reference to the server link
-server_link = None
+server_link       = None
 
 # And a reference to the current download
-current_download = None
-current_filename = None
+current_download  = None
+current_filename  = None
+
+# Variables to store download statistics
+download_started  = 0
+download_finished = 0
+download_time     = 0
+transfer_size     = 0
+file_size         = 0
+
 
 # This initialisation is executed when the users chooses
 # to run as a client
 def client(destination_hexhash, configpath):
-	# We need a binary representation of the destination
-	# hash that was entered on the command line
-	try:
-		if len(destination_hexhash) != 20:
-			raise ValueError("Destination length is invalid, must be 20 hexadecimal characters (10 bytes)")
-		destination_hash = bytes.fromhex(destination_hexhash)
-	except:
-		RNS.log("Invalid destination entered. Check your input!\n")
-		exit()
+    # We need a binary representation of the destination
+    # hash that was entered on the command line
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination_hexhash) != dest_len:
+            raise ValueError(
+                "Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2)
+            )
+            
+        destination_hash = bytes.fromhex(destination_hexhash)
+    except:
+        RNS.log("Invalid destination entered. Check your input!\n")
+        exit()
 
-	# We must first initialise Reticulum
-	reticulum = RNS.Reticulum(configpath)
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
 
 
-	# Check if we know a path to the destination
-	if not RNS.Transport.hasPath(destination_hash):
-		RNS.log("Destination is not yet known. Requesting path and waiting for announce to arrive...")
-		RNS.Transport.requestPath(destination_hash)
-		while not RNS.Transport.hasPath(destination_hash):
-			time.sleep(0.1)
+    # Check if we know a path to the destination
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.log("Destination is not yet known. Requesting path and waiting for announce to arrive...")
+        RNS.Transport.request_path(destination_hash)
+        while not RNS.Transport.has_path(destination_hash):
+            time.sleep(0.1)
 
-	# Recall the server identity
-	server_identity = RNS.Identity.recall(destination_hash)
+    # Recall the server identity
+    server_identity = RNS.Identity.recall(destination_hash)
 
-	# Inform the user that we'll begin connecting
-	RNS.log("Establishing link with server...")
+    # Inform the user that we'll begin connecting
+    RNS.log("Establishing link with server...")
 
-	# When the server identity is known, we set
-	# up a destination
-	server_destination = RNS.Destination(server_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "filetransfer", "server")
+    # When the server identity is known, we set
+    # up a destination
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.OUT,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "filetransfer",
+        "server"
+    )
 
-	# We also want to automatically prove incoming packets
-	server_destination.set_proof_strategy(RNS.Destination.PROVE_ALL)
+    # We also want to automatically prove incoming packets
+    server_destination.set_proof_strategy(RNS.Destination.PROVE_ALL)
 
-	# And create a link
-	link = RNS.Link(server_destination)
+    # And create a link
+    link = RNS.Link(server_destination)
 
-	# We expect any normal data packets on the link
-	# to contain a list of served files, so we set
-	# a callback accordingly
-	link.packet_callback(filelist_received)
+    # We expect any normal data packets on the link
+    # to contain a list of served files, so we set
+    # a callback accordingly
+    link.set_packet_callback(filelist_received)
 
-	# We'll also set up functions to inform the
-	# user when the link is established or closed
-	link.link_established_callback(link_established)
-	link.link_closed_callback(link_closed)
+    # We'll also set up functions to inform the
+    # user when the link is established or closed
+    link.set_link_established_callback(link_established)
+    link.set_link_closed_callback(link_closed)
 
-	# And set the link to automatically begin
-	# downloading advertised resources
-	link.set_resource_strategy(RNS.Link.ACCEPT_ALL)
-	link.resource_started_callback(download_began)
-	link.resource_concluded_callback(download_concluded)
+    # And set the link to automatically begin
+    # downloading advertised resources
+    link.set_resource_strategy(RNS.Link.ACCEPT_ALL)
+    link.set_resource_started_callback(download_began)
+    link.set_resource_concluded_callback(download_concluded)
 
-	menu()
+    menu()
 
 # Requests the specified file from the server
 def download(filename):
-	global server_link, menu_mode, current_filename
-	current_filename = filename
+    global server_link, menu_mode, current_filename, transfer_size, download_started
+    current_filename = filename
+    download_started = 0
+    transfer_size    = 0
 
-	# We just create a packet containing the
-	# requested filename, and send it down the
-	# link. We also specify we don't need a
-	# packet receipt.
-	request_packet = RNS.Packet(server_link, filename.encode("utf-8"), create_receipt=False)
-	request_packet.send()
-	
-	print("")
-	print(("Requested \""+filename+"\" from server, waiting for download to begin..."))
-	menu_mode = "download_started"
+    # We just create a packet containing the
+    # requested filename, and send it down the
+    # link. We also specify we don't need a
+    # packet receipt.
+    request_packet = RNS.Packet(server_link, filename.encode("utf-8"), create_receipt=False)
+    request_packet.send()
+    
+    print("")
+    print(("Requested \""+filename+"\" from server, waiting for download to begin..."))
+    menu_mode = "download_started"
 
 # This function runs a simple menu for the user
 # to select which files to download, or quit
 menu_mode = None
 def menu():
-	global server_files, server_link
-	# Wait until we have a filelist
-	while len(server_files) == 0:
-		time.sleep(0.1)
-	RNS.log("Ready!")
-	time.sleep(0.5)
+    global server_files, server_link
+    # Wait until we have a filelist
+    while len(server_files) == 0:
+        time.sleep(0.1)
+    RNS.log("Ready!")
+    time.sleep(0.5)
 
-	global menu_mode
-	menu_mode = "main"
-	should_quit = False
-	while (not should_quit):
-		print_menu()
+    global menu_mode
+    menu_mode = "main"
+    should_quit = False
+    while (not should_quit):
+        print_menu()
 
-		while not menu_mode == "main":
-			# Wait
-			time.sleep(0.25)
+        while not menu_mode == "main":
+            # Wait
+            time.sleep(0.25)
 
-		user_input = input()
-		if user_input == "q" or user_input == "quit" or user_input == "exit":
-			should_quit = True
-			print("")
-		else:
-			if user_input in server_files:
-				download(user_input)
-			else:
-				try:
-					if 0 <= int(user_input) < len(server_files):
-						download(server_files[int(user_input)])
-				except:
-					pass
+        user_input = input()
+        if user_input == "q" or user_input == "quit" or user_input == "exit":
+            should_quit = True
+            print("")
+        else:
+            if user_input in server_files:
+                download(user_input)
+            else:
+                try:
+                    if 0 <= int(user_input) < len(server_files):
+                        download(server_files[int(user_input)])
+                except:
+                    pass
 
-	if should_quit:
-		server_link.teardown()
+    if should_quit:
+        server_link.teardown()
 
 # Prints out menus or screens for the
 # various states of the client program.
@@ -292,164 +340,204 @@ def menu():
 # I won't go into detail here. Just
 # strings basically.
 def print_menu():
-	global menu_mode
+    global menu_mode, download_time, download_started, download_finished, transfer_size, file_size
 
-	if menu_mode == "main":
-		clear_screen()
-		print_filelist()
-		print("")
-		print("Select a file to download by entering name or number, or q to quit")
-		print(("> "), end=' ')
-	elif menu_mode == "download_started":
-		download_began = time.time()
-		while menu_mode == "download_started":
-			time.sleep(0.1)
-			if time.time() > download_began+APP_TIMEOUT:
-				print("The download timed out")
-				time.sleep(1)
-				server_link.teardown()
+    if menu_mode == "main":
+        clear_screen()
+        print_filelist()
+        print("")
+        print("Select a file to download by entering name or number, or q to quit")
+        print(("> "), end=' ')
+    elif menu_mode == "download_started":
+        download_began = time.time()
+        while menu_mode == "download_started":
+            time.sleep(0.1)
+            if time.time() > download_began+APP_TIMEOUT:
+                print("The download timed out")
+                time.sleep(1)
+                server_link.teardown()
 
-	if menu_mode == "downloading":
-		print("Download started")
-		print("")
-		while menu_mode == "downloading":
-			global current_download
-			percent = round(current_download.progress() * 100.0, 1)
-			print(("\rProgress: "+str(percent)+" %   "), end=' ')
-			sys.stdout.flush()
-			time.sleep(0.1)
+    if menu_mode == "downloading":
+        print("Download started")
+        print("")
+        while menu_mode == "downloading":
+            global current_download
+            percent = round(current_download.get_progress() * 100.0, 1)
+            print(("\rProgress: "+str(percent)+" %   "), end=' ')
+            sys.stdout.flush()
+            time.sleep(0.1)
 
-	if menu_mode == "save_error":
-		print(("\rProgress: 100.0 %"), end=' ')
-		sys.stdout.flush()
-		print("")
-		print("Could not write downloaded file to disk")
-		current_download.status = RNS.Resource.FAILED
-		menu_mode = "download_concluded"
+    if menu_mode == "save_error":
+        print(("\rProgress: 100.0 %"), end=' ')
+        sys.stdout.flush()
+        print("")
+        print("Could not write downloaded file to disk")
+        current_download.status = RNS.Resource.FAILED
+        menu_mode = "download_concluded"
 
-	if menu_mode == "download_concluded":
-		if current_download.status == RNS.Resource.COMPLETE:
-			print(("\rProgress: 100.0 %"), end=' ')
-			sys.stdout.flush()
-			print("")
-			print("The download completed! Press enter to return to the menu.")
-			input()
+    if menu_mode == "download_concluded":
+        if current_download.status == RNS.Resource.COMPLETE:
+            print(("\rProgress: 100.0 %"), end=' ')
+            sys.stdout.flush()
 
-		else:
-			print("")
-			print("The download failed! Press enter to return to the menu.")
-			input()
+            # Print statistics
+            hours, rem = divmod(download_time, 3600)
+            minutes, seconds = divmod(rem, 60)
+            timestring = "{:0>2}:{:0>2}:{:05.2f}".format(int(hours),int(minutes),seconds)
+            print("")
+            print("")
+            print("--- Statistics -----")
+            print("\tTime taken       : "+timestring)
+            print("\tFile size        : "+size_str(file_size))
+            print("\tData transferred : "+size_str(transfer_size))
+            print("\tEffective rate   : "+size_str(file_size/download_time, suffix='b')+"/s")
+            print("\tTransfer rate    : "+size_str(transfer_size/download_time, suffix='b')+"/s")
+            print("")
+            print("The download completed! Press enter to return to the menu.")
+            print("")
+            input()
 
-		current_download = None
-		menu_mode = "main"
-		print_menu()
+        else:
+            print("")
+            print("The download failed! Press enter to return to the menu.")
+            input()
+
+        current_download = None
+        menu_mode = "main"
+        print_menu()
 
 # This function prints out a list of files
 # on the connected server.
 def print_filelist():
-	global server_files
-	print("Files on server:")
-	for index,file in enumerate(server_files):
-		print("\t("+str(index)+")\t"+file)
+    global server_files
+    print("Files on server:")
+    for index,file in enumerate(server_files):
+        print("\t("+str(index)+")\t"+file)
 
 def filelist_received(filelist_data, packet):
-	global server_files, menu_mode
-	try:
-		# Unpack the list and extend our
-		# local list of available files
-		filelist = umsgpack.unpackb(filelist_data)
-		for file in filelist:
-			if not file in server_files:
-				server_files.append(file)
+    global server_files, menu_mode
+    try:
+        # Unpack the list and extend our
+        # local list of available files
+        filelist = umsgpack.unpackb(filelist_data)
+        for file in filelist:
+            if not file in server_files:
+                server_files.append(file)
 
-		# If the menu is already visible,
-		# we'll update it with what was
-		# just received
-		if menu_mode == "main":
-			print_menu()
-	except:
-		RNS.log("Invalid file list data received, closing link")
-		packet.link.teardown()
+        # If the menu is already visible,
+        # we'll update it with what was
+        # just received
+        if menu_mode == "main":
+            print_menu()
+    except:
+        RNS.log("Invalid file list data received, closing link")
+        packet.link.teardown()
 
 # This function is called when a link
 # has been established with the server
 def link_established(link):
-	# We store a reference to the link
-	# instance for later use
-	global server_link
-	server_link = link
+    # We store a reference to the link
+    # instance for later use
+    global server_link
+    server_link = link
 
-	# Inform the user that the server is
-	# connected
-	RNS.log("Link established with server")
-	RNS.log("Waiting for filelist...")
+    # Inform the user that the server is
+    # connected
+    RNS.log("Link established with server")
+    RNS.log("Waiting for filelist...")
 
-	# And set up a small job to check for
-	# a potential timeout in receiving the
-	# file list
-	thread = threading.Thread(target=filelist_timeout_job)
-	thread.setDaemon(True)
-	thread.start()
+    # And set up a small job to check for
+    # a potential timeout in receiving the
+    # file list
+    thread = threading.Thread(target=filelist_timeout_job)
+    thread.setDaemon(True)
+    thread.start()
 
 # This job just sleeps for the specified
 # time, and then checks if the file list
 # was received. If not, the program will
 # exit.
 def filelist_timeout_job():
-	time.sleep(APP_TIMEOUT)
+    time.sleep(APP_TIMEOUT)
 
-	global server_files
-	if len(server_files) == 0:
-		RNS.log("Timed out waiting for filelist, exiting")
-		os._exit(0)
+    global server_files
+    if len(server_files) == 0:
+        RNS.log("Timed out waiting for filelist, exiting")
+        os._exit(0)
 
 
 # When a link is closed, we'll inform the
 # user, and exit the program
 def link_closed(link):
-	if link.teardown_reason == RNS.Link.TIMEOUT:
-		RNS.log("The link timed out, exiting now")
-	elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
-		RNS.log("The link was closed by the server, exiting now")
-	else:
-		RNS.log("Link closed, exiting now")
-	
-	RNS.Reticulum.exit_handler()
-	time.sleep(1.5)
-	os._exit(0)
+    if link.teardown_reason == RNS.Link.TIMEOUT:
+        RNS.log("The link timed out, exiting now")
+    elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
+        RNS.log("The link was closed by the server, exiting now")
+    else:
+        RNS.log("Link closed, exiting now")
+    
+    RNS.Reticulum.exit_handler()
+    time.sleep(1.5)
+    os._exit(0)
 
 # When RNS detects that the download has
 # started, we'll update our menu state
 # so the user can be shown a progress of
 # the download.
 def download_began(resource):
-	global menu_mode, current_download
-	current_download = resource
-	menu_mode = "downloading"
+    global menu_mode, current_download, download_started, transfer_size, file_size
+    current_download = resource
+    
+    if download_started == 0:
+        download_started = time.time()
+    
+    transfer_size += resource.size
+    file_size = resource.total_size
+    
+    menu_mode = "downloading"
 
 # When the download concludes, successfully
 # or not, we'll update our menu state and 
 # inform the user about how it all went.
 def download_concluded(resource):
-	global menu_mode, current_filename
-	saved_filename = current_filename
+    global menu_mode, current_filename, download_started, download_finished, download_time
+    download_finished = time.time()
+    download_time = download_finished - download_started
 
-	if resource.status == RNS.Resource.COMPLETE:
-		counter = 0
-		while os.path.isfile(saved_filename):
-			counter += 1
-			saved_filename = current_filename+"."+str(counter)
+    saved_filename = current_filename
 
-		try:
-			file = open(saved_filename, "wb")
-			file.write(resource.data)
-			file.close()
-			menu_mode = "download_concluded"
-		except:
-			menu_mode = "save_error"
-	else:
-		menu_mode = "download_concluded"
+    if resource.status == RNS.Resource.COMPLETE:
+        counter = 0
+        while os.path.isfile(saved_filename):
+            counter += 1
+            saved_filename = current_filename+"."+str(counter)
 
+        try:
+            file = open(saved_filename, "wb")
+            file.write(resource.data.read())
+            file.close()
+            menu_mode = "download_concluded"
+        except:
+            menu_mode = "save_error"
+    else:
+        menu_mode = "download_concluded"
+
+# A convenience function for printing a human-
+# readable file size
+def size_str(num, suffix='B'):
+    units = ['','Ki','Mi','Gi','Ti','Pi','Ei','Zi']
+    last_unit = 'Yi'
+
+    if suffix == 'b':
+        num *= 8
+        units = ['','K','M','G','T','P','E','Z']
+        last_unit = 'Y'
+
+    for unit in units:
+        if abs(num) < 1024.0:
+            return "%3.2f %s%s" % (num, unit, suffix)
+        num /= 1024.0
+    return "%.2f %s%s" % (num, last_unit, suffix)
 
 # A convenience function for clearing the screen
 def clear_screen():
@@ -463,31 +551,55 @@ def clear_screen():
 # and parses input of from the user, and then
 # starts up the desired program mode.
 if __name__ == "__main__":
-	try:
-		parser = argparse.ArgumentParser(description="Simple file transfer server and client utility")
-		parser.add_argument("-s", "--serve", action="store", metavar="dir", help="serve a directory of files to clients")
-		parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
-		parser.add_argument("destination", nargs="?", default=None, help="hexadecimal hash of the server destination", type=str)
-		args = parser.parse_args()
+    try:
+        parser = argparse.ArgumentParser(
+            description="Simple file transfer server and client utility"
+        )
 
-		if args.config:
-			configarg = args.config
-		else:
-			configarg = None
+        parser.add_argument(
+            "-s",
+            "--serve",
+            action="store",
+            metavar="dir",
+            help="serve a directory of files to clients"
+        )
 
-		if args.serve:
-			if os.path.isdir(args.serve):
-				server(configarg, args.serve)
-			else:
-				RNS.log("The specified directory does not exist")
-		else:
-			if (args.destination == None):
-				print("")
-				parser.print_help()
-				print("")
-			else:
-				client(args.destination, configarg)
+        parser.add_argument(
+            "--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
 
-	except KeyboardInterrupt:
-		print("")
-		exit()
+        parser.add_argument(
+            "destination",
+            nargs="?",
+            default=None,
+            help="hexadecimal hash of the server destination",
+            type=str
+        )
+
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        if args.serve:
+            if os.path.isdir(args.serve):
+                server(configarg, args.serve)
+            else:
+                RNS.log("The specified directory does not exist")
+        else:
+            if (args.destination == None):
+                print("")
+                parser.print_help()
+                print("")
+            else:
+                client(args.destination, configarg)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()

Examples/Identify.py

@@ -0,0 +1,314 @@
+##########################################################
+# This RNS example demonstrates how to set up a link to  #
+# a destination, and identify the initiator to it's peer #
+##########################################################
+
+import os
+import sys
+import time
+import argparse
+import RNS
+
+# Let's define an app name. We'll use this for all
+# destinations we create. Since this echo example
+# is part of a range of example utilities, we'll put
+# them all within the app namespace "example_utilities"
+APP_NAME = "example_utilities"
+
+##########################################################
+#### Server Part #########################################
+##########################################################
+
+# A reference to the latest client link that connected
+latest_client_link = None
+
+# This initialisation is executed when the users chooses
+# to run as a server
+def server(configpath):
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # Randomly create a new identity for our link example
+    server_identity = RNS.Identity()
+
+    # We create a destination that clients can connect to. We
+    # want clients to create links to this destination, so we
+    # need to create a "single" destination type.
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "identifyexample"
+    )
+
+    # We configure a function that will get called every time
+    # a new client creates a link to this destination.
+    server_destination.set_link_established_callback(client_connected)
+
+    # Everything's ready!
+    # Let's Wait for client requests or user input
+    server_loop(server_destination)
+
+def server_loop(destination):
+    # Let the user know that everything is ready
+    RNS.log(
+        "Link identification example "+
+        RNS.prettyhexrep(destination.hash)+
+        " running, waiting for a connection."
+    )
+
+    RNS.log("Hit enter to manually send an announce (Ctrl-C to quit)")
+
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will announce our server
+    # destination on the network, which will let clients
+    # know how to create messages directed towards it.
+    while True:
+        entered = input()
+        destination.announce()
+        RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
+
+# When a client establishes a link to our server
+# destination, this function will be called with
+# a reference to the link.
+def client_connected(link):
+    global latest_client_link
+
+    RNS.log("Client connected")
+    link.set_link_closed_callback(client_disconnected)
+    link.set_packet_callback(server_packet_received)
+    link.set_remote_identified_callback(remote_identified)
+    latest_client_link = link
+
+def client_disconnected(link):
+    RNS.log("Client disconnected")
+
+def remote_identified(link, identity):
+    RNS.log("Remote identified as: "+str(identity))
+
+def server_packet_received(message, packet):
+    global latest_client_link
+
+    # Get the originating identity for display
+    remote_peer =  "unidentified peer"
+    if packet.link.get_remote_identity() != None:
+        remote_peer = str(packet.link.get_remote_identity())
+
+    # When data is received over any active link,
+    # it will all be directed to the last client
+    # that connected.
+    text = message.decode("utf-8")
+
+    RNS.log("Received data from "+remote_peer+": "+text)
+    
+    reply_text = "I received \""+text+"\" over the link from "+remote_peer
+    reply_data = reply_text.encode("utf-8")
+    RNS.Packet(latest_client_link, reply_data).send()
+
+
+##########################################################
+#### Client Part #########################################
+##########################################################
+
+# A reference to the server link
+server_link = None
+
+# A reference to the client identity
+client_identity = None
+
+# This initialisation is executed when the users chooses
+# to run as a client
+def client(destination_hexhash, configpath):
+    global client_identity
+    # We need a binary representation of the destination
+    # hash that was entered on the command line
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination_hexhash) != dest_len:
+            raise ValueError(
+                "Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2)
+            )
+
+        destination_hash = bytes.fromhex(destination_hexhash)
+    except:
+        RNS.log("Invalid destination entered. Check your input!\n")
+        exit()
+
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+
+    # Create a new client identity
+    client_identity = RNS.Identity()
+    RNS.log(
+        "Client created new identity "+
+        str(client_identity)
+    )
+
+    # Check if we know a path to the destination
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.log("Destination is not yet known. Requesting path and waiting for announce to arrive...")
+        RNS.Transport.request_path(destination_hash)
+        while not RNS.Transport.has_path(destination_hash):
+            time.sleep(0.1)
+
+    # Recall the server identity
+    server_identity = RNS.Identity.recall(destination_hash)
+
+    # Inform the user that we'll begin connecting
+    RNS.log("Establishing link with server...")
+
+    # When the server identity is known, we set
+    # up a destination
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.OUT,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "identifyexample"
+    )
+
+    # And create a link
+    link = RNS.Link(server_destination)
+
+    # We set a callback that will get executed
+    # every time a packet is received over the
+    # link
+    link.set_packet_callback(client_packet_received)
+
+    # We'll also set up functions to inform the
+    # user when the link is established or closed
+    link.set_link_established_callback(link_established)
+    link.set_link_closed_callback(link_closed)
+
+    # Everything is set up, so let's enter a loop
+    # for the user to interact with the example
+    client_loop()
+
+def client_loop():
+    global server_link
+
+    # Wait for the link to become active
+    while not server_link:
+        time.sleep(0.1)
+
+    should_quit = False
+    while not should_quit:
+        try:
+            print("> ", end=" ")
+            text = input()
+
+            # Check if we should quit the example
+            if text == "quit" or text == "q" or text == "exit":
+                should_quit = True
+                server_link.teardown()
+
+            # If not, send the entered text over the link
+            if text != "":
+                data = text.encode("utf-8")
+                if len(data) <= RNS.Link.MDU:
+                    RNS.Packet(server_link, data).send()
+                else:
+                    RNS.log(
+                        "Cannot send this packet, the data size of "+
+                        str(len(data))+" bytes exceeds the link packet MDU of "+
+                        str(RNS.Link.MDU)+" bytes",
+                        RNS.LOG_ERROR
+                    )
+
+        except Exception as e:
+            RNS.log("Error while sending data over the link: "+str(e))
+            should_quit = True
+            server_link.teardown()
+
+# This function is called when a link
+# has been established with the server
+def link_established(link):
+    # We store a reference to the link
+    # instance for later use
+    global server_link, client_identity
+    server_link = link
+
+    # Inform the user that the server is
+    # connected
+    RNS.log("Link established with server, identifying to remote peer...")
+
+    link.identify(client_identity)
+
+# When a link is closed, we'll inform the
+# user, and exit the program
+def link_closed(link):
+    if link.teardown_reason == RNS.Link.TIMEOUT:
+        RNS.log("The link timed out, exiting now")
+    elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
+        RNS.log("The link was closed by the server, exiting now")
+    else:
+        RNS.log("Link closed, exiting now")
+    
+    RNS.Reticulum.exit_handler()
+    time.sleep(1.5)
+    os._exit(0)
+
+# When a packet is received over the link, we
+# simply print out the data.
+def client_packet_received(message, packet):
+    text = message.decode("utf-8")
+    RNS.log("Received data on the link: "+text)
+    print("> ", end=" ")
+    sys.stdout.flush()
+
+
+##########################################################
+#### Program Startup #####################################
+##########################################################
+
+# This part of the program runs at startup,
+# and parses input of from the user, and then
+# starts up the desired program mode.
+if __name__ == "__main__":
+    try:
+        parser = argparse.ArgumentParser(description="Simple link example")
+
+        parser.add_argument(
+            "-s",
+            "--server",
+            action="store_true",
+            help="wait for incoming link requests from clients"
+        )
+
+        parser.add_argument(
+            "--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
+
+        parser.add_argument(
+            "destination",
+            nargs="?",
+            default=None,
+            help="hexadecimal hash of the server destination",
+            type=str
+        )
+
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        if args.server:
+            server(configarg)
+        else:
+            if (args.destination == None):
+                print("")
+                parser.print_help()
+                print("")
+            else:
+                client(args.destination, configarg)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()

Examples/Link.py

@@ -13,7 +13,7 @@ import RNS
 # destinations we create. Since this echo example
 # is part of a range of example utilities, we'll put
 # them all within the app namespace "example_utilities"
-APP_NAME = "example_utilitites"
+APP_NAME = "example_utilities"
 
 ##########################################################
 #### Server Part #########################################
@@ -25,65 +25,76 @@ latest_client_link = None
 # This initialisation is executed when the users chooses
 # to run as a server
 def server(configpath):
-	# We must first initialise Reticulum
-	reticulum = RNS.Reticulum(configpath)
-	
-	# Randomly create a new identity for our link example
-	server_identity = RNS.Identity()
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # Randomly create a new identity for our link example
+    server_identity = RNS.Identity()
 
-	# We create a destination that clients can connect to. We
-	# want clients to create links to this destination, so we
-	# need to create a "single" destination type.
-	server_destination = RNS.Destination(server_identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, "linkexample")
+    # We create a destination that clients can connect to. We
+    # want clients to create links to this destination, so we
+    # need to create a "single" destination type.
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "linkexample"
+    )
 
-	# We configure a function that will get called every time
-	# a new client creates a link to this destination.
-	server_destination.link_established_callback(client_connected)
+    # We configure a function that will get called every time
+    # a new client creates a link to this destination.
+    server_destination.set_link_established_callback(client_connected)
 
-	# Everything's ready!
-	# Let's Wait for client requests or user input
-	server_loop(server_destination)
+    # Everything's ready!
+    # Let's Wait for client requests or user input
+    server_loop(server_destination)
 
 def server_loop(destination):
-	# Let the user know that everything is ready
-	RNS.log("Link example "+RNS.prettyhexrep(destination.hash)+" running, waiting for a connection.")
-	RNS.log("Hit enter to manually send an announce (Ctrl-C to quit)")
+    # Let the user know that everything is ready
+    RNS.log(
+        "Link example "+
+        RNS.prettyhexrep(destination.hash)+
+        " running, waiting for a connection."
+    )
 
-	# We enter a loop that runs until the users exits.
-	# If the user hits enter, we will announce our server
-	# destination on the network, which will let clients
-	# know how to create messages directed towards it.
-	while True:
-		entered = input()
-		destination.announce()
-		RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
+    RNS.log("Hit enter to manually send an announce (Ctrl-C to quit)")
+
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will announce our server
+    # destination on the network, which will let clients
+    # know how to create messages directed towards it.
+    while True:
+        entered = input()
+        destination.announce()
+        RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
 
 # When a client establishes a link to our server
 # destination, this function will be called with
 # a reference to the link.
 def client_connected(link):
-	global latest_client_link
+    global latest_client_link
 
-	RNS.log("Client connected")
-	link.link_closed_callback(client_disconnected)
-	link.packet_callback(server_packet_received)
-	latest_client_link = link
+    RNS.log("Client connected")
+    link.set_link_closed_callback(client_disconnected)
+    link.set_packet_callback(server_packet_received)
+    latest_client_link = link
 
 def client_disconnected(link):
-	RNS.log("Client disconnected")
+    RNS.log("Client disconnected")
 
 def server_packet_received(message, packet):
-	global latest_client_link
+    global latest_client_link
 
-	# When data is received over any active link,
-	# it will all be directed to the last client
-	# that connected.
-	text = message.decode("utf-8")
-	RNS.log("Received data on the link: "+text)
-	
-	reply_text = "I received \""+text+"\" over the link"
-	reply_data = reply_text.encode("utf-8")
-	RNS.Packet(latest_client_link, reply_data).send()
+    # When data is received over any active link,
+    # it will all be directed to the last client
+    # that connected.
+    text = message.decode("utf-8")
+    RNS.log("Received data on the link: "+text)
+    
+    reply_text = "I received \""+text+"\" over the link"
+    reply_data = reply_text.encode("utf-8")
+    RNS.Packet(latest_client_link, reply_data).send()
 
 
 ##########################################################
@@ -96,112 +107,132 @@ server_link = None
 # This initialisation is executed when the users chooses
 # to run as a client
 def client(destination_hexhash, configpath):
-	# We need a binary representation of the destination
-	# hash that was entered on the command line
-	try:
-		if len(destination_hexhash) != 20:
-			raise ValueError("Destination length is invalid, must be 20 hexadecimal characters (10 bytes)")
-		destination_hash = bytes.fromhex(destination_hexhash)
-	except:
-		RNS.log("Invalid destination entered. Check your input!\n")
-		exit()
+    # We need a binary representation of the destination
+    # hash that was entered on the command line
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination_hexhash) != dest_len:
+            raise ValueError(
+                "Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2)
+            )
+            
+        destination_hash = bytes.fromhex(destination_hexhash)
+    except:
+        RNS.log("Invalid destination entered. Check your input!\n")
+        exit()
 
-	# We must first initialise Reticulum
-	reticulum = RNS.Reticulum(configpath)
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
 
-	# Check if we know a path to the destination
-	if not RNS.Transport.hasPath(destination_hash):
-		RNS.log("Destination is not yet known. Requesting path and waiting for announce to arrive...")
-		RNS.Transport.requestPath(destination_hash)
-		while not RNS.Transport.hasPath(destination_hash):
-			time.sleep(0.1)
+    # Check if we know a path to the destination
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.log("Destination is not yet known. Requesting path and waiting for announce to arrive...")
+        RNS.Transport.request_path(destination_hash)
+        while not RNS.Transport.has_path(destination_hash):
+            time.sleep(0.1)
 
-	# Recall the server identity
-	server_identity = RNS.Identity.recall(destination_hash)
+    # Recall the server identity
+    server_identity = RNS.Identity.recall(destination_hash)
 
-	# Inform the user that we'll begin connecting
-	RNS.log("Establishing link with server...")
+    # Inform the user that we'll begin connecting
+    RNS.log("Establishing link with server...")
 
-	# When the server identity is known, we set
-	# up a destination
-	server_destination = RNS.Destination(server_identity, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "linkexample")
+    # When the server identity is known, we set
+    # up a destination
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.OUT,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "linkexample"
+    )
 
-	# And create a link
-	link = RNS.Link(server_destination)
+    # And create a link
+    link = RNS.Link(server_destination)
 
-	# We set a callback that will get executed
-	# every time a packet is received over the
-	# link
-	link.packet_callback(client_packet_received)
+    # We set a callback that will get executed
+    # every time a packet is received over the
+    # link
+    link.set_packet_callback(client_packet_received)
 
-	# We'll also set up functions to inform the
-	# user when the link is established or closed
-	link.link_established_callback(link_established)
-	link.link_closed_callback(link_closed)
+    # We'll also set up functions to inform the
+    # user when the link is established or closed
+    link.set_link_established_callback(link_established)
+    link.set_link_closed_callback(link_closed)
 
-	# Everything is set up, so let's enter a loop
-	# for the user to interact with the example
-	client_loop()
+    # Everything is set up, so let's enter a loop
+    # for the user to interact with the example
+    client_loop()
 
 def client_loop():
-	global server_link
+    global server_link
 
-	# Wait for the link to become active
-	while not server_link:
-		time.sleep(0.1)
+    # Wait for the link to become active
+    while not server_link:
+        time.sleep(0.1)
 
-	should_quit = False
-	while not should_quit:
-		try:
-			print("> ", end=" ")
-			text = input()
+    should_quit = False
+    while not should_quit:
+        try:
+            print("> ", end=" ")
+            text = input()
 
-			# Check if we should quit the example
-			if text == "quit" or text == "q" or text == "exit":
-				should_quit = True
-				server_link.teardown()
+            # Check if we should quit the example
+            if text == "quit" or text == "q" or text == "exit":
+                should_quit = True
+                server_link.teardown()
 
-			# If not, send the entered text over the link
-			if text != "":
-				data = text.encode("utf-8")
-				RNS.Packet(server_link, data).send()
-		except Exception as e:
-			should_quit = True
-			server_link.teardown()
+            # If not, send the entered text over the link
+            if text != "":
+                data = text.encode("utf-8")
+                if len(data) <= RNS.Link.MDU:
+                    RNS.Packet(server_link, data).send()
+                else:
+                    RNS.log(
+                        "Cannot send this packet, the data size of "+
+                        str(len(data))+" bytes exceeds the link packet MDU of "+
+                        str(RNS.Link.MDU)+" bytes",
+                        RNS.LOG_ERROR
+                    )
+
+        except Exception as e:
+            RNS.log("Error while sending data over the link: "+str(e))
+            should_quit = True
+            server_link.teardown()
 
 # This function is called when a link
 # has been established with the server
 def link_established(link):
-	# We store a reference to the link
-	# instance for later use
-	global server_link
-	server_link = link
+    # We store a reference to the link
+    # instance for later use
+    global server_link
+    server_link = link
 
-	# Inform the user that the server is
-	# connected
-	RNS.log("Link established with server, enter some text to send, or \"quit\" to quit")
+    # Inform the user that the server is
+    # connected
+    RNS.log("Link established with server, enter some text to send, or \"quit\" to quit")
 
 # When a link is closed, we'll inform the
 # user, and exit the program
 def link_closed(link):
-	if link.teardown_reason == RNS.Link.TIMEOUT:
-		RNS.log("The link timed out, exiting now")
-	elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
-		RNS.log("The link was closed by the server, exiting now")
-	else:
-		RNS.log("Link closed, exiting now")
-	
-	RNS.Reticulum.exit_handler()
-	time.sleep(1.5)
-	os._exit(0)
+    if link.teardown_reason == RNS.Link.TIMEOUT:
+        RNS.log("The link timed out, exiting now")
+    elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
+        RNS.log("The link was closed by the server, exiting now")
+    else:
+        RNS.log("Link closed, exiting now")
+    
+    RNS.Reticulum.exit_handler()
+    time.sleep(1.5)
+    os._exit(0)
 
 # When a packet is received over the link, we
 # simply print out the data.
 def client_packet_received(message, packet):
-	text = message.decode("utf-8")
-	RNS.log("Received data on the link: "+text)
-	print("> ", end=" ")
-	sys.stdout.flush()
+    text = message.decode("utf-8")
+    RNS.log("Received data on the link: "+text)
+    print("> ", end=" ")
+    sys.stdout.flush()
 
 
 ##########################################################
@@ -212,28 +243,49 @@ def client_packet_received(message, packet):
 # and parses input of from the user, and then
 # starts up the desired program mode.
 if __name__ == "__main__":
-	try:
-		parser = argparse.ArgumentParser(description="Simple link example")
-		parser.add_argument("-s", "--server", action="store_true", help="wait for incoming link requests from clients")
-		parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
-		parser.add_argument("destination", nargs="?", default=None, help="hexadecimal hash of the server destination", type=str)
-		args = parser.parse_args()
+    try:
+        parser = argparse.ArgumentParser(description="Simple link example")
 
-		if args.config:
-			configarg = args.config
-		else:
-			configarg = None
+        parser.add_argument(
+            "-s",
+            "--server",
+            action="store_true",
+            help="wait for incoming link requests from clients"
+        )
 
-		if args.server:
-			server(configarg)
-		else:
-			if (args.destination == None):
-				print("")
-				parser.print_help()
-				print("")
-			else:
-				client(args.destination, configarg)
+        parser.add_argument(
+            "--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
 
-	except KeyboardInterrupt:
-		print("")
-		exit()
+        parser.add_argument(
+            "destination",
+            nargs="?",
+            default=None,
+            help="hexadecimal hash of the server destination",
+            type=str
+        )
+
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        if args.server:
+            server(configarg)
+        else:
+            if (args.destination == None):
+                print("")
+                parser.print_help()
+                print("")
+            else:
+                client(args.destination, configarg)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()

Examples/Minimal.py

@@ -11,49 +11,59 @@ import RNS
 # destinations we create. Since this basic example
 # is part of a range of example utilities, we'll put
 # them all within the app namespace "example_utilities"
-APP_NAME = "example_utilitites"
+APP_NAME = "example_utilities"
 
 # This initialisation is executed when the program is started
 def program_setup(configpath):
-	# We must first initialise Reticulum
-	reticulum = RNS.Reticulum(configpath)
-	
-	# Randomly create a new identity for our example
-	identity = RNS.Identity()
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # Randomly create a new identity for our example
+    identity = RNS.Identity()
 
-	# Using the identity we just created, we create a destination.
-	# Destinations are endpoints in Reticulum, that can be addressed
-	# and communicated with. Destinations can also announce their
-	# existence, which will let the network know they are reachable
-	# and autoomatically create paths to them, from anywhere else
-	# in the network.
-	destination = RNS.Destination(identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, "minimalsample")
+    # Using the identity we just created, we create a destination.
+    # Destinations are endpoints in Reticulum, that can be addressed
+    # and communicated with. Destinations can also announce their
+    # existence, which will let the network know they are reachable
+    # and autoomatically create paths to them, from anywhere else
+    # in the network.
+    destination = RNS.Destination(
+        identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "minimalsample"
+    )
 
-	# We configure the destination to automatically prove all
-	# packets adressed to it. By doing this, RNS will automatically
-	# generate a proof for each incoming packet and transmit it
-	# back to the sender of that packet. This will let anyone that
-	# tries to communicate with the destination know whether their
-	# communication was received correctly.
-	destination.set_proof_strategy(RNS.Destination.PROVE_ALL)
-	
-	# Everything's ready!
-	# Let's hand over control to the announce loop
-	announceLoop(destination)
+    # We configure the destination to automatically prove all
+    # packets adressed to it. By doing this, RNS will automatically
+    # generate a proof for each incoming packet and transmit it
+    # back to the sender of that packet. This will let anyone that
+    # tries to communicate with the destination know whether their
+    # communication was received correctly.
+    destination.set_proof_strategy(RNS.Destination.PROVE_ALL)
+    
+    # Everything's ready!
+    # Let's hand over control to the announce loop
+    announceLoop(destination)
 
 
 def announceLoop(destination):
-	# Let the user know that everything is ready
-	RNS.log("Minimal example "+RNS.prettyhexrep(destination.hash)+" running, hit enter to manually send an announce (Ctrl-C to quit)")
+    # Let the user know that everything is ready
+    RNS.log(
+        "Minimal example "+
+        RNS.prettyhexrep(destination.hash)+
+        " running, hit enter to manually send an announce (Ctrl-C to quit)"
+    )
 
-	# We enter a loop that runs until the users exits.
-	# If the user hits enter, we will announce our server
-	# destination on the network, which will let clients
-	# know how to create messages directed towards it.
-	while True:
-		entered = input()
-		destination.announce()
-		RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will announce our server
+    # destination on the network, which will let clients
+    # know how to create messages directed towards it.
+    while True:
+        entered = input()
+        destination.announce()
+        RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
 
 
 ##########################################################
@@ -64,18 +74,28 @@ def announceLoop(destination):
 # and parses input from the user, and then starts
 # the desired program mode.
 if __name__ == "__main__":
-	try:
-		parser = argparse.ArgumentParser(description="Bare minimum example to start Reticulum and create a destination")
-		parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
-		args = parser.parse_args()
+    try:
+        parser = argparse.ArgumentParser(
+            description="Minimal example to start Reticulum and create a destination"
+        )
 
-		if args.config:
-			configarg = args.config
-		else:
-			configarg = None
+        parser.add_argument(
+            "--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
 
-		program_setup(configarg)
+        args = parser.parse_args()
 
-	except KeyboardInterrupt:
-		print("")
-		exit()
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        program_setup(configarg)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()

Examples/Request.py

@@ -0,0 +1,287 @@
+##########################################################
+# This RNS example demonstrates how to set perform       #
+# requests and receive responses over a link.            #
+##########################################################
+
+import os
+import sys
+import time
+import random
+import argparse
+import RNS
+
+# Let's define an app name. We'll use this for all
+# destinations we create. Since this echo example
+# is part of a range of example utilities, we'll put
+# them all within the app namespace "example_utilities"
+APP_NAME = "example_utilities"
+
+##########################################################
+#### Server Part #########################################
+##########################################################
+
+# A reference to the latest client link that connected
+latest_client_link = None
+
+def random_text_generator(path, data, request_id, remote_identity, requested_at):
+    RNS.log("Generating response to request "+RNS.prettyhexrep(request_id))
+    texts = ["They looked up", "On each full moon", "Becky was upset", "I’ll stay away from it", "The pet shop stocks everything"]
+    return texts[random.randint(0, len(texts)-1)]
+
+# This initialisation is executed when the users chooses
+# to run as a server
+def server(configpath):
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # Randomly create a new identity for our link example
+    server_identity = RNS.Identity()
+
+    # We create a destination that clients can connect to. We
+    # want clients to create links to this destination, so we
+    # need to create a "single" destination type.
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "requestexample"
+    )
+
+    # We configure a function that will get called every time
+    # a new client creates a link to this destination.
+    server_destination.set_link_established_callback(client_connected)
+
+    # We register a request handler for handling incoming
+    # requests over any established links.
+    server_destination.register_request_handler(
+        "/random/text",
+        response_generator = random_text_generator,
+        allow = RNS.Destination.ALLOW_ALL
+    )
+
+    # Everything's ready!
+    # Let's Wait for client requests or user input
+    server_loop(server_destination)
+
+def server_loop(destination):
+    # Let the user know that everything is ready
+    RNS.log(
+        "Request example "+
+        RNS.prettyhexrep(destination.hash)+
+        " running, waiting for a connection."
+    )
+
+    RNS.log("Hit enter to manually send an announce (Ctrl-C to quit)")
+
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will announce our server
+    # destination on the network, which will let clients
+    # know how to create messages directed towards it.
+    while True:
+        entered = input()
+        destination.announce()
+        RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
+
+# When a client establishes a link to our server
+# destination, this function will be called with
+# a reference to the link.
+def client_connected(link):
+    global latest_client_link
+
+    RNS.log("Client connected")
+    link.set_link_closed_callback(client_disconnected)
+    latest_client_link = link
+
+def client_disconnected(link):
+    RNS.log("Client disconnected")
+
+
+##########################################################
+#### Client Part #########################################
+##########################################################
+
+# A reference to the server link
+server_link = None
+
+# This initialisation is executed when the users chooses
+# to run as a client
+def client(destination_hexhash, configpath):
+    # We need a binary representation of the destination
+    # hash that was entered on the command line
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination_hexhash) != dest_len:
+            raise ValueError(
+                "Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2)
+            )
+            
+        destination_hash = bytes.fromhex(destination_hexhash)
+    except:
+        RNS.log("Invalid destination entered. Check your input!\n")
+        exit()
+
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+
+    # Check if we know a path to the destination
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.log("Destination is not yet known. Requesting path and waiting for announce to arrive...")
+        RNS.Transport.request_path(destination_hash)
+        while not RNS.Transport.has_path(destination_hash):
+            time.sleep(0.1)
+
+    # Recall the server identity
+    server_identity = RNS.Identity.recall(destination_hash)
+
+    # Inform the user that we'll begin connecting
+    RNS.log("Establishing link with server...")
+
+    # When the server identity is known, we set
+    # up a destination
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.OUT,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "requestexample"
+    )
+
+    # And create a link
+    link = RNS.Link(server_destination)
+
+    # We'll set up functions to inform the
+    # user when the link is established or closed
+    link.set_link_established_callback(link_established)
+    link.set_link_closed_callback(link_closed)
+
+    # Everything is set up, so let's enter a loop
+    # for the user to interact with the example
+    client_loop()
+
+def client_loop():
+    global server_link
+
+    # Wait for the link to become active
+    while not server_link:
+        time.sleep(0.1)
+
+    should_quit = False
+    while not should_quit:
+        try:
+            print("> ", end=" ")
+            text = input()
+
+            # Check if we should quit the example
+            if text == "quit" or text == "q" or text == "exit":
+                should_quit = True
+                server_link.teardown()
+
+            else:
+                server_link.request(
+                    "/random/text",
+                    data = None,
+                    response_callback = got_response,
+                    failed_callback = request_failed
+                )
+
+
+        except Exception as e:
+            RNS.log("Error while sending request over the link: "+str(e))
+            should_quit = True
+            server_link.teardown()
+
+def got_response(request_receipt):
+    request_id = request_receipt.request_id
+    response = request_receipt.response
+
+    RNS.log("Got response for request "+RNS.prettyhexrep(request_id)+": "+str(response))
+
+def request_received(request_receipt):
+    RNS.log("The request "+RNS.prettyhexrep(request_receipt.request_id)+" was received by the remote peer.")
+
+def request_failed(request_receipt):
+    RNS.log("The request "+RNS.prettyhexrep(request_receipt.request_id)+" failed.")
+
+
+# This function is called when a link
+# has been established with the server
+def link_established(link):
+    # We store a reference to the link
+    # instance for later use
+    global server_link
+    server_link = link
+
+    # Inform the user that the server is
+    # connected
+    RNS.log("Link established with server, hit enter to perform a request, or type in \"quit\" to quit")
+
+# When a link is closed, we'll inform the
+# user, and exit the program
+def link_closed(link):
+    if link.teardown_reason == RNS.Link.TIMEOUT:
+        RNS.log("The link timed out, exiting now")
+    elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
+        RNS.log("The link was closed by the server, exiting now")
+    else:
+        RNS.log("Link closed, exiting now")
+    
+    RNS.Reticulum.exit_handler()
+    time.sleep(1.5)
+    os._exit(0)
+
+
+##########################################################
+#### Program Startup #####################################
+##########################################################
+
+# This part of the program runs at startup,
+# and parses input of from the user, and then
+# starts up the desired program mode.
+if __name__ == "__main__":
+    try:
+        parser = argparse.ArgumentParser(description="Simple request/response example")
+
+        parser.add_argument(
+            "-s",
+            "--server",
+            action="store_true",
+            help="wait for incoming requests from clients"
+        )
+
+        parser.add_argument(
+            "--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
+
+        parser.add_argument(
+            "destination",
+            nargs="?",
+            default=None,
+            help="hexadecimal hash of the server destination",
+            type=str
+        )
+
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        if args.server:
+            server(configarg)
+        else:
+            if (args.destination == None):
+                print("")
+                parser.print_help()
+                print("")
+            else:
+                client(args.destination, configarg)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()

Examples/Speedtest.py

@@ -0,0 +1,347 @@
+##########################################################
+# This RNS example demonstrates a simple speedtest       #
+# program to measure link throughput.                    #
+#                                                        #
+# The current configuration is suited for testing fast   #
+# links. If you want to measure slow links like LoRa or  #
+# packet radio, you must significantly lower the         #
+# data_cap variable, which defines how much data is sent #
+# for each test.                                         #
+##########################################################
+
+import os
+import sys
+import time
+import argparse
+import RNS
+
+# Let's define an app name. We'll use this for all
+# destinations we create.
+APP_NAME = "example_utilities"
+
+##########################################################
+#### Server Part #########################################
+##########################################################
+
+latest_client_link = None
+first_packet_at = None
+last_packet_at = None
+received_data = 0
+rc = 0
+data_cap = 2*1024*1024
+printed = False
+
+# This initialisation is executed when the users chooses
+# to run as a server
+def server(configpath):
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+    
+    # Randomly create a new identity for our link example
+    server_identity = RNS.Identity()
+
+    # We create a destination that clients can connect to. We
+    # want clients to create links to this destination, so we
+    # need to create a "single" destination type.
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.IN,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "speedtest"
+    )
+
+    # We configure a function that will get called every time
+    # a new client creates a link to this destination.
+    server_destination.set_link_established_callback(client_connected)
+
+    # Everything's ready!
+    # Let's Wait for client requests or user input
+    server_loop(server_destination)
+
+def server_loop(destination):
+    # Let the user know that everything is ready
+    RNS.log(
+        "Speedtest "+
+        RNS.prettyhexrep(destination.hash)+
+        " running, waiting for a connection."
+    )
+
+    RNS.log("Hit enter to manually send an announce (Ctrl-C to quit)")
+
+    # We enter a loop that runs until the users exits.
+    # If the user hits enter, we will announce our server
+    # destination on the network, which will let clients
+    # know how to create messages directed towards it.
+    while True:
+        entered = input()
+        destination.announce()
+        RNS.log("Sent announce from "+RNS.prettyhexrep(destination.hash))
+
+# When a client establishes a link to our server
+# destination, this function will be called with
+# a reference to the link.
+def client_connected(link):
+    global latest_client_link, first_packet_at, rc
+
+    RNS.log("Client connected")
+    first_packet_at = time.time()
+    rc = 0
+    link.set_link_closed_callback(client_disconnected)
+    link.set_packet_callback(server_packet_received)
+    latest_client_link = link
+
+def client_disconnected(link):
+    RNS.log("Client disconnected")
+
+
+# A convenience function for printing a human-
+# readable file size
+def size_str(num, suffix='B'):
+    units = ['','Ki','Mi','Gi','Ti','Pi','Ei','Zi']
+    last_unit = 'Yi'
+
+    if suffix == 'b':
+        num *= 8
+        units = ['','K','M','G','T','P','E','Z']
+        last_unit = 'Y'
+
+    for unit in units:
+        if abs(num) < 1024.0:
+            return "%3.2f %s%s" % (num, unit, suffix)
+        num /= 1024.0
+    return "%.2f %s%s" % (num, last_unit, suffix)
+
+
+def server_packet_received(message, packet):
+    global latest_client_link, first_packet_at, last_packet_at, received_data, rc, data_cap
+    
+    received_data += len(packet.data)
+    
+    rc += 1
+    if rc >= 50:
+        RNS.log(size_str(received_data))
+        rc = 0
+
+    if received_data > data_cap:
+        rcv_d = received_data
+        received_data = 0
+        rc = 0
+
+        last_packet_at = time.time()
+        
+        # Print statistics
+        download_time = last_packet_at-first_packet_at
+        hours, rem = divmod(download_time, 3600)
+        minutes, seconds = divmod(rem, 60)
+        timestring = "{:0>2}:{:0>2}:{:05.2f}".format(int(hours),int(minutes),seconds)
+
+        print("")
+        print("")
+        print("--- Statistics -----")
+        print("\tTime taken       : "+timestring)
+        print("\tData transferred : "+size_str(rcv_d))
+        print("\tTransfer rate    : "+size_str(rcv_d/download_time, suffix='b')+"/s")
+        print("")
+
+        sys.stdout.flush()
+        latest_client_link.teardown()
+        time.sleep(0.2)
+        rc = 0
+        received_data = 0
+        # latest_client_link.teardown()
+        # os._exit(0)
+
+
+##########################################################
+#### Client Part #########################################
+##########################################################
+
+# A reference to the server link
+server_link = None
+
+# This initialisation is executed when the users chooses
+# to run as a client
+def client(destination_hexhash, configpath):
+    # We need a binary representation of the destination
+    # hash that was entered on the command line
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination_hexhash) != dest_len:
+            raise ValueError(
+                "Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2)
+            )
+            
+        destination_hash = bytes.fromhex(destination_hexhash)
+    except:
+        RNS.log("Invalid destination entered. Check your input!\n")
+        exit()
+
+    # We must first initialise Reticulum
+    reticulum = RNS.Reticulum(configpath)
+
+    # Check if we know a path to the destination
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.log("Destination is not yet known. Requesting path and waiting for announce to arrive...")
+        RNS.Transport.request_path(destination_hash)
+        while not RNS.Transport.has_path(destination_hash):
+            time.sleep(0.1)
+
+    # Recall the server identity
+    server_identity = RNS.Identity.recall(destination_hash)
+
+    # Inform the user that we'll begin connecting
+    RNS.log("Establishing link with server...")
+
+    # When the server identity is known, we set
+    # up a destination
+    server_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.OUT,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "speedtest"
+    )
+
+    # And create a link
+    link = RNS.Link(server_destination)
+
+    # We'll also set up functions to inform the
+    # user when the link is established or closed
+    link.set_link_established_callback(link_established)
+    link.set_link_closed_callback(link_closed)
+
+    # Everything is set up, so let's enter a loop
+    # for the user to interact with the example
+    client_loop()
+
+def client_loop():
+    global server_link
+
+    # Wait for the link to become active
+    while not server_link:
+        time.sleep(0.1)
+
+    should_quit = False
+    while not should_quit:
+        try:
+            text = input()
+
+            # Check if we should quit the example
+            if text == "quit" or text == "q" or text == "exit":
+                should_quit = True
+                server_link.teardown()
+
+        except Exception as e:
+            raise e
+
+# This function is called when a link
+# has been established with the server
+def link_established(link):
+    # We store a reference to the link
+    # instance for later use
+    global server_link, data_cap, printed
+    server_link = link
+    data_sent = 0
+
+    # Inform the user that the server is
+    # connected
+    RNS.log("Link established with server,sending...")
+    rd = os.urandom(RNS.Link.MDU)
+    started = time.time()
+    while link.status == RNS.Link.ACTIVE and data_sent < data_cap*1.25:
+        RNS.Packet(server_link, rd, create_receipt=False).send()
+        data_sent += len(rd)
+
+        if data_sent > data_cap and not printed:
+            printed = True
+            ended = time.time()
+            # Print statistics
+            download_time = ended-started
+            hours, rem = divmod(download_time, 3600)
+            minutes, seconds = divmod(rem, 60)
+            timestring = "{:0>2}:{:0>2}:{:05.2f}".format(int(hours),int(minutes),seconds)
+            print("")
+            print("")
+            print("--- Statistics -----")
+            print("\tTime taken       : "+timestring)
+            print("\tData transferred : "+size_str(data_sent))
+            print("\tTransfer rate    : "+size_str(data_sent/download_time, suffix='b')+"/s")
+            print("")
+
+            sys.stdout.flush()
+            time.sleep(0.1)
+
+
+# When a link is closed, we'll inform the
+# user, and exit the program
+def link_closed(link):
+    if link.teardown_reason == RNS.Link.TIMEOUT:
+        RNS.log("The link timed out, exiting now")
+    elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
+        RNS.log("The link was closed by the server, exiting now")
+    else:
+        RNS.log("Link closed, exiting now")
+    
+    RNS.Reticulum.exit_handler()
+
+    time.sleep(1.5)
+    os._exit(0)
+
+def client_packet_received(message, packet):
+    pass
+
+##########################################################
+#### Program Startup #####################################
+##########################################################
+
+# This part of the program runs at startup,
+# and parses input of from the user, and then
+# starts up the desired program mode.
+if __name__ == "__main__":
+    try:
+        parser = argparse.ArgumentParser(description="Speedtest example")
+
+        parser.add_argument(
+            "-s",
+            "--server",
+            action="store_true",
+            help="wait for incoming requests from clients"
+        )
+
+        parser.add_argument(
+            "--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
+
+        parser.add_argument(
+            "destination",
+            nargs="?",
+            default=None,
+            help="hexadecimal hash of the server destination",
+            type=str
+        )
+
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        if args.server:
+            server(configarg)
+        else:
+            if (args.destination == None):
+                print("")
+                parser.print_help()
+                print("")
+            else:
+                client(args.destination, configarg)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()

LICENSE

@@ -1,6 +1,6 @@
 MIT License, unless otherwise noted
 
-Copyright (c) 2018 Mark Qvist / unsigned.io
+Copyright (c) 2016-2022 Mark Qvist / unsigned.io
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

Makefile

@@ -0,0 +1,61 @@
+all: release
+
+test:
+	@echo Running tests...
+	python -m tests.all
+
+clean:
+	@echo Cleaning...
+	@-rm -rf ./build
+	@-rm -rf ./dist
+	@-rm -rf ./*.data
+	@-rm -rf ./__pycache__
+	@-rm -rf ./RNS/__pycache__
+	@-rm -rf ./RNS/Cryptography/__pycache__
+	@-rm -rf ./RNS/Cryptography/aes/__pycache__
+	@-rm -rf ./RNS/Cryptography/pure25519/__pycache__
+	@-rm -rf ./RNS/Interfaces/__pycache__
+	@-rm -rf ./RNS/Utilities/__pycache__
+	@-rm -rf ./RNS/vendor/__pycache__
+	@-rm -rf ./RNS/vendor/i2plib/__pycache__
+	@-rm -rf ./tests/__pycache__
+	@-rm -rf ./tests/rnsconfig/storage
+	@-rm -rf ./*.egg-info
+	@make -C docs clean
+	@echo Done
+
+remove_symlinks:
+	@echo Removing symlinks for build...
+	-rm Examples/RNS
+	-rm RNS/Utilities/RNS
+
+create_symlinks:
+	@echo Creating symlinks...
+	-ln -s ../RNS ./Examples/
+	-ln -s ../../RNS ./RNS/Utilities/
+
+build_sdist_only:
+	python3 setup.py sdist
+
+build_wheel:
+	python3 setup.py sdist bdist_wheel
+
+build_pure_wheel:
+	python3 setup.py sdist bdist_wheel --pure
+
+documentation:
+	make -C docs html
+
+manual:
+	make -C docs latexpdf
+
+release: test remove_symlinks build_wheel build_pure_wheel documentation manual create_symlinks
+
+debug: remove_symlinks build_wheel build_pure_wheel create_symlinks
+
+upload:
+	@echo Ready to publish release, hit enter to continue
+	@read VOID
+	@echo Uploading to PyPi...
+	twine upload dist/*
+	@echo Release published

Notes/Wire Format

@@ -1,54 +0,0 @@
-Reticulum Wire Format
-
-Header Types
------------------
-type 1          00  Two byte header, one 10 byte address field
-type 2          01  Two byte header, two 10 byte address fields
-type 3          10  Reserved
-type 4          11  Reserved for extended header format
-
-
-Propagation Types
------------------
-broadcast       00
-transport       01
-relay           10
-tunnel          11
-
-
-Destination Types
------------------
-single          00
-group           01
-plain           10
-link            11
-
-
-Packet Types
------------------
-data            00
-announce        01
-link request    10
-proof           11
-
-
-+- Packet Example -+
-
-01010000 00000100 [ADDR1, 10 bytes] [ADDR2, 10 bytes] [CONTEXT, 1 byte] [DATA]
- | | | |    |
- | | | |    +-- Hops = 4
- | | | +------- DATA packet
- | | +--------- SINGLE destination
- | +----------- TRANSPORT propagation type
- +------------- HEADER_2, two byte header, two address fields
-
-
- +- Packet Example -+
-
-00000000 00000111 [ADDR1, 10 bytes] [CONTEXT, 1 byte] [DATA]
- | | | |    |
- | | | |    +-- Hops = 7
- | | | +------- DATA packet
- | | +--------- SINGLE destination
- | +----------- BROADCAST propagation type
- +------------- HEADER_1, two byte header, one address field

README.html

@@ -1,837 +0,0 @@
-<!DOCTYPE html><html><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"><style>@font-face {
-  font-family: octicons-anchor;
-  src: url(https://cdnjs.cloudflare.com/ajax/libs/octicons/4.4.0/font/octicons.woff) format('woff');
-}
-
-* {
-    box-sizing: border-box;
-}
-
-body {
-    width: 980px;
-    margin-right: auto;
-    margin-left: auto;
-    color:#333;
-    background:#fff;
-}
-
-body .markdown-body {
-    padding: 45px;
-    border: 1px solid #ddd;
-    border-radius: 3px;
-    word-wrap: break-word;
-}
-
-pre {
-    font: 12px Consolas, "Liberation Mono", Menlo, Courier, monospace;
-}
-
-.markdown-body {
-  -webkit-text-size-adjust: 100%;
-  text-size-adjust: 100%;
-  color: #333;
-  font-family: "Helvetica Neue", Helvetica, "Segoe UI", Arial, freesans, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";
-  font-size: 16px;
-  line-height: 1.6;
-  word-wrap: break-word;
-}
-
-.markdown-body a {
-  background-color: transparent;
-}
-
-.markdown-body a:active,
-.markdown-body a:hover {
-  outline: 0;
-}
-
-.markdown-body strong {
-  font-weight: bold;
-}
-
-.markdown-body h1 {
-  font-size: 2em;
-  margin: 0.67em 0;
-}
-
-.markdown-body img {
-  border: 0;
-}
-
-.markdown-body hr {
-  box-sizing: content-box;
-  height: 0;
-}
-
-.markdown-body pre {
-  overflow: auto;
-}
-
-.markdown-body code,
-.markdown-body kbd,
-.markdown-body pre {
-  font-family: monospace, monospace;
-  font-size: 1em;
-}
-
-.markdown-body input {
-  color: inherit;
-  font: inherit;
-  margin: 0;
-}
-
-.markdown-body html input[disabled] {
-  cursor: default;
-}
-
-.markdown-body input {
-  line-height: normal;
-}
-
-.markdown-body input[type="checkbox"] {
-  box-sizing: border-box;
-  padding: 0;
-}
-
-.markdown-body table {
-  border-collapse: collapse;
-  border-spacing: 0;
-}
-
-.markdown-body td,
-.markdown-body th {
-  padding: 0;
-}
-
-.markdown-body input {
-  font: 13px / 1.4 Helvetica, arial, nimbussansl, liberationsans, freesans, clean, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";
-}
-
-.markdown-body a {
-  color: #4078c0;
-  text-decoration: none;
-}
-
-.markdown-body a:hover,
-.markdown-body a:active {
-  text-decoration: underline;
-}
-
-.markdown-body hr {
-  height: 0;
-  margin: 15px 0;
-  overflow: hidden;
-  background: transparent;
-  border: 0;
-  border-bottom: 1px solid #ddd;
-}
-
-.markdown-body hr:before {
-  display: table;
-  content: "";
-}
-
-.markdown-body hr:after {
-  display: table;
-  clear: both;
-  content: "";
-}
-
-.markdown-body h1,
-.markdown-body h2,
-.markdown-body h3,
-.markdown-body h4,
-.markdown-body h5,
-.markdown-body h6 {
-  margin-top: 15px;
-  margin-bottom: 15px;
-  line-height: 1.1;
-}
-
-.markdown-body h1 {
-  font-size: 30px;
-}
-
-.markdown-body h2 {
-  font-size: 21px;
-}
-
-.markdown-body h3 {
-  font-size: 16px;
-}
-
-.markdown-body h4 {
-  font-size: 14px;
-}
-
-.markdown-body h5 {
-  font-size: 12px;
-}
-
-.markdown-body h6 {
-  font-size: 11px;
-}
-
-.markdown-body blockquote {
-  margin: 0;
-}
-
-.markdown-body ul,
-.markdown-body ol {
-  padding: 0;
-  margin-top: 0;
-  margin-bottom: 0;
-}
-
-.markdown-body ol ol,
-.markdown-body ul ol {
-  list-style-type: lower-roman;
-}
-
-.markdown-body ul ul ol,
-.markdown-body ul ol ol,
-.markdown-body ol ul ol,
-.markdown-body ol ol ol {
-  list-style-type: lower-alpha;
-}
-
-.markdown-body dd {
-  margin-left: 0;
-}
-
-.markdown-body code {
-  font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace;
-  font-size: 12px;
-}
-
-.markdown-body pre {
-  margin-top: 0;
-  margin-bottom: 0;
-  font: 12px Consolas, "Liberation Mono", Menlo, Courier, monospace;
-}
-
-.markdown-body .select::-ms-expand {
-  opacity: 0;
-}
-
-.markdown-body .octicon {
-  font: normal normal normal 16px/1 octicons-anchor;
-  display: inline-block;
-  text-decoration: none;
-  text-rendering: auto;
-  -webkit-font-smoothing: antialiased;
-  -moz-osx-font-smoothing: grayscale;
-  -webkit-user-select: none;
-  -moz-user-select: none;
-  -ms-user-select: none;
-  user-select: none;
-}
-
-.markdown-body .octicon-link:before {
-  content: '\f05c';
-}
-
-.markdown-body:before {
-  display: table;
-  content: "";
-}
-
-.markdown-body:after {
-  display: table;
-  clear: both;
-  content: "";
-}
-
-.markdown-body>*:first-child {
-  margin-top: 0 !important;
-}
-
-.markdown-body>*:last-child {
-  margin-bottom: 0 !important;
-}
-
-.markdown-body a:not([href]) {
-  color: inherit;
-  text-decoration: none;
-}
-
-.markdown-body .anchor {
-  display: inline-block;
-  padding-right: 2px;
-  margin-left: -18px;
-}
-
-.markdown-body .anchor:focus {
-  outline: none;
-}
-
-.markdown-body h1,
-.markdown-body h2,
-.markdown-body h3,
-.markdown-body h4,
-.markdown-body h5,
-.markdown-body h6 {
-  margin-top: 1em;
-  margin-bottom: 16px;
-  font-weight: bold;
-  line-height: 1.4;
-}
-
-.markdown-body h1 .octicon-link,
-.markdown-body h2 .octicon-link,
-.markdown-body h3 .octicon-link,
-.markdown-body h4 .octicon-link,
-.markdown-body h5 .octicon-link,
-.markdown-body h6 .octicon-link {
-  color: #000;
-  vertical-align: middle;
-  visibility: hidden;
-}
-
-.markdown-body h1:hover .anchor,
-.markdown-body h2:hover .anchor,
-.markdown-body h3:hover .anchor,
-.markdown-body h4:hover .anchor,
-.markdown-body h5:hover .anchor,
-.markdown-body h6:hover .anchor {
-  text-decoration: none;
-}
-
-.markdown-body h1:hover .anchor .octicon-link,
-.markdown-body h2:hover .anchor .octicon-link,
-.markdown-body h3:hover .anchor .octicon-link,
-.markdown-body h4:hover .anchor .octicon-link,
-.markdown-body h5:hover .anchor .octicon-link,
-.markdown-body h6:hover .anchor .octicon-link {
-  visibility: visible;
-}
-
-.markdown-body h1 {
-  padding-bottom: 0.3em;
-  font-size: 2.25em;
-  line-height: 1.2;
-  border-bottom: 1px solid #eee;
-}
-
-.markdown-body h1 .anchor {
-  line-height: 1;
-}
-
-.markdown-body h2 {
-  padding-bottom: 0.3em;
-  font-size: 1.75em;
-  line-height: 1.225;
-  border-bottom: 1px solid #eee;
-}
-
-.markdown-body h2 .anchor {
-  line-height: 1;
-}
-
-.markdown-body h3 {
-  font-size: 1.5em;
-  line-height: 1.43;
-}
-
-.markdown-body h3 .anchor {
-  line-height: 1.2;
-}
-
-.markdown-body h4 {
-  font-size: 1.25em;
-}
-
-.markdown-body h4 .anchor {
-  line-height: 1.2;
-}
-
-.markdown-body h5 {
-  font-size: 1em;
-}
-
-.markdown-body h5 .anchor {
-  line-height: 1.1;
-}
-
-.markdown-body h6 {
-  font-size: 1em;
-  color: #777;
-}
-
-.markdown-body h6 .anchor {
-  line-height: 1.1;
-}
-
-.markdown-body p,
-.markdown-body blockquote,
-.markdown-body ul,
-.markdown-body ol,
-.markdown-body dl,
-.markdown-body table,
-.markdown-body pre {
-  margin-top: 0;
-  margin-bottom: 16px;
-}
-
-.markdown-body hr {
-  height: 4px;
-  padding: 0;
-  margin: 16px 0;
-  background-color: #e7e7e7;
-  border: 0 none;
-}
-
-.markdown-body ul,
-.markdown-body ol {
-  padding-left: 2em;
-}
-
-.markdown-body ul ul,
-.markdown-body ul ol,
-.markdown-body ol ol,
-.markdown-body ol ul {
-  margin-top: 0;
-  margin-bottom: 0;
-}
-
-.markdown-body li>p {
-  margin-top: 16px;
-}
-
-.markdown-body dl {
-  padding: 0;
-}
-
-.markdown-body dl dt {
-  padding: 0;
-  margin-top: 16px;
-  font-size: 1em;
-  font-style: italic;
-  font-weight: bold;
-}
-
-.markdown-body dl dd {
-  padding: 0 16px;
-  margin-bottom: 16px;
-}
-
-.markdown-body blockquote {
-  padding: 0 15px;
-  color: #777;
-  border-left: 4px solid #ddd;
-}
-
-.markdown-body blockquote>:first-child {
-  margin-top: 0;
-}
-
-.markdown-body blockquote>:last-child {
-  margin-bottom: 0;
-}
-
-.markdown-body table {
-  display: block;
-  width: 100%;
-  overflow: auto;
-  word-break: normal;
-  word-break: keep-all;
-}
-
-.markdown-body table th {
-  font-weight: bold;
-}
-
-.markdown-body table th,
-.markdown-body table td {
-  padding: 6px 13px;
-  border: 1px solid #ddd;
-}
-
-.markdown-body table tr {
-  background-color: #fff;
-  border-top: 1px solid #ccc;
-}
-
-.markdown-body table tr:nth-child(2n) {
-  background-color: #f8f8f8;
-}
-
-.markdown-body img {
-  max-width: 100%;
-  box-sizing: content-box;
-  background-color: #fff;
-}
-
-.markdown-body code {
-  padding: 0;
-  padding-top: 0.2em;
-  padding-bottom: 0.2em;
-  margin: 0;
-  font-size: 85%;
-  background-color: rgba(0,0,0,0.04);
-  border-radius: 3px;
-}
-
-.markdown-body code:before,
-.markdown-body code:after {
-  letter-spacing: -0.2em;
-  content: "\00a0";
-}
-
-.markdown-body pre>code {
-  padding: 0;
-  margin: 0;
-  font-size: 100%;
-  word-break: normal;
-  white-space: pre;
-  background: transparent;
-  border: 0;
-}
-
-.markdown-body .highlight {
-  margin-bottom: 16px;
-}
-
-.markdown-body .highlight pre,
-.markdown-body pre {
-  padding: 16px;
-  overflow: auto;
-  font-size: 85%;
-  line-height: 1.45;
-  background-color: #f7f7f7;
-  border-radius: 3px;
-}
-
-.markdown-body .highlight pre {
-  margin-bottom: 0;
-  word-break: normal;
-}
-
-.markdown-body pre {
-  word-wrap: normal;
-}
-
-.markdown-body pre code {
-  display: inline;
-  max-width: initial;
-  padding: 0;
-  margin: 0;
-  overflow: initial;
-  line-height: inherit;
-  word-wrap: normal;
-  background-color: transparent;
-  border: 0;
-}
-
-.markdown-body pre code:before,
-.markdown-body pre code:after {
-  content: normal;
-}
-
-.markdown-body kbd {
-  display: inline-block;
-  padding: 3px 5px;
-  font-size: 11px;
-  line-height: 10px;
-  color: #555;
-  vertical-align: middle;
-  background-color: #fcfcfc;
-  border: solid 1px #ccc;
-  border-bottom-color: #bbb;
-  border-radius: 3px;
-  box-shadow: inset 0 -1px 0 #bbb;
-}
-
-.markdown-body .pl-c {
-  color: #969896;
-}
-
-.markdown-body .pl-c1,
-.markdown-body .pl-s .pl-v {
-  color: #0086b3;
-}
-
-.markdown-body .pl-e,
-.markdown-body .pl-en {
-  color: #795da3;
-}
-
-.markdown-body .pl-s .pl-s1,
-.markdown-body .pl-smi {
-  color: #333;
-}
-
-.markdown-body .pl-ent {
-  color: #63a35c;
-}
-
-.markdown-body .pl-k {
-  color: #a71d5d;
-}
-
-.markdown-body .pl-pds,
-.markdown-body .pl-s,
-.markdown-body .pl-s .pl-pse .pl-s1,
-.markdown-body .pl-sr,
-.markdown-body .pl-sr .pl-cce,
-.markdown-body .pl-sr .pl-sra,
-.markdown-body .pl-sr .pl-sre {
-  color: #183691;
-}
-
-.markdown-body .pl-v {
-  color: #ed6a43;
-}
-
-.markdown-body .pl-id {
-  color: #b52a1d;
-}
-
-.markdown-body .pl-ii {
-  background-color: #b52a1d;
-  color: #f8f8f8;
-}
-
-.markdown-body .pl-sr .pl-cce {
-  color: #63a35c;
-  font-weight: bold;
-}
-
-.markdown-body .pl-ml {
-  color: #693a17;
-}
-
-.markdown-body .pl-mh,
-.markdown-body .pl-mh .pl-en,
-.markdown-body .pl-ms {
-  color: #1d3e81;
-  font-weight: bold;
-}
-
-.markdown-body .pl-mq {
-  color: #008080;
-}
-
-.markdown-body .pl-mi {
-  color: #333;
-  font-style: italic;
-}
-
-.markdown-body .pl-mb {
-  color: #333;
-  font-weight: bold;
-}
-
-.markdown-body .pl-md {
-  background-color: #ffecec;
-  color: #bd2c00;
-}
-
-.markdown-body .pl-mi1 {
-  background-color: #eaffea;
-  color: #55a532;
-}
-
-.markdown-body .pl-mdr {
-  color: #795da3;
-  font-weight: bold;
-}
-
-.markdown-body .pl-mo {
-  color: #1d3e81;
-}
-
-.markdown-body kbd {
-  display: inline-block;
-  padding: 3px 5px;
-  font: 11px Consolas, "Liberation Mono", Menlo, Courier, monospace;
-  line-height: 10px;
-  color: #555;
-  vertical-align: middle;
-  background-color: #fcfcfc;
-  border: solid 1px #ccc;
-  border-bottom-color: #bbb;
-  border-radius: 3px;
-  box-shadow: inset 0 -1px 0 #bbb;
-}
-
-.markdown-body .plan-price-unit {
-  color: #767676;
-  font-weight: normal;
-}
-
-.markdown-body .task-list-item {
-  list-style-type: none;
-}
-
-.markdown-body .task-list-item+.task-list-item {
-  margin-top: 3px;
-}
-
-.markdown-body .task-list-item input {
-  margin: 0 0.35em 0.25em -1.6em;
-  vertical-align: middle;
-}
-
-.markdown-body .plan-choice {
-  padding: 15px;
-  padding-left: 40px;
-  display: block;
-  border: 1px solid #e0e0e0;
-  position: relative;
-  font-weight: normal;
-  background-color: #fafafa;
-}
-
-.markdown-body .plan-choice.open {
-  background-color: #fff;
-}
-
-.markdown-body .plan-choice.open .plan-choice-seat-breakdown {
-  display: block;
-}
-
-.markdown-body .plan-choice-free {
-  border-radius: 3px 3px 0 0;
-}
-
-.markdown-body .plan-choice-paid {
-  border-radius: 0 0 3px 3px;
-  border-top: 0;
-  margin-bottom: 20px;
-}
-
-.markdown-body .plan-choice-radio {
-  position: absolute;
-  left: 15px;
-  top: 18px;
-}
-
-.markdown-body .plan-choice-exp {
-  color: #999;
-  font-size: 12px;
-  margin-top: 5px;
-}
-
-.markdown-body .plan-choice-seat-breakdown {
-  margin-top: 10px;
-  display: none;
-}
-
-.markdown-body :checked+.radio-label {
-  z-index: 1;
-  position: relative;
-  border-color: #4078c0;
-}
-
-@media print {
-  body .markdown-body {
-    padding: 0;
-    border: none;
-  }
-}
-</style><title>README</title></head><body><article class="markdown-body"><h1>
-<a id="user-content-reticulum-network-stack-α" class="anchor" href="#reticulum-network-stack-%CE%B1" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Reticulum Network Stack α</h1>
-<p>Reticulum is a cryptography-based networking stack for high-latency, wide-area networks built on readily available hardware. Reticulum allows you to build very wide-area networks with off-the-shelf tools, and offers end-to-end encryption, autoconfiguring cryptographically backed multi-hop transport, efficient addressing, resource caching, unforgeable packet acknowledgements and much more.</p>
-<p>Reticulum is a complete networking stack, and does not use IP or higher layers, although it can be easily tunnelled through conventional IP networks. This frees up overhead, that has been utilised to implement a networking stack built directly on cryptographic principles, allowing resilience and stable functionality in open and trustless networks.</p>
-<p>No kernel modules or drivers are required. Reticulum runs completely in userland, and can run on practically any system that runs Python 3.</p>
-<p>For more info, see <a href="https://unsigned.io/projects/reticulum/" rel="nofollow">unsigned.io/projects/reticulum</a></p>
-<h2>
-<a id="user-content-where-can-reticulum-be-used" class="anchor" href="#where-can-reticulum-be-used" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Where can Reticulum be used?</h2>
-<p>On practically any hardware that can support at least a half-duplex channel with 1.000 bits per second throughput, and an MTU of 500 bytes. Data radios, modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes, free-space optical links and similar systems are all examples of the types of interfaces Reticulum was designed for.</p>
-<p>An open-source LoRa-based interface called <a href="https://unsigned.io/projects/rnode/" rel="nofollow">RNode</a> has been designed specifically for use with Reticulum. It is possible to build yourself, or can be purchased as a complete transceiver that just needs a USB connection to the host.</p>
-<p>Reticulum can also be encapsulated over existing IP networks, so there's nothing stopping you from using it over wired ethernet or your local WiFi network, where it'll work just as well. In fact, one of the strengths of Reticulum is how easily it allows you to connect different mediums into a self-configuring, resilient and encrypted mesh.</p>
-<p>As an example, it's possible to set up a Raspberry Pi connected to both a LoRa radio, a packet radio TNC and a WiFi network. Once the interfaces are configured, Reticulum will take care of the rest, and any device on the WiFi network can communicate with nodes on the LoRa and packet radio sides of the network, and vice versa.</p>
-<h2>
-<a id="user-content-current-status" class="anchor" href="#current-status" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Current Status</h2>
-<p>Consider Reticulum experimental at this stage. Most features are implemented and working, but at this point the protocol may still change significantly, and is made publicly available for development collaboration, previewing and testing.</p>
-<p>An API- and wireformat-stable alpha release is coming in the near future. Until then expect things to change unexpectedly if something warrants it.</p>
-<h2>
-<a id="user-content-what-is-implemented-at-this-point" class="anchor" href="#what-is-implemented-at-this-point" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>What is implemented at this point?</h2>
-<ul>
-<li>Adressing and identification</li>
-<li>Fully self-configuring multi-hop routing</li>
-<li>RSA assymetric encryption and signatures as basis for all communication</li>
-<li>AES-128 symmetric encryption for group destinations</li>
-<li>Elliptic curve encryption for links (on the SECP256R1 curve)</li>
-<li>Perfect Forward Secrecy on links with ephemereal ECDH keys</li>
-<li>Unforgeable packet delivery confirmations</li>
-<li>A variety of supported interface types</li>
-<li>Efficient and easy resource transfers</li>
-<li>A simple and easy-to-use API</li>
-<li>Some basic programming examples</li>
-</ul>
-<h2>
-<a id="user-content-supported-interface-types-and-devices" class="anchor" href="#supported-interface-types-and-devices" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Supported interface types and devices</h2>
-<p>Reticulum implements a range of generalised interface types that covers most of the communications hardware that Reticulum can run over. If your hardware is not supported, it's relatively simple to implement an interface class. Currently, the following interfaces are supported:</p>
-<ul>
-<li>Any ethernet device</li>
-<li>LoRa using <a href="https://unsigned.io/projects/rnode/" rel="nofollow">RNode</a>
-</li>
-<li>Packet Radio TNCs (with or without AX.25)</li>
-<li>Any device with a serial port</li>
-<li>TCP over IP networks</li>
-<li>UDP over IP networks</li>
-</ul>
-<h2>
-<a id="user-content-what-is-currently-being-worked-on" class="anchor" href="#what-is-currently-being-worked-on" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>What is currently being worked on?</h2>
-<ul>
-<li>Delay/disruption tolerant bundle transfers</li>
-<li>Useful example programs and utilities</li>
-<li>API documentation</li>
-<li>A messaging protocol built on Reticulum, see <a href="https://github.com/markqvist/lxmf">LXMF</a>
-</li>
-<li>A few useful-in-the-real-world apps built with Reticulum</li>
-</ul>
-<h2>
-<a id="user-content-can-i-use-reticulum-on-amateur-radio-spectrum" class="anchor" href="#can-i-use-reticulum-on-amateur-radio-spectrum" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Can I use Reticulum on amateur radio spectrum?</h2>
-<p>Some countries still ban the use of encryption when operating under an amateur radio license. Reticulum offers several encryptionless modes, while still using cryptographic principles for station verification, link establishment, data integrity verification, acknowledgements and routing. It is therefore perfectly possible to include Reticulum in amateur radio use, even if your country bans encryption.</p>
-<h2>
-<a id="user-content-dependencies" class="anchor" href="#dependencies" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Dependencies:</h2>
-<ul>
-<li>Python 3</li>
-<li>cryptography.io</li>
-<li>pyserial</li>
-</ul>
-<h2>
-<a id="user-content-how-do-i-get-started" class="anchor" href="#how-do-i-get-started" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>How do I get started?</h2>
-<p>Full documentation and tutorials are coming with the stable alpha release. Until then, you are mostly on your own. If you want to experiment already, you could take a look in the "Examples" folder, for some well-documented example programs. The default configuration file created by Reticulum on the first run is also worth reading. Be sure to also read the <a href="http://unsigned.io/wp-content/uploads/2018/04/Reticulum_Overview_v0.4.pdf" rel="nofollow">Reticulum Overview Document</a>.</p>
-<p>If you just need Reticulum as a dependency for another application, the easiest way is probably via pip:</p>
-<div class="highlight highlight-source-shell"><pre>pip3 install rns</pre></div>
-<p>For development, you might want to get the latest source from GitHub. In that case, don't use pip, but try this recipe:</p>
-<div class="highlight highlight-source-shell"><pre><span class="pl-c"><span class="pl-c">#</span> Install dependencies</span>
-pip3 install cryptography pyserial
-
-<span class="pl-c"><span class="pl-c">#</span> Clone repository</span>
-git clone https://github.com/markqvist/Reticulum.git
-
-<span class="pl-c"><span class="pl-c">#</span> Move into Reticulum folder and symlink library to examples folder</span>
-<span class="pl-c1">cd</span> Reticulum
-ln -s ../RNS ./Examples/
-
-<span class="pl-c"><span class="pl-c">#</span> Run an example</span>
-python3 Examples/Echo.py -s
-
-<span class="pl-c"><span class="pl-c">#</span> Unless you've manually created a config file, Reticulum will do so now,</span>
-<span class="pl-c"><span class="pl-c">#</span> and immediately exit. Make any necessary changes to the file:</span>
-nano <span class="pl-k">~</span>/.reticulum/config
-
-<span class="pl-c"><span class="pl-c">#</span> ... and launch the example again.</span>
-python3 Examples/Echo.py -s
-
-<span class="pl-c"><span class="pl-c">#</span> You can now repeat the process on another computer,</span>
-<span class="pl-c"><span class="pl-c">#</span> and run the same example with -h to get command line options.</span>
-python3 Examples/Echo.py -h
-
-<span class="pl-c"><span class="pl-c">#</span> Run the example in client mode to "ping" the server.</span>
-<span class="pl-c"><span class="pl-c">#</span> Replace the hash below with the actual destination hash of your server.</span>
-python3 Examples/Echo.py 3e12fc71692f8ec47bc5
-
-<span class="pl-c"><span class="pl-c">#</span> Have a look at another example</span>
-python3 Examples/Filetransfer.py -h</pre></div>
-<p>The default config file contains examples for using Reticulum with LoRa transceivers (specifically <a href="https://unsigned.io/projects/rnode/" rel="nofollow">RNode</a>), packet radio TNCs/modems and UDP. By default a UDP interface is already enabled in the default config, which will enable Reticulum communication in your local ethernet broadcast domain.</p>
-<p>You can use the examples in the config file to expand communication over other mediums such as packet radio or LoRa, or over fast IP links using the UDP interface. I'll add in-depth tutorials and explanations on these topics later. For now, the included examples will hopefully be enough to get started.</p>
-<h2>
-<a id="user-content-caveat-emptor" class="anchor" href="#caveat-emptor" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Caveat Emptor</h2>
-<p>Reticulum is alpha software, and should be considered experimental. While it has been built with cryptography best-practices very foremost in mind, it <em>has not</em> been externally security audited, and there could very well be privacy-breaking bugs. If you want to help out, or help sponsor an audit, please do get in touch.</p>
-</article></body></html>

README.md

@@ -1,114 +1,373 @@
-Reticulum Network Stack α
+Reticulum Network Stack β
 ==========
 
-Reticulum is a cryptography-based networking stack for high-latency, wide-area networks built on readily available hardware. Reticulum allows you to build very wide-area networks with off-the-shelf tools, and offers end-to-end encryption, autoconfiguring cryptographically backed multi-hop transport, efficient addressing, resource caching, unforgeable packet acknowledgements and much more.
+<p align="center"><img width="200" src="https://raw.githubusercontent.com/markqvist/Reticulum/master/docs/source/graphics/rns_logo_512.png"></p>
 
-Reticulum is a complete networking stack, and does not use IP or higher layers, although it can be easily tunnelled through conventional IP networks. This frees up overhead, that has been utilised to implement a networking stack built directly on cryptographic principles, allowing resilience and stable functionality in open and trustless networks.
+Reticulum is the cryptography-based networking stack for wide-area networks
+built on readily available hardware. It can operate even with very high latency
+and extremely low bandwidth. Reticulum allows you to build wide-area networks
+with off-the-shelf tools, and offers end-to-end encryption and connectivity,
+initiator anonymity, autoconfiguring cryptographically backed multi-hop
+transport, efficient addressing, unforgeable delivery acknowledgements and
+more.
 
-No kernel modules or drivers are required. Reticulum runs completely in userland, and can run on practically any system that runs Python 3.
+The vision of Reticulum is to allow anyone to be their own network operator,
+and to make it cheap and easy to cover vast areas with a myriad of independent,
+inter-connectable and autonomous networks. Reticulum **is not** *one* network.
+It is **a tool** for building *thousands of networks*. Networks without
+kill-switches, surveillance, censorship and control. Networks that can freely
+interoperate, associate and disassociate with each other, and require no
+central oversight. Networks for human beings. *Networks for the people*.
+
+Reticulum is a complete networking stack, and does not rely on IP or higher
+layers, but it is possible to use IP as the underlying carrier for Reticulum.
+It is therefore trivial to tunnel Reticulum over the Internet or private IP
+networks.
+
+Having no dependencies on traditional networking stacks frees up overhead that
+has been used to implement a networking stack built directly on cryptographic
+principles, allowing resilience and stable functionality, even in open and
+trustless networks.
+
+No kernel modules or drivers are required. Reticulum runs completely in
+userland, and can run on practically any system that runs Python 3.
+
+## Read The Manual
+The full documentation for Reticulum is available at [markqvist.github.io/Reticulum/manual/](https://markqvist.github.io/Reticulum/manual/).
+
+You can also [download the Reticulum manual as a PDF](https://github.com/markqvist/Reticulum/raw/master/docs/Reticulum%20Manual.pdf)
 
 For more info, see [unsigned.io/projects/reticulum](https://unsigned.io/projects/reticulum/)
 
+## Notable Features
+- Coordination-less globally unique addressing and identification
+- Fully self-configuring multi-hop routing
+- Complete initiator anonymity, communicate without revealing your identity
+- Asymmetric X25519 encryption and Ed25519 signatures as a basis for all communication
+- Forward Secrecy with ephemeral Elliptic Curve Diffie-Hellman keys on Curve25519
+- Reticulum uses the [Fernet](https://github.com/fernet/spec/blob/master/Spec.md) specification for on-the-wire / over-the-air encryption
+  - Keys are ephemeral and derived from an ECDH key exchange on Curve25519
+  - AES-128 in CBC mode with PKCS7 padding
+  - HMAC using SHA256 for authentication
+  - IVs are generated through os.urandom()
+- Unforgeable packet delivery confirmations
+- A variety of supported interface types
+- An intuitive and easy-to-use API
+- Reliable and efficient transfer of arbitrary amounts of data
+  - Reticulum can handle a few bytes of data or files of many gigabytes
+  - Sequencing, transfer coordination and checksumming are automatic
+  - The API is very easy to use, and provides transfer progress
+- Lightweight, flexible and expandable Request/Response mechanism
+- Efficient link establishment
+  - Total bandwidth cost of setting up an encrypted link is 3 packets totaling 297 bytes
+  - Low cost of keeping links open at only 0.44 bits per second
+
+## Examples of Reticulum Applications
+If you want to quickly get an idea of what Reticulum can do, take a look at the
+following resources.
+
+- [LXMF](https://github.com/markqvist/lxmf) is a distributed, delay and disruption tolerant message transfer protocol built on Reticulum
+- For an off-grid, encrypted and resilient mesh communications platform, see [Nomad Network](https://github.com/markqvist/NomadNet)
+- The Android, Linux and macOS app [Sideband](https://unsigned.io/sideband) has a graphical interface and focuses on ease of use.
+
 ## Where can Reticulum be used?
-On practically any hardware that can support at least a half-duplex channel with 1.000 bits per second throughput, and an MTU of 500 bytes. Data radios, modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes, free-space optical links and similar systems are all examples of the types of interfaces Reticulum was designed for.
+Over practically any medium that can support at least a half-duplex channel
+with 500 bits per second throughput, and an MTU of 500 bytes. Data radios,
+modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes,
+WiFi and Ethernet devices, free-space optical links, and similar systems are
+all examples of the types of physical devices Reticulum can use.
 
-An open-source LoRa-based interface called [RNode](https://unsigned.io/projects/rnode/) has been designed specifically for use with Reticulum. It is possible to build yourself, or can be purchased as a complete transceiver that just needs a USB connection to the host.
+An open-source LoRa-based interface called
+[RNode](https://markqvist.github.io/Reticulum/manual/hardware.html#rnode) has
+been designed specifically for use with Reticulum. It is possible to build
+yourself, or it can be purchased as a complete transceiver that just needs a
+USB connection to the host.
 
-Reticulum can also be encapsulated over existing IP networks, so there's nothing stopping you from using it over wired ethernet or your local WiFi network, where it'll work just as well. In fact, one of the strengths of Reticulum is how easily it allows you to connect different mediums into a self-configuring, resilient and encrypted mesh.
+Reticulum can also be encapsulated over existing IP networks, so there's
+nothing stopping you from using it over wired Ethernet, your local WiFi network
+or the Internet, where it'll work just as well. In fact, one of the strengths
+of Reticulum is how easily it allows you to connect different mediums into a
+self-configuring, resilient and encrypted mesh, using any available mixture of
+available infrastructure.
 
-As an example, it's possible to set up a Raspberry Pi connected to both a LoRa radio, a packet radio TNC and a WiFi network. Once the interfaces are configured, Reticulum will take care of the rest, and any device on the WiFi network can communicate with nodes on the LoRa and packet radio sides of the network, and vice versa.
-
-## Current Status
-Consider Reticulum experimental at this stage. Most features are implemented and working, but at this point the protocol may still change significantly, and is made publicly available for development collaboration, previewing and testing.
-
-An API- and wireformat-stable alpha release is coming in the near future. Until then expect things to change unexpectedly if something warrants it.
-
-## What is implemented at this point?
- - Adressing and identification
- - Fully self-configuring multi-hop routing
- - RSA assymetric encryption and signatures as basis for all communication
- - AES-128 symmetric encryption for group destinations
- - Elliptic curve encryption for links (on the SECP256R1 curve)
- - Perfect Forward Secrecy on links with ephemereal ECDH keys
- - Unforgeable packet delivery confirmations
- - A variety of supported interface types
- - Efficient and easy resource transfers
- - A simple and easy-to-use API
- - Some basic programming examples
-
-## Supported interface types and devices
-
-Reticulum implements a range of generalised interface types that covers most of the communications hardware that Reticulum can run over. If your hardware is not supported, it's relatively simple to implement an interface class. Currently, the following interfaces are supported:
-
- - Any ethernet device
- - LoRa using [RNode](https://unsigned.io/projects/rnode/)
- - Packet Radio TNCs (with or without AX.25)
- - Any device with a serial port
- - TCP over IP networks
- - UDP over IP networks
-
-## What is currently being worked on?
- - Delay/disruption tolerant bundle transfers
- - Useful example programs and utilities
- - API documentation
- - A messaging protocol built on Reticulum, see [LXMF](https://github.com/markqvist/lxmf)
- - A few useful-in-the-real-world apps built with Reticulum
-
-## Can I use Reticulum on amateur radio spectrum?
-Some countries still ban the use of encryption when operating under an amateur radio license. Reticulum offers several encryptionless modes, while still using cryptographic principles for station verification, link establishment, data integrity verification, acknowledgements and routing. It is therefore perfectly possible to include Reticulum in amateur radio use, even if your country bans encryption.
-
-## Dependencies:
- - Python 3
- - cryptography.io
- - pyserial
+As an example, it's possible to set up a Raspberry Pi connected to both a LoRa
+radio, a packet radio TNC and a WiFi network. Once the interfaces are
+configured, Reticulum will take care of the rest, and any device on the WiFi
+network can communicate with nodes on the LoRa and packet radio sides of the
+network, and vice versa.
 
 ## How do I get started?
-Full documentation and tutorials are coming with the stable alpha release. Until then, you are mostly on your own. If you want to experiment already, you could take a look in the "Examples" folder, for some well-documented example programs. The default configuration file created by Reticulum on the first run is also worth reading. Be sure to also read the [Reticulum Overview Document](http://unsigned.io/wp-content/uploads/2018/04/Reticulum_Overview_v0.4.pdf).
+The best way to get started with the Reticulum Network Stack depends on what
+you want to do. For full details and examples, have a look at the 
+[Getting Started Fast](https://markqvist.github.io/Reticulum/manual/gettingstartedfast.html) 
+section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
 
-If you just need Reticulum as a dependency for another application, the easiest way is probably via pip:
+To simply install Reticulum and related utilities on your system, the easiest way is via pip:
 
 ```bash
 pip3 install rns
 ```
 
-For development, you might want to get the latest source from GitHub. In that case, don't use pip, but try this recipe:
+You can then start any program that uses Reticulum, or start Reticulum as a
+system service with [the rnsd utility](https://markqvist.github.io/Reticulum/manual/using.html#the-rnsd-utility).
 
-```bash
-# Install dependencies
-pip3 install cryptography pyserial
+When first started, Reticulum will create a default configuration file,
+providing basic connectivity to other Reticulum peers that might be locally
+reachable. The default config file contains a few examples, and references for
+creating a more complex configuration.
 
-# Clone repository
-git clone https://github.com/markqvist/Reticulum.git
+For more detailed examples on how to expand communication over many mediums such 
+as packet radio or LoRa, serial ports, or over fast IP links and the Internet using 
+the UDP and TCP interfaces, take a look at the [Supported Interfaces](https://markqvist.github.io/Reticulum/manual/interfaces.html) 
+section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
 
-# Move into Reticulum folder and symlink library to examples folder
-cd Reticulum
-ln -s ../RNS ./Examples/
+## Included Utilities
+Reticulum includes a range of useful utilities for managing your networks, 
+viewing status and information, and other tasks. You can read more about these 
+programs in the [Included Utility Programs](https://markqvist.github.io/Reticulum/manual/using.html#included-utility-programs) 
+section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
 
-# Run an example
-python3 Examples/Echo.py -s
+- The system daemon `rnsd` for running Reticulum as an always-available service
+- An interface status utility called `rnstatus`, that displays information about interfaces
+- The path lookup and management tool `rnpath` letting you view and modify path tables
+- A diagnostics tool called `rnprobe` for checking connectivity to destinations
+- A simple file transfer program called `rncp` making it easy to copy files to remote systems
+- The remote command execution program `rnx` let's you run commands and
+  programs and retrieve output from remote systems
 
-# Unless you've manually created a config file, Reticulum will do so now,
-# and immediately exit. Make any necessary changes to the file:
-nano ~/.reticulum/config
+All tools, including `rnx` and `rncp`, work reliably and well even over very
+low-bandwidth links like LoRa or Packet Radio.
 
-# ... and launch the example again.
-python3 Examples/Echo.py -s
+## Supported interface types and devices
 
-# You can now repeat the process on another computer,
-# and run the same example with -h to get command line options.
-python3 Examples/Echo.py -h
+Reticulum implements a range of generalised interface types that covers most of
+the communications hardware that Reticulum can run over. If your hardware is
+not supported, it's relatively simple to implement an interface class. I will
+gratefully accept pull requests for custom interfaces if they are generally
+useful.
 
-# Run the example in client mode to "ping" the server.
-# Replace the hash below with the actual destination hash of your server.
-python3 Examples/Echo.py 3e12fc71692f8ec47bc5
+Currently, the following interfaces are supported:
 
-# Have a look at another example
-python3 Examples/Filetransfer.py -h
+- Any Ethernet device
+- LoRa using [RNode](https://unsigned.io/projects/rnode/)
+- Packet Radio TNCs (with or without AX.25)
+- KISS-compatible hardware and software modems
+- Any device with a serial port
+- TCP over IP networks
+- UDP over IP networks
+- External programs via stdio or pipes
+- Custom hardware via stdio or pipes
+
+## Performance
+Reticulum targets a *very* wide usable performance envelope, but prioritises
+functionality and performance over low-bandwidth mediums. The goal is to
+provide a dynamic performance envelope from 250 bits per second, to 1 gigabit
+per second on normal hardware.
+
+Currently, the usable performance envelope is approximately 500 bits per second
+to 20 megabits per second, with physical mediums faster than that not being
+saturated. Performance beyond the current level is intended for future
+upgrades, but not highly prioritised at this point in time.
+
+## Current Status
+Reticulum should currently be considered beta software. All core protocol
+features are implemented and functioning, but additions will probably occur as
+real-world use is explored. There will be bugs. The API and wire-format can be
+considered relatively stable at the moment, but could change if warranted.
+
+## Development Roadmap
+- Version 0.4.0
+  - Improving [the manual](https://markqvist.github.io/Reticulum/manual/) with sections specifically for beginners
+  - Performance and memory optimisations
+  - Utilities for managing identities, signing and encryption
+  - User friendly interface configuration tool
+  - Support for radio and modem interfaces on Android
+  - More interface types for even broader compatibility
+    - Plain ESP32 devices (ESP-Now, WiFi, Bluetooth, etc.)
+    - More LoRa transceivers
+    - IR Transceivers
+- Planned, but not yet scheduled
+  - OpenWRT support
+  - Metric-based path selection
+  - Distributed Destination Naming System
+  - Network-wide path balancing
+  - Globally routable multicast
+  - Bindings for other programming languages
+  - Multiple paths in path table for quick recovery on link failures
+  - A portable Reticulum implementation in C, see [#21](https://github.com/markqvist/Reticulum/discussions/21)
+  - Easy way to share interface configurations, see [#19](https://github.com/markqvist/Reticulum/discussions/19)
+  - More interface types
+    - AT-compatible modems
+    - AWDL / OWL
+    - HF Modems
+    - CAN-bus
+    - ZeroMQ
+    - MQTT
+    - XBee
+    - SPI
+    - i²c
+    - Tor
+
+## Dependencies
+The installation of the default `rns` package requires the dependencies listed
+below. Almost all systems and distributions have readily available packages for
+these dependencies, and when the `rns` package is installed with `pip`, they
+will be downloaded and installed as well.
+
+- [PyCA/cryptography](https://github.com/pyca/cryptography)
+- [netifaces](https://github.com/al45tair/netifaces)
+- [pyserial](https://github.com/pyserial/pyserial)
+
+On more unusual systems, and in some rare cases, it might not be possible to
+install or even compile one or more of the above modules. In such situations,
+you can use the `rnspure` package instead, which require no external
+dependencies for installation. Please note that the contents of the `rns` and
+`rnspure` packages are *identical*. The only difference is that the `rnspure`
+package lists no dependencies required for installation.
+
+No matter how Reticulum is installed and started, it will load external
+dependencies only if they are *needed* and *available*. If for example you want
+to use Reticulum on a system that cannot support
+[pyserial](https://github.com/pyserial/pyserial), it is perfectly possible to
+do so using the `rnspure` package, but Reticulum will not be able to use
+serial-based interfaces. All other available modules will still be loaded when
+needed.
+
+**Please Note!** If you use the `rnspure` package to run Reticulum on systems
+that do not support [PyCA/cryptography](https://github.com/pyca/cryptography),
+it is important that you read and understand the [Cryptographic
+Primitives](#cryptographic-primitives) section of this document.
+
+## Public Testnet
+If you just want to get started experimenting without building any physical
+networks, you are welcome to join the Unsigned.io RNS Testnet. The testnet is
+just that, an informal network for testing and experimenting. It will be up
+most of the time, and anyone can join, but it also means that there's no
+guarantees for service availability.
+
+The testnet runs the very latest version of Reticulum (often even a short while
+before it is publicly released). Sometimes experimental versions of Reticulum
+might be deployed to nodes on the testnet, which means strange behaviour might
+occur. If none of that scares you, you can join the testnet via either TCP or
+I2P. Just add one of the following interfaces to your Reticulum configuration
+file:
+
+```
+# TCP/IP interface to the Dublin Hub
+  [[RNS Testnet Dublin]]
+    type = TCPClientInterface
+    enabled = yes
+    target_host = dublin.connect.reticulum.network
+    target_port = 4965
+
+# TCP/IP interface to the Frankfurt Hub
+  [[RNS Testnet Frankfurt]]
+    type = TCPClientInterface
+    enabled = yes
+    target_host = frankfurt.connect.reticulum.network
+    target_port = 5377
+
+# Interface to I2P Hub A
+  [[RNS Testnet I2P Hub A]]
+    type = I2PInterface
+    enabled = yes
+    peers = mrwqlsioq4hoo2lmeeud7dkfscnm7yxak7dmiyvsrnpfag3z5tsq.b32.i2p
+
+# Interface to I2P Hub B
+  [[RNS Testnet I2P Hub B]]
+    type = I2PInterface
+    enabled = yes
+    peers = iwoqtz22dsr73aemwpw7guocplsjjoamyl7sogj33qtcd6ds4mza.b32.i2p
 ```
 
-The default config file contains examples for using Reticulum with LoRa transceivers (specifically [RNode](https://unsigned.io/projects/rnode/)), packet radio TNCs/modems and UDP. By default a UDP interface is already enabled in the default config, which will enable Reticulum communication in your local ethernet broadcast domain.
+The testnet also contains a number of [Nomad Network](https://github.com/markqvist/nomadnet) nodes, and LXMF propagation nodes.
 
-You can use the examples in the config file to expand communication over other mediums such as packet radio or LoRa, or over fast IP links using the UDP interface. I'll add in-depth tutorials and explanations on these topics later. For now, the included examples will hopefully be enough to get started.
+## Support Reticulum
+You can help support the continued development of open, free and private communications systems by donating via one of the following channels:
 
-## Caveat Emptor
-Reticulum is alpha software, and should be considered experimental. While it has been built with cryptography best-practices very foremost in mind, it _has not_ been externally security audited, and there could very well be privacy-breaking bugs. If you want to help out, or help sponsor an audit, please do get in touch.
+- Monero:
+ ```
+  84FpY1QbxHcgdseePYNmhTHcrgMX4nFfBYtz2GKYToqHVVhJp8Eaw1Z1EedRnKD19b3B8NiLCGVxzKV17UMmmeEsCrPyA5w
+  ```
+- Ethereum
+  ```
+  0x81F7B979fEa6134bA9FD5c701b3501A2e61E897a
+  ```
+- Bitcoin
+  ```
+  3CPmacGm34qYvR6XWLVEJmi2aNe3PZqUuq
+  ```
+- Ko-Fi: https://ko-fi.com/markqvist
+
+Are certain features in the development roadmap are important to you or your
+organisation? Make them a reality quickly by sponsoring their implementation.
+
+## Cryptographic Primitives
+Reticulum uses a simple suite of efficient, strong and modern cryptographic
+primitives, with widely available implementations that can be used both on
+general-purpose CPUs and on microcontrollers. The necessary primitives are:
+
+- Ed25519 for signatures
+- X22519 for ECDH key exchanges
+- HKDF for key derivation
+- Modified Fernet for encrypted tokens
+  - AES-128 in CBC mode
+  - HMAC for message authentication
+  - No Fernet version and timestamp fields
+- SHA-256
+- SHA-512
+
+In the default installation configuration, the `X25519`, `Ed25519` and
+`AES-128-CBC` primitives are provided by [OpenSSL](https://www.openssl.org/)
+(via the [PyCA/cryptography](https://github.com/pyca/cryptography) package).
+The hashing functions `SHA-256` and `SHA-512` are provided by the standard
+Python [hashlib](https://docs.python.org/3/library/hashlib.html). The `HKDF`,
+`HMAC`, `Fernet` primitives, and the `PKCS7` padding function are always
+provided by the following internal implementations:
+
+- [HKDF.py](RNS/Cryptography/HKDF.py)
+- [HMAC.py](RNS/Cryptography/HMAC.py)
+- [Fernet.py](RNS/Cryptography/Fernet.py)
+- [PKCS7.py](RNS/Cryptography/PKCS7.py)
+
+
+Reticulum also includes a complete implementation of all necessary primitives
+in pure Python. If OpenSSL & PyCA are not available on the system when
+Reticulum is started, Reticulum will instead use the internal pure-python
+primitives. A trivial consequence of this is performance, with the OpenSSL
+backend being *much* faster. The most important consequence however, is the
+potential loss of security by using primitives that has not seen the same
+amount of scrutiny, testing and review as those from OpenSSL.
+
+If you want to use the internal pure-python primitives, it is **highly
+advisable** that you have a good understanding of the risks that this pose, and
+make an informed decision on whether those risks are acceptable to you.
+
+Reticulum is relatively young software, and should be considered as such. While
+it has been built with cryptography best-practices very foremost in mind, it
+_has not_ been externally security audited, and there could very well be
+privacy or security breaking bugs. If you want to help out, or help sponsor an
+audit, please do get in touch.
+
+## Acknowledgements & Credits
+Reticulum can only exist because of the mountain of Open Source work it was
+built on top of, the contributions of everyone involved, and everyone that has
+supported the project through the years. To everyone who has helped, thank you
+so much.
+
+A number of other modules and projects are either part of, or used by
+Reticulum. Sincere thanks to the authors and contributors of the following
+projects:
+
+- [PyCA/cryptography](https://github.com/pyca/cryptography), *BSD License*
+- [Pure-25519](https://github.com/warner/python-pure25519) by [Brian Warner](https://github.com/warner), *MIT License*
+- [Pysha2](https://github.com/thomdixon/pysha2) by [Thom Dixon](https://github.com/thomdixon), *MIT License*
+- [Python-AES](https://github.com/orgurar/python-aes) by [Or Gur Arie](https://github.com/orgurar), *MIT License*
+- [Curve25519.py](https://gist.github.com/nickovs/cc3c22d15f239a2640c185035c06f8a3#file-curve25519-py) by [Nicko van Someren](https://gist.github.com/nickovs), *Public Domain*
+- [I2Plib](https://github.com/l-n-s/i2plib) by [Viktor Villainov](https://github.com/l-n-s)
+- [PySerial](https://github.com/pyserial/pyserial) by Chris Liechti, *BSD License*
+- [Netifaces](https://github.com/al45tair/netifaces) by [Alastair Houghton](https://github.com/al45tair), *MIT License*
+- [Configobj](https://github.com/DiffSK/configobj) by Michael Foord, Nicola Larosa, Rob Dennis & Eli Courtwright, *BSD License*
+- [Six](https://github.com/benjaminp/six) by [Benjamin Peterson](https://github.com/benjaminp), *MIT License*
+- [Umsgpack.py](https://github.com/vsergeev/u-msgpack-python) by [Ivan A. Sergeev](https://github.com/vsergeev)
+- [Python](https://www.python.org)

RNS/Cryptography/AES.py

@@ -0,0 +1,68 @@
+# MIT License
+#
+# Copyright (c) 2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import RNS.Cryptography.Provider as cp
+import RNS.vendor.platformutils as pu
+
+if cp.PROVIDER == cp.PROVIDER_INTERNAL:
+    from .aes import AES
+    
+elif cp.PROVIDER == cp.PROVIDER_PYCA:
+    from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
+
+    if pu.cryptography_old_api():
+        from cryptography.hazmat.backends import default_backend
+
+
+class AES_128_CBC:
+
+    @staticmethod
+    def encrypt(plaintext, key, iv):
+        if cp.PROVIDER == cp.PROVIDER_INTERNAL:
+            cipher = AES(key)
+            return cipher.encrypt(plaintext, iv)
+
+        elif cp.PROVIDER == cp.PROVIDER_PYCA:
+            if not pu.cryptography_old_api():
+                cipher = Cipher(algorithms.AES(key), modes.CBC(iv))
+            else:
+                cipher = Cipher(algorithms.AES(key), modes.CBC(iv), backend=default_backend())
+            
+            encryptor = cipher.encryptor()
+            ciphertext = encryptor.update(plaintext) + encryptor.finalize()
+            return ciphertext
+
+    @staticmethod
+    def decrypt(ciphertext, key, iv):
+        if cp.PROVIDER == cp.PROVIDER_INTERNAL:
+            cipher = AES(key)
+            return cipher.decrypt(ciphertext, iv)
+
+        elif cp.PROVIDER == cp.PROVIDER_PYCA:
+            if not pu.cryptography_old_api():
+                cipher = Cipher(algorithms.AES(key), modes.CBC(iv))
+            else:
+                cipher = Cipher(algorithms.AES(key), modes.CBC(iv), backend=default_backend())
+
+            decryptor = cipher.decryptor()
+            plaintext = decryptor.update(ciphertext) + decryptor.finalize()
+            return plaintext

RNS/Cryptography/Ed25519.py

@@ -0,0 +1,41 @@
+import os
+from .pure25519 import ed25519_oop as ed25519
+
+class Ed25519PrivateKey:
+    def __init__(self, seed):
+        self.seed = seed
+        self.sk = ed25519.SigningKey(self.seed)
+        #self.vk = self.sk.get_verifying_key()
+
+    @classmethod
+    def generate(cls):
+        return cls.from_private_bytes(os.urandom(32))
+
+    @classmethod
+    def from_private_bytes(cls, data):
+        return cls(seed=data)
+
+    def private_bytes(self):
+        return self.seed
+
+    def public_key(self):
+        return Ed25519PublicKey.from_public_bytes(self.sk.vk_s)
+
+    def sign(self, message):
+        return self.sk.sign(message)
+
+
+class Ed25519PublicKey:
+    def __init__(self, seed):
+        self.seed = seed
+        self.vk = ed25519.VerifyingKey(self.seed)
+
+    @classmethod
+    def from_public_bytes(cls, data):
+        return cls(data)
+
+    def public_bytes(self):
+        return self.vk.to_bytes()
+
+    def verify(self, signature, message):
+        self.vk.verify(signature, message)

RNS/Cryptography/Fernet.py

@@ -0,0 +1,110 @@
+# MIT License
+#
+# Copyright (c) 2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import os
+import time
+
+from RNS.Cryptography import HMAC
+from RNS.Cryptography import PKCS7
+from RNS.Cryptography.AES import AES_128_CBC
+
+class Fernet():
+    """
+    This class provides a slightly modified implementation of the Fernet spec
+    found at: https://github.com/fernet/spec/blob/master/Spec.md
+
+    According to the spec, a Fernet token includes a one byte VERSION and
+    eight byte TIMESTAMP field at the start of each token. These fields are
+    not relevant to Reticulum. They are therefore stripped from this
+    implementation, since they incur overhead and leak initiator metadata.
+    """
+    FERNET_OVERHEAD  = 48 # Bytes
+
+    @staticmethod
+    def generate_key():
+        return os.urandom(32)
+
+    def __init__(self, key = None):
+        if key == None:
+            raise ValueError("Fernet key cannot be None")
+
+        if len(key) != 32:
+            raise ValueError("Fernet key must be 32 bytes, not "+str(len(key)))
+            
+        self._signing_key = key[:16]
+        self._encryption_key = key[16:]
+
+
+    def verify_hmac(self, token):
+        if len(token) <= 32:
+            raise ValueError("Cannot verify HMAC on token of only "+str(len(token))+" bytes")
+        else:
+            received_hmac = token[-32:]
+            expected_hmac = HMAC.new(self._signing_key, token[:-32]).digest()
+
+            if received_hmac == expected_hmac:
+                return True
+            else:
+                return False
+
+
+    def encrypt(self, data = None):
+        iv = os.urandom(16)
+        current_time = int(time.time())
+
+        if not isinstance(data, bytes):
+            raise TypeError("Fernet token plaintext input must be bytes")
+
+        ciphertext = AES_128_CBC.encrypt(
+            plaintext = PKCS7.pad(data),
+            key = self._encryption_key,
+            iv = iv,
+        )
+
+        signed_parts = iv+ciphertext
+
+        return signed_parts + HMAC.new(self._signing_key, signed_parts).digest()
+
+
+    def decrypt(self, token = None):
+        if not isinstance(token, bytes):
+            raise TypeError("Fernet token must be bytes")
+
+        if not self.verify_hmac(token):
+            raise ValueError("Fernet token HMAC was invalid")
+
+        iv = token[:16]
+        ciphertext = token[16:-32]
+
+        try:
+            plaintext = PKCS7.unpad(
+                AES_128_CBC.decrypt(
+                    ciphertext,
+                    self._encryption_key,
+                    iv,
+                )
+            )
+
+            return plaintext
+
+        except Exception as e:
+            raise ValueError("Could not decrypt Fernet token")

RNS/Cryptography/HKDF.py

@@ -0,0 +1,57 @@
+# MIT License
+#
+# Copyright (c) 2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import hashlib
+from math import ceil
+from RNS.Cryptography import HMAC
+
+def hkdf(length=None, derive_from=None, salt=None, context=None):
+    hash_len = 32
+
+    def hmac_sha256(key, data):
+        return HMAC.new(key, data).digest()
+
+    if length == None or length < 1:
+        raise ValueError("Invalid output key length")
+
+    if derive_from == "None" or derive_from == "":
+        raise ValueError("Cannot derive key from empty input material")
+
+    if salt == None or len(salt) == 0:
+        salt = bytes([0] * hash_len)
+
+    if salt == None:
+        salt = b""
+
+    if context == None:
+        context = b""
+
+    pseudorandom_key = hmac_sha256(salt, derive_from)
+
+    block = b""
+    derived = b""
+
+    for i in range(ceil(length / hash_len)):
+        block = hmac_sha256(pseudorandom_key, block + context + bytes([i + 1]))
+        derived += block
+
+    return derived[:length]

RNS/Cryptography/HMAC.py

@@ -0,0 +1,183 @@
+# This HMAC implementation comes directly from the HMAC implementation
+# included in Python 3.10.4, and is almost completely identical. It has
+# been modified to be a pure Python implementation, that is not dependent
+# on the system having OpenSSL binaries installed.
+
+import warnings as _warnings
+import hashlib as _hashlib
+
+trans_5C = bytes((x ^ 0x5C) for x in range(256))
+trans_36 = bytes((x ^ 0x36) for x in range(256))
+
+# The size of the digests returned by HMAC depends on the underlying
+# hashing module used.  Use digest_size from the instance of HMAC instead.
+digest_size = None
+
+
+class HMAC:
+    """RFC 2104 HMAC class.  Also complies with RFC 4231.
+    This supports the API for Cryptographic Hash Functions (PEP 247).
+    """
+    blocksize = 64  # 512-bit HMAC; can be changed in subclasses.
+
+    __slots__ = (
+        "_hmac", "_inner", "_outer", "block_size", "digest_size"
+    )
+
+    def __init__(self, key, msg=None, digestmod=_hashlib.sha256):
+        """Create a new HMAC object.
+        key: bytes or buffer, key for the keyed hash object.
+        msg: bytes or buffer, Initial input for the hash or None.
+        digestmod: A hash name suitable for hashlib.new(). *OR*
+                   A hashlib constructor returning a new hash object. *OR*
+                   A module supporting PEP 247.
+                   Required as of 3.8, despite its position after the optional
+                   msg argument.  Passing it as a keyword argument is
+                   recommended, though not required for legacy API reasons.
+        """
+
+        if not isinstance(key, (bytes, bytearray)):
+            raise TypeError("key: expected bytes or bytearray, but got %r" % type(key).__name__)
+
+        if not digestmod:
+            raise TypeError("Missing required parameter 'digestmod'.")
+
+        self._hmac_init(key, msg, digestmod)
+
+    def _hmac_init(self, key, msg, digestmod):
+        if callable(digestmod):
+            digest_cons = digestmod
+        elif isinstance(digestmod, str):
+            digest_cons = lambda d=b'': _hashlib.new(digestmod, d)
+        else:
+            digest_cons = lambda d=b'': digestmod.new(d)
+
+        self._hmac = None
+        self._outer = digest_cons()
+        self._inner = digest_cons()
+        self.digest_size = self._inner.digest_size
+
+        if hasattr(self._inner, 'block_size'):
+            blocksize = self._inner.block_size
+            if blocksize < 16:
+                _warnings.warn('block_size of %d seems too small; using our '
+                               'default of %d.' % (blocksize, self.blocksize),
+                               RuntimeWarning, 2)
+                blocksize = self.blocksize
+        else:
+            _warnings.warn('No block_size attribute on given digest object; '
+                           'Assuming %d.' % (self.blocksize),
+                           RuntimeWarning, 2)
+            blocksize = self.blocksize
+
+        if len(key) > blocksize:
+            key = digest_cons(key).digest()
+
+        # self.blocksize is the default blocksize. self.block_size is
+        # effective block size as well as the public API attribute.
+        self.block_size = blocksize
+
+        key = key.ljust(blocksize, b'\0')
+        self._outer.update(key.translate(trans_5C))
+        self._inner.update(key.translate(trans_36))
+        if msg is not None:
+            self.update(msg)
+
+    @property
+    def name(self):
+        if self._hmac:
+            return self._hmac.name
+        else:
+            return f"hmac-{self._inner.name}"
+
+    def update(self, msg):
+        """Feed data from msg into this hashing object."""
+        inst = self._hmac or self._inner
+        inst.update(msg)
+
+    def copy(self):
+        """Return a separate copy of this hashing object.
+        An update to this copy won't affect the original object.
+        """
+        # Call __new__ directly to avoid the expensive __init__.
+        other = self.__class__.__new__(self.__class__)
+        other.digest_size = self.digest_size
+        if self._hmac:
+            other._hmac = self._hmac.copy()
+            other._inner = other._outer = None
+        else:
+            other._hmac = None
+            other._inner = self._inner.copy()
+            other._outer = self._outer.copy()
+        return other
+
+    def _current(self):
+        """Return a hash object for the current state.
+        To be used only internally with digest() and hexdigest().
+        """
+        if self._hmac:
+            return self._hmac
+        else:
+            h = self._outer.copy()
+            h.update(self._inner.digest())
+            return h
+
+    def digest(self):
+        """Return the hash value of this hashing object.
+        This returns the hmac value as bytes.  The object is
+        not altered in any way by this function; you can continue
+        updating the object after calling this function.
+        """
+        h = self._current()
+        return h.digest()
+
+    def hexdigest(self):
+        """Like digest(), but returns a string of hexadecimal digits instead.
+        """
+        h = self._current()
+        return h.hexdigest()
+
+def new(key, msg=None, digestmod=_hashlib.sha256):
+    """Create a new hashing object and return it.
+    key: bytes or buffer, The starting key for the hash.
+    msg: bytes or buffer, Initial input for the hash, or None.
+    digestmod: A hash name suitable for hashlib.new(). *OR*
+               A hashlib constructor returning a new hash object. *OR*
+               A module supporting PEP 247.
+               Required as of 3.8, despite its position after the optional
+               msg argument.  Passing it as a keyword argument is
+               recommended, though not required for legacy API reasons.
+    You can now feed arbitrary bytes into the object using its update()
+    method, and can ask for the hash value at any time by calling its digest()
+    or hexdigest() methods.
+    """
+    return HMAC(key, msg, digestmod)
+
+
+def digest(key, msg, digest):
+    """Fast inline implementation of HMAC.
+    key: bytes or buffer, The key for the keyed hash object.
+    msg: bytes or buffer, Input message.
+    digest: A hash name suitable for hashlib.new() for best performance. *OR*
+            A hashlib constructor returning a new hash object. *OR*
+            A module supporting PEP 247.
+    """
+    if callable(digest):
+        digest_cons = digest
+    elif isinstance(digest, str):
+        digest_cons = lambda d=b'': _hashlib.new(digest, d)
+    else:
+        digest_cons = lambda d=b'': digest.new(d)
+
+    inner = digest_cons()
+    outer = digest_cons()
+    blocksize = getattr(inner, 'block_size', 64)
+    if len(key) > blocksize:
+        key = digest_cons(key).digest()
+
+    key = key + b'\x00' * (blocksize - len(key))
+    inner.update(key.translate(trans_36))
+    outer.update(key.translate(trans_5C))
+    inner.update(msg)
+    outer.update(inner.digest())
+    return outer.digest()

RNS/Cryptography/Hashes.py

@@ -0,0 +1,34 @@
+import importlib
+if importlib.util.find_spec('hashlib') != None:
+    import hashlib
+else:
+    hashlib = None
+
+if hasattr(hashlib, "sha512"):
+    from hashlib import sha512 as ext_sha512
+else:
+    from .SHA512 import sha512 as ext_sha512
+
+if hasattr(hashlib, "sha256"):
+    from hashlib import sha256 as ext_sha256
+else:
+    from .SHA256 import sha256 as ext_sha256
+
+"""
+The SHA primitives are abstracted here to allow platform-
+aware hardware acceleration in the future. Currently only
+uses Python's internal SHA-256 implementation. All SHA-256
+calls in RNS end up here.
+"""
+
+def sha256(data):
+    digest = ext_sha256()
+    digest.update(data)
+
+    return digest.digest()
+
+def sha512(data):
+    digest = ext_sha512()
+    digest.update(data)
+
+    return digest.digest()

RNS/Cryptography/PKCS7.py

@@ -0,0 +1,40 @@
+# MIT License
+#
+# Copyright (c) 2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+class PKCS7:
+    BLOCKSIZE = 16
+
+    @staticmethod
+    def pad(data, bs=BLOCKSIZE):
+        l = len(data)
+        n = bs-l%bs
+        v = bytes([n])
+        return data+v*n
+
+    @staticmethod
+    def unpad(data, bs=BLOCKSIZE):
+        l = len(data)
+        n = data[-1]
+        if n > bs:
+            raise ValueError("Cannot unpad, invalid padding length of "+str(n)+" bytes")
+        else:
+            return data[:l-n]

RNS/Cryptography/Provider.py

@@ -0,0 +1,38 @@
+import importlib
+
+PROVIDER_NONE     = 0x00
+PROVIDER_INTERNAL = 0x01
+PROVIDER_PYCA     = 0x02
+
+PROVIDER = PROVIDER_NONE
+
+pyca_v = None
+use_pyca = False
+
+try:
+    if importlib.util.find_spec('cryptography') != None:
+        import cryptography
+        pyca_v = cryptography.__version__
+        v = pyca_v.split(".")
+
+        if int(v[0]) == 2:
+            if int(v[1]) >= 8:
+                use_pyca = True
+        elif int(v[0]) >= 3:
+            use_pyca = True
+
+except Exception as e:
+    pass
+
+if use_pyca:
+    PROVIDER = PROVIDER_PYCA
+else:
+    PROVIDER = PROVIDER_INTERNAL
+
+def backend():
+    if PROVIDER == PROVIDER_NONE:
+        return "none"
+    elif PROVIDER == PROVIDER_INTERNAL:
+        return "internal"
+    elif PROVIDER == PROVIDER_PYCA:
+        return "openssl, PyCA "+str(pyca_v)

RNS/Cryptography/Proxies.py

@@ -0,0 +1,90 @@
+from cryptography.hazmat.primitives import serialization
+from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey, Ed25519PublicKey
+from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey, X25519PublicKey
+
+# These proxy classes exist to create a uniform API accross
+# cryptography primitive providers.
+
+class X25519PrivateKeyProxy:
+    def __init__(self, real):
+        self.real = real
+
+    @classmethod
+    def generate(cls):
+        return cls(X25519PrivateKey.generate())
+
+    @classmethod
+    def from_private_bytes(cls, data):
+        return cls(X25519PrivateKey.from_private_bytes(data))
+
+    def private_bytes(self):
+        return self.real.private_bytes(
+            encoding=serialization.Encoding.Raw,
+            format=serialization.PrivateFormat.Raw,
+            encryption_algorithm=serialization.NoEncryption(),
+        )
+
+    def public_key(self):
+        return X25519PublicKeyProxy(self.real.public_key())
+
+    def exchange(self, peer_public_key):
+        return self.real.exchange(peer_public_key.real)
+
+
+class X25519PublicKeyProxy:
+    def __init__(self, real):
+        self.real = real
+
+    @classmethod
+    def from_public_bytes(cls, data):
+        return cls(X25519PublicKey.from_public_bytes(data))
+
+    def public_bytes(self):
+        return self.real.public_bytes(
+            encoding=serialization.Encoding.Raw,
+            format=serialization.PublicFormat.Raw
+        )
+
+
+class Ed25519PrivateKeyProxy:
+    def __init__(self, real):
+        self.real = real
+
+    @classmethod
+    def generate(cls):
+        return cls(Ed25519PrivateKey.generate())
+
+    @classmethod
+    def from_private_bytes(cls, data):
+        return cls(Ed25519PrivateKey.from_private_bytes(data))
+
+    def private_bytes(self):
+        return self.real.private_bytes(
+            encoding=serialization.Encoding.Raw,
+            format=serialization.PrivateFormat.Raw,
+            encryption_algorithm=serialization.NoEncryption()
+        )
+
+    def public_key(self):
+        return Ed25519PublicKeyProxy(self.real.public_key())
+
+    def sign(self, message):
+        return self.real.sign(message)
+
+
+class Ed25519PublicKeyProxy:
+    def __init__(self, real):
+        self.real = real
+
+    @classmethod
+    def from_public_bytes(cls, data):
+        return cls(Ed25519PublicKey.from_public_bytes(data))
+
+    def public_bytes(self):
+        return self.real.public_bytes(
+            encoding=serialization.Encoding.Raw,
+            format=serialization.PublicFormat.Raw
+        )
+
+    def verify(self, signature, message):
+        self.real.verify(signature, message)

RNS/Cryptography/SHA256.py

@@ -0,0 +1,129 @@
+# MIT License
+#
+# Copyright (c) 2017 Thomas Dixon
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import copy
+import struct
+import sys
+
+
+def new(m=None):
+    return sha256(m)
+
+class sha256(object):
+    _k = (0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5,
+          0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5,
+          0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3,
+          0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174,
+          0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc,
+          0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da,
+          0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7,
+          0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967,
+          0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13,
+          0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85,
+          0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3,
+          0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070,
+          0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5,
+          0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
+          0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208,
+          0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2)
+    _h = (0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a,
+          0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19)
+    _output_size = 8
+    
+    blocksize = 1
+    block_size = 64
+    digest_size = 32
+    
+    def __init__(self, m=None):        
+        self._buffer = b""
+        self._counter = 0
+        
+        if m is not None:
+            if type(m) is not bytes:
+                raise TypeError('%s() argument 1 must be bytes, not %s' % (self.__class__.__name__, type(m).__name__))
+            self.update(m)
+        
+    def _rotr(self, x, y):
+        return ((x >> y) | (x << (32-y))) & 0xFFFFFFFF
+                    
+    def _sha256_process(self, c):
+        w = [0]*64
+        w[0:16] = struct.unpack('!16L', c)
+        
+        for i in range(16, 64):
+            s0 = self._rotr(w[i-15], 7) ^ self._rotr(w[i-15], 18) ^ (w[i-15] >> 3)
+            s1 = self._rotr(w[i-2], 17) ^ self._rotr(w[i-2], 19) ^ (w[i-2] >> 10)
+            w[i] = (w[i-16] + s0 + w[i-7] + s1) & 0xFFFFFFFF
+        
+        a,b,c,d,e,f,g,h = self._h
+        
+        for i in range(64):
+            s0 = self._rotr(a, 2) ^ self._rotr(a, 13) ^ self._rotr(a, 22)
+            maj = (a & b) ^ (a & c) ^ (b & c)
+            t2 = s0 + maj
+            s1 = self._rotr(e, 6) ^ self._rotr(e, 11) ^ self._rotr(e, 25)
+            ch = (e & f) ^ ((~e) & g)
+            t1 = h + s1 + ch + self._k[i] + w[i]
+            
+            h = g
+            g = f
+            f = e
+            e = (d + t1) & 0xFFFFFFFF
+            d = c
+            c = b
+            b = a
+            a = (t1 + t2) & 0xFFFFFFFF
+            
+        self._h = [(x+y) & 0xFFFFFFFF for x,y in zip(self._h, [a,b,c,d,e,f,g,h])]
+        
+    def update(self, m):
+        if not m:
+            return
+
+        if type(m) is not bytes:
+            raise TypeError('%s() argument 1 must be bytes, not %s' % (sys._getframe().f_code.co_name, type(m).__name__))
+        
+        self._buffer += m
+        self._counter += len(m)
+        
+        while len(self._buffer) >= 64:
+            self._sha256_process(self._buffer[:64])
+            self._buffer = self._buffer[64:]
+            
+    def digest(self):
+        mdi = self._counter & 0x3F
+        length = struct.pack('!Q', self._counter<<3)
+        
+        if mdi < 56:
+            padlen = 55-mdi
+        else:
+            padlen = 119-mdi
+        
+        r = self.copy()
+        r.update(b'\x80'+(b'\x00'*padlen)+length)
+        return b''.join([struct.pack('!L', i) for i in r._h[:self._output_size]])
+        
+    def hexdigest(self):
+        return self.digest().encode('hex')
+        
+    def copy(self):
+        return copy.deepcopy(self)

RNS/Cryptography/SHA512.py

@@ -0,0 +1,129 @@
+# MIT License
+#
+# Copyright (c) 2017 Thomas Dixon
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import copy, struct, sys
+
+def new(m=None):
+    return sha512(m)
+
+class sha512(object):
+    _k = (0x428a2f98d728ae22, 0x7137449123ef65cd, 0xb5c0fbcfec4d3b2f, 0xe9b5dba58189dbbc,
+          0x3956c25bf348b538, 0x59f111f1b605d019, 0x923f82a4af194f9b, 0xab1c5ed5da6d8118,
+          0xd807aa98a3030242, 0x12835b0145706fbe, 0x243185be4ee4b28c, 0x550c7dc3d5ffb4e2,
+          0x72be5d74f27b896f, 0x80deb1fe3b1696b1, 0x9bdc06a725c71235, 0xc19bf174cf692694,
+          0xe49b69c19ef14ad2, 0xefbe4786384f25e3, 0x0fc19dc68b8cd5b5, 0x240ca1cc77ac9c65,
+          0x2de92c6f592b0275, 0x4a7484aa6ea6e483, 0x5cb0a9dcbd41fbd4, 0x76f988da831153b5,
+          0x983e5152ee66dfab, 0xa831c66d2db43210, 0xb00327c898fb213f, 0xbf597fc7beef0ee4,
+          0xc6e00bf33da88fc2, 0xd5a79147930aa725, 0x06ca6351e003826f, 0x142929670a0e6e70,
+          0x27b70a8546d22ffc, 0x2e1b21385c26c926, 0x4d2c6dfc5ac42aed, 0x53380d139d95b3df,
+          0x650a73548baf63de, 0x766a0abb3c77b2a8, 0x81c2c92e47edaee6, 0x92722c851482353b,
+          0xa2bfe8a14cf10364, 0xa81a664bbc423001, 0xc24b8b70d0f89791, 0xc76c51a30654be30,
+          0xd192e819d6ef5218, 0xd69906245565a910, 0xf40e35855771202a, 0x106aa07032bbd1b8,
+          0x19a4c116b8d2d0c8, 0x1e376c085141ab53, 0x2748774cdf8eeb99, 0x34b0bcb5e19b48a8,
+          0x391c0cb3c5c95a63, 0x4ed8aa4ae3418acb, 0x5b9cca4f7763e373, 0x682e6ff3d6b2b8a3,
+          0x748f82ee5defb2fc, 0x78a5636f43172f60, 0x84c87814a1f0ab72, 0x8cc702081a6439ec,
+          0x90befffa23631e28, 0xa4506cebde82bde9, 0xbef9a3f7b2c67915, 0xc67178f2e372532b,
+          0xca273eceea26619c, 0xd186b8c721c0c207, 0xeada7dd6cde0eb1e, 0xf57d4f7fee6ed178,
+          0x06f067aa72176fba, 0x0a637dc5a2c898a6, 0x113f9804bef90dae, 0x1b710b35131c471b,
+          0x28db77f523047d84, 0x32caab7b40c72493, 0x3c9ebe0a15c9bebc, 0x431d67c49c100d4c,
+          0x4cc5d4becb3e42b6, 0x597f299cfc657e2a, 0x5fcb6fab3ad6faec, 0x6c44198c4a475817)
+    _h = (0x6a09e667f3bcc908, 0xbb67ae8584caa73b, 0x3c6ef372fe94f82b, 0xa54ff53a5f1d36f1,
+          0x510e527fade682d1, 0x9b05688c2b3e6c1f, 0x1f83d9abfb41bd6b, 0x5be0cd19137e2179)
+    _output_size = 8
+
+    blocksize = 1
+    block_size = 128
+    digest_size = 64
+
+    def __init__(self, m=None):
+        self._buffer = b''
+        self._counter = 0
+
+        if m is not None:
+            if type(m) is not bytes:
+                raise TypeError('%s() argument 1 must be bytes, not %s' % (self.__class__.__name__, type(m).__name__))
+            self.update(m)
+
+    def _rotr(self, x, y):
+        return ((x >> y) | (x << (64-y))) & 0xFFFFFFFFFFFFFFFF
+
+    def _sha512_process(self, chunk):
+        w = [0]*80
+        w[0:16] = struct.unpack('!16Q', chunk)
+
+        for i in range(16, 80):
+            s0 = self._rotr(w[i-15], 1) ^ self._rotr(w[i-15], 8) ^ (w[i-15] >> 7)
+            s1 = self._rotr(w[i-2], 19) ^ self._rotr(w[i-2], 61) ^ (w[i-2] >> 6)
+            w[i] = (w[i-16] + s0 + w[i-7] + s1) & 0xFFFFFFFFFFFFFFFF
+
+        a,b,c,d,e,f,g,h = self._h
+
+        for i in range(80):
+            s0 = self._rotr(a, 28) ^ self._rotr(a, 34) ^ self._rotr(a, 39)
+            maj = (a & b) ^ (a & c) ^ (b & c)
+            t2 = s0 + maj
+            s1 = self._rotr(e, 14) ^ self._rotr(e, 18) ^ self._rotr(e, 41)
+            ch = (e & f) ^ ((~e) & g)
+            t1 = h + s1 + ch + self._k[i] + w[i]
+
+            h = g
+            g = f
+            f = e
+            e = (d + t1) & 0xFFFFFFFFFFFFFFFF
+            d = c
+            c = b
+            b = a
+            a = (t1 + t2) & 0xFFFFFFFFFFFFFFFF
+
+        self._h = [(x+y) & 0xFFFFFFFFFFFFFFFF for x,y in zip(self._h, [a,b,c,d,e,f,g,h])]
+
+    def update(self, m):
+        if not m:
+            return
+        if type(m) is not bytes:
+            raise TypeError('%s() argument 1 must be bytes, not %s' % (sys._getframe().f_code.co_name, type(m).__name__))
+
+        self._buffer += m
+        self._counter += len(m)
+
+        while len(self._buffer) >= 128:
+            self._sha512_process(self._buffer[:128])
+            self._buffer = self._buffer[128:]
+
+    def digest(self):
+        mdi = self._counter & 0x7F
+        length = struct.pack('!Q', self._counter<<3)
+
+        if mdi < 112:
+            padlen = 111-mdi
+        else:
+            padlen = 239-mdi
+
+        r = self.copy()
+        r.update(b'\x80'+(b'\x00'*(padlen+8))+length)
+        return b''.join([struct.pack('!Q', i) for i in r._h[:self._output_size]])
+
+    def hexdigest(self):
+        return self.digest().encode('hex')
+
+    def copy(self):
+        return copy.deepcopy(self)

RNS/Cryptography/X25519.py

@@ -0,0 +1,171 @@
+# By Nicko van Someren, 2021. This code is released into the public domain.
+# Small modifications for use in Reticulum, and constant time key exchange
+# added by Mark Qvist in 2022.
+
+# WARNING! Only the X25519PrivateKey.exchange() method attempts to hide execution time.
+# In the context of Reticulum, this is sufficient, but it may not be in other systems. If
+# this code is to be used to provide cryptographic security in an environment where the
+# start and end times of the execution can be guessed, inferred or measured then it is
+# critical that steps are taken to hide the execution time, for instance by adding a
+# delay so that encrypted packets are not sent until a fixed time after the _start_ of
+# execution.
+
+
+import os
+import time
+
+P = 2 ** 255 - 19
+_A = 486662
+
+
+def _point_add(point_n, point_m, point_diff):
+    """Given the projection of two points and their difference, return their sum"""
+    (xn, zn) = point_n
+    (xm, zm) = point_m
+    (x_diff, z_diff) = point_diff
+    x = (z_diff << 2) * (xm * xn - zm * zn) ** 2
+    z = (x_diff << 2) * (xm * zn - zm * xn) ** 2
+    return x % P, z % P
+
+
+def _point_double(point_n):
+    """Double a point provided in projective coordinates"""
+    (xn, zn) = point_n
+    xn2 = xn ** 2
+    zn2 = zn ** 2
+    x = (xn2 - zn2) ** 2
+    xzn = xn * zn
+    z = 4 * xzn * (xn2 + _A * xzn + zn2)
+    return x % P, z % P
+
+
+def _const_time_swap(a, b, swap):
+    """Swap two values in constant time"""
+    index = int(swap) * 2
+    temp = (a, b, b, a)
+    return temp[index:index+2]
+
+
+def _raw_curve25519(base, n):
+    """Raise the point base to the power n"""
+    zero = (1, 0)
+    one = (base, 1)
+    mP, m1P = zero, one
+
+    for i in reversed(range(256)):
+        bit = bool(n & (1 << i))
+        mP, m1P = _const_time_swap(mP, m1P, bit)
+        mP, m1P = _point_double(mP), _point_add(mP, m1P, one)
+        mP, m1P = _const_time_swap(mP, m1P, bit)
+
+    x, z = mP
+    inv_z = pow(z, P - 2, P)
+    return (x * inv_z) % P
+
+
+def _unpack_number(s):
+    """Unpack 32 bytes to a 256 bit value"""
+    if len(s) != 32:
+        raise ValueError('Curve25519 values must be 32 bytes')
+    return int.from_bytes(s, "little")
+
+
+def _pack_number(n):
+    """Pack a value into 32 bytes"""
+    return n.to_bytes(32, "little")
+
+
+def _fix_secret(n):
+    """Mask a value to be an acceptable exponent"""
+    n &= ~7
+    n &= ~(128 << 8 * 31)
+    n |= 64 << 8 * 31
+    return n
+
+
+def curve25519(base_point_raw, secret_raw):
+    """Raise the base point to a given power"""
+    base_point = _unpack_number(base_point_raw)
+    secret = _fix_secret(_unpack_number(secret_raw))
+    return _pack_number(_raw_curve25519(base_point, secret))
+
+
+def curve25519_base(secret_raw):
+    """Raise the generator point to a given power"""
+    secret = _fix_secret(_unpack_number(secret_raw))
+    return _pack_number(_raw_curve25519(9, secret))
+
+
+class X25519PublicKey:
+    def __init__(self, x):
+        self.x = x
+
+    @classmethod
+    def from_public_bytes(cls, data):
+        return cls(_unpack_number(data))
+
+    def public_bytes(self):
+        return _pack_number(self.x)
+
+
+class X25519PrivateKey:
+    MIN_EXEC_TIME = 0.002
+    MAX_EXEC_TIME = 0.5
+    DELAY_WINDOW = 10
+
+    T_CLEAR = None
+    T_MAX = 0
+
+    def __init__(self, a):
+        self.a = a
+
+    @classmethod
+    def generate(cls):
+        return cls.from_private_bytes(os.urandom(32))
+
+    @classmethod
+    def from_private_bytes(cls, data):
+        return cls(_fix_secret(_unpack_number(data)))
+
+    def private_bytes(self):
+        return _pack_number(self.a)
+
+    def public_key(self):
+        return X25519PublicKey.from_public_bytes(_pack_number(_raw_curve25519(9, self.a)))
+
+    def exchange(self, peer_public_key):
+        if isinstance(peer_public_key, bytes):
+            peer_public_key = X25519PublicKey.from_public_bytes(peer_public_key)
+
+        start = time.time()
+        
+        shared = _pack_number(_raw_curve25519(peer_public_key.x, self.a))
+        
+        end = time.time()
+        duration = end-start
+
+        if X25519PrivateKey.T_CLEAR == None:
+            X25519PrivateKey.T_CLEAR = end + X25519PrivateKey.DELAY_WINDOW
+
+        if end > X25519PrivateKey.T_CLEAR:
+            X25519PrivateKey.T_CLEAR = end + X25519PrivateKey.DELAY_WINDOW
+            X25519PrivateKey.T_MAX = 0
+        
+        if duration < X25519PrivateKey.T_MAX or duration < X25519PrivateKey.MIN_EXEC_TIME:
+            target = start+X25519PrivateKey.T_MAX
+
+            if target > start+X25519PrivateKey.MAX_EXEC_TIME:
+                target = start+X25519PrivateKey.MAX_EXEC_TIME
+
+            if target < start+X25519PrivateKey.MIN_EXEC_TIME:
+                target = start+X25519PrivateKey.MIN_EXEC_TIME
+
+            try:
+                time.sleep(target-time.time())
+            except Exception as e:
+                pass
+
+        elif duration > X25519PrivateKey.T_MAX:
+            X25519PrivateKey.T_MAX = duration
+
+        return shared

RNS/Cryptography/__init__.py

@@ -0,0 +1,24 @@
+import os
+import glob
+
+from .Hashes import sha256
+from .Hashes import sha512
+from .HKDF import hkdf
+from .PKCS7 import PKCS7
+from .Fernet import Fernet
+from .Provider import backend
+
+import RNS.Cryptography.Provider as cp
+
+if cp.PROVIDER == cp.PROVIDER_INTERNAL:
+    from RNS.Cryptography.X25519 import X25519PrivateKey, X25519PublicKey
+    from RNS.Cryptography.Ed25519 import Ed25519PrivateKey, Ed25519PublicKey
+
+elif cp.PROVIDER == cp.PROVIDER_PYCA:
+    from RNS.Cryptography.Proxies import X25519PrivateKeyProxy as X25519PrivateKey
+    from RNS.Cryptography.Proxies import X25519PublicKeyProxy as X25519PublicKey
+    from RNS.Cryptography.Proxies import Ed25519PrivateKeyProxy as Ed25519PrivateKey
+    from RNS.Cryptography.Proxies import Ed25519PublicKeyProxy as Ed25519PublicKey
+
+modules = glob.glob(os.path.dirname(__file__)+"/*.py")
+__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]

RNS/Cryptography/aes/__init__.py

@@ -0,0 +1 @@
+from .aes import AES

RNS/Cryptography/aes/aes.py

@@ -0,0 +1,271 @@
+# MIT License
+
+# Copyright (c) 2021 Or Gur Arie
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from .utils import *
+
+
+class AES:
+    # AES-128 block size
+    block_size = 16
+    # AES-128 encrypts messages with 10 rounds
+    _rounds = 10
+
+
+    # initiate the AES objecy
+    def __init__(self, key):
+        """
+        Initializes the object with a given key.
+        """
+        # make sure key length is right
+        assert len(key) == AES.block_size
+
+        # ExpandKey
+        self._round_keys = self._expand_key(key)
+
+
+    # will perform the AES ExpandKey phase
+    def _expand_key(self, master_key):
+        """
+        Expands and returns a list of key matrices for the given master_key.
+        """
+
+        # Initialize round keys with raw key material.
+        key_columns = bytes2matrix(master_key)
+        iteration_size = len(master_key) // 4
+
+        # Each iteration has exactly as many columns as the key material.
+        i = 1
+        while len(key_columns) < (self._rounds + 1) * 4:
+            # Copy previous word.
+            word = list(key_columns[-1])
+
+            # Perform schedule_core once every "row".
+            if len(key_columns) % iteration_size == 0:
+                # Circular shift.
+                word.append(word.pop(0))
+                # Map to S-BOX.
+                word = [s_box[b] for b in word]
+                # XOR with first byte of R-CON, since the others bytes of R-CON are 0.
+                word[0] ^= r_con[i]
+                i += 1
+            elif len(master_key) == 32 and len(key_columns) % iteration_size == 4:
+                # Run word through S-box in the fourth iteration when using a
+                # 256-bit key.
+                word = [s_box[b] for b in word]
+
+            # XOR with equivalent word from previous iteration.
+            word = bytes(i^j for i, j in zip(word, key_columns[-iteration_size]))
+            key_columns.append(word)
+
+        # Group key words in 4x4 byte matrices.
+        return [key_columns[4*i : 4*(i+1)] for i in range(len(key_columns) // 4)]
+
+
+    # encrypt a single block of data with AES
+    def _encrypt_block(self, plaintext):
+        """
+        Encrypts a single block of 16 byte long plaintext.
+        """
+        # length of a single block
+        assert len(plaintext) == AES.block_size
+
+        # perform on a matrix
+        state = bytes2matrix(plaintext)
+
+        # AddRoundKey
+        add_round_key(state, self._round_keys[0])
+
+        # 9 main rounds
+        for i in range(1, self._rounds):
+            # SubBytes
+            sub_bytes(state)
+            # ShiftRows
+            shift_rows(state)
+            # MixCols
+            mix_columns(state)
+            # AddRoundKey
+            add_round_key(state, self._round_keys[i])
+
+        # last round, w/t AddRoundKey step
+        sub_bytes(state)
+        shift_rows(state)
+        add_round_key(state, self._round_keys[-1])
+
+        # return the encrypted matrix as bytes
+        return matrix2bytes(state)
+
+
+    # decrypt a single block of data with AES
+    def _decrypt_block(self, ciphertext):
+        """
+        Decrypts a single block of 16 byte long ciphertext.
+        """
+        # length of a single block
+        assert len(ciphertext) == AES.block_size
+
+        # perform on a matrix
+        state = bytes2matrix(ciphertext)
+
+        # in reverse order, last round is first
+        add_round_key(state, self._round_keys[-1])
+        inv_shift_rows(state)
+        inv_sub_bytes(state)
+
+        for i in range(self._rounds - 1, 0, -1):
+            # nain rounds
+            add_round_key(state, self._round_keys[i])
+            inv_mix_columns(state)
+            inv_shift_rows(state)
+            inv_sub_bytes(state)
+
+        # initial AddRoundKey phase
+        add_round_key(state, self._round_keys[0])
+
+        # return bytes
+        return matrix2bytes(state)
+
+
+    # will encrypt the entire data 
+    def encrypt(self, plaintext, iv):
+        """
+        Encrypts `plaintext` using CBC mode and PKCS#7 padding, with the given
+        initialization vector (iv).
+        """
+        # iv length must be same as block size
+        assert len(iv) == AES.block_size
+
+        assert len(plaintext) % AES.block_size == 0
+
+        ciphertext_blocks = []
+
+        previous = iv
+        for plaintext_block in split_blocks(plaintext):
+            # in CBC mode every block is XOR'd with the previous block
+            xorred = xor_bytes(plaintext_block, previous)
+
+            # encrypt current block
+            block = self._encrypt_block(xorred)
+            previous = block
+
+            # append to ciphertext
+            ciphertext_blocks.append(block)
+
+        # return as bytes
+        return b''.join(ciphertext_blocks)
+
+
+    # will decrypt the entire data 
+    def decrypt(self, ciphertext, iv):
+        """
+        Decrypts `ciphertext` using CBC mode and PKCS#7 padding, with the given
+        initialization vector (iv).
+        """
+        # iv length must be same as block size
+        assert len(iv) == AES.block_size
+
+        plaintext_blocks = []
+
+        previous = iv
+        for ciphertext_block in split_blocks(ciphertext):
+            # in CBC mode every block is XOR'd with the previous block
+            xorred = xor_bytes(previous, self._decrypt_block(ciphertext_block))
+            
+            # append plaintext
+            plaintext_blocks.append(xorred)
+            previous = ciphertext_block
+
+        return b''.join(plaintext_blocks)
+
+
+def test():
+    # modules and classes requiered for test only
+    import os
+    class bcolors:
+        OK = '\033[92m' #GREEN
+        WARNING = '\033[93m' #YELLOW
+        FAIL = '\033[91m' #RED
+        RESET = '\033[0m' #RESET COLOR
+
+    # will test AES class by performing an encryption / decryption
+    print("AES Tests")
+    print("=========")
+
+    # generate a secret key and print details
+    key = os.urandom(AES.block_size)
+    _aes = AES(key)
+    print(f"Algorithm: AES-CBC-{AES.block_size*8}")
+    print(f"Secret Key: {key.hex()}")
+    print()
+
+    # test single block encryption / decryption
+    iv = os.urandom(AES.block_size)
+
+    single_block_text = b"SingleBlock Text"
+    print("Single Block Tests")
+    print("------------------")
+    print(f"iv: {iv.hex()}")
+    
+    print(f"plain text: '{single_block_text.decode()}'")
+    ciphertext_block = _aes._encrypt_block(single_block_text)
+    plaintext_block = _aes._decrypt_block(ciphertext_block)
+    print(f"Ciphertext Hex: {ciphertext_block.hex()}")
+    print(f"Plaintext: {plaintext_block.decode()}")
+    assert plaintext_block == single_block_text
+    print(bcolors.OK + "Single Block Test Passed Successfully" + bcolors.RESET)
+    print()
+
+    # test a less than a block length phrase
+    iv = os.urandom(AES.block_size)
+
+    short_text = b"Just Text"
+    print("Short Text Tests")
+    print("----------------")
+    print(f"iv: {iv.hex()}")
+    print(f"plain text: '{short_text.decode()}'")
+    ciphertext_short = _aes.encrypt(short_text, iv)
+    plaintext_short = _aes.decrypt(ciphertext_short, iv)
+    print(f"Ciphertext Hex: {ciphertext_short.hex()}")
+    print(f"Plaintext: {plaintext_short.decode()}")
+    assert short_text == plaintext_short
+    print(bcolors.OK + "Short Text Test Passed Successfully" + bcolors.RESET)
+    print()
+
+    # test an arbitrary length phrase
+    iv = os.urandom(AES.block_size)
+
+    text = b"This Text is longer than one block"
+    print("Arbitrary Length Tests")
+    print("----------------------")
+    print(f"iv: {iv.hex()}")
+    print(f"plain text: '{text.decode()}'")
+    ciphertext = _aes.encrypt(text, iv)
+    plaintext = _aes.decrypt(ciphertext, iv)
+    print(f"Ciphertext Hex: {ciphertext.hex()}")
+    print(f"Plaintext: {plaintext.decode()}")
+    assert text == plaintext
+    print(bcolors.OK + "Arbitrary Length Text Test Passed Successfully" + bcolors.RESET)
+    print()
+
+
+if __name__ == "__main__":
+    # test AES class
+    test()    

RNS/Cryptography/aes/utils.py

@@ -0,0 +1,159 @@
+# MIT License
+
+# Copyright (c) 2021 Or Gur Arie
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+'''
+Utils class for AES encryption / decryption
+'''
+
+## AES lookup tables
+# resource: https://en.wikipedia.org/wiki/Rijndael_S-box
+s_box = (
+    0x63, 0x7C, 0x77, 0x7B, 0xF2, 0x6B, 0x6F, 0xC5, 0x30, 0x01, 0x67, 0x2B, 0xFE, 0xD7, 0xAB, 0x76,
+    0xCA, 0x82, 0xC9, 0x7D, 0xFA, 0x59, 0x47, 0xF0, 0xAD, 0xD4, 0xA2, 0xAF, 0x9C, 0xA4, 0x72, 0xC0,
+    0xB7, 0xFD, 0x93, 0x26, 0x36, 0x3F, 0xF7, 0xCC, 0x34, 0xA5, 0xE5, 0xF1, 0x71, 0xD8, 0x31, 0x15,
+    0x04, 0xC7, 0x23, 0xC3, 0x18, 0x96, 0x05, 0x9A, 0x07, 0x12, 0x80, 0xE2, 0xEB, 0x27, 0xB2, 0x75,
+    0x09, 0x83, 0x2C, 0x1A, 0x1B, 0x6E, 0x5A, 0xA0, 0x52, 0x3B, 0xD6, 0xB3, 0x29, 0xE3, 0x2F, 0x84,
+    0x53, 0xD1, 0x00, 0xED, 0x20, 0xFC, 0xB1, 0x5B, 0x6A, 0xCB, 0xBE, 0x39, 0x4A, 0x4C, 0x58, 0xCF,
+    0xD0, 0xEF, 0xAA, 0xFB, 0x43, 0x4D, 0x33, 0x85, 0x45, 0xF9, 0x02, 0x7F, 0x50, 0x3C, 0x9F, 0xA8,
+    0x51, 0xA3, 0x40, 0x8F, 0x92, 0x9D, 0x38, 0xF5, 0xBC, 0xB6, 0xDA, 0x21, 0x10, 0xFF, 0xF3, 0xD2,
+    0xCD, 0x0C, 0x13, 0xEC, 0x5F, 0x97, 0x44, 0x17, 0xC4, 0xA7, 0x7E, 0x3D, 0x64, 0x5D, 0x19, 0x73,
+    0x60, 0x81, 0x4F, 0xDC, 0x22, 0x2A, 0x90, 0x88, 0x46, 0xEE, 0xB8, 0x14, 0xDE, 0x5E, 0x0B, 0xDB,
+    0xE0, 0x32, 0x3A, 0x0A, 0x49, 0x06, 0x24, 0x5C, 0xC2, 0xD3, 0xAC, 0x62, 0x91, 0x95, 0xE4, 0x79,
+    0xE7, 0xC8, 0x37, 0x6D, 0x8D, 0xD5, 0x4E, 0xA9, 0x6C, 0x56, 0xF4, 0xEA, 0x65, 0x7A, 0xAE, 0x08,
+    0xBA, 0x78, 0x25, 0x2E, 0x1C, 0xA6, 0xB4, 0xC6, 0xE8, 0xDD, 0x74, 0x1F, 0x4B, 0xBD, 0x8B, 0x8A,
+    0x70, 0x3E, 0xB5, 0x66, 0x48, 0x03, 0xF6, 0x0E, 0x61, 0x35, 0x57, 0xB9, 0x86, 0xC1, 0x1D, 0x9E,
+    0xE1, 0xF8, 0x98, 0x11, 0x69, 0xD9, 0x8E, 0x94, 0x9B, 0x1E, 0x87, 0xE9, 0xCE, 0x55, 0x28, 0xDF,
+    0x8C, 0xA1, 0x89, 0x0D, 0xBF, 0xE6, 0x42, 0x68, 0x41, 0x99, 0x2D, 0x0F, 0xB0, 0x54, 0xBB, 0x16,
+)
+
+inv_s_box = (
+    0x52, 0x09, 0x6A, 0xD5, 0x30, 0x36, 0xA5, 0x38, 0xBF, 0x40, 0xA3, 0x9E, 0x81, 0xF3, 0xD7, 0xFB,
+    0x7C, 0xE3, 0x39, 0x82, 0x9B, 0x2F, 0xFF, 0x87, 0x34, 0x8E, 0x43, 0x44, 0xC4, 0xDE, 0xE9, 0xCB,
+    0x54, 0x7B, 0x94, 0x32, 0xA6, 0xC2, 0x23, 0x3D, 0xEE, 0x4C, 0x95, 0x0B, 0x42, 0xFA, 0xC3, 0x4E,
+    0x08, 0x2E, 0xA1, 0x66, 0x28, 0xD9, 0x24, 0xB2, 0x76, 0x5B, 0xA2, 0x49, 0x6D, 0x8B, 0xD1, 0x25,
+    0x72, 0xF8, 0xF6, 0x64, 0x86, 0x68, 0x98, 0x16, 0xD4, 0xA4, 0x5C, 0xCC, 0x5D, 0x65, 0xB6, 0x92,
+    0x6C, 0x70, 0x48, 0x50, 0xFD, 0xED, 0xB9, 0xDA, 0x5E, 0x15, 0x46, 0x57, 0xA7, 0x8D, 0x9D, 0x84,
+    0x90, 0xD8, 0xAB, 0x00, 0x8C, 0xBC, 0xD3, 0x0A, 0xF7, 0xE4, 0x58, 0x05, 0xB8, 0xB3, 0x45, 0x06,
+    0xD0, 0x2C, 0x1E, 0x8F, 0xCA, 0x3F, 0x0F, 0x02, 0xC1, 0xAF, 0xBD, 0x03, 0x01, 0x13, 0x8A, 0x6B,
+    0x3A, 0x91, 0x11, 0x41, 0x4F, 0x67, 0xDC, 0xEA, 0x97, 0xF2, 0xCF, 0xCE, 0xF0, 0xB4, 0xE6, 0x73,
+    0x96, 0xAC, 0x74, 0x22, 0xE7, 0xAD, 0x35, 0x85, 0xE2, 0xF9, 0x37, 0xE8, 0x1C, 0x75, 0xDF, 0x6E,
+    0x47, 0xF1, 0x1A, 0x71, 0x1D, 0x29, 0xC5, 0x89, 0x6F, 0xB7, 0x62, 0x0E, 0xAA, 0x18, 0xBE, 0x1B,
+    0xFC, 0x56, 0x3E, 0x4B, 0xC6, 0xD2, 0x79, 0x20, 0x9A, 0xDB, 0xC0, 0xFE, 0x78, 0xCD, 0x5A, 0xF4,
+    0x1F, 0xDD, 0xA8, 0x33, 0x88, 0x07, 0xC7, 0x31, 0xB1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xEC, 0x5F,
+    0x60, 0x51, 0x7F, 0xA9, 0x19, 0xB5, 0x4A, 0x0D, 0x2D, 0xE5, 0x7A, 0x9F, 0x93, 0xC9, 0x9C, 0xEF,
+    0xA0, 0xE0, 0x3B, 0x4D, 0xAE, 0x2A, 0xF5, 0xB0, 0xC8, 0xEB, 0xBB, 0x3C, 0x83, 0x53, 0x99, 0x61,
+    0x17, 0x2B, 0x04, 0x7E, 0xBA, 0x77, 0xD6, 0x26, 0xE1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0C, 0x7D,
+)
+
+
+## AES AddRoundKey
+# Round constants https://en.wikipedia.org/wiki/AES_key_schedule#Round_constants
+r_con = (
+    0x00, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40,
+    0x80, 0x1B, 0x36, 0x6C, 0xD8, 0xAB, 0x4D, 0x9A,
+    0x2F, 0x5E, 0xBC, 0x63, 0xC6, 0x97, 0x35, 0x6A,
+    0xD4, 0xB3, 0x7D, 0xFA, 0xEF, 0xC5, 0x91, 0x39,
+)
+
+def add_round_key(s, k):
+    for i in range(4):
+        for j in range(4):
+            s[i][j] ^= k[i][j]
+
+
+## AES SubBytes
+def sub_bytes(s):
+    for i in range(4):
+        for j in range(4):
+            s[i][j] = s_box[s[i][j]]
+
+
+def inv_sub_bytes(s):
+    for i in range(4):
+        for j in range(4):
+            s[i][j] = inv_s_box[s[i][j]]
+
+
+## AES ShiftRows
+def shift_rows(s):
+    s[0][1], s[1][1], s[2][1], s[3][1] = s[1][1], s[2][1], s[3][1], s[0][1]
+    s[0][2], s[1][2], s[2][2], s[3][2] = s[2][2], s[3][2], s[0][2], s[1][2]
+    s[0][3], s[1][3], s[2][3], s[3][3] = s[3][3], s[0][3], s[1][3], s[2][3]
+
+
+def inv_shift_rows(s):
+    s[0][1], s[1][1], s[2][1], s[3][1] = s[3][1], s[0][1], s[1][1], s[2][1]
+    s[0][2], s[1][2], s[2][2], s[3][2] = s[2][2], s[3][2], s[0][2], s[1][2]
+    s[0][3], s[1][3], s[2][3], s[3][3] = s[1][3], s[2][3], s[3][3], s[0][3]
+
+
+## AES MixColumns
+# learned from http://cs.ucsb.edu/~koc/cs178/projects/JT/aes.c
+xtime = lambda a: (((a << 1) ^ 0x1B) & 0xFF) if (a & 0x80) else (a << 1)
+
+
+def mix_single_column(a):
+    # see Sec 4.1.2 in The Design of Rijndael
+    t = a[0] ^ a[1] ^ a[2] ^ a[3]
+    u = a[0]
+    a[0] ^= t ^ xtime(a[0] ^ a[1])
+    a[1] ^= t ^ xtime(a[1] ^ a[2])
+    a[2] ^= t ^ xtime(a[2] ^ a[3])
+    a[3] ^= t ^ xtime(a[3] ^ u)
+
+
+def mix_columns(s):
+    for i in range(4):
+        mix_single_column(s[i])
+
+
+def inv_mix_columns(s):
+    # see Sec 4.1.3 in The Design of Rijndael
+    for i in range(4):
+        u = xtime(xtime(s[i][0] ^ s[i][2]))
+        v = xtime(xtime(s[i][1] ^ s[i][3]))
+        s[i][0] ^= u
+        s[i][1] ^= v
+        s[i][2] ^= u
+        s[i][3] ^= v
+
+    mix_columns(s)
+
+
+## AES Bytes
+def bytes2matrix(text):
+    """ Converts a 16-byte array into a 4x4 matrix.  """
+    return [list(text[i:i+4]) for i in range(0, len(text), 4)]
+
+def matrix2bytes(matrix):
+    """ Converts a 4x4 matrix into a 16-byte array.  """
+    return bytes(sum(matrix, []))
+
+
+def xor_bytes(a, b):
+    """ Returns a new byte array with the elements xor'ed. """
+    return bytes(i^j for i, j in zip(a, b))
+
+
+def split_blocks(message, block_size=16, require_padding=True):
+        assert len(message) % block_size == 0 or not require_padding
+        return [message[i:i+16] for i in range(0, len(message), block_size)]

RNS/Cryptography/pure25519/_ed25519.py

@@ -0,0 +1,58 @@
+# MIT License
+#
+# Copyright (c) 2015 Brian Warner and other contributors
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from . import eddsa
+
+class BadSignatureError(Exception):
+    pass
+
+SECRETKEYBYTES = 64
+PUBLICKEYBYTES = 32
+SIGNATUREKEYBYTES = 64
+
+def publickey(seed32):
+    assert len(seed32) == 32
+    vk32 = eddsa.publickey(seed32)
+    return vk32, seed32+vk32
+
+def sign(msg, skvk):
+    assert len(skvk) == 64
+    sk = skvk[:32]
+    vk = skvk[32:]
+    sig = eddsa.signature(msg, sk, vk)
+    return sig+msg
+
+def open(sigmsg, vk):
+    assert len(vk) == 32
+    sig = sigmsg[:64]
+    msg = sigmsg[64:]
+    try:
+        valid = eddsa.checkvalid(sig, msg, vk)
+    except ValueError as e:
+        raise BadSignatureError(e)
+    except Exception as e:
+        if str(e) == "decoding point that is not on curve":
+            raise BadSignatureError(e)
+        raise
+    if not valid:
+        raise BadSignatureError()
+    return msg

RNS/Cryptography/pure25519/basic.py

@@ -0,0 +1,368 @@
+# MIT License
+#
+# Copyright (c) 2015 Brian Warner and other contributors
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import binascii, hashlib, itertools
+
+Q = 2**255 - 19
+L = 2**252 + 27742317777372353535851937790883648493
+
+def inv(x):
+    return pow(x, Q-2, Q)
+
+d = -121665 * inv(121666)
+I = pow(2,(Q-1)//4,Q)
+
+def xrecover(y):
+    xx = (y*y-1) * inv(d*y*y+1)
+    x = pow(xx,(Q+3)//8,Q)
+    if (x*x - xx) % Q != 0: x = (x*I) % Q
+    if x % 2 != 0: x = Q-x
+    return x
+
+By = 4 * inv(5)
+Bx = xrecover(By)
+B = [Bx % Q,By % Q]
+
+# Extended Coordinates: x=X/Z, y=Y/Z, x*y=T/Z
+# http://www.hyperelliptic.org/EFD/g1p/auto-twisted-extended-1.html
+
+def xform_affine_to_extended(pt):
+    (x, y) = pt
+    return (x%Q, y%Q, 1, (x*y)%Q) # (X,Y,Z,T)
+
+def xform_extended_to_affine(pt):
+    (x, y, z, _) = pt
+    return ((x*inv(z))%Q, (y*inv(z))%Q)
+
+def double_element(pt): # extended->extended
+    # dbl-2008-hwcd
+    (X1, Y1, Z1, _) = pt
+    A = (X1*X1)
+    B = (Y1*Y1)
+    C = (2*Z1*Z1)
+    D = (-A) % Q
+    J = (X1+Y1) % Q
+    E = (J*J-A-B) % Q
+    G = (D+B) % Q
+    F = (G-C) % Q
+    H = (D-B) % Q
+    X3 = (E*F) % Q
+    Y3 = (G*H) % Q
+    Z3 = (F*G) % Q
+    T3 = (E*H) % Q
+    return (X3, Y3, Z3, T3)
+
+def add_elements(pt1, pt2): # extended->extended
+    # add-2008-hwcd-3 . Slightly slower than add-2008-hwcd-4, but -3 is
+    # unified, so it's safe for general-purpose addition
+    (X1, Y1, Z1, T1) = pt1
+    (X2, Y2, Z2, T2) = pt2
+    A = ((Y1-X1)*(Y2-X2)) % Q
+    B = ((Y1+X1)*(Y2+X2)) % Q
+    C = T1*(2*d)*T2 % Q
+    D = Z1*2*Z2 % Q
+    E = (B-A) % Q
+    F = (D-C) % Q
+    G = (D+C) % Q
+    H = (B+A) % Q
+    X3 = (E*F) % Q
+    Y3 = (G*H) % Q
+    T3 = (E*H) % Q
+    Z3 = (F*G) % Q
+    return (X3, Y3, Z3, T3)
+
+def scalarmult_element_safe_slow(pt, n):
+    # this form is slightly slower, but tolerates arbitrary points, including
+    # those which are not in the main 1*L subgroup. This includes points of
+    # order 1 (the neutral element Zero), 2, 4, and 8.
+    assert n >= 0
+    if n==0:
+        return xform_affine_to_extended((0,1))
+    _ = double_element(scalarmult_element_safe_slow(pt, n>>1))
+    return add_elements(_, pt) if n&1 else _
+
+def _add_elements_nonunfied(pt1, pt2): # extended->extended
+    # add-2008-hwcd-4 : NOT unified, only for pt1!=pt2. About 10% faster than
+    # the (unified) add-2008-hwcd-3, and safe to use inside scalarmult if you
+    # aren't using points of order 1/2/4/8
+    (X1, Y1, Z1, T1) = pt1
+    (X2, Y2, Z2, T2) = pt2
+    A = ((Y1-X1)*(Y2+X2)) % Q
+    B = ((Y1+X1)*(Y2-X2)) % Q
+    C = (Z1*2*T2) % Q
+    D = (T1*2*Z2) % Q
+    E = (D+C) % Q
+    F = (B-A) % Q
+    G = (B+A) % Q
+    H = (D-C) % Q
+    X3 = (E*F) % Q
+    Y3 = (G*H) % Q
+    Z3 = (F*G) % Q
+    T3 = (E*H) % Q
+    return (X3, Y3, Z3, T3)
+
+def scalarmult_element(pt, n): # extended->extended
+    # This form only works properly when given points that are a member of
+    # the main 1*L subgroup. It will give incorrect answers when called with
+    # the points of order 1/2/4/8, including point Zero. (it will also work
+    # properly when given points of order 2*L/4*L/8*L)
+    assert n >= 0
+    if n==0:
+        return xform_affine_to_extended((0,1))
+    _ = double_element(scalarmult_element(pt, n>>1))
+    return _add_elements_nonunfied(_, pt) if n&1 else _
+
+# points are encoded as 32-bytes little-endian, b255 is sign, b2b1b0 are 0
+
+def encodepoint(P):
+    x = P[0]
+    y = P[1]
+    # MSB of output equals x.b0 (=x&1)
+    # rest of output is little-endian y
+    assert 0 <= y < (1<<255) # always < 0x7fff..ff
+    if x & 1:
+        y += 1<<255
+    return binascii.unhexlify("%064x" % y)[::-1]
+
+def isoncurve(P):
+    x = P[0]
+    y = P[1]
+    return (-x*x + y*y - 1 - d*x*x*y*y) % Q == 0
+
+class NotOnCurve(Exception):
+    pass
+
+def decodepoint(s):
+    unclamped = int(binascii.hexlify(s[:32][::-1]), 16)
+    clamp = (1 << 255) - 1
+    y = unclamped & clamp # clear MSB
+    x = xrecover(y)
+    if bool(x & 1) != bool(unclamped & (1<<255)): x = Q-x
+    P = [x,y]
+    if not isoncurve(P): raise NotOnCurve("decoding point that is not on curve")
+    return P
+
+# scalars are encoded as 32-bytes little-endian
+
+def bytes_to_scalar(s):
+    assert len(s) == 32, len(s)
+    return int(binascii.hexlify(s[::-1]), 16)
+
+def bytes_to_clamped_scalar(s):
+    # Ed25519 private keys clamp the scalar to ensure two things:
+    #   1: integer value is in L/2 .. L, to avoid small-logarithm
+    #      non-wraparaound
+    #   2: low-order 3 bits are zero, so a small-subgroup attack won't learn
+    #      any information
+    # set the top two bits to 01, and the bottom three to 000
+    a_unclamped = bytes_to_scalar(s)
+    AND_CLAMP = (1<<254) - 1 - 7
+    OR_CLAMP = (1<<254)
+    a_clamped = (a_unclamped & AND_CLAMP) | OR_CLAMP
+    return a_clamped
+
+def random_scalar(entropy_f): # 0..L-1 inclusive
+    # reduce the bias to a safe level by generating 256 extra bits
+    oversized = int(binascii.hexlify(entropy_f(32+32)), 16)
+    return oversized % L
+
+def password_to_scalar(pw):
+    oversized = hashlib.sha512(pw).digest()
+    return int(binascii.hexlify(oversized), 16) % L
+
+def scalar_to_bytes(y):
+    y = y % L
+    assert 0 <= y < 2**256
+    return binascii.unhexlify("%064x" % y)[::-1]
+
+# Elements, of various orders
+
+def is_extended_zero(XYTZ):
+    # catch Zero
+    (X, Y, Z, T) = XYTZ
+    Y = Y % Q
+    Z = Z % Q
+    if X==0 and Y==Z and Y!=0:
+        return True
+    return False
+
+class ElementOfUnknownGroup:
+    # This is used for points of order 2,4,8,2*L,4*L,8*L
+    def __init__(self, XYTZ):
+        assert isinstance(XYTZ, tuple)
+        assert len(XYTZ) == 4
+        self.XYTZ = XYTZ
+
+    def add(self, other):
+        if not isinstance(other, ElementOfUnknownGroup):
+            raise TypeError("elements can only be added to other elements")
+        sum_XYTZ = add_elements(self.XYTZ, other.XYTZ)
+        if is_extended_zero(sum_XYTZ):
+            return Zero
+        return ElementOfUnknownGroup(sum_XYTZ)
+
+    def scalarmult(self, s):
+        if isinstance(s, ElementOfUnknownGroup):
+            raise TypeError("elements cannot be multiplied together")
+        assert s >= 0
+        product = scalarmult_element_safe_slow(self.XYTZ, s)
+        return ElementOfUnknownGroup(product)
+
+    def to_bytes(self):
+        return encodepoint(xform_extended_to_affine(self.XYTZ))
+    def __eq__(self, other):
+        return self.to_bytes() == other.to_bytes()
+    def __ne__(self, other):
+        return not self == other
+
+class Element(ElementOfUnknownGroup):
+    # this only holds elements in the main 1*L subgroup. It never holds Zero,
+    # or elements of order 1/2/4/8, or 2*L/4*L/8*L.
+
+    def add(self, other):
+        if not isinstance(other, ElementOfUnknownGroup):
+            raise TypeError("elements can only be added to other elements")
+        sum_element = ElementOfUnknownGroup.add(self, other)
+        if sum_element is Zero:
+            return sum_element
+        if isinstance(other, Element):
+            # adding two subgroup elements results in another subgroup
+            # element, or Zero, and we've already excluded Zero
+            return Element(sum_element.XYTZ)
+        # not necessarily a subgroup member, so assume not
+        return sum_element
+
+    def scalarmult(self, s):
+        if isinstance(s, ElementOfUnknownGroup):
+            raise TypeError("elements cannot be multiplied together")
+        # scalarmult of subgroup members can be done modulo the subgroup
+        # order, and using the faster non-unified function.
+        s = s % L
+        # scalarmult(s=0) gets you Zero
+        if s == 0:
+            return Zero
+        # scalarmult(s=1) gets you self, which is a subgroup member
+        # scalarmult(s<grouporder) gets you a different subgroup member
+        return Element(scalarmult_element(self.XYTZ, s))
+
+    # negation and subtraction only make sense for the main subgroup
+    def negate(self):
+        # slow. Prefer e.scalarmult(-pw) to e.scalarmult(pw).negate()
+        return Element(scalarmult_element(self.XYTZ, L-2))
+    def subtract(self, other):
+        return self.add(other.negate())
+
+class _ZeroElement(ElementOfUnknownGroup):
+    def add(self, other):
+        return other # zero+anything = anything
+    def scalarmult(self, s):
+        return self # zero*anything = zero
+    def negate(self):
+        return self # -zero = zero
+    def subtract(self, other):
+        return self.add(other.negate())
+
+
+Base = Element(xform_affine_to_extended(B))
+Zero = _ZeroElement(xform_affine_to_extended((0,1))) # the neutral (identity) element
+
+_zero_bytes = Zero.to_bytes()
+
+
+def arbitrary_element(seed): # unknown DL
+    # TODO: if we don't need uniformity, maybe use just sha256 here?
+    hseed = hashlib.sha512(seed).digest()
+    y = int(binascii.hexlify(hseed), 16) % Q
+
+    # we try successive Y values until we find a valid point
+    for plus in itertools.count(0):
+        y_plus = (y + plus) % Q
+        x = xrecover(y_plus)
+        Pa = [x,y_plus] # no attempt to use both "positive" and "negative" X
+
+        # only about 50% of Y coordinates map to valid curve points (I think
+        # the other half give you points on the "twist").
+        if not isoncurve(Pa):
+            continue
+
+        P = ElementOfUnknownGroup(xform_affine_to_extended(Pa))
+        # even if the point is on our curve, it may not be in our particular
+        # (order=L) subgroup. The curve has order 8*L, so an arbitrary point
+        # could have order 1,2,4,8,1*L,2*L,4*L,8*L (everything which divides
+        # the group order).
+
+        # [I MAY BE COMPLETELY WRONG ABOUT THIS, but my brief statistical
+        # tests suggest it's not too far off] There are phi(x) points with
+        # order x, so:
+        #  1 element of order 1: [(x=0,y=1)=Zero]
+        #  1 element of order 2 [(x=0,y=-1)]
+        #  2 elements of order 4
+        #  4 elements of order 8
+        #  L-1 elements of order L (including Base)
+        #  L-1 elements of order 2*L
+        #  2*(L-1) elements of order 4*L
+        #  4*(L-1) elements of order 8*L
+
+        # So 50% of random points will have order 8*L, 25% will have order
+        # 4*L, 13% order 2*L, and 13% will have our desired order 1*L (and a
+        # vanishingly small fraction will have 1/2/4/8). If we multiply any
+        # of the 8*L points by 2, we're sure to get an 4*L point (and
+        # multiplying a 4*L point by 2 gives us a 2*L point, and so on).
+        # Multiplying a 1*L point by 2 gives us a different 1*L point. So
+        # multiplying by 8 gets us from almost any point into a uniform point
+        # on the correct 1*L subgroup.
+
+        P8 = P.scalarmult(8)
+
+        # if we got really unlucky and picked one of the 8 low-order points,
+        # multiplying by 8 will get us to the identity (Zero), which we check
+        # for explicitly.
+        if is_extended_zero(P8.XYTZ):
+            continue
+
+        # Test that we're finally in the right group. We want to scalarmult
+        # by L, and we want to *not* use the trick in Group.scalarmult()
+        # which does x%L, because that would bypass the check we care about.
+        # P is still an _ElementOfUnknownGroup, which doesn't use x%L because
+        # that's not correct for points outside the main group.
+        assert is_extended_zero(P8.scalarmult(L).XYTZ)
+
+        return Element(P8.XYTZ)
+    # never reached
+
+def bytes_to_unknown_group_element(bytes):
+    # this accepts all elements, including Zero and wrong-subgroup ones
+    if bytes == _zero_bytes:
+        return Zero
+    XYTZ = xform_affine_to_extended(decodepoint(bytes))
+    return ElementOfUnknownGroup(XYTZ)
+
+def bytes_to_element(bytes):
+    # this strictly only accepts elements in the right subgroup
+    P = bytes_to_unknown_group_element(bytes)
+    if P is Zero:
+        raise ValueError("element was Zero")
+    if not is_extended_zero(P.scalarmult(L).XYTZ):
+        raise ValueError("element is not in the right group")
+    # the point is in the expected 1*L subgroup, not in the 2/4/8 groups,
+    # or in the 2*L/4*L/8*L groups. Promote it to a correct-group Element.
+    return Element(P.XYTZ)

RNS/Cryptography/pure25519/ed25519_oop.py

@@ -0,0 +1,213 @@
+# MIT License
+#
+# Copyright (c) 2015 Brian Warner and other contributors
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import os
+import base64
+from . import _ed25519
+BadSignatureError = _ed25519.BadSignatureError
+
+def create_keypair(entropy=os.urandom):
+    SEEDLEN = int(_ed25519.SECRETKEYBYTES/2)
+    assert SEEDLEN == 32
+    seed = entropy(SEEDLEN)
+    sk = SigningKey(seed)
+    vk = sk.get_verifying_key()
+    return sk, vk
+
+class BadPrefixError(Exception):
+    pass
+
+def remove_prefix(s_bytes, prefix):
+    assert(type(s_bytes) == type(prefix))
+    if s_bytes[:len(prefix)] != prefix:
+        raise BadPrefixError("did not see expected '%s' prefix" % (prefix,))
+    return s_bytes[len(prefix):]
+
+def to_ascii(s_bytes, prefix="", encoding="base64"):
+    """Return a version-prefixed ASCII representation of the given binary
+    string. 'encoding' indicates how to do the encoding, and can be one of:
+     * base64
+     * base32
+     * base16 (or hex)
+
+    This function handles bytes, not bits, so it does not append any trailing
+    '=' (unlike standard base64.b64encode). It also lowercases the base32
+    output.
+
+    'prefix' will be prepended to the encoded form, and is useful for
+    distinguishing the purpose and version of the binary string. E.g. you
+    could prepend 'pub0-' to a VerifyingKey string to allow the receiving
+    code to raise a useful error if someone pasted in a signature string by
+    mistake.
+    """
+    assert isinstance(s_bytes, bytes)
+    if not isinstance(prefix, bytes):
+        prefix = prefix.encode('ascii')
+    if encoding == "base64":
+        s_ascii = base64.b64encode(s_bytes).decode('ascii').rstrip("=")
+    elif encoding == "base32":
+        s_ascii = base64.b32encode(s_bytes).decode('ascii').rstrip("=").lower()
+    elif encoding in ("base16", "hex"):
+        s_ascii = base64.b16encode(s_bytes).decode('ascii').lower()
+    else:
+        raise NotImplementedError
+    return prefix+s_ascii.encode('ascii')
+
+def from_ascii(s_ascii, prefix="", encoding="base64"):
+    """This is the opposite of to_ascii. It will throw BadPrefixError if
+    the prefix is not found.
+    """
+    if isinstance(s_ascii, bytes):
+        s_ascii = s_ascii.decode('ascii')
+    if isinstance(prefix, bytes):
+        prefix = prefix.decode('ascii')
+    s_ascii = remove_prefix(s_ascii.strip(), prefix)
+    if encoding == "base64":
+        s_ascii += "="*((4 - len(s_ascii)%4)%4)
+        s_bytes = base64.b64decode(s_ascii)
+    elif encoding == "base32":
+        s_ascii += "="*((8 - len(s_ascii)%8)%8)
+        s_bytes = base64.b32decode(s_ascii.upper())
+    elif encoding in ("base16", "hex"):
+        s_bytes = base64.b16decode(s_ascii.upper())
+    else:
+        raise NotImplementedError
+    return s_bytes
+
+class SigningKey(object):
+    # this can only be used to reconstruct a key created by create_keypair().
+    def __init__(self, sk_s, prefix="", encoding=None):
+        assert isinstance(sk_s, bytes)
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        sk_s = remove_prefix(sk_s, prefix)
+        if encoding is not None:
+            sk_s = from_ascii(sk_s, encoding=encoding)
+        if len(sk_s) == 32:
+            # create from seed
+            vk_s, sk_s = _ed25519.publickey(sk_s)
+        else:
+            if len(sk_s) != 32+32:
+                raise ValueError("SigningKey takes 32-byte seed or 64-byte string")
+        self.sk_s = sk_s # seed+pubkey
+        self.vk_s = sk_s[32:] # just pubkey
+
+    def to_bytes(self, prefix=""):
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        return prefix+self.sk_s
+
+    def to_ascii(self, prefix="", encoding=None):
+        assert encoding
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        return to_ascii(self.to_seed(), prefix, encoding)
+
+    def to_seed(self, prefix=""):
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        return prefix+self.sk_s[:32]
+
+    def __eq__(self, them):
+        if not isinstance(them, object): return False
+        return (them.__class__ == self.__class__
+                and them.sk_s == self.sk_s)
+
+    def get_verifying_key(self):
+        return VerifyingKey(self.vk_s)
+
+    def sign(self, msg, prefix="", encoding=None):
+        assert isinstance(msg, bytes)
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        sig_and_msg = _ed25519.sign(msg, self.sk_s)
+        # the response is R+S+msg
+        sig_R = sig_and_msg[0:32]
+        sig_S = sig_and_msg[32:64]
+        msg_out = sig_and_msg[64:]
+        sig_out = sig_R + sig_S
+        assert msg_out == msg
+        if encoding:
+            return to_ascii(sig_out, prefix, encoding)
+        return prefix+sig_out
+
+class VerifyingKey(object):
+    def __init__(self, vk_s, prefix="", encoding=None):
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        if not isinstance(vk_s, bytes):
+            vk_s = vk_s.encode('ascii')
+        assert isinstance(vk_s, bytes)
+        vk_s = remove_prefix(vk_s, prefix)
+        if encoding is not None:
+            vk_s = from_ascii(vk_s, encoding=encoding)
+
+        assert len(vk_s) == 32
+        self.vk_s = vk_s
+
+    def to_bytes(self, prefix=""):
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        return prefix+self.vk_s
+
+    def to_ascii(self, prefix="", encoding=None):
+        assert encoding
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        return to_ascii(self.vk_s, prefix, encoding)
+
+    def __eq__(self, them):
+        if not isinstance(them, object): return False
+        return (them.__class__ == self.__class__
+                and them.vk_s == self.vk_s)
+
+    def verify(self, sig, msg, prefix="", encoding=None):
+        if not isinstance(sig, bytes):
+            sig = sig.encode('ascii')
+        if not isinstance(prefix, bytes):
+            prefix = prefix.encode('ascii')
+        assert isinstance(sig, bytes)
+        assert isinstance(msg, bytes)
+        if encoding:
+            sig = from_ascii(sig, prefix, encoding)
+        else:
+            sig = remove_prefix(sig, prefix)
+        assert len(sig) == 64
+        sig_R = sig[:32]
+        sig_S = sig[32:]
+        sig_and_msg = sig_R + sig_S + msg
+        # this might raise BadSignatureError
+        msg2 = _ed25519.open(sig_and_msg, self.vk_s)
+        assert msg2 == msg
+
+def selftest():
+    message = b"crypto libraries should always test themselves at powerup"
+    sk = SigningKey(b"priv0-VIsfn5OFGa09Un2MR6Hm7BQ5++xhcQskU2OGXG8jSJl4cWLZrRrVcSN2gVYMGtZT+3354J5jfmqAcuRSD9KIyg",
+                    prefix="priv0-", encoding="base64")
+    vk = VerifyingKey(b"pub0-eHFi2a0a1XEjdoFWDBrWU/t9+eCeY35qgHLkUg/SiMo",
+                      prefix="pub0-", encoding="base64")
+    assert sk.get_verifying_key() == vk
+    sig = sk.sign(message, prefix="sig0-", encoding="base64")
+    assert sig == b"sig0-E/QrwtSF52x8+q0l4ahA7eJbRKc777ClKNg217Q0z4fiYMCdmAOI+rTLVkiFhX6k3D+wQQfKdJYMxaTUFfv1DQ", sig
+    vk.verify(sig, message, prefix="sig0-", encoding="base64")
+
+selftest()

RNS/Cryptography/pure25519/eddsa.py

@@ -0,0 +1,94 @@
+# MIT License
+#
+# Copyright (c) 2015 Brian Warner and other contributors
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from RNS.Cryptography.Hashes import sha512
+from .basic import (bytes_to_clamped_scalar,
+                    bytes_to_scalar, scalar_to_bytes,
+                    bytes_to_element, Base)
+import hashlib, binascii
+
+def H(m):
+    return sha512(m)
+
+def publickey(seed):
+    # turn first half of SHA512(seed) into scalar, then into point
+    assert len(seed) == 32
+    a = bytes_to_clamped_scalar(H(seed)[:32])
+    A = Base.scalarmult(a)
+    return A.to_bytes()
+
+def Hint(m):
+    h = H(m)
+    return int(binascii.hexlify(h[::-1]), 16)
+
+def signature(m,sk,pk):
+    assert len(sk) == 32 # seed
+    assert len(pk) == 32
+    h = H(sk[:32])
+    a_bytes, inter = h[:32], h[32:]
+    a = bytes_to_clamped_scalar(a_bytes)
+    r = Hint(inter + m)
+    R = Base.scalarmult(r)
+    R_bytes = R.to_bytes()
+    S = r + Hint(R_bytes + pk + m) * a
+    return R_bytes + scalar_to_bytes(S)
+
+def checkvalid(s, m, pk):
+    if len(s) != 64: raise Exception("signature length is wrong")
+    if len(pk) != 32: raise Exception("public-key length is wrong")
+    R = bytes_to_element(s[:32])
+    A = bytes_to_element(pk)
+    S = bytes_to_scalar(s[32:])
+    h = Hint(s[:32] + pk + m)
+    v1 = Base.scalarmult(S)
+    v2 = R.add(A.scalarmult(h))
+    return v1==v2
+
+# wrappers
+
+import os
+
+def create_signing_key():
+    seed = os.urandom(32)
+    return seed
+
+def create_verifying_key(signing_key):
+    return publickey(signing_key)
+
+def sign(skbytes, msg):
+    """Return just the signature, given the message and just the secret
+    key."""
+    if len(skbytes) != 32:
+        raise ValueError("Bad signing key length %d" % len(skbytes))
+    vkbytes = create_verifying_key(skbytes)
+    sig = signature(msg, skbytes, vkbytes)
+    return sig
+
+def verify(vkbytes, sig, msg):
+    if len(vkbytes) != 32:
+        raise ValueError("Bad verifying key length %d" % len(vkbytes))
+    if len(sig) != 64:
+        raise ValueError("Bad signature length %d" % len(sig))
+    rc = checkvalid(sig, msg, vkbytes)
+    if not rc:
+        raise ValueError("rc != 0", rc)
+    return True

RNS/Destination.py

@@ -1,237 +1,455 @@
-import base64
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 import math
+import time
 import RNS
 
-from cryptography.fernet import Fernet
-from cryptography.hazmat.primitives import hashes
-from cryptography.hazmat.backends import default_backend
-from cryptography.hazmat.primitives import serialization
-from cryptography.hazmat.primitives.asymmetric import rsa
-from cryptography.hazmat.primitives.asymmetric import padding
+from RNS.Cryptography import Fernet
 
 class Callbacks:
-	def __init__(self):
-		self.link_established = None
-		self.packet = None
-		self.proof_requested = None
+    def __init__(self):
+        self.link_established = None
+        self.packet = None
+        self.proof_requested = None
 
 class Destination:
-	KEYSIZE    = RNS.Identity.KEYSIZE;
-	PADDINGSIZE= RNS.Identity.PADDINGSIZE;
+    """
+    A class used to describe endpoints in a Reticulum Network. Destination
+    instances are used both to create outgoing and incoming endpoints. The
+    destination type will decide if encryption, and what type, is used in
+    communication with the endpoint. A destination can also announce its
+    presence on the network, which will also distribute necessary keys for
+    encrypted communication with it.
 
-	# Constants
-	SINGLE     = 0x00
-	GROUP      = 0x01
-	PLAIN      = 0x02
-	LINK       = 0x03
-	types      = [SINGLE, GROUP, PLAIN, LINK]
+    :param identity: An instance of :ref:`RNS.Identity<api-identity>`. Can hold only public keys for an outgoing destination, or holding private keys for an ingoing.
+    :param direction: ``RNS.Destination.IN`` or ``RNS.Destination.OUT``.
+    :param type: ``RNS.Destination.SINGLE``, ``RNS.Destination.GROUP`` or ``RNS.Destination.PLAIN``.
+    :param app_name: A string specifying the app name.
+    :param \*aspects: Any non-zero number of string arguments.
+    """
 
-	PROVE_NONE = 0x21
-	PROVE_APP  = 0x22
-	PROVE_ALL  = 0x23
-	proof_strategies = [PROVE_NONE, PROVE_APP, PROVE_ALL]
+    # Constants
+    SINGLE     = 0x00
+    GROUP      = 0x01
+    PLAIN      = 0x02
+    LINK       = 0x03
+    types      = [SINGLE, GROUP, PLAIN, LINK]
 
-	IN         = 0x11;
-	OUT        = 0x12;
-	directions = [IN, OUT]
+    PROVE_NONE = 0x21
+    PROVE_APP  = 0x22
+    PROVE_ALL  = 0x23
+    proof_strategies = [PROVE_NONE, PROVE_APP, PROVE_ALL]
 
-	@staticmethod
-	def getDestinationName(app_name, *aspects):
-		# Check input values and build name string
-		if "." in app_name: raise ValueError("Dots can't be used in app names")
+    ALLOW_NONE = 0x00
+    ALLOW_ALL  = 0x01
+    ALLOW_LIST = 0x02
+    request_policies = [ALLOW_NONE, ALLOW_ALL, ALLOW_LIST]
 
-		name = app_name
-		for aspect in aspects:
-			if "." in aspect: raise ValueError("Dots can't be used in aspects")
-			name = name + "." + aspect
+    IN         = 0x11;
+    OUT        = 0x12;
+    directions = [IN, OUT]
 
-		return name
+    @staticmethod
+    def expand_name(identity, app_name, *aspects):
+        """
+        :returns: A string containing the full human-readable name of the destination, for an app_name and a number of aspects.
+        """
+
+        # Check input values and build name string
+        if "." in app_name: raise ValueError("Dots can't be used in app names")
+
+        name = app_name
+        for aspect in aspects:
+            if "." in aspect: raise ValueError("Dots can't be used in aspects")
+            name += "." + aspect
+
+        if identity != None:
+            name += "." + identity.hexhash
+
+        return name
 
 
-	@staticmethod
-	def getDestinationHash(app_name, *aspects):
-		name = Destination.getDestinationName(app_name, *aspects)
+    @staticmethod
+    def hash(identity, app_name, *aspects):
+        """
+        :returns: A destination name in adressable hash form, for an app_name and a number of aspects.
+        """
+        name_hash = RNS.Identity.full_hash(Destination.expand_name(None, app_name, *aspects).encode("utf-8"))
+        addr_hash_material = name_hash
+        if identity != None:
+            addr_hash_material += identity.hash
 
-		# Create a digest for the destination
-		digest = hashes.Hash(hashes.SHA256(), backend=default_backend())
-		digest.update(name.encode("UTF-8"))
+        return RNS.Identity.full_hash(addr_hash_material)[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8]
 
-		return digest.finalize()[:10]
+    @staticmethod
+    def app_and_aspects_from_name(full_name):
+        """
+        :returns: A tuple containing the app name and a list of aspects, for a full-name string.
+        """
+        components = full_name.split(".")
+        return (components[0], components[1:])
+
+    @staticmethod
+    def hash_from_name_and_identity(full_name, identity):
+        """
+        :returns: A destination name in adressable hash form, for a full name string and Identity instance.
+        """
+        app_name, aspects = Destination.app_and_aspects_from_name(full_name)
+
+        return Destination.hash(identity, app_name, *aspects)
+
+    def __init__(self, identity, direction, type, app_name, *aspects):
+        # Check input values and build name string
+        if "." in app_name: raise ValueError("Dots can't be used in app names") 
+        if not type in Destination.types: raise ValueError("Unknown destination type")
+        if not direction in Destination.directions: raise ValueError("Unknown destination direction")
+
+        self.accept_link_requests = True
+        self.callbacks = Callbacks()
+        self.request_handlers = {}
+        self.type = type
+        self.direction = direction
+        self.proof_strategy = Destination.PROVE_NONE
+        self.mtu = 0
+
+        self.links = []
+
+        if identity == None and direction == Destination.IN and self.type != Destination.PLAIN:
+            identity = RNS.Identity()
+            aspects = aspects+(identity.hexhash,)
+
+        if identity != None and self.type == Destination.PLAIN:
+            raise TypeError("Selected destination type PLAIN cannot hold an identity")
+
+        self.identity = identity
+        self.name = Destination.expand_name(identity, app_name, *aspects)
+
+        # Generate the destination address hash
+        self.hash = Destination.hash(self.identity, app_name, *aspects)
+        self.name_hash = RNS.Identity.full_hash(self.expand_name(None, app_name, *aspects).encode("utf-8"))
+        self.hexhash = self.hash.hex()
+
+        self.default_app_data = None
+        self.callback = None
+        self.proofcallback = None
+
+        RNS.Transport.register_destination(self)
 
 
-	def __init__(self, identity, direction, type, app_name, *aspects):
-		# Check input values and build name string
-		if "." in app_name: raise ValueError("Dots can't be used in app names") 
-		if not type in Destination.types: raise ValueError("Unknown destination type")
-		if not direction in Destination.directions: raise ValueError("Unknown destination direction")
-		self.callbacks = Callbacks()
-		self.type = type
-		self.direction = direction
-		self.proof_strategy = Destination.PROVE_NONE
-		self.mtu = 0
-
-		self.links = []
-
-		if identity != None and type == Destination.SINGLE:
-			aspects = aspects+(identity.hexhash,)
-
-		if identity == None and direction == Destination.IN and self.type != Destination.PLAIN:
-			identity = RNS.Identity()
-			aspects = aspects+(identity.hexhash,)
-
-		self.identity = identity
-
-		self.name = Destination.getDestinationName(app_name, *aspects)		
-		self.hash = Destination.getDestinationHash(app_name, *aspects)
-		self.hexhash = self.hash.hex()
-
-		self.callback = None
-		self.proofcallback = None
-
-		RNS.Transport.registerDestination(self)
+    def __str__(self):
+        """
+        :returns: A human-readable representation of the destination including addressable hash and full name.
+        """
+        return "<"+self.name+"/"+self.hexhash+">"
 
 
-	def __str__(self):
-		return "<"+self.name+"/"+self.hexhash+">"
+    def announce(self, app_data=None, path_response=False, send=True):
+        """
+        Creates an announce packet for this destination and broadcasts it on all
+        relevant interfaces. Application specific data can be added to the announce.
+
+        :param app_data: *bytes* containing the app_data.
+        :param path_response: Internal flag used by :ref:`RNS.Transport<api-transport>`. Ignore.
+        """
+        if self.type != Destination.SINGLE:
+            raise TypeError("Only SINGLE destination types can be announced")
+            
+        destination_hash = self.hash
+        random_hash = RNS.Identity.get_random_hash()[0:5]+int(time.time()).to_bytes(5, "big")
+
+        if app_data == None and self.default_app_data != None:
+            if isinstance(self.default_app_data, bytes):
+                app_data = self.default_app_data
+            elif callable(self.default_app_data):
+                returned_app_data = self.default_app_data()
+                if isinstance(returned_app_data, bytes):
+                    app_data = returned_app_data
+        
+        signed_data = self.hash+self.identity.get_public_key()+self.name_hash+random_hash
+        if app_data != None:
+            signed_data += app_data
+
+        signature = self.identity.sign(signed_data)
+
+        announce_data = self.identity.get_public_key()+self.name_hash+random_hash+signature
+
+        if app_data != None:
+            announce_data += app_data
+
+        if path_response:
+            announce_context = RNS.Packet.PATH_RESPONSE
+        else:
+            announce_context = RNS.Packet.NONE
+
+        announce_packet = RNS.Packet(self, announce_data, RNS.Packet.ANNOUNCE, context = announce_context)
+
+        if send:
+            announce_packet.send()
+        else:
+            return announce_packet
+
+    def accepts_links(self, accepts = None):
+        """
+        Set or query whether the destination accepts incoming link requests.
+
+        :param accepts: If ``True`` or ``False``, this method sets whether the destination accepts incoming link requests. If not provided or ``None``, the method returns whether the destination currently accepts link requests.
+        :returns: ``True`` or ``False`` depending on whether the destination accepts incoming link requests, if the *accepts* parameter is not provided or ``None``.
+        """
+        if accepts == None:
+            return self.accept_link_requests
+
+        if accepts:
+            self.accept_link_requests = True
+        else:
+            self.accept_link_requests = False
+
+    def set_link_established_callback(self, callback):
+        """
+        Registers a function to be called when a link has been established to
+        this destination.
+
+        :param callback: A function or method with the signature *callback(link)* to be called when a new link is established with this destination.
+        """
+        self.callbacks.link_established = callback
+
+    def set_packet_callback(self, callback):
+        """
+        Registers a function to be called when a packet has been received by
+        this destination.
+
+        :param callback: A function or method with the signature *callback(data, packet)* to be called when this destination receives a packet.
+        """
+        self.callbacks.packet = callback
+
+    def set_proof_requested_callback(self, callback):
+        """
+        Registers a function to be called when a proof has been requested for
+        a packet sent to this destination. Allows control over when and if
+        proofs should be returned for received packets.
+
+        :param callback: A function or method to with the signature *callback(packet)* be called when a packet that requests a proof is received. The callback must return one of True or False. If the callback returns True, a proof will be sent. If it returns False, a proof will not be sent.
+        """
+        self.callbacks.proof_requested = callback
+
+    def set_proof_strategy(self, proof_strategy):
+        """
+        Sets the destinations proof strategy.
+
+        :param proof_strategy: One of ``RNS.Destination.PROVE_NONE``, ``RNS.Destination.PROVE_ALL`` or ``RNS.Destination.PROVE_APP``. If ``RNS.Destination.PROVE_APP`` is set, the `proof_requested_callback` will be called to determine whether a proof should be sent or not.
+        """
+        if not proof_strategy in Destination.proof_strategies:
+            raise TypeError("Unsupported proof strategy")
+        else:
+            self.proof_strategy = proof_strategy
 
 
-	def link_established_callback(self, callback):
-		self.callbacks.link_established = callback
+    def register_request_handler(self, path, response_generator = None, allow = ALLOW_NONE, allowed_list = None):
+        """
+        Registers a request handler.
 
-	def packet_callback(self, callback):
-		self.callbacks.packet = callback
-
-	def proof_requested_callback(self, callback):
-		self.callbacks.proof_requested = callback
-
-	def set_proof_strategy(self, proof_strategy):
-		if not proof_strategy in Destination.proof_strategies:
-			raise TypeError("Unsupported proof strategy")
-		else:
-			self.proof_strategy = proof_strategy
-
-	def receive(self, packet):
-		plaintext = self.decrypt(packet.data)
-		if plaintext != None:
-			if packet.packet_type == RNS.Packet.LINKREQUEST:
-				self.incomingLinkRequest(plaintext, packet)
-
-			if packet.packet_type == RNS.Packet.DATA:
-				if self.callbacks.packet != None:
-					self.callbacks.packet(plaintext, packet)
-
-	def incomingLinkRequest(self, data, packet):
-		link = RNS.Link.validateRequest(self, data, packet)
-		if link != None:
-			self.links.append(link)
-
-	def createKeys(self):
-		if self.type == Destination.PLAIN:
-			raise TypeError("A plain destination does not hold any keys")
-
-		if self.type == Destination.SINGLE:
-			raise TypeError("A single destination holds keys through an Identity instance")
-
-		if self.type == Destination.GROUP:
-			self.prv_bytes = Fernet.generate_key()
-			self.prv = Fernet(self.prv_bytes)
+        :param path: The path for the request handler to be registered.
+        :param response_generator: A function or method with the signature *response_generator(path, data, request_id, remote_identity, requested_at)* to be called. Whatever this funcion returns will be sent as a response to the requester. If the function returns ``None``, no response will be sent.
+        :param allow: One of ``RNS.Destination.ALLOW_NONE``, ``RNS.Destination.ALLOW_ALL`` or ``RNS.Destination.ALLOW_LIST``. If ``RNS.Destination.ALLOW_LIST`` is set, the request handler will only respond to requests for identified peers in the supplied list.
+        :param allowed_list: A list of *bytes-like* :ref:`RNS.Identity<api-identity>` hashes.
+        :raises: ``ValueError`` if any of the supplied arguments are invalid.
+        """
+        if path == None or path == "":
+            raise ValueError("Invalid path specified")
+        elif not callable(response_generator):
+            raise ValueError("Invalid response generator specified")
+        elif not allow in Destination.request_policies:
+            raise ValueError("Invalid request policy")
+        else:
+            path_hash = RNS.Identity.truncated_hash(path.encode("utf-8"))
+            request_handler = [path, response_generator, allow, allowed_list]
+            self.request_handlers[path_hash] = request_handler
 
 
-	def getPrivateKey(self):
-		if self.type == Destination.PLAIN:
-			raise TypeError("A plain destination does not hold any keys")
-		elif self.type == Destination.SINGLE:
-			raise TypeError("A single destination holds keys through an Identity instance")
-		else:
-			return self.prv_bytes
+    def deregister_request_handler(self, path):
+        """
+        Deregisters a request handler.
+
+        :param path: The path for the request handler to be deregistered.
+        :returns: True if the handler was deregistered, otherwise False.
+        """
+        path_hash = RNS.Identity.truncated_hash(path.encode("utf-8"))
+        if path_hash in self.request_handlers:
+            self.request_handlers.pop(path_hash)
+            return True
+        else:
+            return False
+
+        
+
+    def receive(self, packet):
+        if packet.packet_type == RNS.Packet.LINKREQUEST:
+            plaintext = packet.data
+            self.incoming_link_request(plaintext, packet)
+        else:
+            plaintext = self.decrypt(packet.data)
+            if plaintext != None:
+                if packet.packet_type == RNS.Packet.DATA:
+                    if self.callbacks.packet != None:
+                        try:
+                            self.callbacks.packet(plaintext, packet)
+                        except Exception as e:
+                            RNS.log("Error while executing receive callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
 
 
-	def loadPrivateKey(self, key):
-		if self.type == Destination.PLAIN:
-			raise TypeError("A plain destination does not hold any keys")
+    def incoming_link_request(self, data, packet):
+        if self.accept_link_requests:
+            link = RNS.Link.validate_request(self, data, packet)
+            if link != None:
+                self.links.append(link)
 
-		if self.type == Destination.SINGLE:
-			raise TypeError("A single destination holds keys through an Identity instance")
+    def create_keys(self):
+        """
+        For a ``RNS.Destination.GROUP`` type destination, creates a new symmetric key.
 
-		if self.type == Destination.GROUP:
-			self.prv_bytes = key
-			self.prv = Fernet(self.prv_bytes)
+        :raises: ``TypeError`` if called on an incompatible type of destination.
+        """
+        if self.type == Destination.PLAIN:
+            raise TypeError("A plain destination does not hold any keys")
 
-	def loadPublicKey(self, key):
-		if self.type != Destination.SINGLE:
-			raise TypeError("Only the \"single\" destination type can hold a public key")
-		else:
-			raise TypeError("A single destination holds keys through an Identity instance")
+        if self.type == Destination.SINGLE:
+            raise TypeError("A single destination holds keys through an Identity instance")
+
+        if self.type == Destination.GROUP:
+            self.prv_bytes = Fernet.generate_key()
+            self.prv = Fernet(self.prv_bytes)
 
 
-	def encrypt(self, plaintext):
-		if self.type == Destination.PLAIN:
-			return plaintext
+    def get_private_key(self):
+        """
+        For a ``RNS.Destination.GROUP`` type destination, returns the symmetric private key.
 
-		if self.type == Destination.SINGLE and self.identity != None:
-			return self.identity.encrypt(plaintext)
+        :raises: ``TypeError`` if called on an incompatible type of destination.
+        """
+        if self.type == Destination.PLAIN:
+            raise TypeError("A plain destination does not hold any keys")
+        elif self.type == Destination.SINGLE:
+            raise TypeError("A single destination holds keys through an Identity instance")
+        else:
+            return self.prv_bytes
 
-		if self.type == Destination.GROUP:
-			if hasattr(self, "prv") and self.prv != None:
-				try:
-					return base64.urlsafe_b64decode(self.prv.encrypt(plaintext))
-				except Exception as e:
-					RNS.log("The GROUP destination could not encrypt data", RNS.LOG_ERROR)
-					RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
-			else:
-				raise ValueError("No private key held by GROUP destination. Did you create or load one?")
+
+    def load_private_key(self, key):
+        """
+        For a ``RNS.Destination.GROUP`` type destination, loads a symmetric private key.
+
+        :param key: A *bytes-like* containing the symmetric key.
+        :raises: ``TypeError`` if called on an incompatible type of destination.
+        """
+        if self.type == Destination.PLAIN:
+            raise TypeError("A plain destination does not hold any keys")
+
+        if self.type == Destination.SINGLE:
+            raise TypeError("A single destination holds keys through an Identity instance")
+
+        if self.type == Destination.GROUP:
+            self.prv_bytes = key
+            self.prv = Fernet(self.prv_bytes)
+
+    def load_public_key(self, key):
+        if self.type != Destination.SINGLE:
+            raise TypeError("Only the \"single\" destination type can hold a public key")
+        else:
+            raise TypeError("A single destination holds keys through an Identity instance")
+
+
+    def encrypt(self, plaintext):
+        """
+        Encrypts information for ``RNS.Destination.SINGLE`` or ``RNS.Destination.GROUP`` type destination.
+
+        :param plaintext: A *bytes-like* containing the plaintext to be encrypted.
+        :raises: ``ValueError`` if destination does not hold a necessary key for encryption.
+        """
+        if self.type == Destination.PLAIN:
+            return plaintext
+
+        if self.type == Destination.SINGLE and self.identity != None:
+            return self.identity.encrypt(plaintext)
+
+        if self.type == Destination.GROUP:
+            if hasattr(self, "prv") and self.prv != None:
+                try:
+                    return self.prv.encrypt(plaintext)
+                except Exception as e:
+                    RNS.log("The GROUP destination could not encrypt data", RNS.LOG_ERROR)
+                    RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+            else:
+                raise ValueError("No private key held by GROUP destination. Did you create or load one?")
 
 
 
-	def decrypt(self, ciphertext):
-		if self.type == Destination.PLAIN:
-			return ciphertext
+    def decrypt(self, ciphertext):
+        """
+        Decrypts information for ``RNS.Destination.SINGLE`` or ``RNS.Destination.GROUP`` type destination.
 
-		if self.type == Destination.SINGLE and self.identity != None:
-			return self.identity.decrypt(ciphertext)
+        :param ciphertext: *Bytes* containing the ciphertext to be decrypted.
+        :raises: ``ValueError`` if destination does not hold a necessary key for decryption.
+        """
+        if self.type == Destination.PLAIN:
+            return ciphertext
 
-		if self.type == Destination.GROUP:
-			if hasattr(self, "prv") and self.prv != None:
-				try:
-					return self.prv.decrypt(base64.urlsafe_b64encode(ciphertext))
-				except Exception as e:
-					RNS.log("The GROUP destination could not decrypt data", RNS.LOG_ERROR)
-					RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
-			else:
-				raise ValueError("No private key held by GROUP destination. Did you create or load one?")
+        if self.type == Destination.SINGLE and self.identity != None:
+            return self.identity.decrypt(ciphertext)
+
+        if self.type == Destination.GROUP:
+            if hasattr(self, "prv") and self.prv != None:
+                try:
+                    return self.prv.decrypt(ciphertext)
+                except Exception as e:
+                    RNS.log("The GROUP destination could not decrypt data", RNS.LOG_ERROR)
+                    RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+            else:
+                raise ValueError("No private key held by GROUP destination. Did you create or load one?")
 
 
-	def sign(self, message):
-		if self.type == Destination.SINGLE and self.identity != None:
-			return self.identity.sign(message)
-		else:
-			return None
+    def sign(self, message):
+        """
+        Signs information for ``RNS.Destination.SINGLE`` type destination.
 
+        :param message: *Bytes* containing the message to be signed.
+        :returns: A *bytes-like* containing the message signature, or *None* if the destination could not sign the message.
+        """
+        if self.type == Destination.SINGLE and self.identity != None:
+            return self.identity.sign(message)
+        else:
+            return None
 
-	# Creates an announce packet for this destination.
-	# Application specific data can be added to the announce.
-	def announce(self, app_data=None, path_response=False):
-		destination_hash = self.hash
-		random_hash = RNS.Identity.getRandomHash()
-		
-		signed_data = self.hash+self.identity.getPublicKey()+random_hash
-		if app_data != None:
-			signed_data += app_data
+    def set_default_app_data(self, app_data=None):
+        """
+        Sets the default app_data for the destination. If set, the default
+        app_data will be included in every announce sent by the destination,
+        unless other app_data is specified in the *announce* method.
 
-		signature = self.identity.sign(signed_data)
-
-		# TODO: Check if this could be optimised by only
-		# carrying the hash in the destination field, not
-		# also redundantly inside the signed blob as here
-		announce_data = self.hash+self.identity.getPublicKey()+random_hash+signature
-
-		if app_data != None:
-			announce_data += app_data
-
-		if path_response:
-			announce_context = RNS.Packet.PATH_RESPONSE
-		else:
-			announce_context = RNS.Packet.NONE
-
-		RNS.Packet(self, announce_data, RNS.Packet.ANNOUNCE, context = announce_context).send()
+        :param app_data: A *bytes-like* containing the default app_data, or a *callable* returning a *bytes-like* containing the app_data.
+        """
+        self.default_app_data = app_data
 
+    def clear_default_app_data(self):
+        """
+        Clears default app_data previously set for the destination.
+        """
+        self.set_default_app_data(app_data=None)

RNS/Identity.py

@@ -1,326 +1,551 @@
-import base64
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 import math
 import os
 import RNS
 import time
 import atexit
+import hashlib
+
 from .vendor import umsgpack as umsgpack
-from cryptography.hazmat.primitives import hashes
-from cryptography.hazmat.backends import default_backend
-from cryptography.hazmat.primitives import serialization
-from cryptography.hazmat.primitives.serialization import load_der_public_key
-from cryptography.hazmat.primitives.serialization import load_der_private_key
-from cryptography.hazmat.primitives.asymmetric import rsa
-from cryptography.hazmat.primitives.asymmetric import padding
+
+from RNS.Cryptography import X25519PrivateKey, X25519PublicKey, Ed25519PrivateKey, Ed25519PublicKey
+from RNS.Cryptography import Fernet
+
 
 class Identity:
-   #KEYSIZE     = 1536
-	KEYSIZE     = 1024
-	DERKEYSIZE  = KEYSIZE+272
+    """
+    This class is used to manage identities in Reticulum. It provides methods
+    for encryption, decryption, signatures and verification, and is the basis
+    for all encrypted communication over Reticulum networks.
 
-	# Non-configurable constants
-	PADDINGSIZE = 336		# In bits
-	HASHLENGTH  = 256		# In bits
-	SIGLENGTH   = KEYSIZE
+    :param create_keys: Specifies whether new encryption and signing keys should be generated.
+    """
 
-	ENCRYPT_CHUNKSIZE = (KEYSIZE-PADDINGSIZE)//8
-	DECRYPT_CHUNKSIZE = KEYSIZE//8
+    CURVE = "Curve25519"
+    """
+    The curve used for Elliptic Curve DH key exchanges
+    """
 
-	TRUNCATED_HASHLENGTH = 80 # In bits
+    KEYSIZE     = 256*2
+    """
+    X25519 key size in bits. A complete key is the concatenation of a 256 bit encryption key, and a 256 bit signing key.
+    """   
 
-	# Storage
-	known_destinations = {}
+    # Non-configurable constants
+    FERNET_OVERHEAD           = RNS.Cryptography.Fernet.FERNET_OVERHEAD
+    AES128_BLOCKSIZE          = 16          # In bytes
+    HASHLENGTH                = 256         # In bits
+    SIGLENGTH                 = KEYSIZE     # In bits
 
-	@staticmethod
-	def remember(packet_hash, destination_hash, public_key, app_data = None):
-		Identity.known_destinations[destination_hash] = [time.time(), packet_hash, public_key, app_data]
+    TRUNCATED_HASHLENGTH = RNS.Reticulum.TRUNCATED_HASHLENGTH
+    """
+    Constant specifying the truncated hash length (in bits) used by Reticulum
+    for addressable hashes and other purposes. Non-configurable.
+    """
+
+    # Storage
+    known_destinations = {}
+
+    @staticmethod
+    def remember(packet_hash, destination_hash, public_key, app_data = None):
+        if len(public_key) != Identity.KEYSIZE//8:
+            raise TypeError("Can't remember "+RNS.prettyhexrep(destination_hash)+", the public key size of "+str(len(public_key))+" is not valid.", RNS.LOG_ERROR)
+        else:
+            Identity.known_destinations[destination_hash] = [time.time(), packet_hash, public_key, app_data]
 
 
-	@staticmethod
-	def recall(destination_hash):
-		RNS.log("Searching for "+RNS.prettyhexrep(destination_hash)+"...", RNS.LOG_EXTREME)
-		if destination_hash in Identity.known_destinations:
-			identity_data = Identity.known_destinations[destination_hash]
-			identity = Identity(public_only=True)
-			identity.loadPublicKey(identity_data[2])
-			RNS.log("Found "+RNS.prettyhexrep(destination_hash)+" in known destinations", RNS.LOG_EXTREME)
-			return identity
-		else:
-			RNS.log("Could not find "+RNS.prettyhexrep(destination_hash)+" in known destinations", RNS.LOG_EXTREME)
-			return None
+    @staticmethod
+    def recall(destination_hash):
+        """
+        Recall identity for a destination hash.
 
-	@staticmethod
-	def saveKnownDestinations():
-		RNS.log("Saving known destinations to storage...", RNS.LOG_VERBOSE)
-		file = open(RNS.Reticulum.storagepath+"/known_destinations","wb")
-		umsgpack.dump(Identity.known_destinations, file)
-		file.close()
-		RNS.log("Done saving known destinations to storage", RNS.LOG_VERBOSE)
+        :param destination_hash: Destination hash as *bytes*.
+        :returns: An :ref:`RNS.Identity<api-identity>` instance that can be used to create an outgoing :ref:`RNS.Destination<api-destination>`, or *None* if the destination is unknown.
+        """
+        if destination_hash in Identity.known_destinations:
+            identity_data = Identity.known_destinations[destination_hash]
+            identity = Identity(create_keys=False)
+            identity.load_public_key(identity_data[2])
+            identity.app_data = identity_data[3]
+            return identity
+        else:
+            return None
 
-	@staticmethod
-	def loadKnownDestinations():
-		if os.path.isfile(RNS.Reticulum.storagepath+"/known_destinations"):
-			try:
-				file = open(RNS.Reticulum.storagepath+"/known_destinations","rb")
-				Identity.known_destinations = umsgpack.load(file)
-				file.close()
-				RNS.log("Loaded "+str(len(Identity.known_destinations))+" known destination from storage", RNS.LOG_VERBOSE)
-			except:
-				RNS.log("Error loading known destinations from disk, file will be recreated on exit", RNS.LOG_ERROR)
-		else:
-			RNS.log("Destinations file does not exist, so no known destinations loaded", RNS.LOG_VERBOSE)
+    @staticmethod
+    def recall_app_data(destination_hash):
+        """
+        Recall last heard app_data for a destination hash.
 
-	@staticmethod
-	def fullHash(data):
-		digest = hashes.Hash(hashes.SHA256(), backend=default_backend())
-		digest.update(data)
+        :param destination_hash: Destination hash as *bytes*.
+        :returns: *Bytes* containing app_data, or *None* if the destination is unknown.
+        """
+        if destination_hash in Identity.known_destinations:
+            app_data = Identity.known_destinations[destination_hash][3]
+            return app_data
+        else:
+            return None
 
-		return digest.finalize()
+    @staticmethod
+    def save_known_destinations():
+        # TODO: Improve the storage method so we don't have to
+        # deserialize and serialize the entire table on every
+        # save, but the only changes. It might be possible to
+        # simply overwrite on exit now that every local client
+        # disconnect triggers a data persist.
+        
+        try:
+            if hasattr(Identity, "saving_known_destinations"):
+                wait_interval = 0.2
+                wait_timeout = 5
+                wait_start = time.time()
+                while Identity.saving_known_destinations:
+                    time.sleep(wait_interval)
+                    if time.time() > wait_start+wait_timeout:
+                        RNS.log("Could not save known destinations to storage, waiting for previous save operation timed out.", RNS.LOG_ERROR)
+                        return False
 
-	@staticmethod
-	def truncatedHash(data):
-		digest = hashes.Hash(hashes.SHA256(), backend=default_backend())
-		digest.update(data)
+            Identity.saving_known_destinations = True
+            save_start = time.time()
 
-		return digest.finalize()[:(Identity.TRUNCATED_HASHLENGTH//8)]
+            storage_known_destinations = {}
+            if os.path.isfile(RNS.Reticulum.storagepath+"/known_destinations"):
+                try:
+                    file = open(RNS.Reticulum.storagepath+"/known_destinations","rb")
+                    storage_known_destinations = umsgpack.load(file)
+                    file.close()
+                except:
+                    pass
 
-	@staticmethod
-	def getRandomHash():
-		return Identity.truncatedHash(os.urandom(10))
+            for destination_hash in storage_known_destinations:
+                if not destination_hash in Identity.known_destinations:
+                    Identity.known_destinations[destination_hash] = storage_known_destinations[destination_hash]
 
-	@staticmethod
-	def validateAnnounce(packet):
-		if packet.packet_type == RNS.Packet.ANNOUNCE:
-			RNS.log("Validating announce from "+RNS.prettyhexrep(packet.destination_hash), RNS.LOG_DEBUG)
-			destination_hash = packet.destination_hash
-			public_key = packet.data[10:Identity.DERKEYSIZE//8+10]
-			random_hash = packet.data[Identity.DERKEYSIZE//8+10:Identity.DERKEYSIZE//8+20]
-			signature = packet.data[Identity.DERKEYSIZE//8+20:Identity.DERKEYSIZE//8+20+Identity.KEYSIZE//8]
-			app_data = b""
-			if len(packet.data) > Identity.DERKEYSIZE//8+20+Identity.KEYSIZE//8:
-				app_data = packet.data[Identity.DERKEYSIZE//8+20+Identity.KEYSIZE//8:]
+            RNS.log("Saving "+str(len(Identity.known_destinations))+" known destinations to storage...", RNS.LOG_DEBUG)
+            file = open(RNS.Reticulum.storagepath+"/known_destinations","wb")
+            umsgpack.dump(Identity.known_destinations, file)
+            file.close()
 
-			signed_data = destination_hash+public_key+random_hash+app_data
+            save_time = time.time() - save_start
+            if save_time < 1:
+                time_str = str(round(save_time*1000,2))+"ms"
+            else:
+                time_str = str(round(save_time,2))+"s"
 
-			announced_identity = Identity(public_only=True)
-			announced_identity.loadPublicKey(public_key)
+            RNS.log("Saved known destinations to storage in "+time_str, RNS.LOG_DEBUG)
 
-			if announced_identity.pub != None and announced_identity.validate(signature, signed_data):
-				RNS.Identity.remember(packet.getHash(), destination_hash, public_key)
-				RNS.log("Stored valid announce from "+RNS.prettyhexrep(destination_hash), RNS.LOG_DEBUG)
-				del announced_identity
-				return True
-			else:
-				RNS.log("Received invalid announce", RNS.LOG_DEBUG)
-				del announced_identity
-				return False
+        except Exception as e:
+            RNS.log("Error while saving known destinations to disk, the contained exception was: "+str(e), RNS.LOG_ERROR)
 
-	@staticmethod
-	def exitHandler():
-		Identity.saveKnownDestinations()
+        Identity.saving_known_destinations = False
+
+    @staticmethod
+    def load_known_destinations():
+        if os.path.isfile(RNS.Reticulum.storagepath+"/known_destinations"):
+            try:
+                file = open(RNS.Reticulum.storagepath+"/known_destinations","rb")
+                loaded_known_destinations = umsgpack.load(file)
+                file.close()
+
+                Identity.known_destinations = {}
+                for known_destination in loaded_known_destinations:
+                    if len(known_destination) == RNS.Reticulum.TRUNCATED_HASHLENGTH//8:
+                        Identity.known_destinations[known_destination] = loaded_known_destinations[known_destination]
+
+                RNS.log("Loaded "+str(len(Identity.known_destinations))+" known destination from storage", RNS.LOG_VERBOSE)
+            except:
+                RNS.log("Error loading known destinations from disk, file will be recreated on exit", RNS.LOG_ERROR)
+        else:
+            RNS.log("Destinations file does not exist, no known destinations loaded", RNS.LOG_VERBOSE)
+
+    @staticmethod
+    def full_hash(data):
+        """
+        Get a SHA-256 hash of passed data.
+
+        :param data: Data to be hashed as *bytes*.
+        :returns: SHA-256 hash as *bytes*
+        """
+        return RNS.Cryptography.sha256(data)
+
+    @staticmethod
+    def truncated_hash(data):
+        """
+        Get a truncated SHA-256 hash of passed data.
+
+        :param data: Data to be hashed as *bytes*.
+        :returns: Truncated SHA-256 hash as *bytes*
+        """
+        return Identity.full_hash(data)[:(Identity.TRUNCATED_HASHLENGTH//8)]
+
+    @staticmethod
+    def get_random_hash():
+        """
+        Get a random SHA-256 hash.
+
+        :param data: Data to be hashed as *bytes*.
+        :returns: Truncated SHA-256 hash of random data as *bytes*
+        """
+        return Identity.truncated_hash(os.urandom(Identity.TRUNCATED_HASHLENGTH//8))
+
+    @staticmethod
+    def validate_announce(packet):
+        try:
+            if packet.packet_type == RNS.Packet.ANNOUNCE:
+                destination_hash = packet.destination_hash
+                public_key = packet.data[:Identity.KEYSIZE//8]
+                name_hash = packet.data[Identity.KEYSIZE//8:Identity.KEYSIZE//8+Identity.HASHLENGTH//8]
+                random_hash = packet.data[Identity.KEYSIZE//8+Identity.HASHLENGTH//8:Identity.KEYSIZE//8+Identity.HASHLENGTH//8+10]
+                signature = packet.data[Identity.KEYSIZE//8+Identity.HASHLENGTH//8+10:Identity.KEYSIZE//8+Identity.HASHLENGTH//8+10+Identity.SIGLENGTH//8]
+                app_data = b""
+                if len(packet.data) > Identity.KEYSIZE//8+Identity.HASHLENGTH//8+10+Identity.SIGLENGTH//8:
+                    app_data = packet.data[Identity.KEYSIZE//8+Identity.HASHLENGTH//8+10+Identity.SIGLENGTH//8:]
+
+                signed_data = destination_hash+public_key+name_hash+random_hash+app_data
+
+                if not len(packet.data) > Identity.KEYSIZE//8+Identity.HASHLENGTH//8+10+Identity.SIGLENGTH//8:
+                    app_data = None
+
+                announced_identity = Identity(create_keys=False)
+                announced_identity.load_public_key(public_key)
+
+                if announced_identity.pub != None and announced_identity.validate(signature, signed_data):
+                    hash_material = name_hash+announced_identity.hash
+                    expected_hash = RNS.Identity.full_hash(hash_material)[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8]
+
+                    if destination_hash == expected_hash:
+                        RNS.Identity.remember(packet.get_hash(), destination_hash, public_key, app_data)
+                        del announced_identity
+
+                        if hasattr(packet, "transport_id") and packet.transport_id != None:
+                            RNS.log("Valid announce for "+RNS.prettyhexrep(destination_hash)+" "+str(packet.hops)+" hops away, received via "+RNS.prettyhexrep(packet.transport_id)+" on "+str(packet.receiving_interface), RNS.LOG_EXTREME)
+                        else:
+                            RNS.log("Valid announce for "+RNS.prettyhexrep(destination_hash)+" "+str(packet.hops)+" hops away, received on "+str(packet.receiving_interface), RNS.LOG_EXTREME)
+
+                        return True
+
+                    else:
+                        RNS.log("Received invalid announce for "+RNS.prettyhexrep(destination_hash)+". Destination mismatch.", RNS.LOG_DEBUG)
+                        return False
+
+                else:
+                    RNS.log("Received invalid announce for "+RNS.prettyhexrep(destination_hash), RNS.LOG_DEBUG)
+                    del announced_identity
+                    return False
+        
+        except Exception as e:
+            RNS.log("Error occurred while validating announce. The contained exception was: "+str(e), RNS.LOG_ERROR)
+            return False
+
+    @staticmethod
+    def persist_data():
+        if not RNS.Transport.owner.is_connected_to_shared_instance:
+            Identity.save_known_destinations()
+
+    @staticmethod
+    def exit_handler():
+        Identity.persist_data()
 
 
-	@staticmethod
-	def from_file(path):
-		identity = Identity(public_only=True)
-		if identity.load(path):
-			return identity
-		else:
-			return None
+    @staticmethod
+    def from_bytes(prv_bytes):
+        """
+        Create a new :ref:`RNS.Identity<api-identity>` instance from *bytes* of private key.
+        Can be used to load previously created and saved identities into Reticulum.
+
+        :param prv_bytes: The *bytes* of private a saved private key. **HAZARD!** Never use this to generate a new key by feeding random data in prv_bytes.
+        :returns: A :ref:`RNS.Identity<api-identity>` instance, or *None* if the *bytes* data was invalid.
+        """
+        identity = Identity(create_keys=False)
+        if identity.load_private_key(prv_bytes):
+            return identity
+        else:
+            return None
 
 
-	def __init__(self,public_only=False):
-		# Initialize keys to none
-		self.prv = None
-		self.pub = None
-		self.prv_bytes = None
-		self.pub_bytes = None
-		self.hash = None
-		self.hexhash = None
+    @staticmethod
+    def from_file(path):
+        """
+        Create a new :ref:`RNS.Identity<api-identity>` instance from a file.
+        Can be used to load previously created and saved identities into Reticulum.
 
-		if not public_only:
-			self.createKeys()
+        :param path: The full path to the saved :ref:`RNS.Identity<api-identity>` data
+        :returns: A :ref:`RNS.Identity<api-identity>` instance, or *None* if the loaded data was invalid.
+        """
+        identity = Identity(create_keys=False)
+        if identity.load(path):
+            return identity
+        else:
+            return None
 
-	def createKeys(self):
-		self.prv = rsa.generate_private_key(
-			public_exponent=65337,
-			key_size=Identity.KEYSIZE,
-			backend=default_backend()
-		)
-		self.prv_bytes = self.prv.private_bytes(
-			encoding=serialization.Encoding.DER,
-			format=serialization.PrivateFormat.PKCS8,
-			encryption_algorithm=serialization.NoEncryption()
-		)
-		self.pub = self.prv.public_key()
-		self.pub_bytes = self.pub.public_bytes(
-			encoding=serialization.Encoding.DER,
-			format=serialization.PublicFormat.SubjectPublicKeyInfo
-		)
+    def to_file(self, path):
+        """
+        Saves the identity to a file. This will write the private key to disk,
+        and anyone with access to this file will be able to decrypt all
+        communication for the identity. Be very careful with this method.
 
-		self.updateHashes()
+        :param path: The full path specifying where to save the identity.
+        :returns: True if the file was saved, otherwise False.
+        """
+        try:
+            with open(path, "wb") as key_file:
+                key_file.write(self.get_private_key())
+                return True
+            return False
+        except Exception as e:
+            RNS.log("Error while saving identity to "+str(path), RNS.LOG_ERROR)
+            RNS.log("The contained exception was: "+str(e))
 
-		RNS.log("Identity keys created for "+RNS.prettyhexrep(self.hash), RNS.LOG_VERBOSE)
+    def __init__(self,create_keys=True):
+        # Initialize keys to none
+        self.prv           = None
+        self.prv_bytes     = None
+        self.sig_prv       = None
+        self.sig_prv_bytes = None
 
-	def getPrivateKey(self):
-		return self.prv_bytes
+        self.pub           = None
+        self.pub_bytes     = None
+        self.sig_pub       = None
+        self.sig_pub_bytes = None
 
-	def getPublicKey(self):
-		return self.pub_bytes
+        self.hash          = None
+        self.hexhash       = None
 
-	def loadPrivateKey(self, prv_bytes):
-		try:
-			self.prv_bytes = prv_bytes
-			self.prv = serialization.load_der_private_key(
-				self.prv_bytes,
-				password=None,
-				backend=default_backend()
-			)
-			self.pub = self.prv.public_key()
-			self.pub_bytes = self.pub.public_bytes(
-				encoding=serialization.Encoding.DER,
-				format=serialization.PublicFormat.SubjectPublicKeyInfo
-			)
-			self.updateHashes()
+        if create_keys:
+            self.create_keys()
 
-			return True
+    def create_keys(self):
+        self.prv           = X25519PrivateKey.generate()
+        self.prv_bytes     = self.prv.private_bytes()
 
-		except Exception as e:
-			RNS.log("Failed to load identity key", RNS.LOG_ERROR)
-			RNS.log("The contained exception was: "+str(e))
-			return False
+        self.sig_prv       = Ed25519PrivateKey.generate()
+        self.sig_prv_bytes = self.sig_prv.private_bytes()
 
-	def loadPublicKey(self, key):
-		try:
-			self.pub_bytes = key
-			self.pub = load_der_public_key(self.pub_bytes, backend=default_backend())
-			self.updateHashes()
-		except Exception as e:
-			RNS.log("Error while loading public key, the contained exception was: "+str(e), RNS.LOG_ERROR)
+        self.pub           = self.prv.public_key()
+        self.pub_bytes     = self.pub.public_bytes()
 
-	def updateHashes(self):
-		self.hash = Identity.truncatedHash(self.pub_bytes)
-		self.hexhash = self.hash.hex()
+        self.sig_pub       = self.sig_prv.public_key()
+        self.sig_pub_bytes = self.sig_pub.public_bytes()
 
-	def save(self, path):
-		try:
-			with open(path, "wb") as key_file:
-				key_file.write(self.prv_bytes)
-				return True
-			return False
-		except Exception as e:
-			RNS.log("Error while saving identity to "+str(path), RNS.LOG_ERROR)
-			RNS.log("The contained exception was: "+str(e))
+        self.update_hashes()
 
-	def load(self, path):
-		try:
-			with open(path, "rb") as key_file:
-				prv_bytes = key_file.read()
-				return self.loadPrivateKey(prv_bytes)
-			return False
-		except Exception as e:
-			RNS.log("Error while loading identity from "+str(path), RNS.LOG_ERROR)
-			RNS.log("The contained exception was: "+str(e))
+        RNS.log("Identity keys created for "+RNS.prettyhexrep(self.hash), RNS.LOG_VERBOSE)
 
-	def encrypt(self, plaintext):
-		if self.pub != None:
-			chunksize = Identity.ENCRYPT_CHUNKSIZE
-			chunks = int(math.ceil(len(plaintext)/(float(chunksize))))
+    def get_private_key(self):
+        """
+        :returns: The private key as *bytes*
+        """
+        return self.prv_bytes+self.sig_prv_bytes
 
-			ciphertext = b"";
-			for chunk in range(chunks):
-				start = chunk*chunksize
-				end = (chunk+1)*chunksize
-				if (chunk+1)*chunksize > len(plaintext):
-					end = len(plaintext)
-				
-				ciphertext += self.pub.encrypt(
-					plaintext[start:end],
-					padding.OAEP(
-						mgf=padding.MGF1(algorithm=hashes.SHA1()),
-						algorithm=hashes.SHA1(),
-						label=None
-					)
-				)
-			return ciphertext
-		else:
-			raise KeyError("Encryption failed because identity does not hold a public key")
+    def get_public_key(self):
+        """
+        :returns: The public key as *bytes*
+        """
+        return self.pub_bytes+self.sig_pub_bytes
+
+    def load_private_key(self, prv_bytes):
+        """
+        Load a private key into the instance.
+
+        :param prv_bytes: The private key as *bytes*.
+        :returns: True if the key was loaded, otherwise False.
+        """
+        try:
+            self.prv_bytes     = prv_bytes[:Identity.KEYSIZE//8//2]
+            self.prv           = X25519PrivateKey.from_private_bytes(self.prv_bytes)
+            self.sig_prv_bytes = prv_bytes[Identity.KEYSIZE//8//2:]
+            self.sig_prv       = Ed25519PrivateKey.from_private_bytes(self.sig_prv_bytes)
+            
+            self.pub           = self.prv.public_key()
+            self.pub_bytes     = self.pub.public_bytes()
+
+            self.sig_pub       = self.sig_prv.public_key()
+            self.sig_pub_bytes = self.sig_pub.public_bytes()
+
+            self.update_hashes()
+
+            return True
+
+        except Exception as e:
+            raise e
+            RNS.log("Failed to load identity key", RNS.LOG_ERROR)
+            RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+            return False
+
+    def load_public_key(self, pub_bytes):
+        """
+        Load a public key into the instance.
+
+        :param pub_bytes: The public key as *bytes*.
+        :returns: True if the key was loaded, otherwise False.
+        """
+        try:
+            self.pub_bytes     = pub_bytes[:Identity.KEYSIZE//8//2]
+            self.sig_pub_bytes = pub_bytes[Identity.KEYSIZE//8//2:]
+
+            self.pub           = X25519PublicKey.from_public_bytes(self.pub_bytes)
+            self.sig_pub       = Ed25519PublicKey.from_public_bytes(self.sig_pub_bytes)
+
+            self.update_hashes()
+        except Exception as e:
+            RNS.log("Error while loading public key, the contained exception was: "+str(e), RNS.LOG_ERROR)
+
+    def update_hashes(self):
+        self.hash = Identity.truncated_hash(self.get_public_key())
+        self.hexhash = self.hash.hex()
+
+    def load(self, path):
+        try:
+            with open(path, "rb") as key_file:
+                prv_bytes = key_file.read()
+                return self.load_private_key(prv_bytes)
+            return False
+        except Exception as e:
+            RNS.log("Error while loading identity from "+str(path), RNS.LOG_ERROR)
+            RNS.log("The contained exception was: "+str(e))
+
+    def get_salt(self):
+        return self.hash
+
+    def get_context(self):
+        return None
+
+    def encrypt(self, plaintext):
+        """
+        Encrypts information for the identity.
+
+        :param plaintext: The plaintext to be encrypted as *bytes*.
+        :returns: Ciphertext token as *bytes*.
+        :raises: *KeyError* if the instance does not hold a public key.
+        """
+        if self.pub != None:
+            ephemeral_key = X25519PrivateKey.generate()
+            ephemeral_pub_bytes = ephemeral_key.public_key().public_bytes()
+
+            shared_key = ephemeral_key.exchange(self.pub)
+            
+            derived_key = RNS.Cryptography.hkdf(
+                length=32,
+                derive_from=shared_key,
+                salt=self.get_salt(),
+                context=self.get_context(),
+            )
+
+            fernet = Fernet(derived_key)
+            ciphertext = fernet.encrypt(plaintext)
+            token = ephemeral_pub_bytes+ciphertext
+
+            return token
+        else:
+            raise KeyError("Encryption failed because identity does not hold a public key")
 
 
-	def decrypt(self, ciphertext):
-		if self.prv != None:
-			plaintext = None
-			try:
-				chunksize = Identity.DECRYPT_CHUNKSIZE
-				chunks = int(math.ceil(len(ciphertext)/(float(chunksize))))
+    def decrypt(self, ciphertext_token):
+        """
+        Decrypts information for the identity.
 
-				plaintext = b"";
-				for chunk in range(chunks):
-					start = chunk*chunksize
-					end = (chunk+1)*chunksize
-					if (chunk+1)*chunksize > len(ciphertext):
-						end = len(ciphertext)
+        :param ciphertext: The ciphertext to be decrypted as *bytes*.
+        :returns: Plaintext as *bytes*, or *None* if decryption fails.
+        :raises: *KeyError* if the instance does not hold a private key.
+        """
+        if self.prv != None:
+            if len(ciphertext_token) > Identity.KEYSIZE//8//2:
+                plaintext = None
+                try:
+                    peer_pub_bytes = ciphertext_token[:Identity.KEYSIZE//8//2]
+                    peer_pub = X25519PublicKey.from_public_bytes(peer_pub_bytes)
 
-					plaintext += self.prv.decrypt(
-						ciphertext[start:end],
-						padding.OAEP(
-							mgf=padding.MGF1(algorithm=hashes.SHA1()),
-							algorithm=hashes.SHA1(),
-							label=None
-						)
-					)
-			except:
-				RNS.log("Decryption by "+RNS.prettyhexrep(self.hash)+" failed", RNS.LOG_VERBOSE)
-				
-			return plaintext;
-		else:
-			raise KeyError("Decryption failed because identity does not hold a private key")
+                    shared_key = self.prv.exchange(peer_pub)
+
+                    derived_key = RNS.Cryptography.hkdf(
+                        length=32,
+                        derive_from=shared_key,
+                        salt=self.get_salt(),
+                        context=self.get_context(),
+                    )
+
+                    fernet = Fernet(derived_key)
+                    ciphertext = ciphertext_token[Identity.KEYSIZE//8//2:]
+                    plaintext = fernet.decrypt(ciphertext)
+
+                except Exception as e:
+                    RNS.log("Decryption by "+RNS.prettyhexrep(self.hash)+" failed: "+str(e), RNS.LOG_DEBUG)
+                    
+                return plaintext;
+            else:
+                RNS.log("Decryption failed because the token size was invalid.", RNS.LOG_DEBUG)
+                return None
+        else:
+            raise KeyError("Decryption failed because identity does not hold a private key")
 
 
-	def sign(self, message):
-		if self.prv != None:
-			signature = self.prv.sign(
-				message,
-				padding.PSS(
-					mgf=padding.MGF1(hashes.SHA256()),
-					salt_length=padding.PSS.MAX_LENGTH
-				),
-				hashes.SHA256()
-			)
-			return signature
-		else:
-			raise KeyError("Signing failed because identity does not hold a private key")
+    def sign(self, message):
+        """
+        Signs information by the identity.
 
-	def validate(self, signature, message):
-		if self.pub != None:
-			try:
-				self.pub.verify(
-					signature,
-					message,
-					padding.PSS(
-						mgf=padding.MGF1(hashes.SHA256()),
-						salt_length=padding.PSS.MAX_LENGTH
-					),
-					hashes.SHA256()
-				)
-				return True
-			except Exception as e:
-				return False
-		else:
-			raise KeyError("Signature validation failed because identity does not hold a public key")
+        :param message: The message to be signed as *bytes*.
+        :returns: Signature as *bytes*.
+        :raises: *KeyError* if the instance does not hold a private key.
+        """
+        if self.sig_prv != None:
+            try:
+                return self.sig_prv.sign(message)    
+            except Exception as e:
+                RNS.log("The identity "+str(self)+" could not sign the requested message. The contained exception was: "+str(e), RNS.LOG_ERROR)
+                raise e
+        else:
+            raise KeyError("Signing failed because identity does not hold a private key")
 
-	def prove(self, packet, destination=None):
-		signature = self.sign(packet.packet_hash)
-		if RNS.Reticulum.should_use_implicit_proof():
-			proof_data = signature
-		else:
-			proof_data = packet.packet_hash + signature
-		
-		if destination == None:
-			destination = packet.generateProofDestination()
+    def validate(self, signature, message):
+        """
+        Validates the signature of a signed message.
 
-		proof = RNS.Packet(destination, proof_data, RNS.Packet.PROOF, attached_interface = packet.receiving_interface)
-		proof.send()
+        :param signature: The signature to be validated as *bytes*.
+        :param message: The message to be validated as *bytes*.
+        :returns: True if the signature is valid, otherwise False.
+        :raises: *KeyError* if the instance does not hold a public key.
+        """
+        if self.pub != None:
+            try:
+                self.sig_pub.verify(signature, message)
+                return True
+            except Exception as e:
+                return False
+        else:
+            raise KeyError("Signature validation failed because identity does not hold a public key")
 
-	def __str__(self):
-		return RNS.prettyhexrep(self.hash)
+    def prove(self, packet, destination=None):
+        signature = self.sign(packet.packet_hash)
+        if RNS.Reticulum.should_use_implicit_proof():
+            proof_data = signature
+        else:
+            proof_data = packet.packet_hash + signature
+        
+        if destination == None:
+            destination = packet.generate_proof_destination()
+
+        proof = RNS.Packet(destination, proof_data, RNS.Packet.PROOF, attached_interface = packet.receiving_interface)
+        proof.send()
+
+    def __str__(self):
+        return RNS.prettyhexrep(self.hash)

RNS/Interfaces/AX25KISSInterface.py

@@ -1,305 +1,371 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
 
 from .Interface import Interface
 from time import sleep
 import sys
-import serial
 import threading
 import time
 import RNS
 
 class KISS():
-	FEND			  = 0xC0
-	FESC			  = 0xDB
-	TFEND			  = 0xDC
-	TFESC			  = 0xDD
-	CMD_UNKNOWN		  = 0xFE
-	CMD_DATA		  = 0x00
-	CMD_TXDELAY		  = 0x01
-	CMD_P			  = 0x02
-	CMD_SLOTTIME	  = 0x03
-	CMD_TXTAIL		  = 0x04
-	CMD_FULLDUPLEX	  = 0x05
-	CMD_SETHARDWARE	  = 0x06
-	CMD_READY         = 0x0F
-	CMD_RETURN		  = 0xFF
+    FEND              = 0xC0
+    FESC              = 0xDB
+    TFEND             = 0xDC
+    TFESC             = 0xDD
+    CMD_UNKNOWN       = 0xFE
+    CMD_DATA          = 0x00
+    CMD_TXDELAY       = 0x01
+    CMD_P             = 0x02
+    CMD_SLOTTIME      = 0x03
+    CMD_TXTAIL        = 0x04
+    CMD_FULLDUPLEX    = 0x05
+    CMD_SETHARDWARE   = 0x06
+    CMD_READY         = 0x0F
+    CMD_RETURN        = 0xFF
 
-	@staticmethod
-	def escape(data):
-		data = data.replace(bytes([0xdb]), bytes([0xdb, 0xdd]))
-		data = data.replace(bytes([0xc0]), bytes([0xdb, 0xdc]))
-		return data
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([0xdb]), bytes([0xdb, 0xdd]))
+        data = data.replace(bytes([0xc0]), bytes([0xdb, 0xdc]))
+        return data
 
 class AX25():
-	PID_NOLAYER3	= 0xF0
-	CTRL_UI			= 0x03
-	CRC_CORRECT     = bytes([0xF0])+bytes([0xB8])
-	HEADER_SIZE		= 16
+    PID_NOLAYER3    = 0xF0
+    CTRL_UI         = 0x03
+    CRC_CORRECT     = bytes([0xF0])+bytes([0xB8])
+    HEADER_SIZE     = 16
 
 
 class AX25KISSInterface(Interface):
-	MAX_CHUNK = 32768
+    MAX_CHUNK = 32768
+    BITRATE_GUESS = 1200
 
-	owner    = None
-	port     = None
-	speed    = None
-	databits = None
-	parity   = None
-	stopbits = None
-	serial   = None
+    owner    = None
+    port     = None
+    speed    = None
+    databits = None
+    parity   = None
+    stopbits = None
+    serial   = None
 
-	def __init__(self, owner, name, callsign, ssid, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control):
-		self.serial   = None
-		self.owner    = owner
-		self.name	  = name
-		self.src_call = callsign.upper().encode("ascii")
-		self.src_ssid = ssid
-		self.dst_call = "APZRNS".encode("ascii")
-		self.dst_ssid = 0
-		self.port     = port
-		self.speed    = speed
-		self.databits = databits
-		self.parity   = serial.PARITY_NONE
-		self.stopbits = stopbits
-		self.timeout  = 100
-		self.online   = False
-		# TODO: Sane default and make this configurable
-		# TODO: Changed to 25ms instead of 100ms, check it
-		self.txdelay  = 0.025
+    def __init__(self, owner, name, callsign, ssid, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control):
+        import importlib
+        if importlib.util.find_spec('serial') != None:
+            import serial
+        else:
+            RNS.log("Using the AX.25 KISS interface requires a serial communication module to be installed.", RNS.LOG_CRITICAL)
+            RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
+            RNS.panic()
 
-		self.packet_queue    = []
-		self.flow_control    = flow_control
-		self.interface_ready = False
+        self.rxb = 0
+        self.txb = 0
 
-		if (len(self.src_call) < 3 or len(self.src_call) > 6):
-			raise ValueError("Invalid callsign for "+str(self))
+        self.HW_MTU = 564
+        
+        self.pyserial = serial
+        self.serial   = None
+        self.owner    = owner
+        self.name     = name
+        self.src_call = callsign.upper().encode("ascii")
+        self.src_ssid = ssid
+        self.dst_call = "APZRNS".encode("ascii")
+        self.dst_ssid = 0
+        self.port     = port
+        self.speed    = speed
+        self.databits = databits
+        self.parity   = serial.PARITY_NONE
+        self.stopbits = stopbits
+        self.timeout  = 100
+        self.online   = False
+        self.bitrate  = KISSInterface.BITRATE_GUESS
 
-		if (self.src_ssid < 0 or self.src_ssid > 15):
-			raise ValueError("Invalid SSID for "+str(self))
+        self.packet_queue    = []
+        self.flow_control    = flow_control
+        self.interface_ready = False
+        self.flow_control_timeout = 5
+        self.flow_control_locked  = time.time()
 
-		self.preamble    = preamble if preamble != None else 350;
-		self.txtail      = txtail if txtail != None else 20;
-		self.persistence = persistence if persistence != None else 64;
-		self.slottime    = slottime if slottime != None else 20;
+        if (len(self.src_call) < 3 or len(self.src_call) > 6):
+            raise ValueError("Invalid callsign for "+str(self))
 
-		if parity.lower() == "e" or parity.lower() == "even":
-			self.parity = serial.PARITY_EVEN
+        if (self.src_ssid < 0 or self.src_ssid > 15):
+            raise ValueError("Invalid SSID for "+str(self))
 
-		if parity.lower() == "o" or parity.lower() == "odd":
-			self.parity = serial.PARITY_ODD
+        self.preamble    = preamble if preamble != None else 350;
+        self.txtail      = txtail if txtail != None else 20;
+        self.persistence = persistence if persistence != None else 64;
+        self.slottime    = slottime if slottime != None else 20;
 
-		try:
-			RNS.log("Opening serial port "+self.port+"...")
-			self.serial = serial.Serial(
-				port = self.port,
-				baudrate = self.speed,
-				bytesize = self.databits,
-				parity = self.parity,
-				stopbits = self.stopbits,
-				xonxoff = False,
-				rtscts = False,
-				timeout = 0,
-				inter_byte_timeout = None,
-				write_timeout = None,
-				dsrdtr = False,
-			)
-		except Exception as e:
-			RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
-			raise e
+        if parity.lower() == "e" or parity.lower() == "even":
+            self.parity = serial.PARITY_EVEN
 
-		if self.serial.is_open:
-			# Allow time for interface to initialise before config
-			sleep(2.0)
-			thread = threading.Thread(target=self.readLoop)
-			thread.setDaemon(True)
-			thread.start()
-			self.online = True
-			RNS.log("Serial port "+self.port+" is now open")
-			RNS.log("Configuring AX.25 KISS interface parameters...")
-			self.setPreamble(self.preamble)
-			self.setTxTail(self.txtail)
-			self.setPersistence(self.persistence)
-			self.setSlotTime(self.slottime)
-			self.setFlowControl(self.flow_control)
-			self.interface_ready = True
-			RNS.log("AX.25 KISS interface configured")
-			sleep(2)
-		else:
-			raise IOError("Could not open serial port")
+        if parity.lower() == "o" or parity.lower() == "odd":
+            self.parity = serial.PARITY_ODD
+
+        try:
+            self.open_port()
+        except Exception as e:
+            RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
+            raise e
+
+        if self.serial.is_open:
+            self.configure_device()
+        else:
+            raise IOError("Could not open serial port")
+
+    def open_port(self):
+        RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
+        self.serial = self.pyserial.Serial(
+            port = self.port,
+            baudrate = self.speed,
+            bytesize = self.databits,
+            parity = self.parity,
+            stopbits = self.stopbits,
+            xonxoff = False,
+            rtscts = False,
+            timeout = 0,
+            inter_byte_timeout = None,
+            write_timeout = None,
+            dsrdtr = False,
+        )
+
+    def configure_device(self):
+        # Allow time for interface to initialise before config
+        sleep(2.0)
+        thread = threading.Thread(target=self.readLoop)
+        thread.daemon = True
+        thread.start()
+        self.online = True
+        RNS.log("Serial port "+self.port+" is now open")
+        RNS.log("Configuring AX.25 KISS interface parameters...")
+        self.setPreamble(self.preamble)
+        self.setTxTail(self.txtail)
+        self.setPersistence(self.persistence)
+        self.setSlotTime(self.slottime)
+        self.setFlowControl(self.flow_control)
+        self.interface_ready = True
+        RNS.log("AX.25 KISS interface configured")
+
+    def setPreamble(self, preamble):
+        preamble_ms = preamble
+        preamble = int(preamble_ms / 10)
+        if preamble < 0:
+            preamble = 0
+        if preamble > 255:
+            preamble = 255
+
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_TXDELAY])+bytes([preamble])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("Could not configure AX.25 KISS interface preamble to "+str(preamble_ms)+" (command value "+str(preamble)+")")
+
+    def setTxTail(self, txtail):
+        txtail_ms = txtail
+        txtail = int(txtail_ms / 10)
+        if txtail < 0:
+            txtail = 0
+        if txtail > 255:
+            txtail = 255
+
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_TXTAIL])+bytes([txtail])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("Could not configure AX.25 KISS interface TX tail to "+str(txtail_ms)+" (command value "+str(txtail)+")")
+
+    def setPersistence(self, persistence):
+        if persistence < 0:
+            persistence = 0
+        if persistence > 255:
+            persistence = 255
+
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_P])+bytes([persistence])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("Could not configure AX.25 KISS interface persistence to "+str(persistence))
+
+    def setSlotTime(self, slottime):
+        slottime_ms = slottime
+        slottime = int(slottime_ms / 10)
+        if slottime < 0:
+            slottime = 0
+        if slottime > 255:
+            slottime = 255
+
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SLOTTIME])+bytes([slottime])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("Could not configure AX.25 KISS interface slot time to "+str(slottime_ms)+" (command value "+str(slottime)+")")
+
+    def setFlowControl(self, flow_control):
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_READY])+bytes([0x01])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            if (flow_control):
+                raise IOError("Could not enable AX.25 KISS interface flow control")
+            else:
+                raise IOError("Could not enable AX.25 KISS interface flow control")
 
 
-	def setPreamble(self, preamble):
-		preamble_ms = preamble
-		preamble = int(preamble_ms / 10)
-		if preamble < 0:
-			preamble = 0
-		if preamble > 255:
-			preamble = 255
-
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_TXDELAY])+bytes([preamble])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			raise IOError("Could not configure AX.25 KISS interface preamble to "+str(preamble_ms)+" (command value "+str(preamble)+")")
-
-	def setTxTail(self, txtail):
-		txtail_ms = txtail
-		txtail = int(txtail_ms / 10)
-		if txtail < 0:
-			txtail = 0
-		if txtail > 255:
-			txtail = 255
-
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_TXTAIL])+bytes([txtail])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			raise IOError("Could not configure AX.25 KISS interface TX tail to "+str(txtail_ms)+" (command value "+str(txtail)+")")
-
-	def setPersistence(self, persistence):
-		if persistence < 0:
-			persistence = 0
-		if persistence > 255:
-			persistence = 255
-
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_P])+bytes([persistence])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			raise IOError("Could not configure AX.25 KISS interface persistence to "+str(persistence))
-
-	def setSlotTime(self, slottime):
-		slottime_ms = slottime
-		slottime = int(slottime_ms / 10)
-		if slottime < 0:
-			slottime = 0
-		if slottime > 255:
-			slottime = 255
-
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SLOTTIME])+bytes([slottime])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			raise IOError("Could not configure AX.25 KISS interface slot time to "+str(slottime_ms)+" (command value "+str(slottime)+")")
-
-	def setFlowControl(self, flow_control):
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_READY])+bytes([0x01])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			if (flow_control):
-				raise IOError("Could not enable AX.25 KISS interface flow control")
-			else:
-				raise IOError("Could not enable AX.25 KISS interface flow control")
+    def processIncoming(self, data):
+        if (len(data) > AX25.HEADER_SIZE):
+            self.rxb += len(data)
+            self.owner.inbound(data[AX25.HEADER_SIZE:], self)
 
 
-	def processIncoming(self, data):
-		if (len(data) > AX25.HEADER_SIZE):
-			self.owner.inbound(data[AX25.HEADER_SIZE:], self)
+    def processOutgoing(self,data):
+        datalen = len(data)
+        if self.online:
+            if self.interface_ready:
+                if self.flow_control:
+                    self.interface_ready = False
+                    self.flow_control_locked = time.time()
 
+                encoded_dst_ssid = bytes([0x60 | (self.dst_ssid << 1)])
+                encoded_src_ssid = bytes([0x60 | (self.src_ssid << 1) | 0x01])
 
-	def processOutgoing(self,data):
-		if self.online:
-			if self.interface_ready:
-				if self.flow_control:
-					self.interface_ready = False
+                addr = b""
 
-				encoded_dst_ssid = bytes([0x60 | (self.dst_ssid << 1)])
-				encoded_src_ssid = bytes([0x60 | (self.src_ssid << 1) | 0x01])
+                for i in range(0,6):
+                    if (i < len(self.dst_call)):
+                        addr += bytes([self.dst_call[i]<<1])
+                    else:
+                        addr += bytes([0x20])
+                addr += encoded_dst_ssid
 
-				addr = b""
+                for i in range(0,6):
+                    if (i < len(self.src_call)):
+                        addr += bytes([self.src_call[i]<<1])
+                    else:
+                        addr += bytes([0x20])
+                addr += encoded_src_ssid
 
-				for i in range(0,6):
-					if (i < len(self.dst_call)):
-						addr += bytes([self.dst_call[i]<<1])
-					else:
-						addr += bytes([0x20])
-				addr += encoded_dst_ssid
+                data = addr+bytes([AX25.CTRL_UI])+bytes([AX25.PID_NOLAYER3])+data
 
-				for i in range(0,6):
-					if (i < len(self.src_call)):
-						addr += bytes([self.src_call[i]<<1])
-					else:
-						addr += bytes([0x20])
-				addr += encoded_src_ssid
+                data = data.replace(bytes([0xdb]), bytes([0xdb])+bytes([0xdd]))
+                data = data.replace(bytes([0xc0]), bytes([0xdb])+bytes([0xdc]))
+                kiss_frame = bytes([KISS.FEND])+bytes([0x00])+data+bytes([KISS.FEND])
 
-				data = addr+bytes([AX25.CTRL_UI])+bytes([AX25.PID_NOLAYER3])+data
+                written = self.serial.write(kiss_frame)
+                self.txb += datalen
 
-				data = data.replace(bytes([0xdb]), bytes([0xdb])+bytes([0xdd]))
-				data = data.replace(bytes([0xc0]), bytes([0xdb])+bytes([0xdc]))
-				kiss_frame = bytes([KISS.FEND])+bytes([0x00])+data+bytes([KISS.FEND])
+                if written != len(kiss_frame):
+                    if self.flow_control:
+                        self.interface_ready = True
+                    raise IOError("AX.25 interface only wrote "+str(written)+" bytes of "+str(len(kiss_frame)))
+            else:
+                self.queue(data)
 
-				if (self.txdelay > 0):
-					RNS.log(str(self.name)+" delaying TX for "+str(self.txdelay)+" seconds", RNS.LOG_EXTREME)
-					sleep(self.txdelay)
+    def queue(self, data):
+        self.packet_queue.append(data)
 
-				written = self.serial.write(kiss_frame)
-				if written != len(kiss_frame):
-					if self.flow_control:
-						self.interface_ready = True
-					raise IOError("AX.25 interface only wrote "+str(written)+" bytes of "+str(len(kiss_frame)))
-			else:
-				self.queue(data)
+    def process_queue(self):
+        if len(self.packet_queue) > 0:
+            data = self.packet_queue.pop(0)
+            self.interface_ready = True
+            self.processOutgoing(data)
+        elif len(self.packet_queue) == 0:
+            self.interface_ready = True
 
-	def queue(self, data):
-		self.packet_queue.append(data)
+    def readLoop(self):
+        try:
+            in_frame = False
+            escape = False
+            command = KISS.CMD_UNKNOWN
+            data_buffer = b""
+            last_read_ms = int(time.time()*1000)
 
-	def process_queue(self):
-		if len(self.packet_queue) > 0:
-			data = self.packet_queue.pop(0)
-			self.interface_ready = True
-			self.processOutgoing(data)
-		elif len(self.packet_queue) == 0:
-			self.interface_ready = True
+            while self.serial.is_open:
+                if self.serial.in_waiting:
+                    byte = ord(self.serial.read(1))
+                    last_read_ms = int(time.time()*1000)
 
-	def readLoop(self):
-		try:
-			in_frame = False
-			escape = False
-			command = KISS.CMD_UNKNOWN
-			data_buffer = b""
-			last_read_ms = int(time.time()*1000)
+                    if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
+                        in_frame = False
+                        self.processIncoming(data_buffer)
+                    elif (byte == KISS.FEND):
+                        in_frame = True
+                        command = KISS.CMD_UNKNOWN
+                        data_buffer = b""
+                    elif (in_frame and len(data_buffer) < self.HW_MTU+AX25.HEADER_SIZE):
+                        if (len(data_buffer) == 0 and command == KISS.CMD_UNKNOWN):
+                            # We only support one HDLC port for now, so
+                            # strip off the port nibble
+                            byte = byte & 0x0F
+                            command = byte
+                        elif (command == KISS.CMD_DATA):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                data_buffer = data_buffer+bytes([byte])
+                        elif (command == KISS.CMD_READY):
+                            self.process_queue()
+                else:
+                    time_since_last = int(time.time()*1000) - last_read_ms
+                    if len(data_buffer) > 0 and time_since_last > self.timeout:
+                        data_buffer = b""
+                        in_frame = False
+                        command = KISS.CMD_UNKNOWN
+                        escape = False
+                    sleep(0.05)
 
-			while self.serial.is_open:
-				if self.serial.in_waiting:
-					byte = ord(self.serial.read(1))
-					last_read_ms = int(time.time()*1000)
+                    if self.flow_control:
+                        if not self.interface_ready:
+                            if time.time() > self.flow_control_locked + self.flow_control_timeout:
+                                RNS.log("Interface "+str(self)+" is unlocking flow control due to time-out. This should not happen. Your hardware might have missed a flow-control READY command, or maybe it does not support flow-control.", RNS.LOG_WARNING)
+                                self.process_queue()
 
-					if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
-						in_frame = False
-						self.processIncoming(data_buffer)
-					elif (byte == KISS.FEND):
-						in_frame = True
-						command = KISS.CMD_UNKNOWN
-						data_buffer = b""
-					elif (in_frame and len(data_buffer) < RNS.Reticulum.MTU+AX25.HEADER_SIZE):
-						if (len(data_buffer) == 0 and command == KISS.CMD_UNKNOWN):
-							# We only support one HDLC port for now, so
-							# strip off the port nibble
-							byte = byte & 0x0F
-							command = byte
-						elif (command == KISS.CMD_DATA):
-							if (byte == KISS.FESC):
-								escape = True
-							else:
-								if (escape):
-									if (byte == KISS.TFEND):
-										byte = KISS.FEND
-									if (byte == KISS.TFESC):
-										byte = KISS.FESC
-									escape = False
-								data_buffer = data_buffer+bytes([byte])
-						elif (command == KISS.CMD_READY):
-							# TODO: add timeout and reset if ready
-							# command never arrives
-							self.process_queue()
-				else:
-					time_since_last = int(time.time()*1000) - last_read_ms
-					if len(data_buffer) > 0 and time_since_last > self.timeout:
-			 			data_buffer = b""
-			 			in_frame = False
-			 			command = KISS.CMD_UNKNOWN
-			 			escape = False
-					sleep(0.08)
+        except Exception as e:
+            self.online = False
+            RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
+            
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
 
-		except Exception as e:
-			self.online = False
-			RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
-			RNS.log("The interface "+str(self.name)+" is now offline. Restart Reticulum to attempt reconnection.", RNS.LOG_ERROR)
+            RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
 
-	def __str__(self):
-		return "AX25KISSInterface["+self.name+"]"
+        self.online = False
+        self.serial.close()
+        self.reconnect_port()
+
+    def reconnect_port(self):
+        while not self.online:
+            try:
+                time.sleep(5)
+                RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
+                self.open_port()
+                if self.serial.is_open:
+                    self.configure_device()
+            except Exception as e:
+                RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
+
+        RNS.log("Reconnected serial port for "+str(self))
+
+    def __str__(self):
+        return "AX25KISSInterface["+self.name+"]"

RNS/Interfaces/AutoInterface.py

@@ -0,0 +1,359 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from .Interface import Interface
+import socketserver
+import threading
+import socket
+import struct
+import time
+import sys
+import RNS
+
+
+class AutoInterface(Interface):
+    DEFAULT_DISCOVERY_PORT = 29716
+    DEFAULT_DATA_PORT      = 42671
+    DEFAULT_GROUP_ID       = "reticulum".encode("utf-8")
+
+    SCOPE_LINK         = "2"
+    SCOPE_ADMIN        = "4"
+    SCOPE_SITE         = "5"
+    SCOPE_ORGANISATION = "8"
+    SCOPE_GLOBAL       = "e"
+
+    PEERING_TIMEOUT    = 7.5
+
+    DARWIN_IGNORE_IFS  = ["awdl0", "llw0", "lo0", "en5"]
+    ANDROID_IGNORE_IFS = ["dummy0", "lo", "tun0"]
+
+    BITRATE_GUESS      = 10*1000*1000
+
+    def __init__(self, owner, name, group_id=None, discovery_scope=None, discovery_port=None, data_port=None, allowed_interfaces=None, ignored_interfaces=None, configured_bitrate=None):
+        import importlib
+        if importlib.util.find_spec('netifaces') != None:
+            import netifaces
+        else:
+            RNS.log("Using AutoInterface requires the netifaces module.", RNS.LOG_CRITICAL)
+            RNS.log("You can install it with the command: python3 -m pip install netifaces", RNS.LOG_CRITICAL)
+            RNS.panic()
+
+        self.netifaces = netifaces
+        self.rxb = 0
+        self.txb = 0
+
+        self.HW_MTU = 1064
+
+        self.IN  = True
+        self.OUT = False
+        self.name = name
+        self.online = False
+        self.peers = {}
+        self.link_local_addresses = []
+        self.adopted_interfaces = {}
+        self.multicast_echoes = {}
+        self.timed_out_interfaces = {}
+
+        self.outbound_udp_socket = None
+
+        self.announce_interval = AutoInterface.PEERING_TIMEOUT/6.0
+        self.peer_job_interval = AutoInterface.PEERING_TIMEOUT*1.1
+        self.peering_timeout   = AutoInterface.PEERING_TIMEOUT
+        self.multicast_echo_timeout = AutoInterface.PEERING_TIMEOUT/2
+
+        if allowed_interfaces == None:
+            self.allowed_interfaces = []
+        else:
+            self.allowed_interfaces = allowed_interfaces
+
+        if ignored_interfaces == None:
+            self.ignored_interfaces = []
+        else:
+            self.ignored_interfaces = ignored_interfaces
+
+        if group_id == None:
+            self.group_id = AutoInterface.DEFAULT_GROUP_ID
+        else:
+            self.group_id = group_id.encode("utf-8")
+
+        if discovery_port == None:
+            self.discovery_port = AutoInterface.DEFAULT_DISCOVERY_PORT
+        else:
+            self.discovery_port = discovery_port
+
+        if data_port == None:
+            self.data_port = AutoInterface.DEFAULT_DATA_PORT
+        else:
+            self.data_port = data_port
+
+        if discovery_scope == None:
+            self.discovery_scope = AutoInterface.SCOPE_LINK
+        elif str(discovery_scope).lower() == "link":
+            self.discovery_scope = AutoInterface.SCOPE_LINK
+        elif str(discovery_scope).lower() == "admin":
+            self.discovery_scope = AutoInterface.SCOPE_ADMIN
+        elif str(discovery_scope).lower() == "site":
+            self.discovery_scope = AutoInterface.SCOPE_SITE
+        elif str(discovery_scope).lower() == "organisation":
+            self.discovery_scope = AutoInterface.SCOPE_ORGANISATION
+        elif str(discovery_scope).lower() == "global":
+            self.discovery_scope = AutoInterface.SCOPE_GLOBAL
+
+        self.group_hash = RNS.Identity.full_hash(self.group_id)
+        g = self.group_hash
+        #gt  = "{:02x}".format(g[1]+(g[0]<<8))
+        gt  = "0"
+        gt += ":"+"{:02x}".format(g[3]+(g[2]<<8))
+        gt += ":"+"{:02x}".format(g[5]+(g[4]<<8))
+        gt += ":"+"{:02x}".format(g[7]+(g[6]<<8))
+        gt += ":"+"{:02x}".format(g[9]+(g[8]<<8))
+        gt += ":"+"{:02x}".format(g[11]+(g[10]<<8))
+        gt += ":"+"{:02x}".format(g[13]+(g[12]<<8))
+        self.mcast_discovery_address = "ff1"+self.discovery_scope+":"+gt
+
+        suitable_interfaces = 0
+        for ifname in self.netifaces.interfaces():
+            if RNS.vendor.platformutils.is_darwin() and ifname in AutoInterface.DARWIN_IGNORE_IFS and not ifname in self.allowed_interfaces:
+                RNS.log(str(self)+" skipping Darwin AWDL or tethering interface "+str(ifname), RNS.LOG_EXTREME)
+            elif RNS.vendor.platformutils.is_darwin() and ifname == "lo0":
+                RNS.log(str(self)+" skipping Darwin loopback interface "+str(ifname), RNS.LOG_EXTREME)
+            elif RNS.vendor.platformutils.is_android() and ifname in AutoInterface.ANDROID_IGNORE_IFS and not ifname in self.allowed_interfaces:
+                RNS.log(str(self)+" skipping Android system interface "+str(ifname), RNS.LOG_EXTREME)
+            elif ifname in self.ignored_interfaces:
+                RNS.log(str(self)+" ignoring disallowed interface "+str(ifname), RNS.LOG_EXTREME)
+            else:
+                if len(self.allowed_interfaces) > 0 and not ifname in self.allowed_interfaces:
+                    RNS.log(str(self)+" ignoring interface "+str(ifname)+" since it was not allowed", RNS.LOG_EXTREME)
+                else:
+                    addresses = self.netifaces.ifaddresses(ifname)
+                    if self.netifaces.AF_INET6 in addresses:
+                        link_local_addr = None
+                        for address in addresses[self.netifaces.AF_INET6]:
+                            if "addr" in address:
+                                if address["addr"].startswith("fe80:"):
+                                    link_local_addr = address["addr"]
+                                    self.link_local_addresses.append(link_local_addr.split("%")[0])
+                                    self.adopted_interfaces[ifname] = link_local_addr.split("%")[0]
+                                    self.multicast_echoes[ifname] = time.time()
+                                    RNS.log(str(self)+" Selecting link-local address "+str(link_local_addr)+" for interface "+str(ifname), RNS.LOG_EXTREME)
+
+                        if link_local_addr == None:
+                            RNS.log(str(self)+" No link-local IPv6 address configured for "+str(ifname)+", skipping interface", RNS.LOG_EXTREME)
+                        else:
+                            mcast_addr = self.mcast_discovery_address
+                            RNS.log(str(self)+" Creating multicast discovery listener on "+str(ifname)+" with address "+str(mcast_addr), RNS.LOG_EXTREME)
+
+                            # Struct with interface index
+                            if_struct = struct.pack("I", socket.if_nametoindex(ifname))
+
+                            # Set up multicast socket
+                            discovery_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
+                            discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+                            if hasattr(socket, "SO_REUSEPORT"):
+                                discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+                            discovery_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_IF, if_struct)
+
+                            # Join multicast group
+                            mcast_group = socket.inet_pton(socket.AF_INET6, mcast_addr) + if_struct
+                            discovery_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_JOIN_GROUP, mcast_group)
+
+                            # Bind socket
+                            addr_info = socket.getaddrinfo(mcast_addr+"%"+ifname, self.discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
+                            discovery_socket.bind(addr_info[0][4])
+
+                            # Set up thread for discovery packets
+                            def discovery_loop():
+                                self.discovery_handler(discovery_socket, ifname)
+
+                            thread = threading.Thread(target=discovery_loop)
+                            thread.daemon = True
+                            thread.start()
+
+                            suitable_interfaces += 1
+
+        if suitable_interfaces == 0:
+            RNS.log(str(self)+" could not autoconfigure. This interface currently provides no connectivity.", RNS.LOG_WARNING)
+        else:
+            self.receives = True
+
+            peering_wait = self.announce_interval*1.2
+            RNS.log(str(self)+" discovering peers for "+str(round(peering_wait, 2))+" seconds...", RNS.LOG_VERBOSE)
+
+            def handlerFactory(callback):
+                def createHandler(*args, **keys):
+                    return AutoInterfaceHandler(callback, *args, **keys)
+                return createHandler
+
+            self.owner = owner
+            socketserver.UDPServer.address_family = socket.AF_INET6
+
+            for ifname in self.adopted_interfaces:
+                local_addr = self.adopted_interfaces[ifname]+"%"+ifname
+                addr_info = socket.getaddrinfo(local_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
+                address = addr_info[0][4]
+
+                self.server = socketserver.UDPServer(address, handlerFactory(self.processIncoming))
+                
+                thread = threading.Thread(target=self.server.serve_forever)
+                thread.daemon = True
+                thread.start()
+
+            job_thread = threading.Thread(target=self.peer_jobs)
+            job_thread.daemon = True
+            job_thread.start()
+
+            time.sleep(peering_wait)
+
+            if configured_bitrate != None:
+                self.bitrate = configured_bitrate
+            else:
+                self.bitrate = AutoInterface.BITRATE_GUESS
+
+            self.online = True
+
+
+    def discovery_handler(self, socket, ifname):
+        def announce_loop():
+            self.announce_handler(ifname)
+            
+        thread = threading.Thread(target=announce_loop)
+        thread.daemon = True
+        thread.start()
+        
+        while True:
+            data, ipv6_src = socket.recvfrom(1024)
+            expected_hash = RNS.Identity.full_hash(self.group_id+ipv6_src[0].encode("utf-8"))
+            if data == expected_hash:
+                self.add_peer(ipv6_src[0], ifname)
+            else:
+                RNS.log(str(self)+" received peering packet on "+str(ifname)+" from "+str(ipv6_src[0])+", but authentication hash was incorrect.", RNS.LOG_DEBUG)
+
+    def peer_jobs(self):
+        while True:
+            time.sleep(self.peer_job_interval)
+            now = time.time()
+            timed_out_peers = []
+
+            # Check for timed out peers
+            for peer_addr in self.peers:
+                peer = self.peers[peer_addr]
+                last_heard = peer[1]
+                if now > last_heard+self.peering_timeout:
+                    timed_out_peers.append(peer_addr)
+
+            # Remove any timed out peers
+            for peer_addr in timed_out_peers:
+                removed_peer = self.peers.pop(peer_addr)
+                RNS.log(str(self)+" removed peer "+str(peer_addr)+" on "+str(removed_peer[0]), RNS.LOG_DEBUG)
+
+            for ifname in self.adopted_interfaces:
+                last_multicast_echo = 0
+                if ifname in self.multicast_echoes:
+                    last_multicast_echo = self.multicast_echoes[ifname]
+
+                if now - last_multicast_echo > self.multicast_echo_timeout:
+                    if ifname in self.timed_out_interfaces and self.timed_out_interfaces[ifname] == False:
+                        RNS.log("Multicast echo timeout for "+str(ifname)+". Carrier lost.", RNS.LOG_WARNING)
+                    self.timed_out_interfaces[ifname] = True
+                else:
+                    if ifname in self.timed_out_interfaces and self.timed_out_interfaces[ifname] == True:
+                        RNS.log(str(self)+" Carrier recovered on "+str(ifname), RNS.LOG_WARNING)
+                    self.timed_out_interfaces[ifname] = False
+                
+
+    def announce_handler(self, ifname):
+        while True:
+            self.peer_announce(ifname)
+            time.sleep(self.announce_interval)
+            
+    def peer_announce(self, ifname):
+        try:
+            link_local_address = self.adopted_interfaces[ifname]
+            discovery_token = RNS.Identity.full_hash(self.group_id+link_local_address.encode("utf-8"))
+            announce_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
+            addr_info = socket.getaddrinfo(self.mcast_discovery_address, self.discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
+
+            ifis = struct.pack("I", socket.if_nametoindex(ifname))
+            announce_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_IF, ifis)
+            announce_socket.sendto(discovery_token, addr_info[0][4])
+            announce_socket.close()
+            
+        except Exception as e:
+            if (ifname in self.timed_out_interfaces and self.timed_out_interfaces[ifname] == False) or not ifname in self.timed_out_interfaces:
+                RNS.log(str(self)+" Detected possible carrier loss on "+str(ifname)+": "+str(e), RNS.LOG_WARNING)
+            else:
+                pass
+
+    def add_peer(self, addr, ifname):
+        if addr in self.link_local_addresses:
+            ifname = None
+            for interface_name in self.adopted_interfaces:
+                if self.adopted_interfaces[interface_name] == addr:
+                    ifname = interface_name
+
+            if ifname != None:
+                self.multicast_echoes[ifname] = time.time()
+            else:
+                RNS.log(str(self)+" received multicast echo on unexpected interface "+str(ifname), RNS.LOG_WARNING)
+
+        else:
+            if not addr in self.peers:
+                self.peers[addr] = [ifname, time.time()]
+                RNS.log(str(self)+" added peer "+str(addr)+" on "+str(ifname), RNS.LOG_DEBUG)
+            else:
+                self.refresh_peer(addr)
+
+    def refresh_peer(self, addr):
+        self.peers[addr][1] = time.time()
+
+    def processIncoming(self, data):
+        self.rxb += len(data)
+        self.owner.inbound(data, self)
+
+    def processOutgoing(self,data):
+            for peer in self.peers:
+                try:
+                    if self.outbound_udp_socket == None:
+                        self.outbound_udp_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
+                    
+                    peer_addr = str(peer)+"%"+str(self.peers[peer][0])
+                    addr_info = socket.getaddrinfo(peer_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
+                    self.outbound_udp_socket.sendto(data, addr_info[0][4])
+
+                except Exception as e:
+                    RNS.log("Could not transmit on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+
+            
+            self.txb += len(data)
+            
+
+    def __str__(self):
+        return "AutoInterface["+self.name+"]"
+
+class AutoInterfaceHandler(socketserver.BaseRequestHandler):
+    def __init__(self, callback, *args, **keys):
+        self.callback = callback
+        socketserver.BaseRequestHandler.__init__(self, *args, **keys)
+
+    def handle(self):
+        data = self.request[0]
+        self.callback(data)

RNS/Interfaces/I2PInterface.py

@@ -0,0 +1,882 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from .Interface import Interface
+import socketserver
+import threading
+import platform
+import socket
+import time
+import sys
+import os
+import RNS
+import asyncio
+
+class HDLC():
+    FLAG              = 0x7E
+    ESC               = 0x7D
+    ESC_MASK          = 0x20
+
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
+        data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
+        return data
+
+class KISS():
+    FEND              = 0xC0
+    FESC              = 0xDB
+    TFEND             = 0xDC
+    TFESC             = 0xDD
+    CMD_DATA          = 0x00
+    CMD_UNKNOWN       = 0xFE
+
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([0xdb]), bytes([0xdb, 0xdd]))
+        data = data.replace(bytes([0xc0]), bytes([0xdb, 0xdc]))
+        return data
+
+# TODO: Neater shutdown of the event loop and
+# better error handling is needed. Sometimes
+# errors occur in I2P that leave tunnel setup
+# hanging indefinitely, and right now we have
+# no way of catching it. Sometimes the server
+# and client tasks are also not cancelled on
+# shutdown, which leads to errors dumped to
+# the console. This should also be remedied.
+
+class I2PController:
+    def __init__(self, rns_storagepath):
+        import RNS.vendor.i2plib as i2plib
+        import RNS.vendor.i2plib.utils
+
+        self.client_tunnels = {}
+        self.server_tunnels = {}
+        self.i2plib_tunnels = {}
+        self.loop = None
+        self.i2plib = i2plib
+        self.utils = i2plib.utils
+        self.sam_address = i2plib.get_sam_address()
+        self.ready = False
+
+        self.storagepath = rns_storagepath+"/i2p"
+        if not os.path.isdir(self.storagepath):
+            os.makedirs(self.storagepath)
+
+
+    def start(self):
+        asyncio.set_event_loop(asyncio.new_event_loop())
+        self.loop = asyncio.get_event_loop()
+
+        time.sleep(0.10)
+        if self.loop == None:
+            RNS.log("Could not get event loop for "+str(self)+", waiting for event loop to appear", RNS.LOG_VERBOSE)
+
+        while self.loop == None:
+            self.loop = asyncio.get_event_loop()
+            sleep(0.25)
+
+        try:
+            self.ready = True
+            self.loop.run_forever()
+        except Exception as e:
+            self.ready = False
+            RNS.log("Exception on event loop for "+str(self)+": "+str(e), RNS.LOG_ERROR)
+        finally:
+            self.loop.close()
+
+
+    def stop(self):
+        for i2ptunnel in self.i2plib_tunnels:
+            if hasattr(i2ptunnel, "stop") and callable(i2ptunnel.stop):
+                i2ptunnel.stop()
+
+        if hasattr(asyncio.Task, "all_tasks") and callable(asyncio.Task.all_tasks):
+            for task in asyncio.Task.all_tasks(loop=self.loop):
+                task.cancel()
+
+        time.sleep(0.2)
+
+        self.loop.stop()
+
+
+    def get_free_port(self):
+        return self.i2plib.utils.get_free_port()
+
+
+    def stop_tunnel(self, i2ptunnel):
+        if hasattr(i2ptunnel, "stop") and callable(i2ptunnel.stop):
+            i2ptunnel.stop()
+
+    def client_tunnel(self, owner, i2p_destination):
+        self.client_tunnels[i2p_destination] = False
+        self.i2plib_tunnels[i2p_destination] = None
+
+        while True:
+            if not self.client_tunnels[i2p_destination]:
+                try:
+                    async def tunnel_up():
+                        RNS.log("Bringing up I2P tunnel to "+str(owner)+", this may take a while...", RNS.LOG_INFO)
+                        tunnel = self.i2plib.ClientTunnel(i2p_destination, owner.local_addr, sam_address=self.sam_address, loop=self.loop)
+                        self.i2plib_tunnels[i2p_destination] = tunnel
+                        await tunnel.run()
+
+                    self.loop.ext_owner = self
+                    result = asyncio.run_coroutine_threadsafe(tunnel_up(), self.loop).result()
+                    
+                    if not i2p_destination in self.i2plib_tunnels:
+                        raise IOError("No tunnel control instance was created")
+
+                    else: 
+                        tn = self.i2plib_tunnels[i2p_destination]
+                        if tn != None and hasattr(tn, "status"):
+
+                            RNS.log("Waiting for status from I2P control process", RNS.LOG_EXTREME)
+                            while not tn.status["setup_ran"]:
+                                time.sleep(0.1)
+                            RNS.log("Got status from I2P control process", RNS.LOG_EXTREME)
+
+                            if tn.status["setup_failed"]:
+                                self.stop_tunnel(tn)
+                                raise tn.status["exception"]
+
+                            else:
+                                self.client_tunnels[i2p_destination] = True
+                                owner.awaiting_i2p_tunnel = False
+                                if owner.socket != None:
+                                    if hasattr(owner.socket, "close"):
+                                        if callable(owner.socket.close):
+                                            try:
+                                                owner.socket.shutdown(socket.SHUT_RDWR)
+                                            except Exception as e:
+                                                RNS.log("Error while shutting down socket for "+str(owner)+": "+str(e))
+
+                                            try:
+                                                owner.socket.close()
+                                            except Exception as e:
+                                                RNS.log("Error while closing socket for "+str(owner)+": "+str(e))
+
+                                RNS.log(str(owner)+" tunnel setup complete", RNS.LOG_VERBOSE)
+
+                        else:
+                            raise IOError("Got no status response from SAM API")
+
+                except ConnectionRefusedError as e:
+                    raise e
+                    
+                except ConnectionAbortedError as e:
+                    raise e
+
+                except Exception as e:
+                    RNS.log("Unexpected error type from I2P SAM: "+str(e), RNS.LOG_ERROR)
+                    raise e
+
+            else:
+                i2ptunnel = self.i2plib_tunnels[i2p_destination]
+                if hasattr(i2ptunnel, "status"):
+                    i2p_exception = i2ptunnel.status["exception"]
+
+                    if i2ptunnel.status["setup_ran"] == False:
+                        RNS.log(str(self)+" I2P tunnel setup did not complete", RNS.LOG_ERROR)
+
+                        self.stop_tunnel(i2ptunnel)
+                        return False
+
+                    elif i2p_exception != None:
+                        RNS.log("An error ocurred while setting up I2P tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
+
+                        if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer):
+                            RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest):
+                            RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId):
+                            RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId):
+                            RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey):
+                            RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound):
+                            RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
+                        
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound):
+                            RNS.log("The I2P daemon mould not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
+                        
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError):
+                            RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
+                        
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout):
+                            RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
+
+                        RNS.log("Resetting I2P tunnel and retrying later", RNS.LOG_ERROR)
+
+                        self.stop_tunnel(i2ptunnel)
+                        return False
+
+                    elif i2ptunnel.status["setup_failed"] == True:
+                        RNS.log(str(self)+" Unspecified I2P tunnel setup error, resetting I2P tunnel", RNS.LOG_ERROR)
+
+                        self.stop_tunnel(i2ptunnel)
+                        return False
+
+                else:
+                    RNS.log(str(self)+" Got no status from SAM API, resetting I2P tunnel", RNS.LOG_ERROR)
+
+                    self.stop_tunnel(i2ptunnel)
+                    return False
+
+            # Wait for status from I2P control process
+            time.sleep(5)
+
+
+    def server_tunnel(self, owner):
+        while RNS.Transport.identity == None:
+            time.sleep(1)
+
+        # Old format
+        i2p_dest_hash_of = RNS.Identity.full_hash(RNS.Identity.full_hash(owner.name.encode("utf-8")))
+        i2p_keyfile_of   = self.storagepath+"/"+RNS.hexrep(i2p_dest_hash_of, delimit=False)+".i2p"
+
+        # New format
+        i2p_dest_hash_nf = RNS.Identity.full_hash(RNS.Identity.full_hash(owner.name.encode("utf-8"))+RNS.Identity.full_hash(RNS.Transport.identity.hash))
+        i2p_keyfile_nf   = self.storagepath+"/"+RNS.hexrep(i2p_dest_hash_nf, delimit=False)+".i2p"
+
+        # Use old format if a key is already present
+        if os.path.isfile(i2p_keyfile_of):
+            i2p_keyfile = i2p_keyfile_of
+        else:
+            i2p_keyfile = i2p_keyfile_nf
+
+        i2p_dest = None
+        if not os.path.isfile(i2p_keyfile):
+            coro = self.i2plib.new_destination(sam_address=self.sam_address, loop=self.loop)
+            i2p_dest = asyncio.run_coroutine_threadsafe(coro, self.loop).result()
+            key_file = open(i2p_keyfile, "w")
+            key_file.write(i2p_dest.private_key.base64)
+            key_file.close()
+        else:
+            key_file = open(i2p_keyfile, "r")
+            prvd = key_file.read()
+            key_file.close()
+            i2p_dest = self.i2plib.Destination(data=prvd, has_private_key=True)
+
+        i2p_b32 = i2p_dest.base32
+        owner.b32 = i2p_b32
+
+        self.server_tunnels[i2p_b32] = False
+        self.i2plib_tunnels[i2p_b32] = None
+
+        while True:
+            if self.server_tunnels[i2p_b32] == False:
+                try:
+                    async def tunnel_up():
+                        RNS.log(str(owner)+" Bringing up I2P endpoint, this may take a while...", RNS.LOG_INFO)
+                        tunnel = self.i2plib.ServerTunnel((owner.bind_ip, owner.bind_port), loop=self.loop, destination=i2p_dest, sam_address=self.sam_address)
+                        self.i2plib_tunnels[i2p_b32] = tunnel
+                        await tunnel.run()
+                        owner.online = True
+                        RNS.log(str(owner)+ " endpoint setup complete. Now reachable at: "+str(i2p_dest.base32)+".b32.i2p", RNS.LOG_VERBOSE)
+
+                    asyncio.run_coroutine_threadsafe(tunnel_up(), self.loop).result()
+                    self.server_tunnels[i2p_b32] = True
+
+                except Exception as e:
+                    raise e
+
+            else:
+                i2ptunnel = self.i2plib_tunnels[i2p_b32]
+                if hasattr(i2ptunnel, "status"):
+                    i2p_exception = i2ptunnel.status["exception"]
+
+                    if i2ptunnel.status["setup_ran"] == False:
+                        RNS.log(str(self)+" I2P tunnel setup did not complete", RNS.LOG_ERROR)
+
+                        self.stop_tunnel(i2ptunnel)
+                        return False
+
+                    elif i2p_exception != None:
+                        RNS.log("An error ocurred while setting up I2P tunnel", RNS.LOG_ERROR)
+                        
+                        if isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.CantReachPeer):
+                            RNS.log("The I2P daemon can't reach peer "+str(i2p_destination), RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedDest):
+                            RNS.log("The I2P daemon reported that the destination is already in use", RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.DuplicatedId):
+                            RNS.log("The I2P daemon reported that the ID is arleady in use", RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidId):
+                            RNS.log("The I2P daemon reported that the stream session ID doesn't exist", RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.InvalidKey):
+                            RNS.log("The I2P daemon reported that the key for "+str(i2p_destination)+" is invalid", RNS.LOG_ERROR)
+
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.KeyNotFound):
+                            RNS.log("The I2P daemon could not find the key for "+str(i2p_destination), RNS.LOG_ERROR)
+                        
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.PeerNotFound):
+                            RNS.log("The I2P daemon mould not find the peer "+str(i2p_destination), RNS.LOG_ERROR)
+                        
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.I2PError):
+                            RNS.log("The I2P daemon experienced an unspecified error", RNS.LOG_ERROR)
+                        
+                        elif isinstance(i2p_exception, RNS.vendor.i2plib.exceptions.Timeout):
+                            RNS.log("I2P daemon timed out while setting up client tunnel to "+str(i2p_destination), RNS.LOG_ERROR)
+
+                        RNS.log("Resetting I2P tunnel and retrying later", RNS.LOG_ERROR)
+
+                        self.stop_tunnel(i2ptunnel)
+                        return False
+
+                    elif i2ptunnel.status["setup_failed"] == True:
+                        RNS.log(str(self)+" Unspecified I2P tunnel setup error, resetting I2P tunnel", RNS.LOG_ERROR)
+
+                        self.stop_tunnel(i2ptunnel)
+                        return False
+
+                else:
+                    RNS.log(str(self)+" Got no status from SAM API, resetting I2P tunnel", RNS.LOG_ERROR)
+
+                    self.stop_tunnel(i2ptunnel)
+                    return False
+
+            time.sleep(5)
+
+    def get_loop(self):
+        return asyncio.get_event_loop()
+
+
+class ThreadingI2PServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
+    pass
+
+class I2PInterfacePeer(Interface):
+    RECONNECT_WAIT = 15
+    RECONNECT_MAX_TRIES = None
+
+    # TCP socket options
+    I2P_USER_TIMEOUT = 45
+    I2P_PROBE_AFTER = 10
+    I2P_PROBE_INTERVAL = 9
+    I2P_PROBES = 5
+
+    def __init__(self, parent_interface, owner, name, target_i2p_dest=None, connected_socket=None, max_reconnect_tries=None):
+        self.rxb = 0
+        self.txb = 0
+
+        self.HW_MTU = 1064
+        
+        self.IN               = True
+        self.OUT              = False
+        self.socket           = None
+        self.parent_interface = parent_interface
+        self.parent_count     = True
+        self.name             = name
+        self.initiator        = False
+        self.reconnecting     = False
+        self.never_connected  = True
+        self.owner            = owner
+        self.writing          = False
+        self.online           = False
+        self.detached         = False
+        self.kiss_framing     = False
+        self.i2p_tunneled     = True
+        self.i2p_dest         = None
+        self.i2p_tunnel_ready = False
+        self.mode             = RNS.Interfaces.Interface.Interface.MODE_FULL
+        self.bitrate          = I2PInterface.BITRATE_GUESS
+
+        self.announce_rate_target  = None
+        self.announce_rate_grace   = None
+        self.announce_rate_penalty = None
+
+        if max_reconnect_tries == None:
+            self.max_reconnect_tries = I2PInterfacePeer.RECONNECT_MAX_TRIES
+        else:
+            self.max_reconnect_tries = max_reconnect_tries
+
+        if connected_socket != None:
+            self.receives    = True
+            self.target_ip   = None
+            self.target_port = None
+            self.socket      = connected_socket
+
+            if platform.system() == "Linux":
+                self.set_timeouts_linux()
+            elif platform.system() == "Darwin":
+                self.set_timeouts_osx()
+
+        elif target_i2p_dest != None:
+            self.receives    = True
+            self.initiator   = True
+
+            self.bind_ip     = "127.0.0.1"
+
+            self.awaiting_i2p_tunnel = True
+
+            def tunnel_job():
+                while self.awaiting_i2p_tunnel:
+                    try:
+                        self.bind_port   = self.parent_interface.i2p.get_free_port()
+                        self.local_addr  = (self.bind_ip, self.bind_port)
+                        self.target_ip = self.bind_ip
+                        self.target_port = self.bind_port
+
+                        if not self.parent_interface.i2p.client_tunnel(self, target_i2p_dest):
+                            RNS.log(str(self)+" I2P control process experienced an error, requesting new tunnel...", RNS.LOG_ERROR)
+                            self.awaiting_i2p_tunnel = True
+
+                    except Exception as e:
+                        RNS.log("Error while while configuring "+str(self)+": "+str(e), RNS.LOG_ERROR)
+                        RNS.log("Check that I2P is installed and running, and that SAM is enabled. Retrying tunnel setup later.", RNS.LOG_ERROR)
+
+                    time.sleep(15)
+
+            thread = threading.Thread(target=tunnel_job)
+            thread.daemon = True
+            thread.start()
+
+            def wait_job():
+                while self.awaiting_i2p_tunnel:
+                    time.sleep(0.25)
+                
+                if not self.kiss_framing:
+                    self.wants_tunnel = True
+
+                if not self.connect(initial=True):
+                    thread = threading.Thread(target=self.reconnect)
+                    thread.daemon = True
+                    thread.start()
+                else:
+                    thread = threading.Thread(target=self.read_loop)
+                    thread.daemon = True
+                    thread.start()
+
+            thread = threading.Thread(target=wait_job)
+            thread.daemon = True
+            thread.start()
+
+
+    def set_timeouts_linux(self):
+        if not self.i2p_tunneled:
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, int(I2PInterfacePeer.TCP_USER_TIMEOUT * 1000))
+            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, int(I2PInterfacePeer.TCP_PROBE_AFTER))
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, int(I2PInterfacePeer.TCP_PROBE_INTERVAL))
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, int(I2PInterfacePeer.TCP_PROBES))
+        else:
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, int(I2PInterfacePeer.I2P_USER_TIMEOUT * 1000))
+            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, int(I2PInterfacePeer.I2P_PROBE_AFTER))
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, int(I2PInterfacePeer.I2P_PROBE_INTERVAL))
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, int(I2PInterfacePeer.I2P_PROBES))
+
+    def set_timeouts_osx(self):
+        if hasattr(socket, "TCP_KEEPALIVE"):
+            TCP_KEEPIDLE = socket.TCP_KEEPALIVE
+        else:
+            TCP_KEEPIDLE = 0x10
+
+        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
+        
+        if not self.i2p_tunneled:
+            self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(I2PInterfacePeer.TCP_PROBE_AFTER))
+        else:
+            self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(I2PInterfacePeer.I2P_PROBE_AFTER))
+    
+    def shutdown_socket(self, socket):
+        if callable(socket.close):
+            
+            try:
+                socket.shutdown(socket.SHUT_RDWR)
+            except Exception as e:
+                RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
+
+            try:
+                socket.close()
+            except Exception as e:
+                RNS.log("Error while closing socket for "+str(self)+": "+str(e))    
+    
+    def detach(self):
+        RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
+        if self.socket != None:
+            if hasattr(self.socket, "close"):
+                if callable(self.socket.close):
+                    self.detached = True
+                    
+                    try:
+                        self.socket.shutdown(socket.SHUT_RDWR)
+                    except Exception as e:
+                        RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
+
+                    try:
+                        self.socket.close()
+                    except Exception as e:
+                        RNS.log("Error while closing socket for "+str(self)+": "+str(e))
+
+                    self.socket = None
+
+    def connect(self, initial=False):
+        try:
+            self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            self.socket.connect((self.target_ip, self.target_port))
+            self.online  = True
+        
+        except Exception as e:
+            if initial:
+                if not self.awaiting_i2p_tunnel:
+                    RNS.log("Initial connection for "+str(self)+" could not be established: "+str(e), RNS.LOG_ERROR)
+                    RNS.log("Leaving unconnected and retrying connection in "+str(I2PInterfacePeer.RECONNECT_WAIT)+" seconds.", RNS.LOG_ERROR)
+
+                return False
+            
+            else:
+                raise e
+
+        if platform.system() == "Linux":
+            self.set_timeouts_linux()
+        elif platform.system() == "Darwin":
+            self.set_timeouts_osx()
+        
+        self.online  = True
+        self.writing = False
+        self.never_connected = False
+
+        if not self.kiss_framing and self.wants_tunnel:
+            RNS.Transport.synthesize_tunnel(self)
+
+        return True
+
+
+    def reconnect(self):
+        if self.initiator:
+            if not self.reconnecting:
+                self.reconnecting = True
+                attempts = 0
+                while not self.online:
+                    time.sleep(I2PInterfacePeer.RECONNECT_WAIT)
+                    attempts += 1
+
+                    if self.max_reconnect_tries != None and attempts > self.max_reconnect_tries:
+                        RNS.log("Max reconnection attempts reached for "+str(self), RNS.LOG_ERROR)
+                        self.teardown()
+                        break
+
+                    try:
+                        self.connect()
+
+                    except Exception as e:
+                        if not self.awaiting_i2p_tunnel:
+                            RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
+                        else:
+                            RNS.log(str(self)+" still waiting for I2P tunnel to appear", RNS.LOG_VERBOSE)
+
+                if not self.never_connected:
+                    RNS.log(str(self)+" Re-established connection via I2P tunnel", RNS.LOG_INFO)
+
+                self.reconnecting = False
+                thread = threading.Thread(target=self.read_loop)
+                thread.daemon = True
+                thread.start()
+                if not self.kiss_framing:
+                    RNS.Transport.synthesize_tunnel(self)
+
+        else:
+            RNS.log("Attempt to reconnect on a non-initiator I2P interface. This should not happen.", RNS.LOG_ERROR)
+            raise IOError("Attempt to reconnect on a non-initiator I2P interface")
+
+    def processIncoming(self, data):
+        self.rxb += len(data)
+        if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count:
+            self.parent_interface.rxb += len(data)
+                    
+        self.owner.inbound(data, self)
+
+    def processOutgoing(self, data):
+        if self.online:
+            while self.writing:
+                time.sleep(0.001)
+
+            try:
+                self.writing = True
+
+                if self.kiss_framing:
+                    data = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(data)+bytes([KISS.FEND])
+                else:
+                    data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+
+                self.socket.sendall(data)
+                self.writing = False
+                self.txb += len(data)
+                
+                if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count:
+                    self.parent_interface.txb += len(data)
+
+            except Exception as e:
+                RNS.log("Exception occurred while transmitting via "+str(self)+", tearing down interface", RNS.LOG_ERROR)
+                RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+                self.teardown()
+
+
+    def read_loop(self):
+        try:
+            in_frame = False
+            escape = False
+            data_buffer = b""
+            command = KISS.CMD_UNKNOWN
+
+            while True:
+                data_in = self.socket.recv(4096)
+                if len(data_in) > 0:
+                    pointer = 0
+                    while pointer < len(data_in):
+                        byte = data_in[pointer]
+                        pointer += 1
+
+                        if self.kiss_framing:
+                            # Read loop for KISS framing
+                            if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
+                                in_frame = False
+                                self.processIncoming(data_buffer)
+                            elif (byte == KISS.FEND):
+                                in_frame = True
+                                command = KISS.CMD_UNKNOWN
+                                data_buffer = b""
+                            elif (in_frame and len(data_buffer) < self.HW_MTU):
+                                if (len(data_buffer) == 0 and command == KISS.CMD_UNKNOWN):
+                                    # We only support one HDLC port for now, so
+                                    # strip off the port nibble
+                                    byte = byte & 0x0F
+                                    command = byte
+                                elif (command == KISS.CMD_DATA):
+                                    if (byte == KISS.FESC):
+                                        escape = True
+                                    else:
+                                        if (escape):
+                                            if (byte == KISS.TFEND):
+                                                byte = KISS.FEND
+                                            if (byte == KISS.TFESC):
+                                                byte = KISS.FESC
+                                            escape = False
+                                        data_buffer = data_buffer+bytes([byte])
+
+                        else:
+                            # Read loop for HDLC framing
+                            if (in_frame and byte == HDLC.FLAG):
+                                in_frame = False
+                                self.processIncoming(data_buffer)
+                            elif (byte == HDLC.FLAG):
+                                in_frame = True
+                                data_buffer = b""
+                            elif (in_frame and len(data_buffer) < self.HW_MTU):
+                                if (byte == HDLC.ESC):
+                                    escape = True
+                                else:
+                                    if (escape):
+                                        if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
+                                            byte = HDLC.FLAG
+                                        if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
+                                            byte = HDLC.ESC
+                                        escape = False
+                                    data_buffer = data_buffer+bytes([byte])
+                else:
+                    self.online = False
+                    if self.initiator and not self.detached:
+                        RNS.log("Socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
+                        self.reconnect()
+                    else:
+                        RNS.log("Socket for remote client "+str(self)+" was closed.", RNS.LOG_VERBOSE)
+                        self.teardown()
+
+                    break
+
+                
+        except Exception as e:
+            self.online = False
+            RNS.log("An interface error occurred for "+str(self)+", the contained exception was: "+str(e), RNS.LOG_WARNING)
+
+            if self.initiator:
+                RNS.log("Attempting to reconnect...", RNS.LOG_WARNING)
+                self.reconnect()
+            else:
+                self.teardown()
+
+    def teardown(self):
+        if self.initiator and not self.detached:
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+        else:
+            RNS.log("The interface "+str(self)+" is being torn down.", RNS.LOG_VERBOSE)
+
+        self.online = False
+        self.OUT = False
+        self.IN = False
+
+        if hasattr(self, "parent_interface") and self.parent_interface != None:
+            if self.parent_interface.clients > 0:
+                self.parent_interface.clients -= 1
+
+        if self in RNS.Transport.interfaces:
+            if not self.initiator:
+                RNS.Transport.interfaces.remove(self)
+
+
+    def __str__(self):
+        return "I2PInterfacePeer["+str(self.name)+"]"
+
+
+class I2PInterface(Interface):
+    BITRATE_GUESS      = 256*1000
+
+    def __init__(self, owner, name, rns_storagepath, peers, connectable = False):
+        self.rxb = 0
+        self.txb = 0
+        
+        self.HW_MTU = 1064
+
+        self.online = False
+        self.clients = 0
+        self.owner = owner
+        self.connectable = connectable
+        self.i2p_tunneled = True
+        self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
+
+        self.b32 = None
+        self.i2p = I2PController(rns_storagepath)
+
+        self.IN  = True
+        self.OUT = False
+        self.name = name
+
+
+        self.receives = True
+        self.bind_ip     = "127.0.0.1"
+        self.bind_port   = self.i2p.get_free_port()
+        self.address = (self.bind_ip, self.bind_port)
+        self.bitrate = I2PInterface.BITRATE_GUESS
+
+        self.online = False
+
+        i2p_thread = threading.Thread(target=self.i2p.start)
+        i2p_thread.daemon = True
+        i2p_thread.start()
+
+        i2p_notready_warning = False
+        time.sleep(0.25)
+
+        if not self.i2p.ready:
+            RNS.log("I2P controller did not become available in time, waiting for controller", RNS.LOG_VERBOSE)
+            i2p_notready_warning = True
+
+        while not self.i2p.ready:
+            time.sleep(0.25)
+
+        if i2p_notready_warning == True:
+            RNS.log("I2P controller ready, continuing setup", RNS.LOG_VERBOSE)
+
+        def handlerFactory(callback):
+            def createHandler(*args, **keys):
+                return I2PInterfaceHandler(callback, *args, **keys)
+            return createHandler
+        
+        ThreadingI2PServer.allow_reuse_address = True
+        self.server = ThreadingI2PServer(self.address, handlerFactory(self.incoming_connection))
+
+        thread = threading.Thread(target=self.server.serve_forever)
+        thread.daemon = True
+        thread.start()
+
+        if self.connectable:
+            def tunnel_job():
+                while True:
+                    try:
+                        if not self.i2p.server_tunnel(self):
+                            RNS.log(str(self)+" I2P control process experienced an error, requesting new tunnel...", RNS.LOG_ERROR)
+                            self.online = False
+
+                    except Exception as e:
+                        RNS.log("Error while while configuring "+str(self)+": "+str(e), RNS.LOG_ERROR)
+                        RNS.log("Check that I2P is installed and running, and that SAM is enabled. Retrying tunnel setup later.", RNS.LOG_ERROR)
+
+                    time.sleep(15)
+
+
+            thread = threading.Thread(target=tunnel_job)
+            thread.daemon = True
+            thread.start()
+
+        if peers != None:
+            for peer_addr in peers:
+                interface_name = self.name+" to "+peer_addr
+                peer_interface = I2PInterfacePeer(self, self.owner, interface_name, peer_addr)
+                peer_interface.OUT = True
+                peer_interface.IN  = True
+                peer_interface.parent_interface = self
+                peer_interface.parent_count = False
+                RNS.Transport.interfaces.append(peer_interface)
+
+    def incoming_connection(self, handler):
+        RNS.log("Accepting incoming I2P connection", RNS.LOG_VERBOSE)
+        interface_name = "Connected peer on "+self.name
+        spawned_interface = I2PInterfacePeer(self, self.owner, interface_name, connected_socket=handler.request)
+        spawned_interface.OUT = True
+        spawned_interface.IN  = True
+        spawned_interface.parent_interface = self
+        spawned_interface.online = True
+        spawned_interface.bitrate = self.bitrate
+        spawned_interface.ifac_size = self.ifac_size
+        spawned_interface.ifac_netname = self.ifac_netname
+        spawned_interface.ifac_netkey = self.ifac_netkey
+        spawned_interface.announce_rate_target = self.announce_rate_target
+        spawned_interface.announce_rate_grace = self.announce_rate_grace
+        spawned_interface.announce_rate_penalty = self.announce_rate_penalty
+        spawned_interface.mode = self.mode
+        spawned_interface.HW_MTU = self.HW_MTU
+        RNS.log("Spawned new I2PInterface Peer: "+str(spawned_interface), RNS.LOG_VERBOSE)
+        RNS.Transport.interfaces.append(spawned_interface)
+        self.clients += 1
+        spawned_interface.read_loop()
+
+    def processOutgoing(self, data):
+        pass
+
+    def detach(self):
+        RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
+        self.i2p.stop()
+
+    def __str__(self):
+        return "I2PInterface["+self.name+"]"
+
+class I2PInterfaceHandler(socketserver.BaseRequestHandler):
+    def __init__(self, callback, *args, **keys):
+        self.callback = callback
+        socketserver.BaseRequestHandler.__init__(self, *args, **keys)
+
+    def handle(self):
+        self.callback(handler=self)

RNS/Interfaces/Interface.py

@@ -1,4 +1,28 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 import RNS
+import time
+import threading
 
 class Interface:
     IN  = False
@@ -7,9 +31,66 @@ class Interface:
     RPT = False
     name = None
 
+    # Interface mode definitions
+    MODE_FULL           = 0x01
+    MODE_POINT_TO_POINT = 0x02
+    MODE_ACCESS_POINT   = 0x03
+    MODE_ROAMING        = 0x04
+    MODE_BOUNDARY       = 0x05
+    MODE_GATEWAY        = 0x06
+
+    # Which interface modes a Transport Node
+    # should actively discover paths for.
+    DISCOVER_PATHS_FOR  = [MODE_ACCESS_POINT, MODE_GATEWAY]
+
     def __init__(self):
-        pass
+        self.rxb = 0
+        self.txb = 0
+        self.online = False
 
     def get_hash(self):
-    	# TODO: Maybe expand this to something more unique
-    	return RNS.Identity.fullHash(str(self).encode("utf-8"))
+        return RNS.Identity.full_hash(str(self).encode("utf-8"))
+
+    def process_announce_queue(self):
+        if not hasattr(self, "announce_cap"):
+            self.announce_cap = RNS.Reticulum.ANNOUNCE_CAP
+
+        if hasattr(self, "announce_queue"):
+            try:
+                now = time.time()
+                stale = []
+                for a in self.announce_queue:
+                    if now > a["time"]+RNS.Reticulum.QUEUED_ANNOUNCE_LIFE:
+                        stale.append(a)
+
+                for s in stale:
+                    if s in self.announce_queue:
+                        self.announce_queue.remove(s)
+
+                if len(self.announce_queue) > 0:
+                    min_hops = min(entry["hops"] for entry in self.announce_queue)
+                    entries = list(filter(lambda e: e["hops"] == min_hops, self.announce_queue))
+                    entries.sort(key=lambda e: e["time"])
+                    selected = entries[0]
+
+                    now       = time.time()
+                    tx_time   = (len(selected["raw"])*8) / self.bitrate
+                    wait_time = (tx_time / self.announce_cap)
+                    self.announce_allowed_at = now + wait_time
+
+                    self.processOutgoing(selected["raw"])
+
+                    if selected in self.announce_queue:
+                        self.announce_queue.remove(selected)
+
+                    if len(self.announce_queue) > 0:
+                        timer = threading.Timer(wait_time, self.process_announce_queue)
+                        timer.start()
+
+            except Exception as e:
+                self.announce_queue = []
+                RNS.log("Error while processing announce queue on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+                RNS.log("The announce queue for this interface has been cleared.", RNS.LOG_ERROR)
+
+    def detach(self):
+        pass

RNS/Interfaces/KISSInterface.py

@@ -1,256 +1,353 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 from .Interface import Interface
 from time import sleep
 import sys
-import serial
 import threading
 import time
 import RNS
 
 class KISS():
-	FEND			  = 0xC0
-	FESC			  = 0xDB
-	TFEND			  = 0xDC
-	TFESC			  = 0xDD
-	CMD_UNKNOWN		  = 0xFE
-	CMD_DATA		  = 0x00
-	CMD_TXDELAY		  = 0x01
-	CMD_P			  = 0x02
-	CMD_SLOTTIME	  = 0x03
-	CMD_TXTAIL		  = 0x04
-	CMD_FULLDUPLEX	  = 0x05
-	CMD_SETHARDWARE	  = 0x06
-	CMD_READY         = 0x0F
-	CMD_RETURN		  = 0xFF
+    FEND              = 0xC0
+    FESC              = 0xDB
+    TFEND             = 0xDC
+    TFESC             = 0xDD
+    CMD_UNKNOWN       = 0xFE
+    CMD_DATA          = 0x00
+    CMD_TXDELAY       = 0x01
+    CMD_P             = 0x02
+    CMD_SLOTTIME      = 0x03
+    CMD_TXTAIL        = 0x04
+    CMD_FULLDUPLEX    = 0x05
+    CMD_SETHARDWARE   = 0x06
+    CMD_READY         = 0x0F
+    CMD_RETURN        = 0xFF
 
-	@staticmethod
-	def escape(data):
-		data = data.replace(bytes([0xdb]), bytes([0xdb, 0xdd]))
-		data = data.replace(bytes([0xc0]), bytes([0xdb, 0xdc]))
-		return data
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([0xdb]), bytes([0xdb, 0xdd]))
+        data = data.replace(bytes([0xc0]), bytes([0xdb, 0xdc]))
+        return data
 
 class KISSInterface(Interface):
-	MAX_CHUNK = 32768
+    MAX_CHUNK = 32768
+    BITRATE_GUESS = 1200
 
-	owner    = None
-	port     = None
-	speed    = None
-	databits = None
-	parity   = None
-	stopbits = None
-	serial   = None
+    owner    = None
+    port     = None
+    speed    = None
+    databits = None
+    parity   = None
+    stopbits = None
+    serial   = None
 
-	def __init__(self, owner, name, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control):
-		self.serial   = None
-		self.owner    = owner
-		self.name     = name
-		self.port     = port
-		self.speed    = speed
-		self.databits = databits
-		self.parity   = serial.PARITY_NONE
-		self.stopbits = stopbits
-		self.timeout  = 100
-		self.online   = False
+    def __init__(self, owner, name, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control, beacon_interval, beacon_data):
+        import importlib
+        if importlib.util.find_spec('serial') != None:
+            import serial
+        else:
+            RNS.log("Using the KISS interface requires a serial communication module to be installed.", RNS.LOG_CRITICAL)
+            RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
+            RNS.panic()
 
-		self.packet_queue    = []
-		self.flow_control    = flow_control
-		self.interface_ready = False
+        self.rxb = 0
+        self.txb = 0
+        
+        self.HW_MTU = 564
+        
+        if beacon_data == None:
+            beacon_data = ""
 
-		self.preamble    = preamble if preamble != None else 350;
-		self.txtail      = txtail if txtail != None else 20;
-		self.persistence = persistence if persistence != None else 64;
-		self.slottime    = slottime if slottime != None else 20;
+        self.pyserial = serial
+        self.serial   = None
+        self.owner    = owner
+        self.name     = name
+        self.port     = port
+        self.speed    = speed
+        self.databits = databits
+        self.parity   = serial.PARITY_NONE
+        self.stopbits = stopbits
+        self.timeout  = 100
+        self.online   = False
+        self.beacon_i = beacon_interval
+        self.beacon_d = beacon_data.encode("utf-8")
+        self.first_tx = None
+        self.bitrate  = KISSInterface.BITRATE_GUESS
 
-		if parity.lower() == "e" or parity.lower() == "even":
-			self.parity = serial.PARITY_EVEN
+        self.packet_queue    = []
+        self.flow_control    = flow_control
+        self.interface_ready = False
+        self.flow_control_timeout = 5
+        self.flow_control_locked  = time.time()
 
-		if parity.lower() == "o" or parity.lower() == "odd":
-			self.parity = serial.PARITY_ODD
+        self.preamble    = preamble if preamble != None else 350;
+        self.txtail      = txtail if txtail != None else 20;
+        self.persistence = persistence if persistence != None else 64;
+        self.slottime    = slottime if slottime != None else 20;
 
-		try:
-			RNS.log("Opening serial port "+self.port+"...")
-			self.serial = serial.Serial(
-				port = self.port,
-				baudrate = self.speed,
-				bytesize = self.databits,
-				parity = self.parity,
-				stopbits = self.stopbits,
-				xonxoff = False,
-				rtscts = False,
-				timeout = 0,
-				inter_byte_timeout = None,
-				write_timeout = None,
-				dsrdtr = False,
-			)
-		except Exception as e:
-			RNS.log("Could not open serial port "+self.port, RNS.LOG_ERROR)
-			raise e
+        if parity.lower() == "e" or parity.lower() == "even":
+            self.parity = serial.PARITY_EVEN
 
-		if self.serial.is_open:
-			# Allow time for interface to initialise before config
-			sleep(2.0)
-			thread = threading.Thread(target=self.readLoop)
-			thread.setDaemon(True)
-			thread.start()
-			self.online = True
-			RNS.log("Serial port "+self.port+" is now open")
-			RNS.log("Configuring KISS interface parameters...")
-			self.setPreamble(self.preamble)
-			self.setTxTail(self.txtail)
-			self.setPersistence(self.persistence)
-			self.setSlotTime(self.slottime)
-			self.setFlowControl(self.flow_control)
-			self.interface_ready = True
-			RNS.log("KISS interface configured")
-		else:
-			raise IOError("Could not open serial port")
+        if parity.lower() == "o" or parity.lower() == "odd":
+            self.parity = serial.PARITY_ODD
+
+        try:
+            self.open_port()
+        except Exception as e:
+            RNS.log("Could not open serial port "+self.port, RNS.LOG_ERROR)
+            raise e
+
+        if self.serial.is_open:
+            self.configure_device()
+        else:
+            raise IOError("Could not open serial port")
 
 
-	def setPreamble(self, preamble):
-		preamble_ms = preamble
-		preamble = int(preamble_ms / 10)
-		if preamble < 0:
-			preamble = 0
-		if preamble > 255:
-			preamble = 255
-
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_TXDELAY])+bytes([preamble])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			raise IOError("Could not configure KISS interface preamble to "+str(preamble_ms)+" (command value "+str(preamble)+")")
-
-	def setTxTail(self, txtail):
-		txtail_ms = txtail
-		txtail = int(txtail_ms / 10)
-		if txtail < 0:
-			txtail = 0
-		if txtail > 255:
-			txtail = 255
-
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_TXTAIL])+bytes([txtail])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			raise IOError("Could not configure KISS interface TX tail to "+str(txtail_ms)+" (command value "+str(txtail)+")")
-
-	def setPersistence(self, persistence):
-		if persistence < 0:
-			persistence = 0
-		if persistence > 255:
-			persistence = 255
-
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_P])+bytes([persistence])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			raise IOError("Could not configure KISS interface persistence to "+str(persistence))
-
-	def setSlotTime(self, slottime):
-		slottime_ms = slottime
-		slottime = int(slottime_ms / 10)
-		if slottime < 0:
-			slottime = 0
-		if slottime > 255:
-			slottime = 255
-
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SLOTTIME])+bytes([slottime])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			raise IOError("Could not configure KISS interface slot time to "+str(slottime_ms)+" (command value "+str(slottime)+")")
-
-	def setFlowControl(self, flow_control):
-		kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_READY])+bytes([0x01])+bytes([KISS.FEND])
-		written = self.serial.write(kiss_command)
-		if written != len(kiss_command):
-			if (flow_control):
-				raise IOError("Could not enable KISS interface flow control")
-			else:
-				raise IOError("Could not enable KISS interface flow control")
+    def open_port(self):
+        RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
+        self.serial = self.pyserial.Serial(
+            port = self.port,
+            baudrate = self.speed,
+            bytesize = self.databits,
+            parity = self.parity,
+            stopbits = self.stopbits,
+            xonxoff = False,
+            rtscts = False,
+            timeout = 0,
+            inter_byte_timeout = None,
+            write_timeout = None,
+            dsrdtr = False,
+        )
 
 
-	def processIncoming(self, data):
-		self.owner.inbound(data, self)
+    def configure_device(self):
+        # Allow time for interface to initialise before config
+        sleep(2.0)
+        thread = threading.Thread(target=self.readLoop)
+        thread.daemon = True
+        thread.start()
+        self.online = True
+        RNS.log("Serial port "+self.port+" is now open")
+        RNS.log("Configuring KISS interface parameters...")
+        self.setPreamble(self.preamble)
+        self.setTxTail(self.txtail)
+        self.setPersistence(self.persistence)
+        self.setSlotTime(self.slottime)
+        self.setFlowControl(self.flow_control)
+        self.interface_ready = True
+        RNS.log("KISS interface configured")
 
 
-	def processOutgoing(self,data):
-		if self.online:
-			if self.interface_ready:
-				if self.flow_control:
-					self.interface_ready = False
+    def setPreamble(self, preamble):
+        preamble_ms = preamble
+        preamble = int(preamble_ms / 10)
+        if preamble < 0:
+            preamble = 0
+        if preamble > 255:
+            preamble = 255
 
-				data = data.replace(bytes([0xdb]), bytes([0xdb])+bytes([0xdd]))
-				data = data.replace(bytes([0xc0]), bytes([0xdb])+bytes([0xdc]))
-				frame = bytes([KISS.FEND])+bytes([0x00])+data+bytes([KISS.FEND])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_TXDELAY])+bytes([preamble])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("Could not configure KISS interface preamble to "+str(preamble_ms)+" (command value "+str(preamble)+")")
 
-				written = self.serial.write(frame)
-				if written != len(frame):
-					raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
+    def setTxTail(self, txtail):
+        txtail_ms = txtail
+        txtail = int(txtail_ms / 10)
+        if txtail < 0:
+            txtail = 0
+        if txtail > 255:
+            txtail = 255
 
-			else:
-				self.queue(data)
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_TXTAIL])+bytes([txtail])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("Could not configure KISS interface TX tail to "+str(txtail_ms)+" (command value "+str(txtail)+")")
 
-	def queue(self, data):
-		self.packet_queue.append(data)
+    def setPersistence(self, persistence):
+        if persistence < 0:
+            persistence = 0
+        if persistence > 255:
+            persistence = 255
 
-	def process_queue(self):
-		if len(self.packet_queue) > 0:
-			data = self.packet_queue.pop(0)
-			self.interface_ready = True
-			self.processOutgoing(data)
-		elif len(self.packet_queue) == 0:
-			self.interface_ready = True
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_P])+bytes([persistence])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("Could not configure KISS interface persistence to "+str(persistence))
 
-	def readLoop(self):
-		try:
-			in_frame = False
-			escape = False
-			command = KISS.CMD_UNKNOWN
-			data_buffer = b""
-			last_read_ms = int(time.time()*1000)
+    def setSlotTime(self, slottime):
+        slottime_ms = slottime
+        slottime = int(slottime_ms / 10)
+        if slottime < 0:
+            slottime = 0
+        if slottime > 255:
+            slottime = 255
 
-			while self.serial.is_open:
-				if self.serial.in_waiting:
-					byte = ord(self.serial.read(1))
-					last_read_ms = int(time.time()*1000)
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SLOTTIME])+bytes([slottime])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("Could not configure KISS interface slot time to "+str(slottime_ms)+" (command value "+str(slottime)+")")
 
-					if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
-						in_frame = False
-						self.processIncoming(data_buffer)
-					elif (byte == KISS.FEND):
-						in_frame = True
-						command = KISS.CMD_UNKNOWN
-						data_buffer = b""
-					elif (in_frame and len(data_buffer) < RNS.Reticulum.MTU):
-						if (len(data_buffer) == 0 and command == KISS.CMD_UNKNOWN):
-							# We only support one HDLC port for now, so
-							# strip off the port nibble
-							byte = byte & 0x0F
-							command = byte
-						elif (command == KISS.CMD_DATA):
-							if (byte == KISS.FESC):
-								escape = True
-							else:
-								if (escape):
-									if (byte == KISS.TFEND):
-										byte = KISS.FEND
-									if (byte == KISS.TFESC):
-										byte = KISS.FESC
-									escape = False
-								data_buffer = data_buffer+bytes([byte])
-						elif (command == KISS.CMD_READY):
-							# TODO: add timeout and reset if ready
-							# command never arrives
-							self.process_queue()
-				else:
-					time_since_last = int(time.time()*1000) - last_read_ms
-					if len(data_buffer) > 0 and time_since_last > self.timeout:
-						data_buffer = b""
-						in_frame = False
-						command = KISS.CMD_UNKNOWN
-						escape = False
-					sleep(0.08)
+    def setFlowControl(self, flow_control):
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_READY])+bytes([0x01])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            if (flow_control):
+                raise IOError("Could not enable KISS interface flow control")
+            else:
+                raise IOError("Could not enable KISS interface flow control")
 
-		except Exception as e:
-			self.online = False
-			RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
-			RNS.log("The interface "+str(self.name)+" is now offline. Restart Reticulum to attempt reconnection.", RNS.LOG_ERROR)
 
-	def __str__(self):
-		return "KISSInterface["+self.name+"]"
+    def processIncoming(self, data):
+        self.rxb += len(data)  
+        self.owner.inbound(data, self)
+
+
+    def processOutgoing(self,data):
+        datalen = len(data)
+        if self.online:
+            if self.interface_ready:
+                if self.flow_control:
+                    self.interface_ready = False
+                    self.flow_control_locked = time.time()
+
+                data = data.replace(bytes([0xdb]), bytes([0xdb])+bytes([0xdd]))
+                data = data.replace(bytes([0xc0]), bytes([0xdb])+bytes([0xdc]))
+                frame = bytes([KISS.FEND])+bytes([0x00])+data+bytes([KISS.FEND])
+
+                written = self.serial.write(frame)
+                self.txb += datalen
+
+                if data == self.beacon_d:
+                    self.first_tx = None
+                else:
+                    if self.first_tx == None:
+                        self.first_tx = time.time()
+
+                if written != len(frame):
+                    raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
+
+            else:
+                self.queue(data)
+
+    def queue(self, data):
+        self.packet_queue.append(data)
+
+    def process_queue(self):
+        if len(self.packet_queue) > 0:
+            data = self.packet_queue.pop(0)
+            self.interface_ready = True
+            self.processOutgoing(data)
+        elif len(self.packet_queue) == 0:
+            self.interface_ready = True
+
+    def readLoop(self):
+        try:
+            in_frame = False
+            escape = False
+            command = KISS.CMD_UNKNOWN
+            data_buffer = b""
+            last_read_ms = int(time.time()*1000)
+
+            while self.serial.is_open:
+                if self.serial.in_waiting:
+                    byte = ord(self.serial.read(1))
+                    last_read_ms = int(time.time()*1000)
+
+                    if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
+                        in_frame = False
+                        self.processIncoming(data_buffer)
+                    elif (byte == KISS.FEND):
+                        in_frame = True
+                        command = KISS.CMD_UNKNOWN
+                        data_buffer = b""
+                    elif (in_frame and len(data_buffer) < self.HW_MTU):
+                        if (len(data_buffer) == 0 and command == KISS.CMD_UNKNOWN):
+                            # We only support one HDLC port for now, so
+                            # strip off the port nibble
+                            byte = byte & 0x0F
+                            command = byte
+                        elif (command == KISS.CMD_DATA):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                data_buffer = data_buffer+bytes([byte])
+                        elif (command == KISS.CMD_READY):
+                            self.process_queue()
+                else:
+                    time_since_last = int(time.time()*1000) - last_read_ms
+                    if len(data_buffer) > 0 and time_since_last > self.timeout:
+                        data_buffer = b""
+                        in_frame = False
+                        command = KISS.CMD_UNKNOWN
+                        escape = False
+                    sleep(0.05)
+
+                    if self.flow_control:
+                        if not self.interface_ready:
+                            if time.time() > self.flow_control_locked + self.flow_control_timeout:
+                                RNS.log("Interface "+str(self)+" is unlocking flow control due to time-out. This should not happen. Your hardware might have missed a flow-control READY command, or maybe it does not support flow-control.", RNS.LOG_WARNING)
+                                self.process_queue()
+
+                    if self.beacon_i != None and self.beacon_d != None:
+                        if self.first_tx != None:
+                            if time.time() > self.first_tx + self.beacon_i:
+                                RNS.log("Interface "+str(self)+" is transmitting beacon data: "+str(self.beacon_d.decode("utf-8")), RNS.LOG_DEBUG)
+                                self.first_tx = None
+                                self.processOutgoing(self.beacon_d)
+
+        except Exception as e:
+            self.online = False
+            RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
+            
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+            RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
+
+        self.online = False
+        self.serial.close()
+        self.reconnect_port()
+
+    def reconnect_port(self):
+        while not self.online:
+            try:
+                time.sleep(5)
+                RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
+                self.open_port()
+                if self.serial.is_open:
+                    self.configure_device()
+            except Exception as e:
+                RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
+
+        RNS.log("Reconnected serial port for "+str(self))
+
+    def __str__(self):
+        return "KISSInterface["+self.name+"]"

RNS/Interfaces/LocalInterface.py

@@ -1,3 +1,25 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 from .Interface import Interface
 import socketserver
 import threading
@@ -22,13 +44,28 @@ class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
     pass
 
 class LocalClientInterface(Interface):
+    RECONNECT_WAIT = 3
 
     def __init__(self, owner, name, target_port = None, connected_socket=None):
+        self.rxb = 0
+        self.txb = 0
+
+        # TODO: Remove at some point
+        # self.rxptime = 0
+        
+        self.HW_MTU = 1064
+
+        self.online  = False
+        
         self.IN               = True
         self.OUT              = False
         self.socket           = None
         self.parent_interface = None
+        self.reconnecting     = False
+        self.never_connected  = True
+        self.detached         = False
         self.name             = name
+        self.mode             = RNS.Interfaces.Interface.Interface.MODE_FULL
 
         if connected_socket != None:
             self.receives    = True
@@ -36,38 +73,94 @@ class LocalClientInterface(Interface):
             self.target_port = None
             self.socket      = connected_socket
 
+            self.is_connected_to_shared_instance = False
+
         elif target_port != None:
             self.receives    = True
             self.target_ip   = "127.0.0.1"
             self.target_port = target_port
-
-            self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-            self.socket.connect((self.target_ip, self.target_port))
-
-            self.is_connected_to_shared_instance = True
+            self.connect()
 
         self.owner   = owner
+        self.bitrate = 1000*1000*1000
         self.online  = True
         self.writing = False
 
+        self.announce_rate_target  = None
+        self.announce_rate_grace   = None
+        self.announce_rate_penalty = None
+
         if connected_socket == None:
             thread = threading.Thread(target=self.read_loop)
-            thread.setDaemon(True)
+            thread.daemon = True
             thread.start()
 
+    def connect(self):
+        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        self.socket.connect((self.target_ip, self.target_port))
+
+        self.online = True
+        self.is_connected_to_shared_instance = True
+        self.never_connected = False
+
+        return True
+
+
+    def reconnect(self):
+        if self.is_connected_to_shared_instance:
+            if not self.reconnecting:
+                self.reconnecting = True
+                attempts = 0
+
+                while not self.online:
+                    time.sleep(LocalClientInterface.RECONNECT_WAIT)
+                    attempts += 1
+
+                    try:
+                        self.connect()
+
+                    except Exception as e:
+                        RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
+
+                if not self.never_connected:
+                    RNS.log("Reconnected socket for "+str(self)+".", RNS.LOG_INFO)
+
+                self.reconnecting = False
+                thread = threading.Thread(target=self.read_loop)
+                thread.daemon = True
+                thread.start()
+                RNS.Transport.shared_connection_reappeared()
+        
+        else:
+            RNS.log("Attempt to reconnect on a non-initiator shared local interface. This should not happen.", RNS.LOG_ERROR)
+            raise IOError("Attempt to reconnect on a non-initiator local interface")
+
+
     def processIncoming(self, data):
+        self.rxb += len(data)
+        if hasattr(self, "parent_interface") and self.parent_interface != None:
+            self.parent_interface.rxb += len(data)
+        
+        # TODO: Remove at some point
+        # processing_start = time.time()
+        
         self.owner.inbound(data, self)
 
+        # TODO: Remove at some point
+        # duration = time.time() - processing_start
+        # self.rxptime += duration
+
     def processOutgoing(self, data):
         if self.online:
-            while self.writing:
-                time.sleep(0.01)
-
             try:
                 self.writing = True
                 data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
                 self.socket.sendall(data)
                 self.writing = False
+                self.txb += len(data)
+                if hasattr(self, "parent_interface") and self.parent_interface != None:
+                    self.parent_interface.txb += len(data)
+
             except Exception as e:
                 RNS.log("Exception occurred while transmitting via "+str(self)+", tearing down interface", RNS.LOG_ERROR)
                 RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
@@ -93,7 +186,7 @@ class LocalClientInterface(Interface):
                         elif (byte == HDLC.FLAG):
                             in_frame = True
                             data_buffer = b""
-                        elif (in_frame and len(data_buffer) < RNS.Reticulum.MTU):
+                        elif (in_frame and len(data_buffer) < self.HW_MTU):
                             if (byte == HDLC.ESC):
                                 escape = True
                             else:
@@ -105,8 +198,14 @@ class LocalClientInterface(Interface):
                                     escape = False
                                 data_buffer = data_buffer+bytes([byte])
                 else:
-                    RNS.log("Socket for "+str(self)+" was closed, tearing down interface", RNS.LOG_VERBOSE)
-                    self.teardown()
+                    self.online = False
+                    if self.is_connected_to_shared_instance and not self.detached:
+                        RNS.log("Socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
+                        RNS.Transport.shared_connection_disappeared()
+                        self.reconnect()
+                    else:
+                        self.teardown(nowarning=True)
+
                     break
 
                 
@@ -116,7 +215,26 @@ class LocalClientInterface(Interface):
             RNS.log("Tearing down "+str(self), RNS.LOG_ERROR)
             self.teardown()
 
-    def teardown(self):
+    def detach(self):
+        if self.socket != None:
+            if hasattr(self.socket, "close"):
+                if callable(self.socket.close):
+                    RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
+                    self.detached = True
+                    
+                    try:
+                        self.socket.shutdown(socket.SHUT_RDWR)
+                    except Exception as e:
+                        RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
+
+                    try:
+                        self.socket.close()
+                    except Exception as e:
+                        RNS.log("Error while closing socket for "+str(self)+": "+str(e))
+
+                    self.socket = None
+
+    def teardown(self, nowarning=False):
         self.online = False
         self.OUT = False
         self.IN = False
@@ -126,6 +244,20 @@ class LocalClientInterface(Interface):
 
         if self in RNS.Transport.local_client_interfaces:
             RNS.Transport.local_client_interfaces.remove(self)
+            if hasattr(self, "parent_interface") and self.parent_interface != None:
+                self.parent_interface.clients -= 1
+                RNS.Transport.owner._should_persist_data()
+
+        if nowarning == False:
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+        if self.is_connected_to_shared_instance:
+            if nowarning == False:
+                RNS.log("Permanently lost connection to local shared RNS instance. Exiting now.", RNS.LOG_CRITICAL)
+    
+            RNS.exit()
 
 
     def __str__(self):
@@ -135,9 +267,15 @@ class LocalClientInterface(Interface):
 class LocalServerInterface(Interface):
 
     def __init__(self, owner, bindport=None):
+        self.rxb = 0
+        self.txb = 0
+        self.online = False
+        self.clients = 0
+        
         self.IN  = True
         self.OUT = False
         self.name = "Reticulum"
+        self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
 
         if (bindport != None):
             self.receives = True
@@ -153,12 +291,22 @@ class LocalServerInterface(Interface):
             self.is_local_shared_instance = True
 
             address = (self.bind_ip, self.bind_port)
+
+            ThreadingTCPServer.allow_reuse_address = True
             self.server = ThreadingTCPServer(address, handlerFactory(self.incoming_connection))
 
             thread = threading.Thread(target=self.server.serve_forever)
-            thread.setDaemon(True)
+            thread.daemon = True
             thread.start()
 
+            self.announce_rate_target  = None
+            self.announce_rate_grace   = None
+            self.announce_rate_penalty = None
+
+            self.bitrate = 1000*1000*1000
+            self.online = True
+
+
 
     def incoming_connection(self, handler):
         interface_name = str(str(handler.client_address[1]))
@@ -168,16 +316,18 @@ class LocalServerInterface(Interface):
         spawned_interface.target_ip = handler.client_address[0]
         spawned_interface.target_port = str(handler.client_address[1])
         spawned_interface.parent_interface = self
-        RNS.log("Accepting new connection to shared instance: "+str(spawned_interface), RNS.LOG_VERBOSE)
+        spawned_interface.bitrate = self.bitrate
+        RNS.log("Accepting new connection to shared instance: "+str(spawned_interface), RNS.LOG_EXTREME)
         RNS.Transport.interfaces.append(spawned_interface)
         RNS.Transport.local_client_interfaces.append(spawned_interface)
+        self.clients += 1
         spawned_interface.read_loop()
 
     def processOutgoing(self, data):
         pass
 
     def __str__(self):
-        return "Shared Instance ["+str(self.bind_port)+"]"
+        return "Shared Instance["+str(self.bind_port)+"]"
 
 class LocalInterfaceHandler(socketserver.BaseRequestHandler):
     def __init__(self, callback, *args, **keys):

RNS/Interfaces/PipeInterface.py

@@ -0,0 +1,189 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from .Interface import Interface
+from time import sleep
+import sys
+import threading
+import time
+import RNS
+
+import subprocess
+import shlex
+
+class HDLC():
+    # The Pipe Interface packetizes data using
+    # simplified HDLC framing, similar to PPP
+    FLAG              = 0x7E
+    ESC               = 0x7D
+    ESC_MASK          = 0x20
+
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
+        data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
+        return data
+
+class PipeInterface(Interface):
+    MAX_CHUNK = 32768
+    BITRATE_GUESS = 1*1000*1000
+
+    owner    = None
+    command  = None
+    
+    def __init__(self, owner, name, command, respawn_delay):
+        if respawn_delay == None:
+            respawn_delay = 5
+
+        self.rxb = 0
+        self.txb = 0
+
+        self.HW_MTU = 1064
+        
+        self.owner    = owner
+        self.name     = name
+        self.command  = command
+        self.process  = None
+        self.timeout  = 100
+        self.online   = False
+        self.pipe_is_open = False
+        self.bitrate  = PipeInterface.BITRATE_GUESS
+        self.respawn_delay = respawn_delay
+
+        try:
+            self.open_pipe()
+
+        except Exception as e:
+            RNS.log("Could connect pipe for interface "+str(self), RNS.LOG_ERROR)
+            raise e
+
+        if self.pipe_is_open:
+            self.configure_pipe()
+        else:
+            raise IOError("Could not connect pipe")
+
+
+    def open_pipe(self):
+        RNS.log("Connecting subprocess pipe for "+str(self)+"...", RNS.LOG_VERBOSE)
+        
+        try:
+            self.process = subprocess.Popen(shlex.split(self.command), stdin=subprocess.PIPE, stdout=subprocess.PIPE)
+            self.pipe_is_open = True
+        except Exception as e:
+            raise e
+            self.pipe_is_open = False
+
+
+    def configure_pipe(self):
+        sleep(0.01)
+        thread = threading.Thread(target=self.readLoop)
+        thread.daemon = True
+        thread.start()
+        self.online = True
+        RNS.log("Subprocess pipe for "+str(self)+" is now connected", RNS.LOG_VERBOSE)
+
+
+    def processIncoming(self, data):
+        self.rxb += len(data)            
+        self.owner.inbound(data, self)
+
+
+    def processOutgoing(self,data):
+        if self.online:
+            data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+            written = self.process.stdin.write(data)
+            self.process.stdin.flush()
+            self.txb += len(data)            
+            if written != len(data):
+                raise IOError("Pipe interface only wrote "+str(written)+" bytes of "+str(len(data)))
+
+
+    def readLoop(self):
+        try:
+            in_frame = False
+            escape = False
+            data_buffer = b""
+            last_read_ms = int(time.time()*1000)
+
+            while True:
+                process_output = self.process.stdout.read(1)
+                if len(process_output) == 0 and self.process.poll() is not None:
+                    break
+
+                else:
+                    byte = ord(process_output)
+                    last_read_ms = int(time.time()*1000)
+
+                    if (in_frame and byte == HDLC.FLAG):
+                        in_frame = False
+                        self.processIncoming(data_buffer)
+                    elif (byte == HDLC.FLAG):
+                        in_frame = True
+                        data_buffer = b""
+                    elif (in_frame and len(data_buffer) < self.HW_MTU):
+                        if (byte == HDLC.ESC):
+                            escape = True
+                        else:
+                            if (escape):
+                                if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
+                                    byte = HDLC.FLAG
+                                if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
+                                    byte = HDLC.ESC
+                                escape = False
+                            data_buffer = data_buffer+bytes([byte])
+
+            RNS.log("Subprocess terminated on "+str(self))
+            self.process.kill()
+                    
+        except Exception as e:
+            self.online = False
+            try:
+                self.process.kill()
+            except Exception as e:
+                pass
+
+            RNS.log("A pipe error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
+            
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+            RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
+
+        self.online = False
+        self.reconnect_pipe()
+
+    def reconnect_pipe(self):
+        while not self.online:
+            try:
+                time.sleep(self.respawn_delay)
+                RNS.log("Attempting to respawn subprocess for "+str(self)+"...", RNS.LOG_VERBOSE)
+                self.open_pipe()
+                if self.pipe_is_open:
+                    self.configure_pipe()
+            except Exception as e:
+                RNS.log("Error while spawning subprocess, the contained exception was: "+str(e), RNS.LOG_ERROR)
+
+        RNS.log("Reconnected pipe for "+str(self))
+
+    def __str__(self):
+        return "PipeInterface["+self.name+"]"

RNS/Interfaces/SerialInterface.py

@@ -1,136 +1,205 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 from .Interface import Interface
 from time import sleep
 import sys
-import serial
 import threading
 import time
 import RNS
 
 class HDLC():
-	# The Serial Interface packetizes data using
-	# simplified HDLC framing, similar to PPP
-	FLAG			  = 0x7E
-	ESC               = 0x7D
-	ESC_MASK          = 0x20
+    # The Serial Interface packetizes data using
+    # simplified HDLC framing, similar to PPP
+    FLAG              = 0x7E
+    ESC               = 0x7D
+    ESC_MASK          = 0x20
 
-	@staticmethod
-	def escape(data):
-		data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
-		data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
-		return data
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
+        data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
+        return data
 
 class SerialInterface(Interface):
-	MAX_CHUNK = 32768
+    MAX_CHUNK = 32768
 
-	owner    = None
-	port     = None
-	speed    = None
-	databits = None
-	parity   = None
-	stopbits = None
-	serial   = None
+    owner    = None
+    port     = None
+    speed    = None
+    databits = None
+    parity   = None
+    stopbits = None
+    serial   = None
 
-	def __init__(self, owner, name, port, speed, databits, parity, stopbits):
-		self.serial   = None
-		self.owner    = owner
-		self.name     = name
-		self.port     = port
-		self.speed    = speed
-		self.databits = databits
-		self.parity   = serial.PARITY_NONE
-		self.stopbits = stopbits
-		self.timeout  = 100
-		self.online   = False
+    def __init__(self, owner, name, port, speed, databits, parity, stopbits):
+        import importlib
+        if importlib.util.find_spec('serial') != None:
+            import serial
+        else:
+            RNS.log("Using the Serial interface requires a serial communication module to be installed.", RNS.LOG_CRITICAL)
+            RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
+            RNS.panic()
 
-		if parity.lower() == "e" or parity.lower() == "even":
-			self.parity = serial.PARITY_EVEN
+        self.rxb = 0
+        self.txb = 0
 
-		if parity.lower() == "o" or parity.lower() == "odd":
-			self.parity = serial.PARITY_ODD
+        self.HW_MTU = 564
+        
+        self.pyserial = serial
+        self.serial   = None
+        self.owner    = owner
+        self.name     = name
+        self.port     = port
+        self.speed    = speed
+        self.databits = databits
+        self.parity   = serial.PARITY_NONE
+        self.stopbits = stopbits
+        self.timeout  = 100
+        self.online   = False
+        self.bitrate  = self.speed
 
-		try:
-			RNS.log("Opening serial port "+self.port+"...")
-			self.serial = serial.Serial(
-				port = self.port,
-				baudrate = self.speed,
-				bytesize = self.databits,
-				parity = self.parity,
-				stopbits = self.stopbits,
-				xonxoff = False,
-				rtscts = False,
-				timeout = 0,
-				inter_byte_timeout = None,
-				write_timeout = None,
-				dsrdtr = False,
-			)
-		except Exception as e:
-			RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
-			raise e
+        if parity.lower() == "e" or parity.lower() == "even":
+            self.parity = serial.PARITY_EVEN
 
-		if self.serial.is_open:
-			sleep(0.5)
-			thread = threading.Thread(target=self.readLoop)
-			thread.setDaemon(True)
-			thread.start()
-			self.online = True
-			RNS.log("Serial port "+self.port+" is now open")
-		else:
-			raise IOError("Could not open serial port")
+        if parity.lower() == "o" or parity.lower() == "odd":
+            self.parity = serial.PARITY_ODD
+
+        try:
+            self.open_port()
+        except Exception as e:
+            RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
+            raise e
+
+        if self.serial.is_open:
+            self.configure_device()
+        else:
+            raise IOError("Could not open serial port")
 
 
-	def processIncoming(self, data):
-		self.owner.inbound(data, self)
+    def open_port(self):
+        RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
+        self.serial = self.pyserial.Serial(
+            port = self.port,
+            baudrate = self.speed,
+            bytesize = self.databits,
+            parity = self.parity,
+            stopbits = self.stopbits,
+            xonxoff = False,
+            rtscts = False,
+            timeout = 0,
+            inter_byte_timeout = None,
+            write_timeout = None,
+            dsrdtr = False,
+        )
 
 
-	def processOutgoing(self,data):
-		if self.online:
-			data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
-			written = self.serial.write(data)
-			if written != len(data):
-				raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
+    def configure_device(self):
+        sleep(0.5)
+        thread = threading.Thread(target=self.readLoop)
+        thread.daemon = True
+        thread.start()
+        self.online = True
+        RNS.log("Serial port "+self.port+" is now open", RNS.LOG_VERBOSE)
 
 
-	def readLoop(self):
-		try:
-			in_frame = False
-			escape = False
-			data_buffer = b""
-			last_read_ms = int(time.time()*1000)
+    def processIncoming(self, data):
+        self.rxb += len(data)            
+        self.owner.inbound(data, self)
 
-			while self.serial.is_open:
-				if self.serial.in_waiting:
-					byte = ord(self.serial.read(1))
-					last_read_ms = int(time.time()*1000)
 
-					if (in_frame and byte == HDLC.FLAG):
-						in_frame = False
-						self.processIncoming(data_buffer)
-					elif (byte == HDLC.FLAG):
-						in_frame = True
-						data_buffer = b""
-					elif (in_frame and len(data_buffer) < RNS.Reticulum.MTU):
-						if (byte == HDLC.ESC):
-							escape = True
-						else:
-							if (escape):
-								if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
-									byte = HDLC.FLAG
-								if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
-									byte = HDLC.ESC
-								escape = False
-							data_buffer = data_buffer+bytes([byte])
-						
-				else:
-					time_since_last = int(time.time()*1000) - last_read_ms
-					if len(data_buffer) > 0 and time_since_last > self.timeout:
-						data_buffer = b""
-						in_frame = False
-						escape = False
-					sleep(0.08)
-		except Exception as e:
-			self.online = False
-			RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
-			RNS.log("The interface "+str(self.name)+" is now offline. Restart Reticulum to attempt reconnection.", RNS.LOG_ERROR)
+    def processOutgoing(self,data):
+        if self.online:
+            data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+            written = self.serial.write(data)
+            self.txb += len(data)            
+            if written != len(data):
+                raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
 
-	def __str__(self):
-		return "SerialInterface["+self.name+"]"
+
+    def readLoop(self):
+        try:
+            in_frame = False
+            escape = False
+            data_buffer = b""
+            last_read_ms = int(time.time()*1000)
+
+            while self.serial.is_open:
+                if self.serial.in_waiting:
+                    byte = ord(self.serial.read(1))
+                    last_read_ms = int(time.time()*1000)
+
+                    if (in_frame and byte == HDLC.FLAG):
+                        in_frame = False
+                        self.processIncoming(data_buffer)
+                    elif (byte == HDLC.FLAG):
+                        in_frame = True
+                        data_buffer = b""
+                    elif (in_frame and len(data_buffer) < self.HW_MTU):
+                        if (byte == HDLC.ESC):
+                            escape = True
+                        else:
+                            if (escape):
+                                if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
+                                    byte = HDLC.FLAG
+                                if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
+                                    byte = HDLC.ESC
+                                escape = False
+                            data_buffer = data_buffer+bytes([byte])
+                        
+                else:
+                    time_since_last = int(time.time()*1000) - last_read_ms
+                    if len(data_buffer) > 0 and time_since_last > self.timeout:
+                        data_buffer = b""
+                        in_frame = False
+                        escape = False
+                    sleep(0.08)
+                    
+        except Exception as e:
+            self.online = False
+            RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
+            
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+            RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
+
+        self.online = False
+        self.serial.close()
+        self.reconnect_port()
+
+    def reconnect_port(self):
+        while not self.online:
+            try:
+                time.sleep(5)
+                RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
+                self.open_port()
+                if self.serial.is_open:
+                    self.configure_device()
+            except Exception as e:
+                RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
+
+        RNS.log("Reconnected serial port for "+str(self))
+
+    def __str__(self):
+        return "SerialInterface["+self.name+"]"

RNS/Interfaces/TCPInterface.py

@@ -1,6 +1,29 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 from .Interface import Interface
 import socketserver
 import threading
+import platform
 import socket
 import time
 import sys
@@ -18,17 +41,70 @@ class HDLC():
         data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
         return data
 
+class KISS():
+    FEND              = 0xC0
+    FESC              = 0xDB
+    TFEND             = 0xDC
+    TFESC             = 0xDD
+    CMD_DATA          = 0x00
+    CMD_UNKNOWN       = 0xFE
+
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([0xdb]), bytes([0xdb, 0xdd]))
+        data = data.replace(bytes([0xc0]), bytes([0xdb, 0xdc]))
+        return data
+
 class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
     pass
 
 class TCPClientInterface(Interface):
+    BITRATE_GUESS = 10*1000*1000
 
-    def __init__(self, owner, name, target_ip=None, target_port=None, connected_socket=None):
+    RECONNECT_WAIT = 5
+    RECONNECT_MAX_TRIES = None
+
+    # TCP socket options
+    TCP_USER_TIMEOUT = 24
+    TCP_PROBE_AFTER = 5
+    TCP_PROBE_INTERVAL = 2
+    TCP_PROBES = 12
+
+    INITIAL_CONNECT_TIMEOUT = 5
+    SYNCHRONOUS_START = True
+
+    I2P_USER_TIMEOUT = 45
+    I2P_PROBE_AFTER = 10
+    I2P_PROBE_INTERVAL = 9
+    I2P_PROBES = 5
+
+    def __init__(self, owner, name, target_ip=None, target_port=None, connected_socket=None, max_reconnect_tries=None, kiss_framing=False, i2p_tunneled = False, connect_timeout = None):
+        self.rxb = 0
+        self.txb = 0
+        
+        self.HW_MTU = 1064
+        
         self.IN               = True
         self.OUT              = False
         self.socket           = None
         self.parent_interface = None
         self.name             = name
+        self.initiator        = False
+        self.reconnecting     = False
+        self.never_connected  = True
+        self.owner            = owner
+        self.writing          = False
+        self.online           = False
+        self.detached         = False
+        self.kiss_framing     = kiss_framing
+        self.i2p_tunneled     = i2p_tunneled
+        self.mode             = RNS.Interfaces.Interface.Interface.MODE_FULL
+        self.bitrate          = TCPClientInterface.BITRATE_GUESS
+        
+        if max_reconnect_tries == None:
+            self.max_reconnect_tries = TCPClientInterface.RECONNECT_MAX_TRIES
+        else:
+            self.max_reconnect_tries = max_reconnect_tries
 
         if connected_socket != None:
             self.receives    = True
@@ -36,36 +112,183 @@ class TCPClientInterface(Interface):
             self.target_port = None
             self.socket      = connected_socket
 
+            if platform.system() == "Linux":
+                self.set_timeouts_linux()
+            elif platform.system() == "Darwin":
+                self.set_timeouts_osx()
+
         elif target_ip != None and target_port != None:
             self.receives    = True
             self.target_ip   = target_ip
             self.target_port = target_port
+            self.initiator   = True
+
+            if connect_timeout != None:
+                self.connect_timeout = connect_timeout
+            else:
+                self.connect_timeout = TCPClientInterface.INITIAL_CONNECT_TIMEOUT
+            
+            if TCPClientInterface.SYNCHRONOUS_START:
+                self.initial_connect()
+            else:
+                thread = threading.Thread(target=self.initial_connect)
+                thread.daemon = True
+                thread.start()
+            
+    def initial_connect(self):
+        if not self.connect(initial=True):
+            thread = threading.Thread(target=self.reconnect)
+            thread.daemon = True
+            thread.start()
+        else:
+            thread = threading.Thread(target=self.read_loop)
+            thread.daemon = True
+            thread.start()
+            if not self.kiss_framing:
+                self.wants_tunnel = True
+
+    def set_timeouts_linux(self):
+        if not self.i2p_tunneled:
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, int(TCPClientInterface.TCP_USER_TIMEOUT * 1000))
+            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, int(TCPClientInterface.TCP_PROBE_AFTER))
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, int(TCPClientInterface.TCP_PROBE_INTERVAL))
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, int(TCPClientInterface.TCP_PROBES))
+
+        else:
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, int(TCPClientInterface.I2P_USER_TIMEOUT * 1000))
+            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, int(TCPClientInterface.I2P_PROBE_AFTER))
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, int(TCPClientInterface.I2P_PROBE_INTERVAL))
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, int(TCPClientInterface.I2P_PROBES))
+
+    def set_timeouts_osx(self):
+        if hasattr(socket, "TCP_KEEPALIVE"):
+            TCP_KEEPIDLE = socket.TCP_KEEPALIVE
+        else:
+            TCP_KEEPIDLE = 0x10
+
+        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
+        
+        if not self.i2p_tunneled:
+            self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(TCPClientInterface.TCP_PROBE_AFTER))
+        else:
+            self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(TCPClientInterface.I2P_PROBE_AFTER))
+        
+    def detach(self):
+        if self.socket != None:
+            if hasattr(self.socket, "close"):
+                if callable(self.socket.close):
+                    RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
+                    self.detached = True
+                    
+                    try:
+                        self.socket.shutdown(socket.SHUT_RDWR)
+                    except Exception as e:
+                        RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
+
+                    try:
+                        self.socket.close()
+                    except Exception as e:
+                        RNS.log("Error while closing socket for "+str(self)+": "+str(e))
+
+                    self.socket = None
+
+    def connect(self, initial=False):
+        try:
+            if initial:
+                RNS.log("Establishing TCP connection for "+str(self)+"...", RNS.LOG_DEBUG)
 
             self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            self.socket.settimeout(TCPClientInterface.INITIAL_CONNECT_TIMEOUT)
             self.socket.connect((self.target_ip, self.target_port))
+            self.socket.settimeout(None)
+            self.online  = True
 
-        self.owner   = owner
+            if initial:
+                RNS.log("TCP connection for "+str(self)+" established", RNS.LOG_DEBUG)
+        
+        except Exception as e:
+            if initial:
+                RNS.log("Initial connection for "+str(self)+" could not be established: "+str(e), RNS.LOG_ERROR)
+                RNS.log("Leaving unconnected and retrying connection in "+str(TCPClientInterface.RECONNECT_WAIT)+" seconds.", RNS.LOG_ERROR)
+                return False
+            
+            else:
+                raise e
+
+        if platform.system() == "Linux":
+            self.set_timeouts_linux()
+        elif platform.system() == "Darwin":
+            self.set_timeouts_osx()
+        
         self.online  = True
         self.writing = False
+        self.never_connected = False
 
-        if connected_socket == None:
-            thread = threading.Thread(target=self.read_loop)
-            thread.setDaemon(True)
-            thread.start()
+        return True
+
+
+    def reconnect(self):
+        if self.initiator:
+            if not self.reconnecting:
+                self.reconnecting = True
+                attempts = 0
+                while not self.online:
+                    time.sleep(TCPClientInterface.RECONNECT_WAIT)
+                    attempts += 1
+
+                    if self.max_reconnect_tries != None and attempts > self.max_reconnect_tries:
+                        RNS.log("Max reconnection attempts reached for "+str(self), RNS.LOG_ERROR)
+                        self.teardown()
+                        break
+
+                    try:
+                        self.connect()
+
+                    except Exception as e:
+                        RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
+
+                if not self.never_connected:
+                    RNS.log("Reconnected socket for "+str(self)+".", RNS.LOG_INFO)
+
+                self.reconnecting = False
+                thread = threading.Thread(target=self.read_loop)
+                thread.daemon = True
+                thread.start()
+                if not self.kiss_framing:
+                    RNS.Transport.synthesize_tunnel(self)
+
+        else:
+            RNS.log("Attempt to reconnect on a non-initiator TCP interface. This should not happen.", RNS.LOG_ERROR)
+            raise IOError("Attempt to reconnect on a non-initiator TCP interface")
 
     def processIncoming(self, data):
+        self.rxb += len(data)
+        if hasattr(self, "parent_interface") and self.parent_interface != None:
+            self.parent_interface.rxb += len(data)
+                    
         self.owner.inbound(data, self)
 
     def processOutgoing(self, data):
         if self.online:
-            while self.writing:
-                time.sleep(0.01)
+            # while self.writing:
+            #     time.sleep(0.01)
 
             try:
                 self.writing = True
-                data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+
+                if self.kiss_framing:
+                    data = bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+KISS.escape(data)+bytes([KISS.FEND])
+                else:
+                    data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+
                 self.socket.sendall(data)
                 self.writing = False
+                self.txb += len(data)
+                if hasattr(self, "parent_interface") and self.parent_interface != None:
+                    self.parent_interface.txb += len(data)
+
             except Exception as e:
                 RNS.log("Exception occurred while transmitting via "+str(self)+", tearing down interface", RNS.LOG_ERROR)
                 RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
@@ -77,6 +300,7 @@ class TCPClientInterface(Interface):
             in_frame = False
             escape = False
             data_buffer = b""
+            command = KISS.CMD_UNKNOWN
 
             while True:
                 data_in = self.socket.recv(4096)
@@ -85,41 +309,94 @@ class TCPClientInterface(Interface):
                     while pointer < len(data_in):
                         byte = data_in[pointer]
                         pointer += 1
-                        if (in_frame and byte == HDLC.FLAG):
-                            in_frame = False
-                            self.processIncoming(data_buffer)
-                        elif (byte == HDLC.FLAG):
-                            in_frame = True
-                            data_buffer = b""
-                        elif (in_frame and len(data_buffer) < RNS.Reticulum.MTU):
-                            if (byte == HDLC.ESC):
-                                escape = True
-                            else:
-                                if (escape):
-                                    if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
-                                        byte = HDLC.FLAG
-                                    if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
-                                        byte = HDLC.ESC
-                                    escape = False
-                                data_buffer = data_buffer+bytes([byte])
+
+                        if self.kiss_framing:
+                            # Read loop for KISS framing
+                            if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
+                                in_frame = False
+                                self.processIncoming(data_buffer)
+                            elif (byte == KISS.FEND):
+                                in_frame = True
+                                command = KISS.CMD_UNKNOWN
+                                data_buffer = b""
+                            elif (in_frame and len(data_buffer) < self.HW_MTU):
+                                if (len(data_buffer) == 0 and command == KISS.CMD_UNKNOWN):
+                                    # We only support one HDLC port for now, so
+                                    # strip off the port nibble
+                                    byte = byte & 0x0F
+                                    command = byte
+                                elif (command == KISS.CMD_DATA):
+                                    if (byte == KISS.FESC):
+                                        escape = True
+                                    else:
+                                        if (escape):
+                                            if (byte == KISS.TFEND):
+                                                byte = KISS.FEND
+                                            if (byte == KISS.TFESC):
+                                                byte = KISS.FESC
+                                            escape = False
+                                        data_buffer = data_buffer+bytes([byte])
+
+                        else:
+                            # Read loop for HDLC framing
+                            if (in_frame and byte == HDLC.FLAG):
+                                in_frame = False
+                                self.processIncoming(data_buffer)
+                            elif (byte == HDLC.FLAG):
+                                in_frame = True
+                                data_buffer = b""
+                            elif (in_frame and len(data_buffer) < self.HW_MTU):
+                                if (byte == HDLC.ESC):
+                                    escape = True
+                                else:
+                                    if (escape):
+                                        if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
+                                            byte = HDLC.FLAG
+                                        if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
+                                            byte = HDLC.ESC
+                                        escape = False
+                                    data_buffer = data_buffer+bytes([byte])
                 else:
-                    RNS.log("TCP socket for "+str(self)+" was closed, tearing down interface", RNS.LOG_VERBOSE)
-                    self.teardown()
+                    self.online = False
+                    if self.initiator and not self.detached:
+                        RNS.log("The socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
+                        self.reconnect()
+                    else:
+                        RNS.log("The socket for remote client "+str(self)+" was closed.", RNS.LOG_VERBOSE)
+                        self.teardown()
+
                     break
 
                 
         except Exception as e:
             self.online = False
-            RNS.log("An interface error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
-            RNS.log("Tearing down "+str(self), RNS.LOG_ERROR)
-            self.teardown()
+            RNS.log("An interface error occurred for "+str(self)+", the contained exception was: "+str(e), RNS.LOG_WARNING)
+
+            if self.initiator:
+                RNS.log("Attempting to reconnect...", RNS.LOG_WARNING)
+                self.reconnect()
+            else:
+                self.teardown()
 
     def teardown(self):
+        if self.initiator and not self.detached:
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+        else:
+            RNS.log("The interface "+str(self)+" is being torn down.", RNS.LOG_VERBOSE)
+
         self.online = False
         self.OUT = False
         self.IN = False
+
+        if hasattr(self, "parent_interface") and self.parent_interface != None:
+            self.parent_interface.clients -= 1
+
         if self in RNS.Transport.interfaces:
-            RNS.Transport.interfaces.remove(self)
+            if not self.initiator:
+                RNS.Transport.interfaces.remove(self)
 
 
     def __str__(self):
@@ -127,11 +404,49 @@ class TCPClientInterface(Interface):
 
 
 class TCPServerInterface(Interface):
+    BITRATE_GUESS      = 10*1000*1000
 
-    def __init__(self, owner, name, bindip=None, bindport=None):
+    @staticmethod
+    def get_address_for_if(name):
+        import importlib
+        if importlib.util.find_spec('netifaces') != None:
+            import netifaces
+            return netifaces.ifaddresses(name)[netifaces.AF_INET][0]['addr']
+        else:
+            RNS.log("Getting interface addresses from device names requires the netifaces module.", RNS.LOG_CRITICAL)
+            RNS.log("You can install it with the command: python3 -m pip install netifaces", RNS.LOG_CRITICAL)
+            RNS.panic()
+
+    @staticmethod
+    def get_broadcast_for_if(name):
+        import importlib
+        if importlib.util.find_spec('netifaces') != None:
+            import netifaces
+            return netifaces.ifaddresses(name)[netifaces.AF_INET][0]['broadcast']
+        else:
+            RNS.log("Getting interface addresses from device names requires the netifaces module.", RNS.LOG_CRITICAL)
+            RNS.log("You can install it with the command: python3 -m pip install netifaces", RNS.LOG_CRITICAL)
+            RNS.panic()
+
+    def __init__(self, owner, name, device=None, bindip=None, bindport=None, i2p_tunneled=False):
+        self.rxb = 0
+        self.txb = 0
+
+        self.HW_MTU = 1064
+
+        self.online = False
+        self.clients = 0
+        
         self.IN  = True
         self.OUT = False
         self.name = name
+        self.detached = False
+
+        self.i2p_tunneled = i2p_tunneled
+        self.mode         = RNS.Interfaces.Interface.Interface.MODE_FULL
+
+        if device != None:
+            bindip = TCPServerInterface.get_address_for_if(device)
 
         if (bindip != None and bindport != None):
             self.receives = True
@@ -145,36 +460,69 @@ class TCPServerInterface(Interface):
 
             self.owner = owner
             address = (self.bind_ip, self.bind_port)
+
+            ThreadingTCPServer.allow_reuse_address = True
             self.server = ThreadingTCPServer(address, handlerFactory(self.incoming_connection))
 
+            self.bitrate = TCPServerInterface.BITRATE_GUESS
+
             thread = threading.Thread(target=self.server.serve_forever)
-            thread.setDaemon(True)
+            thread.daemon = True
             thread.start()
 
+            self.online = True
+
 
     def incoming_connection(self, handler):
         RNS.log("Accepting incoming TCP connection", RNS.LOG_VERBOSE)
         interface_name = "Client on "+self.name
-        spawned_interface = TCPClientInterface(self.owner, interface_name, target_ip=None, target_port=None, connected_socket=handler.request)
+        spawned_interface = TCPClientInterface(self.owner, interface_name, target_ip=None, target_port=None, connected_socket=handler.request, i2p_tunneled=self.i2p_tunneled)
         spawned_interface.OUT = self.OUT
         spawned_interface.IN  = self.IN
         spawned_interface.target_ip = handler.client_address[0]
         spawned_interface.target_port = str(handler.client_address[1])
         spawned_interface.parent_interface = self
+        spawned_interface.bitrate = self.bitrate
+        spawned_interface.ifac_size = self.ifac_size
+        spawned_interface.ifac_netname = self.ifac_netname
+        spawned_interface.ifac_netkey = self.ifac_netkey
+        spawned_interface.announce_rate_target = self.announce_rate_target
+        spawned_interface.announce_rate_grace = self.announce_rate_grace
+        spawned_interface.announce_rate_penalty = self.announce_rate_penalty
+        spawned_interface.mode = self.mode
+        spawned_interface.HW_MTU = self.HW_MTU
+        spawned_interface.online = True
         RNS.log("Spawned new TCPClient Interface: "+str(spawned_interface), RNS.LOG_VERBOSE)
         RNS.Transport.interfaces.append(spawned_interface)
+        self.clients += 1
         spawned_interface.read_loop()
 
     def processOutgoing(self, data):
         pass
 
+
+    def detach(self):
+        if self.server != None:
+            if hasattr(self.server, "shutdown"):
+                if callable(self.server.shutdown):
+                    try:
+                        RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
+                        self.server.shutdown()
+                        self.detached = True
+                        self.server = None
+
+                    except Exception as e:
+                        RNS.log("Error while shutting down server for "+str(self)+": "+str(e))
+
+
     def __str__(self):
         return "TCPServerInterface["+self.name+"/"+self.bind_ip+":"+str(self.bind_port)+"]"
 
+
 class TCPInterfaceHandler(socketserver.BaseRequestHandler):
     def __init__(self, callback, *args, **keys):
         self.callback = callback
         socketserver.BaseRequestHandler.__init__(self, *args, **keys)
 
     def handle(self):
-        self.callback(handler=self)
+        self.callback(handler=self)

RNS/Interfaces/UDPInterface.py

@@ -0,0 +1,128 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from .Interface import Interface
+import socketserver
+import threading
+import socket
+import time
+import sys
+import RNS
+
+
+class UDPInterface(Interface):
+    BITRATE_GUESS = 10*1000*1000
+
+    @staticmethod
+    def get_address_for_if(name):
+        import importlib
+        if importlib.util.find_spec('netifaces') != None:
+            import netifaces
+            return netifaces.ifaddresses(name)[netifaces.AF_INET][0]['addr']
+        else:
+            RNS.log("Getting interface addresses from device names requires the netifaces module.", RNS.LOG_CRITICAL)
+            RNS.log("You can install it with the command: python3 -m pip install netifaces", RNS.LOG_CRITICAL)
+            RNS.panic()
+
+    @staticmethod
+    def get_broadcast_for_if(name):
+        import importlib
+        if importlib.util.find_spec('netifaces') != None:
+            import netifaces
+            return netifaces.ifaddresses(name)[netifaces.AF_INET][0]['broadcast']
+        else:
+            RNS.log("Getting interface addresses from device names requires the netifaces module.", RNS.LOG_CRITICAL)
+            RNS.log("You can install it with the command: python3 -m pip install netifaces", RNS.LOG_CRITICAL)
+            RNS.panic()
+
+    def __init__(self, owner, name, device=None, bindip=None, bindport=None, forwardip=None, forwardport=None):
+        self.rxb = 0
+        self.txb = 0
+
+        self.HW_MTU = 1064
+
+        self.IN  = True
+        self.OUT = False
+        self.name = name
+        self.online = False
+        self.bitrate = UDPInterface.BITRATE_GUESS
+
+        if device != None:
+            if bindip == None:
+                bindip = UDPInterface.get_broadcast_for_if(device)
+            if forwardip == None:
+                forwardip = UDPInterface.get_broadcast_for_if(device)
+
+
+        if (bindip != None and bindport != None):
+            self.receives = True
+            self.bind_ip = bindip
+            self.bind_port = bindport
+
+            def handlerFactory(callback):
+                def createHandler(*args, **keys):
+                    return UDPInterfaceHandler(callback, *args, **keys)
+                return createHandler
+
+            self.owner = owner
+            address = (self.bind_ip, self.bind_port)
+            socketserver.UDPServer.address_family = socket.AF_INET
+            self.server = socketserver.UDPServer(address, handlerFactory(self.processIncoming))
+
+            thread = threading.Thread(target=self.server.serve_forever)
+            thread.daemon = True
+            thread.start()
+
+            self.online = True
+
+        if (forwardip != None and forwardport != None):
+            self.forwards = True
+            self.forward_ip = forwardip
+            self.forward_port = forwardport
+
+
+    def processIncoming(self, data):
+        self.rxb += len(data)
+        self.owner.inbound(data, self)
+
+    def processOutgoing(self,data):
+        try:
+            udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+            udp_socket.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
+            udp_socket.sendto(data, (self.forward_ip, self.forward_port))
+            self.txb += len(data)
+            
+        except Exception as e:
+            RNS.log("Could not transmit on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+
+
+    def __str__(self):
+        return "UDPInterface["+self.name+"/"+self.bind_ip+":"+str(self.bind_port)+"]"
+
+class UDPInterfaceHandler(socketserver.BaseRequestHandler):
+    def __init__(self, callback, *args, **keys):
+        self.callback = callback
+        socketserver.BaseRequestHandler.__init__(self, *args, **keys)
+
+    def handle(self):
+        data = self.request[0]
+        self.callback(data)

RNS/Interfaces/UdpInterface.py

@@ -1,64 +0,0 @@
-from .Interface import Interface
-import socketserver
-import threading
-import socket
-import time
-import sys
-import RNS
-
-class UdpInterface(Interface):
-
-    def __init__(self, owner, name, bindip=None, bindport=None, forwardip=None, forwardport=None):
-        self.IN  = True
-        self.OUT = False
-
-        # TODO: Optimise so this is not needed
-        self.transmit_delay = 0.001
-
-        self.name = name
-
-        if (bindip != None and bindport != None):
-            self.receives = True
-            self.bind_ip = bindip
-            self.bind_port = bindport
-
-            def handlerFactory(callback):
-                def createHandler(*args, **keys):
-                    return UdpInterfaceHandler(callback, *args, **keys)
-                return createHandler
-
-            self.owner = owner
-            address = (self.bind_ip, self.bind_port)
-            self.server = socketserver.UDPServer(address, handlerFactory(self.processIncoming))
-
-            thread = threading.Thread(target=self.server.serve_forever)
-            thread.setDaemon(True)
-            thread.start()
-
-        if (forwardip != None and forwardport != None):
-            self.forwards = True
-            self.forward_ip = forwardip
-            self.forward_port = forwardport
-
-
-    def processIncoming(self, data):
-        self.owner.inbound(data, self)
-
-    def processOutgoing(self,data):
-        time.sleep(self.transmit_delay)
-        udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
-        udp_socket.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
-        udp_socket.sendto(data, (self.forward_ip, self.forward_port))
-
-
-    def __str__(self):
-        return "UdpInterface["+self.name+"/"+self.bind_ip+":"+str(self.bind_port)+"]"
-
-class UdpInterfaceHandler(socketserver.BaseRequestHandler):
-    def __init__(self, callback, *args, **keys):
-        self.callback = callback
-        socketserver.BaseRequestHandler.__init__(self, *args, **keys)
-
-    def handle(self):
-        data = self.request[0]
-        self.callback(data)

RNS/Interfaces/__init__.py

@@ -1,3 +1,25 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 import os
 import glob
 

RNS/Packet.py

@@ -1,3 +1,25 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 import threading
 import struct
 import math
@@ -5,407 +27,504 @@ import time
 import RNS
 
 class Packet:
-	# Packet types
-	DATA         = 0x00		# Data packets
-	ANNOUNCE     = 0x01		# Announces
-	LINKREQUEST  = 0x02		# Link requests
-	PROOF        = 0x03		# Proofs
-	types        = [DATA, ANNOUNCE, LINKREQUEST, PROOF]
+    """
+    The Packet class is used to create packet instances that can be sent
+    over a Reticulum network. Packets will automatically be encrypted if
+    they are adressed to a ``RNS.Destination.SINGLE`` destination,
+    ``RNS.Destination.GROUP`` destination or a :ref:`RNS.Link<api-link>`.
 
-	# Header types
-	HEADER_1     = 0x00		# Normal header format
-	HEADER_2     = 0x01		# Header format used for packets in transport
-	HEADER_3     = 0x02		# Reserved
-	HEADER_4     = 0x03		# Reserved
-	header_types = [HEADER_1, HEADER_2, HEADER_3, HEADER_4]
+    For ``RNS.Destination.GROUP`` destinations, Reticulum will use the
+    pre-shared key configured for the destination. All packets to group
+    destinations are encrypted with the same AES-128 key.
 
-	# Data packet context types
-	NONE 		   = 0x00	# Generic data packet
-	RESOURCE       = 0x01	# Packet is part of a resource
-	RESOURCE_ADV   = 0x02	# Packet is a resource advertisement
-	RESOURCE_REQ   = 0x03	# Packet is a resource part request
-	RESOURCE_HMU   = 0x04	# Packet is a resource hashmap update
-	RESOURCE_PRF   = 0x05	# Packet is a resource proof
-	RESOURCE_ICL   = 0x06	# Packet is a resource initiator cancel message
-	RESOURCE_RCL   = 0x07	# Packet is a resource receiver cancel message
-	CACHE_REQUEST  = 0x08	# Packet is a cache request
-	REQUEST        = 0x09	# Packet is a request
-	RESPONSE       = 0x0A	# Packet is a response to a request
-	PATH_RESPONSE  = 0x0B	# Packet is a response to a path request
-	COMMAND        = 0x0C	# Packet is a command
-	COMMAND_STATUS = 0x0D	# Packet is a status of an executed command
-	KEEPALIVE      = 0xFB	# Packet is a keepalive packet
-	LINKCLOSE      = 0xFC	# Packet is a link close message
-	LINKPROOF      = 0xFD	# Packet is a link packet proof
-	LRRTT		   = 0xFE	# Packet is a link request round-trip time measurement
-	LRPROOF        = 0xFF	# Packet is a link request proof
+    For ``RNS.Destination.SINGLE`` destinations, Reticulum will use a newly
+    derived ephemeral AES-128 key for every packet.
 
-	# This is used to calculate allowable
-	# payload sizes
-	HEADER_MAXSIZE = 23
-	MDU            = RNS.Reticulum.MDU
+    For :ref:`RNS.Link<api-link>` destinations, Reticulum will use per-link
+    ephemeral keys, and offers **Forward Secrecy**.
 
-	# With an MTU of 500, the maximum RSA-encrypted
-	# amount of data we can send in a single packet
-	# is given by the below calculation; 258 bytes.
-	RSA_MDU   = math.floor(MDU/RNS.Identity.DECRYPT_CHUNKSIZE)*RNS.Identity.ENCRYPT_CHUNKSIZE
-	PLAIN_MDU = MDU
+    :param destination: A :ref:`RNS.Destination<api-destination>` instance to which the packet will be sent.
+    :param data: The data payload to be included in the packet as *bytes*.
+    :param create_receipt: Specifies whether a :ref:`RNS.PacketReceipt<api-packetreceipt>` should be created when instantiating the packet.
+    """
 
-	# TODO: This should be calculated
-	# more intelligently
-	# Default packet timeout
-	TIMEOUT 	 = 60
+    # Packet types
+    DATA         = 0x00     # Data packets
+    ANNOUNCE     = 0x01     # Announces
+    LINKREQUEST  = 0x02     # Link requests
+    PROOF        = 0x03     # Proofs
+    types        = [DATA, ANNOUNCE, LINKREQUEST, PROOF]
 
-	def __init__(self, destination, data, packet_type = DATA, context = NONE, transport_type = RNS.Transport.BROADCAST, header_type = HEADER_1, transport_id = None, attached_interface = None, create_receipt = True):
-		if destination != None:
-			if transport_type == None:
-				transport_type = RNS.Transport.BROADCAST
+    # Header types
+    HEADER_1     = 0x00     # Normal header format
+    HEADER_2     = 0x01     # Header format used for packets in transport
+    HEADER_3     = 0x02     # Reserved
+    HEADER_4     = 0x03     # Reserved
+    header_types = [HEADER_1, HEADER_2, HEADER_3, HEADER_4]
 
-			self.header_type    = header_type
-			self.packet_type    = packet_type
-			self.transport_type = transport_type
-			self.context        = context
+    # Packet context types
+    NONE           = 0x00   # Generic data packet
+    RESOURCE       = 0x01   # Packet is part of a resource
+    RESOURCE_ADV   = 0x02   # Packet is a resource advertisement
+    RESOURCE_REQ   = 0x03   # Packet is a resource part request
+    RESOURCE_HMU   = 0x04   # Packet is a resource hashmap update
+    RESOURCE_PRF   = 0x05   # Packet is a resource proof
+    RESOURCE_ICL   = 0x06   # Packet is a resource initiator cancel message
+    RESOURCE_RCL   = 0x07   # Packet is a resource receiver cancel message
+    CACHE_REQUEST  = 0x08   # Packet is a cache request
+    REQUEST        = 0x09   # Packet is a request
+    RESPONSE       = 0x0A   # Packet is a response to a request
+    PATH_RESPONSE  = 0x0B   # Packet is a response to a path request
+    COMMAND        = 0x0C   # Packet is a command
+    COMMAND_STATUS = 0x0D   # Packet is a status of an executed command
+    KEEPALIVE      = 0xFA   # Packet is a keepalive packet
+    LINKIDENTIFY   = 0xFB   # Packet is a link peer identification proof
+    LINKCLOSE      = 0xFC   # Packet is a link close message
+    LINKPROOF      = 0xFD   # Packet is a link packet proof
+    LRRTT          = 0xFE   # Packet is a link request round-trip time measurement
+    LRPROOF        = 0xFF   # Packet is a link request proof
 
-			self.hops           = 0;
-			self.destination    = destination
-			self.transport_id   = transport_id
-			self.data           = data
-			self.flags          = self.getPackedFlags()
+    # This is used to calculate allowable
+    # payload sizes
+    HEADER_MAXSIZE = RNS.Reticulum.HEADER_MAXSIZE
+    MDU            = RNS.Reticulum.MDU
 
-			self.raw            = None
-			self.packed         = False
-			self.sent           = False
-			self.create_receipt = create_receipt
-			self.receipt        = None
-			self.fromPacked     = False
-		else:
-			self.raw            = data
-			self.packed         = True
-			self.fromPacked     = True
-			self.create_receipt = False
+    # With an MTU of 500, the maximum of data we can
+    # send in a single encrypted packet is given by
+    # the below calculation; 383 bytes.
+    ENCRYPTED_MDU  = math.floor((RNS.Reticulum.MDU-RNS.Identity.FERNET_OVERHEAD-RNS.Identity.KEYSIZE//16)/RNS.Identity.AES128_BLOCKSIZE)*RNS.Identity.AES128_BLOCKSIZE - 1
+    """
+    The maximum size of the payload data in a single encrypted packet 
+    """
+    PLAIN_MDU      = MDU
+    """
+    The maximum size of the payload data in a single unencrypted packet
+    """
 
-		self.MTU         = RNS.Reticulum.MTU
-		self.sent_at     = None
-		self.packet_hash = None
+    TIMEOUT_PER_HOP = RNS.Reticulum.DEFAULT_PER_HOP_TIMEOUT
 
-		self.attached_interface = attached_interface
+    def __init__(self, destination, data, packet_type = DATA, context = NONE, transport_type = RNS.Transport.BROADCAST, header_type = HEADER_1, transport_id = None, attached_interface = None, create_receipt = True):
+        if destination != None:
+            if transport_type == None:
+                transport_type = RNS.Transport.BROADCAST
 
-	def getPackedFlags(self):
-		if self.context == Packet.LRPROOF:
-			packed_flags = (self.header_type << 6) | (self.transport_type << 4) | RNS.Destination.LINK | self.packet_type
-		else:
-			packed_flags = (self.header_type << 6) | (self.transport_type << 4) | (self.destination.type << 2) | self.packet_type
-		return packed_flags
+            self.header_type    = header_type
+            self.packet_type    = packet_type
+            self.transport_type = transport_type
+            self.context        = context
 
-	def pack(self):
-		self.destination_hash = self.destination.hash
-		self.header = b""
-		self.header += struct.pack("!B", self.flags)
-		self.header += struct.pack("!B", self.hops)
+            self.hops           = 0;
+            self.destination    = destination
+            self.transport_id   = transport_id
+            self.data           = data
+            self.flags          = self.get_packed_flags()
 
-		if self.context == Packet.LRPROOF:
-			self.header += self.destination.link_id
-			self.ciphertext = self.data
-		else:
-			if self.header_type == Packet.HEADER_1:
-				self.header += self.destination.hash
+            self.raw            = None
+            self.packed         = False
+            self.sent           = False
+            self.create_receipt = create_receipt
+            self.receipt        = None
+            self.fromPacked     = False
+        else:
+            self.raw            = data
+            self.packed         = True
+            self.fromPacked     = True
+            self.create_receipt = False
 
-				if self.packet_type == Packet.ANNOUNCE:
-					# Announce packets are not encrypted
-					self.ciphertext = self.data
-				elif self.packet_type == Packet.PROOF and self.context == Packet.RESOURCE_PRF:
-					# Resource proofs are not encrypted
-					self.ciphertext = self.data
-				elif self.packet_type == Packet.PROOF and self.destination.type == RNS.Destination.LINK:
-					# Packet proofs over links are not encrypted
-					self.ciphertext = self.data
-				elif self.context == Packet.RESOURCE:
-					# A resource takes care of symmetric
-					# encryption by itself
-					self.ciphertext = self.data
-				elif self.context == Packet.KEEPALIVE:
-					# Keepalive packets contain no actual
-					# data
-					self.ciphertext = self.data
-				else:
-					# In all other cases, we encrypt the packet
-					# with the destination's public key
-					self.ciphertext = self.destination.encrypt(self.data)
+        self.MTU         = RNS.Reticulum.MTU
+        self.sent_at     = None
+        self.packet_hash = None
 
-			if self.header_type == Packet.HEADER_2:
-				if self.transport_id != None:
-					self.header += self.transport_id
-					self.header += self.destination.hash
+        self.attached_interface = attached_interface
+        self.receiving_interface = None
+        self.rssi = None
+        self.snr = None
 
-					if self.packet_type == Packet.ANNOUNCE:
-						# Announce packets are not encrypted
-						self.ciphertext = self.data
-				else:
-					raise IOError("Packet with header type 2 must have a transport ID")
+    def get_packed_flags(self):
+        if self.context == Packet.LRPROOF:
+            packed_flags = (self.header_type << 6) | (self.transport_type << 4) | RNS.Destination.LINK | self.packet_type
+        else:
+            packed_flags = (self.header_type << 6) | (self.transport_type << 4) | (self.destination.type << 2) | self.packet_type
+        return packed_flags
+
+    def pack(self):
+        self.destination_hash = self.destination.hash
+        self.header = b""
+        self.header += struct.pack("!B", self.flags)
+        self.header += struct.pack("!B", self.hops)
+
+        if self.context == Packet.LRPROOF:
+            self.header += self.destination.link_id
+            self.ciphertext = self.data
+        else:
+            if self.header_type == Packet.HEADER_1:
+                self.header += self.destination.hash
+
+                if self.packet_type == Packet.ANNOUNCE:
+                    # Announce packets are not encrypted
+                    self.ciphertext = self.data
+                elif self.packet_type == Packet.LINKREQUEST:
+                    # Link request packets are not encrypted
+                    self.ciphertext = self.data
+                elif self.packet_type == Packet.PROOF and self.context == Packet.RESOURCE_PRF:
+                    # Resource proofs are not encrypted
+                    self.ciphertext = self.data
+                elif self.packet_type == Packet.PROOF and self.destination.type == RNS.Destination.LINK:
+                    # Packet proofs over links are not encrypted
+                    self.ciphertext = self.data
+                elif self.context == Packet.RESOURCE:
+                    # A resource takes care of symmetric
+                    # encryption by itself
+                    self.ciphertext = self.data
+                elif self.context == Packet.KEEPALIVE:
+                    # Keepalive packets contain no actual
+                    # data
+                    self.ciphertext = self.data
+                elif self.context == Packet.CACHE_REQUEST:
+                    # Cache-requests are not encrypted
+                    self.ciphertext = self.data
+                else:
+                    # In all other cases, we encrypt the packet
+                    # with the destination's encryption method
+                    self.ciphertext = self.destination.encrypt(self.data)
+
+            if self.header_type == Packet.HEADER_2:
+                if self.transport_id != None:
+                    self.header += self.transport_id
+                    self.header += self.destination.hash
+
+                    if self.packet_type == Packet.ANNOUNCE:
+                        # Announce packets are not encrypted
+                        self.ciphertext = self.data
+                else:
+                    raise IOError("Packet with header type 2 must have a transport ID")
 
 
-		self.header += bytes([self.context])
-		self.raw = self.header + self.ciphertext
+        self.header += bytes([self.context])
+        self.raw = self.header + self.ciphertext
 
-		if len(self.raw) > self.MTU:
-			raise IOError("Packet size of "+str(len(self.raw))+" exceeds MTU of "+str(self.MTU)+" bytes")
+        if len(self.raw) > self.MTU:
+            raise IOError("Packet size of "+str(len(self.raw))+" exceeds MTU of "+str(self.MTU)+" bytes")
 
-		self.packed = True
-		self.updateHash()
+        self.packed = True
+        self.update_hash()
 
-	def unpack(self):
-		self.flags = self.raw[0]
-		self.hops  = self.raw[1]
 
-		self.header_type      = (self.flags & 0b11000000) >> 6
-		self.transport_type   = (self.flags & 0b00110000) >> 4
-		self.destination_type = (self.flags & 0b00001100) >> 2
-		self.packet_type      = (self.flags & 0b00000011)
+    def unpack(self):
+        try:
+            self.flags = self.raw[0]
+            self.hops  = self.raw[1]
 
-		if self.header_type == Packet.HEADER_2:
-			self.transport_id = self.raw[2:12]
-			self.destination_hash = self.raw[12:22]
-			self.context = ord(self.raw[22:23])
-			self.data = self.raw[23:]
-		else:
-			self.transport_id = None
-			self.destination_hash = self.raw[2:12]
-			self.context = ord(self.raw[12:13])
-			self.data = self.raw[13:]
+            self.header_type      = (self.flags & 0b11000000) >> 6
+            self.transport_type   = (self.flags & 0b00110000) >> 4
+            self.destination_type = (self.flags & 0b00001100) >> 2
+            self.packet_type      = (self.flags & 0b00000011)
 
-		self.packed = False
-		self.updateHash()
+            DST_LEN = RNS.Reticulum.TRUNCATED_HASHLENGTH//8
 
-	# Sends the packet. Returns a receipt if one is generated,
-	# or None if no receipt is available. Returns False if the
-	# packet could not be sent.
-	def send(self):
-		if not self.sent:
-			if self.destination.type == RNS.Destination.LINK:
-				if self.destination.status == RNS.Link.CLOSED:
-					raise IOError("Attempt to transmit over a closed link")
-				else:
-					self.destination.last_outbound = time.time()
-					self.destination.tx += 1
-					self.destination.txbytes += len(self.data)
+            if self.header_type == Packet.HEADER_2:
+                self.transport_id = self.raw[2:DST_LEN+2]
+                self.destination_hash = self.raw[DST_LEN+2:2*DST_LEN+2]
+                self.context = ord(self.raw[2*DST_LEN+2:2*DST_LEN+3])
+                self.data = self.raw[2*DST_LEN+3:]
+            else:
+                self.transport_id = None
+                self.destination_hash = self.raw[2:DST_LEN+2]
+                self.context = ord(self.raw[DST_LEN+2:DST_LEN+3])
+                self.data = self.raw[DST_LEN+3:]
 
-			if not self.packed:
-				self.pack()
-	
-			if RNS.Transport.outbound(self):
-				return self.receipt
-			else:
-				RNS.log("No interfaces could process the outbound packet", RNS.LOG_ERROR)
-				self.sent = False
-				self.receipt = None
-				return False
-				
-		else:
-			raise IOError("Packet was already sent")
+            self.packed = False
+            self.update_hash()
+            return True
 
-	def resend(self):
-		if self.sent:
-			if RNS.Transport.outbound(self):
-				return self.receipt
-			else:
-				RNS.log("No interfaces could process the outbound packet", RNS.LOG_ERROR)
-				self.sent = False
-				self.receipt = None
-				return False
-		else:
-			raise IOError("Packet was not sent yet")
+        except Exception as e:
+            RNS.log("Received malformed packet, dropping it. The contained exception was: "+str(e), RNS.LOG_EXTREME)
+            return False
 
-	def prove(self, destination=None):
-		if self.fromPacked and hasattr(self, "destination") and self.destination:
-			if self.destination.identity and self.destination.identity.prv:
-				self.destination.identity.prove(self, destination)
-		elif self.fromPacked and hasattr(self, "link") and self.link:
-			self.link.prove_packet(self)
-		else:
-			RNS.log("Could not prove packet associated with neither a destination nor a link", RNS.LOG_ERROR)
+    def send(self):
+        """
+        Sends the packet.
+        
+        :returns: A :ref:`RNS.PacketReceipt<api-packetreceipt>` instance if *create_receipt* was set to *True* when the packet was instantiated, if not returns *None*. If the packet could not be sent *False* is returned.
+        """
+        if not self.sent:
+            if self.destination.type == RNS.Destination.LINK:
+                if self.destination.status == RNS.Link.CLOSED:
+                    raise IOError("Attempt to transmit over a closed link")
+                else:
+                    self.destination.last_outbound = time.time()
+                    self.destination.tx += 1
+                    self.destination.txbytes += len(self.data)
 
-	# Generates a special destination that allows Reticulum
-	# to direct the proof back to the proved packet's sender
-	def generateProofDestination(self):
-		return ProofDestination(self)
+            if not self.packed:
+                self.pack()
 
-	def validateProofPacket(self, proof_packet):
-		return self.receipt.validateProofPacket(proof_packet)
+            if RNS.Transport.outbound(self):
+                return self.receipt
+            else:
+                RNS.log("No interfaces could process the outbound packet", RNS.LOG_ERROR)
+                self.sent = False
+                self.receipt = None
+                return False
+                
+        else:
+            raise IOError("Packet was already sent")
 
-	def validateProof(self, proof):
-		return self.receipt.validateProof(proof)
+    def resend(self):
+        """
+        Re-sends the packet.
+        
+        :returns: A :ref:`RNS.PacketReceipt<api-packetreceipt>` instance if *create_receipt* was set to *True* when the packet was instantiated, if not returns *None*. If the packet could not be sent *False* is returned.
+        """
+        if self.sent:
+            if RNS.Transport.outbound(self):
+                return self.receipt
+            else:
+                RNS.log("No interfaces could process the outbound packet", RNS.LOG_ERROR)
+                self.sent = False
+                self.receipt = None
+                return False
+        else:
+            raise IOError("Packet was not sent yet")
 
-	def updateHash(self):
-		self.packet_hash = self.getHash()
+    def prove(self, destination=None):
+        if self.fromPacked and hasattr(self, "destination") and self.destination:
+            if self.destination.identity and self.destination.identity.prv:
+                self.destination.identity.prove(self, destination)
+        elif self.fromPacked and hasattr(self, "link") and self.link:
+            self.link.prove_packet(self)
+        else:
+            RNS.log("Could not prove packet associated with neither a destination nor a link", RNS.LOG_ERROR)
 
-	def getHash(self):
-		return RNS.Identity.fullHash(self.getHashablePart())
+    # Generates a special destination that allows Reticulum
+    # to direct the proof back to the proved packet's sender
+    def generate_proof_destination(self):
+        return ProofDestination(self)
 
-	def getTruncatedHash(self):
-		return RNS.Identity.truncatedHash(self.getHashablePart())
+    def validate_proof_packet(self, proof_packet):
+        return self.receipt.validate_proof_packet(proof_packet)
 
-	def getHashablePart(self):
-		hashable_part = bytes([self.raw[0] & 0b00001111])
-		if self.header_type == Packet.HEADER_2:
-			hashable_part += self.raw[12:]
-		else:
-			hashable_part += self.raw[2:]
+    def validate_proof(self, proof):
+        return self.receipt.validate_proof(proof)
 
-		return hashable_part
+    def update_hash(self):
+        self.packet_hash = self.get_hash()
+
+    def get_hash(self):
+        return RNS.Identity.full_hash(self.get_hashable_part())
+
+    def getTruncatedHash(self):
+        return RNS.Identity.truncated_hash(self.get_hashable_part())
+
+    def get_hashable_part(self):
+        hashable_part = bytes([self.raw[0] & 0b00001111])
+        if self.header_type == Packet.HEADER_2:
+            hashable_part += self.raw[(RNS.Identity.TRUNCATED_HASHLENGTH//8)+2:]
+        else:
+            hashable_part += self.raw[2:]
+
+        return hashable_part
 
 class ProofDestination:
-	def __init__(self, packet):
-		self.hash = packet.getHash()[:10];
-		self.type = RNS.Destination.SINGLE
+    def __init__(self, packet):
+        self.hash = packet.get_hash()[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8];
+        self.type = RNS.Destination.SINGLE
 
-	def encrypt(self, plaintext):
-		return plaintext
+    def encrypt(self, plaintext):
+        return plaintext
 
 
 class PacketReceipt:
-	# Receipt status constants
-	FAILED    = 0x00
-	SENT	  = 0x01
-	DELIVERED = 0x02
-	CULLED    = 0xFF
+    """
+    The PacketReceipt class is used to receive notifications about
+    :ref:`RNS.Packet<api-packet>` instances sent over the network. Instances
+    of this class are never created manually, but always returned from
+    the *send()* method of a :ref:`RNS.Packet<api-packet>` instance.
+    """
+    # Receipt status constants
+    FAILED    = 0x00
+    SENT      = 0x01
+    DELIVERED = 0x02
+    CULLED    = 0xFF
 
 
-	EXPL_LENGTH = RNS.Identity.HASHLENGTH//8+RNS.Identity.SIGLENGTH//8
-	IMPL_LENGTH = RNS.Identity.SIGLENGTH//8
+    EXPL_LENGTH = RNS.Identity.HASHLENGTH//8+RNS.Identity.SIGLENGTH//8
+    IMPL_LENGTH = RNS.Identity.SIGLENGTH//8
 
-	# Creates a new packet receipt from a sent packet
-	def __init__(self, packet):
-		self.hash    = packet.getHash()
-		self.sent    = True
-		self.sent_at = time.time()
-		self.timeout = Packet.TIMEOUT
-		self.proved  = False
-		self.status  = PacketReceipt.SENT
-		self.destination = packet.destination
-		self.callbacks   = PacketReceiptCallbacks()
-		self.concluded_at = None
+    # Creates a new packet receipt from a sent packet
+    def __init__(self, packet):
+        self.hash           = packet.get_hash()
+        self.truncated_hash = packet.getTruncatedHash()
+        self.sent           = True
+        self.sent_at        = time.time()
+        self.proved         = False
+        self.status         = PacketReceipt.SENT
+        self.destination    = packet.destination
+        self.callbacks      = PacketReceiptCallbacks()
+        self.concluded_at   = None
+        self.proof_packet   = None
 
-	# Validate a proof packet
-	def validateProofPacket(self, proof_packet):
-		if hasattr(proof_packet, "link") and proof_packet.link:
-			return self.validate_link_proof(proof_packet.data, proof_packet.link)
-		else:
-			return self.validateProof(proof_packet.data)
-
-	# Validate a raw proof for a link
-	def validate_link_proof(self, proof, link):
-		# TODO: Hardcoded as explicit proofs for now
-		if True or len(proof) == PacketReceipt.EXPL_LENGTH:
-			# This is an explicit proof
-			proof_hash = proof[:RNS.Identity.HASHLENGTH//8]
-			signature = proof[RNS.Identity.HASHLENGTH//8:RNS.Identity.HASHLENGTH//8+RNS.Identity.SIGLENGTH//8]
-			if proof_hash == self.hash:
-				proof_valid = link.validate(signature, self.hash)
-				if proof_valid:
-					self.status = PacketReceipt.DELIVERED
-					self.proved = True
-					self.concluded_at = time.time()
-					if self.callbacks.delivery != None:
-						self.callbacks.delivery(self)
-					return True
-				else:
-					return False
-			else:
-				return False
-		elif len(proof) == PacketReceipt.IMPL_LENGTH:
-			pass
-			# TODO: Why is this disabled?
-			# signature = proof[:RNS.Identity.SIGLENGTH//8]
-			# proof_valid = self.link.validate(signature, self.hash)
-			# if proof_valid:
-			# 		self.status = PacketReceipt.DELIVERED
-			# 		self.proved = True
-			# 		self.concluded_at = time.time()
-			# 		if self.callbacks.delivery != None:
-			# 			self.callbacks.delivery(self)
-			# 		RNS.log("valid")
-			# 		return True
-			# else:
-			# 	RNS.log("invalid")
-			# 	return False
-		else:
-			return False
-
-	# Validate a raw proof
-	def validateProof(self, proof):
-		if len(proof) == PacketReceipt.EXPL_LENGTH:
-			# This is an explicit proof
-			proof_hash = proof[:RNS.Identity.HASHLENGTH//8]
-			signature = proof[RNS.Identity.HASHLENGTH//8:RNS.Identity.HASHLENGTH//8+RNS.Identity.SIGLENGTH//8]
-			if proof_hash == self.hash:
-				proof_valid = self.destination.identity.validate(signature, self.hash)
-				if proof_valid:
-					self.status = PacketReceipt.DELIVERED
-					self.proved = True
-					self.concluded_at = time.time()
-					if self.callbacks.delivery != None:
-						self.callbacks.delivery(self)
-					return True
-				else:
-					return False
-			else:
-				return False
-		elif len(proof) == PacketReceipt.IMPL_LENGTH:
-			# This is an implicit proof
-			if self.destination.identity == None:
-				return False
-
-			signature = proof[:RNS.Identity.SIGLENGTH//8]
-			proof_valid = self.destination.identity.validate(signature, self.hash)
-			if proof_valid:
-					self.status = PacketReceipt.DELIVERED
-					self.proved = True
-					self.concluded_at = time.time()
-					if self.callbacks.delivery != None:
-						self.callbacks.delivery(self)
-					return True
-			else:
-				return False
-		else:
-			return False
-
-	def rtt(self):
-		return self.concluded_at - self.sent_at
-
-	def is_timed_out(self):
-		return (self.sent_at+self.timeout < time.time())
-
-	def check_timeout(self):
-		if self.is_timed_out():
-			if self.timeout == -1:
-				self.status = PacketReceipt.CULLED
-			else:
-				self.status = PacketReceipt.FAILED
-
-			self.concluded_at = time.time()
-
-			if self.callbacks.timeout:
-				thread = threading.Thread(target=self.callbacks.timeout, args=(self,))
-				thread.setDaemon(True)
-				thread.start()
-				#self.callbacks.timeout(self)
+        if packet.destination.type == RNS.Destination.LINK:
+            self.timeout    = packet.destination.rtt * packet.destination.traffic_timeout_factor
+        else:
+            self.timeout    = Packet.TIMEOUT_PER_HOP * RNS.Transport.hops_to(self.destination.hash)
 
 
-	# Set the timeout in seconds
-	def set_timeout(self, timeout):
-		self.timeout = float(timeout)
+    def get_status(self):
+        """
+        :returns: The status of the associated :ref:`RNS.Packet<api-packet>` instance. Can be one of ``RNS.PacketReceipt.SENT``, ``RNS.PacketReceipt.DELIVERED``, ``RNS.PacketReceipt.FAILED`` or ``RNS.PacketReceipt.CULLED``. 
+        """
+        return self.status
 
-	# Set a function that gets called when
-	# a successfull delivery has been proved
-	def delivery_callback(self, callback):
-		self.callbacks.delivery = callback
+    # Validate a proof packet
+    def validate_proof_packet(self, proof_packet):
+        if hasattr(proof_packet, "link") and proof_packet.link:
+            return self.validate_link_proof(proof_packet.data, proof_packet.link, proof_packet)
+        else:
+            return self.validate_proof(proof_packet.data, proof_packet)
 
-	# Set a function that gets called if the
-	# delivery times out
-	def timeout_callback(self, callback):
-		self.callbacks.timeout = callback
+    # Validate a raw proof for a link
+    def validate_link_proof(self, proof, link, proof_packet=None):
+        # TODO: Hardcoded as explicit proofs for now
+        if True or len(proof) == PacketReceipt.EXPL_LENGTH:
+            # This is an explicit proof
+            proof_hash = proof[:RNS.Identity.HASHLENGTH//8]
+            signature = proof[RNS.Identity.HASHLENGTH//8:RNS.Identity.HASHLENGTH//8+RNS.Identity.SIGLENGTH//8]
+            if proof_hash == self.hash:
+                proof_valid = link.validate(signature, self.hash)
+                if proof_valid:
+                    self.status = PacketReceipt.DELIVERED
+                    self.proved = True
+                    self.concluded_at = time.time()
+                    self.proof_packet = proof_packet
+
+                    if self.callbacks.delivery != None:
+                        self.callbacks.delivery(self)
+                    return True
+                else:
+                    return False
+            else:
+                return False
+        elif len(proof) == PacketReceipt.IMPL_LENGTH:
+            pass
+            # TODO: Why is this disabled?
+            # signature = proof[:RNS.Identity.SIGLENGTH//8]
+            # proof_valid = self.link.validate(signature, self.hash)
+            # if proof_valid:
+            #       self.status = PacketReceipt.DELIVERED
+            #       self.proved = True
+            #       self.concluded_at = time.time()
+            #       if self.callbacks.delivery != None:
+            #           self.callbacks.delivery(self)
+            #       RNS.log("valid")
+            #       return True
+            # else:
+            #   RNS.log("invalid")
+            #   return False
+        else:
+            return False
+
+    # Validate a raw proof
+    def validate_proof(self, proof, proof_packet=None):
+        if len(proof) == PacketReceipt.EXPL_LENGTH:
+            # This is an explicit proof
+            proof_hash = proof[:RNS.Identity.HASHLENGTH//8]
+            signature = proof[RNS.Identity.HASHLENGTH//8:RNS.Identity.HASHLENGTH//8+RNS.Identity.SIGLENGTH//8]
+            if proof_hash == self.hash:
+                proof_valid = self.destination.identity.validate(signature, self.hash)
+                if proof_valid:
+                    self.status = PacketReceipt.DELIVERED
+                    self.proved = True
+                    self.concluded_at = time.time()
+                    self.proof_packet = proof_packet
+
+                    if self.callbacks.delivery != None:
+                        try:
+                            self.callbacks.delivery(self)
+                        except Exception as e:
+                            RNS.log("Error while executing proof validated callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
+
+                    return True
+                else:
+                    return False
+            else:
+                return False
+        elif len(proof) == PacketReceipt.IMPL_LENGTH:
+            # This is an implicit proof
+            if self.destination.identity == None:
+                return False
+
+            signature = proof[:RNS.Identity.SIGLENGTH//8]
+            proof_valid = self.destination.identity.validate(signature, self.hash)
+            if proof_valid:
+                    self.status = PacketReceipt.DELIVERED
+                    self.proved = True
+                    self.concluded_at = time.time()
+                    self.proof_packet = proof_packet
+
+                    if self.callbacks.delivery != None:
+                        try:
+                            self.callbacks.delivery(self)
+                        except Exception as e:
+                            RNS.log("Error while executing proof validated callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
+                            
+                    return True
+            else:
+                return False
+        else:
+            return False
+
+    def get_rtt(self):
+        """
+        :returns: The round-trip-time in seconds
+        """
+        return self.concluded_at - self.sent_at
+
+    def is_timed_out(self):
+        return (self.sent_at+self.timeout < time.time())
+
+    def check_timeout(self):
+        if self.status == PacketReceipt.SENT and self.is_timed_out():
+            if self.timeout == -1:
+                self.status = PacketReceipt.CULLED
+            else:
+                self.status = PacketReceipt.FAILED
+
+            self.concluded_at = time.time()
+
+            if self.callbacks.timeout:
+                thread = threading.Thread(target=self.callbacks.timeout, args=(self,))
+                thread.daemon = True
+                thread.start()
+
+
+    def set_timeout(self, timeout):
+        """
+        Sets a timeout in seconds
+        
+        :param timeout: The timeout in seconds.
+        """
+        self.timeout = float(timeout)
+
+    def set_delivery_callback(self, callback):
+        """
+        Sets a function that gets called if a successfull delivery has been proven.
+
+        :param callback: A *callable* with the signature *callback(packet_receipt)*
+        """
+        self.callbacks.delivery = callback
+
+    # Set a function that gets called if the
+    # delivery times out
+    def set_timeout_callback(self, callback):
+        """
+        Sets a function that gets called if the delivery times out.
+
+        :param callback: A *callable* with the signature *callback(packet_receipt)*
+        """
+        self.callbacks.timeout = callback
 
 class PacketReceiptCallbacks:
-	def __init__(self):
-		self.delivery = None
-		self.timeout  = None
+    def __init__(self):
+        self.delivery = None
+        self.timeout  = None

RNS/Utilities/__init__.py

@@ -0,0 +1,27 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import os
+import glob
+
+modules = glob.glob(os.path.dirname(__file__)+"/*.py")
+__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]

RNS/Utilities/rncp.py

@@ -0,0 +1,389 @@
+#!/usr/bin/env python3
+
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import RNS
+import argparse
+import time
+import sys
+import os
+
+from RNS._version import __version__
+
+APP_NAME = "rncp"
+allow_all = False
+allowed_identity_hashes = []
+
+def receive(configdir, verbosity = 0, quietness = 0, allowed = [], display_identity = False, limit = None, disable_auth = None, disable_announce = False):
+    global allow_all, allowed_identity_hashes
+    identity = None
+
+    targetloglevel = 3+verbosity-quietness
+    reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
+
+    identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
+    if os.path.isfile(identity_path):
+        identity = RNS.Identity.from_file(identity_path)                
+
+    if identity == None:
+        RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
+        identity = RNS.Identity()
+        identity.to_file(identity_path)
+
+    destination = RNS.Destination(identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, "receive")
+
+    if display_identity:
+        print("Identity     : "+str(identity))
+        print("Receiving on : "+RNS.prettyhexrep(destination.hash))
+        exit(0)
+
+    if disable_auth:
+        allow_all = True
+    else:
+        if allowed != None:
+            for a in allowed:
+                try:
+                    dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+                    if len(a) != dest_len:
+                        raise ValueError("Allowed destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+                    try:
+                        destination_hash = bytes.fromhex(a)
+                        allowed_identity_hashes.append(destination_hash)
+                    except Exception as e:
+                        raise ValueError("Invalid destination entered. Check your input.")
+                except Exception as e:
+                    print(str(e))
+                    exit(1)
+
+    if len(allowed_identity_hashes) < 1 and not disable_auth:
+        print("Warning: No allowed identities configured, rncp will not accept any files!")
+
+    destination.set_link_established_callback(receive_link_established)
+    print("rncp ready to receive on "+RNS.prettyhexrep(destination.hash))
+
+    if not disable_announce:
+        destination.announce()
+    
+    while True:
+        time.sleep(1)
+
+def receive_link_established(link):
+    RNS.log("Incoming link established", RNS.LOG_VERBOSE)
+    link.set_remote_identified_callback(receive_sender_identified)
+    link.set_resource_strategy(RNS.Link.ACCEPT_APP)
+    link.set_resource_callback(receive_resource_callback)
+    link.set_resource_started_callback(receive_resource_started)
+    link.set_resource_concluded_callback(receive_resource_concluded)
+
+def receive_sender_identified(link, identity):
+    global allow_all
+
+    if identity.hash in allowed_identity_hashes:
+        RNS.log("Authenticated sender", RNS.LOG_VERBOSE)
+    else:
+        if not allow_all:
+            RNS.log("Sender not allowed, tearing down link", RNS.LOG_VERBOSE)
+            link.teardown()
+        else:
+            pass
+
+def receive_resource_callback(resource):
+    global allow_all
+    
+    sender_identity = resource.link.get_remote_identity()
+
+    if sender_identity != None:
+        if sender_identity.hash in allowed_identity_hashes:
+            return True
+
+    if allow_all:
+        return True
+
+    return False
+
+def receive_resource_started(resource):
+    if resource.link.get_remote_identity():
+        id_str = " from "+RNS.prettyhexrep(resource.link.get_remote_identity().hash)
+    else:
+        id_str = ""
+
+    print("Starting resource transfer "+RNS.prettyhexrep(resource.hash)+id_str)
+
+def receive_resource_concluded(resource):
+    if resource.status == RNS.Resource.COMPLETE:
+        print(str(resource)+" completed")
+
+        if resource.total_size > 4:
+            filename_len = int.from_bytes(resource.data.read(2), "big")
+            filename = resource.data.read(filename_len).decode("utf-8")
+
+            counter = 0
+            saved_filename = filename
+            while os.path.isfile(saved_filename):
+                counter += 1
+                saved_filename = filename+"."+str(counter)
+            
+            file = open(saved_filename, "wb")
+            file.write(resource.data.read())
+            file.close()
+
+        else:
+            print("Invalid data received, ignoring resource")
+
+    else:
+        print("Resource failed")
+
+resource_done = False
+current_resource = None
+stats = []
+speed = 0.0
+def sender_progress(resource):
+    stats_max = 32
+    global current_resource, stats, speed, resource_done
+    current_resource = resource
+    now = time.time()
+    got = current_resource.get_progress()*current_resource.total_size
+    entry = [now, got]
+    stats.append(entry)
+    while len(stats) > stats_max:
+        stats.pop(0)
+
+    span = now - stats[0][0]
+    if span == 0:
+        speed = 0
+    else:
+        diff = got - stats[0][1]
+        speed = diff/span
+
+    if resource.status < RNS.Resource.COMPLETE:
+        resource_done = False
+    else:
+        resource_done = True
+
+link = None
+def send(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT):
+    global current_resource, resource_done, link, speed
+    from tempfile import TemporaryFile
+    targetloglevel = 3+verbosity-quietness
+
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination) != dest_len:
+            raise ValueError("Allowed destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+        try:
+            destination_hash = bytes.fromhex(destination)
+        except Exception as e:
+            raise ValueError("Invalid destination entered. Check your input.")
+    except Exception as e:
+        print(str(e))
+        exit(1)
+
+    
+    file_path = os.path.expanduser(file)
+    if not os.path.isfile(file_path):
+        print("File not found")
+        exit(1)
+
+    temp_file = TemporaryFile()
+    real_file = open(file_path, "rb")
+    filename_bytes = os.path.basename(file_path).encode("utf-8")
+    filename_len = len(filename_bytes)
+
+    if filename_len > 0xFFFF:
+        print("Filename exceeds max size, cannot send")
+        exit(1)
+    else:
+        print("Preparing file...", end=" ")
+
+    temp_file.write(filename_len.to_bytes(2, "big"))
+    temp_file.write(filename_bytes)
+    temp_file.write(real_file.read())
+    temp_file.seek(0)
+
+    print("\r                                                            \r", end="")
+
+    reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
+
+    identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
+    if os.path.isfile(identity_path):
+        identity = RNS.Identity.from_file(identity_path)                
+
+    if identity == None:
+        RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
+        identity = RNS.Identity()
+        identity.to_file(identity_path)
+
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.Transport.request_path(destination_hash)
+        print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
+        sys.stdout.flush()
+
+    i = 0
+    syms = "⢄⢂⢁⡁⡈⡐⡠"
+    estab_timeout = time.time()+timeout
+    while not RNS.Transport.has_path(destination_hash) and time.time() < estab_timeout:
+        time.sleep(0.1)
+        print(("\b\b"+syms[i]+" "), end="")
+        sys.stdout.flush()
+        i = (i+1)%len(syms)
+
+    if not RNS.Transport.has_path(destination_hash):
+        print("\r                                                            \rPath not found")
+        exit(1)
+    else:
+        print("\r                                                            \rEstablishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=" ")
+
+    receiver_identity = RNS.Identity.recall(destination_hash)
+    receiver_destination = RNS.Destination(
+        receiver_identity,
+        RNS.Destination.OUT,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "receive"
+    )
+
+    link = RNS.Link(receiver_destination)
+    while link.status != RNS.Link.ACTIVE and time.time() < estab_timeout:
+        time.sleep(0.1)
+        print(("\b\b"+syms[i]+" "), end="")
+        sys.stdout.flush()
+        i = (i+1)%len(syms)
+
+    if not RNS.Transport.has_path(destination_hash):
+        print("\r                                                            \rCould not establish link with "+RNS.prettyhexrep(destination_hash))
+        exit(1)
+    else:
+        print("\r                                                            \rAdvertising file resource  ", end=" ")
+
+    link.identify(identity)
+    resource = RNS.Resource(temp_file, link, callback = sender_progress, progress_callback = sender_progress)
+    current_resource = resource
+
+    while resource.status < RNS.Resource.TRANSFERRING:
+        time.sleep(0.1)
+        print(("\b\b"+syms[i]+" "), end="")
+        sys.stdout.flush()
+        i = (i+1)%len(syms)
+
+    
+    if resource.status > RNS.Resource.COMPLETE:
+        print("\r                                                            \rFile was not accepted by "+RNS.prettyhexrep(destination_hash))
+        exit(1)
+    else:
+        print("\r                                                            \rTransferring file  ", end=" ")
+
+    while not resource_done:
+        time.sleep(0.1)
+        prg = current_resource.get_progress()
+        percent = round(prg * 100.0, 1)
+        stat_str = str(percent)+"% - " + size_str(int(prg*current_resource.total_size)) + " of " + size_str(current_resource.total_size) + " - " +size_str(speed, "b")+"ps"
+        print("\r                                                                                  \rTransferring file "+syms[i]+" "+stat_str, end=" ")
+        sys.stdout.flush()
+        i = (i+1)%len(syms)
+
+    if current_resource.status != RNS.Resource.COMPLETE:
+        print("\r                                                            \rThe transfer failed")
+        exit(1)
+    else:
+        print("\r                                                                                  \r"+str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
+        link.teardown()
+        time.sleep(0.25)
+        real_file.close()
+        temp_file.close()
+        exit(0)
+
+def main():
+    try:
+        parser = argparse.ArgumentParser(description="Reticulum File Transfer Utility")
+        parser.add_argument("file", nargs="?", default=None, help="file to be transferred", type=str)
+        parser.add_argument("destination", nargs="?", default=None, help="hexadecimal hash of the receiver", type=str)
+        parser.add_argument("--config", metavar="path", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
+        parser.add_argument('-v', '--verbose', action='count', default=0, help="increase verbosity")
+        parser.add_argument('-q', '--quiet', action='count', default=0, help="decrease verbosity")
+        parser.add_argument('-p', '--print-identity', action='store_true', default=False, help="print identity and destination info and exit")
+        parser.add_argument("-r", '--receive', action='store_true', default=False, help="wait for incoming files")
+        parser.add_argument("-b", '--no-announce', action='store_true', default=False, help="don't announce at program start")
+        parser.add_argument('-a', metavar="allowed_hash", dest="allowed", action='append', help="accept from this identity", type=str)
+        parser.add_argument('-n', '--no-auth', action='store_true', default=False, help="accept files from anyone")
+        parser.add_argument("-w", action="store", metavar="seconds", type=float, help="sender timeout before giving up", default=RNS.Transport.PATH_REQUEST_TIMEOUT)
+        # parser.add_argument("--limit", action="store", metavar="files", type=float, help="maximum number of files to accept", default=None)
+        parser.add_argument("--version", action="version", version="rncp {version}".format(version=__version__))
+        
+        args = parser.parse_args()
+
+        if args.receive or args.print_identity:
+            receive(
+                configdir = args.config,
+                verbosity=args.verbose,
+                quietness=args.quiet,
+                allowed = args.allowed,
+                display_identity=args.print_identity,
+                # limit=args.limit,
+                disable_auth=args.no_auth,
+                disable_announce=args.no_announce,
+            )
+
+        elif args.destination != None and args.file != None:
+            send(
+                configdir = args.config,
+                verbosity = args.verbose,
+                quietness = args.quiet,
+                destination = args.destination,
+                file = args.file,
+                timeout = args.w,
+            )
+
+        else:
+            print("")
+            parser.print_help()
+            print("")
+
+    except KeyboardInterrupt:
+        print("")
+        if resource != None:
+            resource.cancel()
+        if link != None:
+            link.teardown()
+        exit()
+
+def size_str(num, suffix='B'):
+    units = ['','K','M','G','T','P','E','Z']
+    last_unit = 'Y'
+
+    if suffix == 'b':
+        num *= 8
+        units = ['','K','M','G','T','P','E','Z']
+        last_unit = 'Y'
+
+    for unit in units:
+        if abs(num) < 1000.0:
+            if unit == "":
+                return "%.0f %s%s" % (num, unit, suffix)
+            else:
+                return "%.2f %s%s" % (num, unit, suffix)
+        num /= 1000.0
+
+    return "%.2f%s%s" % (num, last_unit, suffix)
+
+if __name__ == "__main__":
+    main()

RNS/Utilities/rnpath.py

@@ -0,0 +1,338 @@
+#!/usr/bin/env python3
+
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import RNS
+import sys
+import time
+import argparse
+
+from RNS._version import __version__
+
+
+def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity, timeout, drop_queues):
+    if table:
+        destination_hash = None
+        if destination_hexhash != None:
+            try:
+                dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+                if len(destination_hexhash) != dest_len:
+                    raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+                try:
+                    destination_hash = bytes.fromhex(destination_hexhash)
+                except Exception as e:
+                    raise ValueError("Invalid destination entered. Check your input.")
+            except Exception as e:
+                print(str(e))
+                sys.exit(1)
+
+        reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+        table = sorted(reticulum.get_path_table(), key=lambda e: (e["interface"], e["hops"]) )
+
+        displayed = 0
+        for path in table:
+            if destination_hash == None or destination_hash == path["hash"]:
+                displayed += 1
+                exp_str = RNS.timestamp_str(path["expires"])
+                if path["hops"] == 1:
+                    m_str = " "
+                else:
+                    m_str = "s"
+                print(RNS.prettyhexrep(path["hash"])+" is "+str(path["hops"])+" hop"+m_str+" away via "+RNS.prettyhexrep(path["via"])+" on "+path["interface"]+" expires "+RNS.timestamp_str(path["expires"]))
+
+        if destination_hash != None and displayed == 0:
+            print("No path known")
+            sys.exit(1)
+
+    elif rates:
+        destination_hash = None
+        if destination_hexhash != None:
+            try:
+                dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+                if len(destination_hexhash) != dest_len:
+                    raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+                try:
+                    destination_hash = bytes.fromhex(destination_hexhash)
+                except Exception as e:
+                    raise ValueError("Invalid destination entered. Check your input.")
+            except Exception as e:
+                print(str(e))
+                sys.exit(1)
+
+        reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+        table = sorted(reticulum.get_rate_table(), key=lambda e: e["last"] )
+
+        if len(table) == 0:
+            print("No information available")
+
+        else:
+            displayed = 0
+            for entry in table:
+                if destination_hash == None or destination_hash == entry["hash"]:
+                    displayed += 1
+                    try:
+                        last_str = pretty_date(int(entry["last"]))
+                        start_ts = entry["timestamps"][0]
+                        span = max(time.time() - start_ts, 3600.0)
+                        span_hours = span/3600.0
+                        span_str = pretty_date(int(entry["timestamps"][0]))
+                        hour_rate = round(len(entry["timestamps"])/span_hours, 3)
+                        if hour_rate-int(hour_rate) == 0:
+                            hour_rate = int(hour_rate)
+                        
+                        if entry["rate_violations"] > 0:
+                            if entry["rate_violations"] == 1:
+                                s_str = ""
+                            else:
+                                s_str = "s"
+
+                            rv_str = ", "+str(entry["rate_violations"])+" active rate violation"+s_str
+                        else:
+                            rv_str = ""
+                        
+                        if entry["blocked_until"] > time.time():
+                            bli = time.time()-(int(entry["blocked_until"])-time.time())
+                            bl_str = ", new announces allowed in "+pretty_date(int(bli))
+                        else:
+                            bl_str = ""
+
+        
+                        print(RNS.prettyhexrep(entry["hash"])+" last heard "+last_str+" ago, "+str(hour_rate)+" announces/hour in the last "+span_str+rv_str+bl_str)
+
+                    except Exception as e:
+                        print("Error while processing entry for "+RNS.prettyhexrep(entry["hash"]))
+                        print(str(e))
+
+            if destination_hash != None and displayed == 0:
+                print("No information available")
+                sys.exit(1)
+
+    elif drop_queues:
+        reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+        RNS.log("Dropping announce queues on all interfaces...")
+        reticulum.drop_announce_queues()
+    
+    elif drop:
+        try:
+            dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+            if len(destination_hexhash) != dest_len:
+                raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+            try:
+                destination_hash = bytes.fromhex(destination_hexhash)
+            except Exception as e:
+                raise ValueError("Invalid destination entered. Check your input.")
+        except Exception as e:
+            print(str(e))
+            sys.exit(1)
+
+
+        reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+
+        if reticulum.drop_path(destination_hash):
+            print("Dropped path to "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("Unable to drop path to "+RNS.prettyhexrep(destination_hash)+". Does it exist?")
+            sys.exit(1)
+
+
+    else:
+        try:
+            dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+            if len(destination_hexhash) != dest_len:
+                raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+            try:
+                destination_hash = bytes.fromhex(destination_hexhash)
+            except Exception as e:
+                raise ValueError("Invalid destination entered. Check your input.")
+        except Exception as e:
+            print(str(e))
+            sys.exit(1)
+            
+
+        reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+
+        if not RNS.Transport.has_path(destination_hash):
+            RNS.Transport.request_path(destination_hash)
+            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
+            sys.stdout.flush()
+
+        i = 0
+        syms = "⢄⢂⢁⡁⡈⡐⡠"
+        limit = time.time()+timeout
+        while not RNS.Transport.has_path(destination_hash) and time.time()<limit:
+            time.sleep(0.1)
+            print(("\b\b"+syms[i]+" "), end="")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
+
+        if RNS.Transport.has_path(destination_hash):
+            hops = RNS.Transport.hops_to(destination_hash)
+            next_hop = RNS.prettyhexrep(reticulum.get_next_hop(destination_hash))
+            next_hop_interface = reticulum.get_next_hop_if_name(destination_hash)
+
+            if hops != 1:
+                ms = "s"
+            else:
+                ms = ""
+
+            print("\rPath found, destination "+RNS.prettyhexrep(destination_hash)+" is "+str(hops)+" hop"+ms+" away via "+next_hop+" on "+next_hop_interface)
+        else:
+            print("\r                                                       \rPath not found")
+            sys.exit(1)
+
+    
+
+def main():
+    try:
+        parser = argparse.ArgumentParser(description="Reticulum Path Discovery Utility")
+
+        parser.add_argument("--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
+
+        parser.add_argument(
+            "--version",
+            action="version",
+            version="rnpath {version}".format(version=__version__)
+        )
+
+        parser.add_argument(
+            "-t",
+            "--table",
+            action="store_true",
+            help="show all known paths",
+            default=False
+        )
+
+        parser.add_argument(
+            "-r",
+            "--rates",
+            action="store_true",
+            help="show announce rate info",
+            default=False
+        )
+
+        parser.add_argument(
+            "-d",
+            "--drop",
+            action="store_true",
+            help="remove the path to a destination",
+            default=False
+        )
+
+        parser.add_argument(
+            "-D",
+            "--drop-announces",
+            action="store_true",
+            help="drop all queued announces",
+            default=False
+        )
+
+        parser.add_argument(
+            "-w",
+            action="store",
+            metavar="seconds",
+            type=float,
+            help="timeout before giving up",
+            default=RNS.Transport.PATH_REQUEST_TIMEOUT
+        )
+
+        parser.add_argument(
+            "destination",
+            nargs="?",
+            default=None,
+            help="hexadecimal hash of the destination",
+            type=str
+        )
+
+        parser.add_argument('-v', '--verbose', action='count', default=0)
+        
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        if not args.drop_announces and not args.table and not args.rates and not args.destination:
+            print("")
+            parser.print_help()
+            print("")
+        else:
+            program_setup(
+                configdir = configarg,
+                table = args.table,
+                rates = args.rates,
+                drop = args.drop,
+                destination_hexhash = args.destination,
+                verbosity = args.verbose,
+                timeout = args.w,
+                drop_queues = args.drop_announces,
+            )
+            sys.exit(0)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()
+
+def pretty_date(time=False):
+    from datetime import datetime
+    now = datetime.now()
+    if type(time) is int:
+        diff = now - datetime.fromtimestamp(time)
+    elif isinstance(time,datetime):
+        diff = now - time
+    elif not time:
+        diff = now - now
+    second_diff = diff.seconds
+    day_diff = diff.days
+    if day_diff < 0:
+        return ''
+    if day_diff == 0:
+        if second_diff < 10:
+            return str(second_diff) + " seconds"
+        if second_diff < 60:
+            return str(second_diff) + " seconds"
+        if second_diff < 120:
+            return "1 minute"
+        if second_diff < 3600:
+            return str(int(second_diff / 60)) + " minutes"
+        if second_diff < 7200:
+            return "an hour"
+        if second_diff < 86400:
+            return str(int(second_diff / 3600)) + " hours"
+    if day_diff == 1:
+        return "1 day"
+    if day_diff < 7:
+        return str(day_diff) + " days"
+    if day_diff < 31:
+        return str(int(day_diff / 7)) + " weeks"
+    if day_diff < 365:
+        return str(int(day_diff / 30)) + " months"
+    return str(int(day_diff / 365)) + " years"
+
+if __name__ == "__main__":
+    main()

RNS/Utilities/rnprobe.py

@@ -0,0 +1,214 @@
+#!/usr/bin/env python3
+
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import RNS
+import os
+import sys
+import time
+import argparse
+
+from RNS._version import __version__
+
+DEFAULT_PROBE_SIZE = 16
+
+def program_setup(configdir, destination_hexhash, size=DEFAULT_PROBE_SIZE, full_name = None, verbosity = 0):
+    if full_name == None:
+        print("The full destination name including application name aspects must be specified for the destination")
+        exit()
+    
+    try:
+        app_name, aspects = RNS.Destination.app_and_aspects_from_name(full_name)
+
+    except Exception as e:
+        print(str(e))
+        exit()
+
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination_hexhash) != dest_len:
+            raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+        try:
+            destination_hash = bytes.fromhex(destination_hexhash)
+        except Exception as e:
+            raise ValueError("Invalid destination entered. Check your input.")
+    except Exception as e:
+        print(str(e))
+        exit()
+
+    if verbosity > 0:
+        more_output = True
+        verbosity -= 1
+    else:
+        more_output = False
+        verbosity -= 1
+
+
+    reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.Transport.request_path(destination_hash)
+        print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
+        sys.stdout.flush()
+
+    i = 0
+    syms = "⢄⢂⢁⡁⡈⡐⡠"
+    while not RNS.Transport.has_path(destination_hash):
+        time.sleep(0.1)
+        print(("\b\b"+syms[i]+" "), end="")
+        sys.stdout.flush()
+        i = (i+1)%len(syms)
+
+    server_identity = RNS.Identity.recall(destination_hash)
+
+    request_destination = RNS.Destination(
+        server_identity,
+        RNS.Destination.OUT,
+        RNS.Destination.SINGLE,
+        app_name,
+        *aspects
+    )
+
+    probe = RNS.Packet(request_destination, os.urandom(size))
+    receipt = probe.send()
+
+    if more_output:
+        more = " via "+RNS.prettyhexrep(reticulum.get_next_hop(destination_hash))+" on "+str(reticulum.get_next_hop_if_name(destination_hash))
+    else:
+        more = ""
+
+    print("\rSent "+str(size)+" byte probe to "+RNS.prettyhexrep(destination_hash)+more+"  ", end=" ")
+
+    i = 0
+    while not receipt.status == RNS.PacketReceipt.DELIVERED:
+        time.sleep(0.1)
+        print(("\b\b"+syms[i]+" "), end="")
+        sys.stdout.flush()
+        i = (i+1)%len(syms)
+
+    print("\b\b ")
+    sys.stdout.flush()
+
+    hops = RNS.Transport.hops_to(destination_hash)
+    if hops != 1:
+        ms = "s"
+    else:
+        ms = ""
+
+    rtt = receipt.get_rtt()
+    if (rtt >= 1):
+        rtt = round(rtt, 3)
+        rttstring = str(rtt)+" seconds"
+    else:
+        rtt = round(rtt*1000, 3)
+        rttstring = str(rtt)+" milliseconds"
+
+    reception_stats = ""
+    if reticulum.is_connected_to_shared_instance:
+        reception_rssi = reticulum.get_packet_rssi(receipt.proof_packet.packet_hash)
+        reception_snr  = reticulum.get_packet_snr(receipt.proof_packet.packet_hash)
+
+        if reception_rssi != None:
+            reception_stats += " [RSSI "+str(reception_rssi)+" dBm]"
+        
+        if reception_snr != None:
+            reception_stats += " [SNR "+str(reception_snr)+" dB]"
+
+    else:
+        if receipt.proof_packet != None:
+            if receipt.proof_packet.rssi != None:
+                reception_stats += " [RSSI "+str(receipt.proof_packet.rssi)+" dBm]"
+            
+            if receipt.proof_packet.snr != None:
+                reception_stats += " [SNR "+str(receipt.proof_packet.snr)+" dB]"
+
+    print(
+        "Valid reply received from "+
+        RNS.prettyhexrep(receipt.destination.hash)+
+        "\nRound-trip time is "+rttstring+
+        " over "+str(hops)+" hop"+ms+
+        reception_stats
+    )
+
+    
+
+def main():
+    try:
+        parser = argparse.ArgumentParser(description="Reticulum Probe Utility")
+
+        parser.add_argument("--config",
+            action="store",
+            default=None,
+            help="path to alternative Reticulum config directory",
+            type=str
+        )
+
+        parser.add_argument(
+            "--version",
+            action="version",
+            version="rnprobe {version}".format(version=__version__)
+        )
+
+        parser.add_argument(
+            "full_name",
+            nargs="?",
+            default=None,
+            help="full destination name in dotted notation",
+            type=str
+        )
+
+        parser.add_argument(
+            "destination_hash",
+            nargs="?",
+            default=None,
+            help="hexadecimal hash of the destination",
+            type=str
+        )
+
+        parser.add_argument('-v', '--verbose', action='count', default=0)
+
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        if not args.destination_hash:
+            print("")
+            parser.print_help()
+            print("")
+        else:
+            program_setup(
+                configdir = configarg,
+                destination_hexhash = args.destination_hash,
+                full_name = args.full_name,
+                verbosity = args.verbose
+            )
+
+    except KeyboardInterrupt:
+        print("")
+        exit()
+
+if __name__ == "__main__":
+    main()

RNS/Utilities/rnsd.py

@@ -0,0 +1,418 @@
+#!/usr/bin/env python3
+
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import RNS
+import argparse
+import time
+
+from RNS._version import __version__
+
+
+def program_setup(configdir, verbosity = 0, quietness = 0, service = False):
+    targetloglevel = 3+verbosity-quietness
+
+    if service:
+        targetlogdest  = RNS.LOG_FILE
+        targetloglevel = None
+    else:
+        targetlogdest  = RNS.LOG_STDOUT
+
+    reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel, logdest=targetlogdest)
+    if reticulum.is_connected_to_shared_instance:
+        RNS.log("Started rnsd version {version} connected to another shared local instance, this is probably NOT what you want!".format(version=__version__), RNS.LOG_WARNING)
+    else:
+        RNS.log("Started rnsd version {version}".format(version=__version__), RNS.LOG_NOTICE)
+
+    while True:
+        time.sleep(1)
+
+def main():
+    try:
+        parser = argparse.ArgumentParser(description="Reticulum Network Stack Daemon")
+        parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
+        parser.add_argument('-v', '--verbose', action='count', default=0)
+        parser.add_argument('-q', '--quiet', action='count', default=0)
+        parser.add_argument('-s', '--service', action='store_true', default=False, help="rnsd is running as a service and should log to file")
+        parser.add_argument("--exampleconfig", action='store_true', default=False, help="print verbose configuration example to stdout and exit")
+        parser.add_argument("--version", action="version", version="rnsd {version}".format(version=__version__))
+        
+        args = parser.parse_args()
+
+        if args.exampleconfig:
+            print(__example_rns_config__)
+            exit()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet, service=args.service)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()
+
+__example_rns_config__ = '''# This is an example Reticulum config file.
+# You should probably edit it to include any additional,
+# interfaces and settings you might need.
+
+[reticulum]
+
+# If you enable Transport, your system will route traffic
+# for other peers, pass announces and serve path requests.
+# This should be done for systems that are suited to act
+# as transport nodes, ie. if they are stationary and
+# always-on. This directive is optional and can be removed
+# for brevity.
+
+enable_transport = False
+
+
+# By default, the first program to launch the Reticulum
+# Network Stack will create a shared instance, that other
+# programs can communicate with. Only the shared instance
+# opens all the configured interfaces directly, and other
+# local programs communicate with the shared instance over
+# a local socket. This is completely transparent to the
+# user, and should generally be turned on. This directive
+# is optional and can be removed for brevity.
+
+share_instance = Yes
+
+
+# If you want to run multiple *different* shared instances
+# on the same system, you will need to specify different
+# shared instance ports for each. The defaults are given
+# below, and again, these options can be left out if you
+# don't need them.
+
+shared_instance_port = 37428
+instance_control_port = 37429
+
+# You can configure Reticulum to panic and forcibly close
+# if an unrecoverable interface error occurs, such as the
+# hardware device for an interface disappearing. This is
+# an optional directive, and can be left out for brevity.
+# This behaviour is disabled by default.
+
+panic_on_interface_error = No
+
+
+[logging]
+# Valid log levels are 0 through 7:
+#   0: Log only critical information
+#   1: Log errors and lower log levels
+#   2: Log warnings and lower log levels
+#   3: Log notices and lower log levels
+#   4: Log info and lower (this is the default)
+#   5: Verbose logging
+#   6: Debug logging
+#   7: Extreme logging
+
+loglevel = 4
+
+
+# The interfaces section defines the physical and virtual
+# interfaces Reticulum will use to communicate on. This
+# section will contain examples for a variety of interface
+# types. You can modify these or use them as a basis for
+# your own config, or simply remove the unused ones.
+
+[interfaces]
+
+  # This interface enables communication with other
+  # link-local Reticulum nodes over UDP. It does not
+  # need any functional IP infrastructure like routers
+  # or DHCP servers, but will require that at least link-
+  # local IPv6 is enabled in your operating system, which
+  # should be enabled by default in almost any OS. See
+  # the Reticulum Manual for more configuration options.
+
+  [[Default Interface]]
+    type = AutoInterface
+    enabled = yes
+
+
+  # The following example enables communication with other
+  # local Reticulum peers using UDP broadcasts.
+  
+  [[UDP Interface]]
+    type = UDPInterface
+    enabled = no
+    listen_ip = 0.0.0.0
+    listen_port = 4242
+    forward_ip = 255.255.255.255
+    forward_port = 4242
+
+    # The above configuration will allow communication
+    # within the local broadcast domains of all local
+    # IP interfaces.
+
+    # Instead of specifying listen_ip, listen_port,
+    # forward_ip and forward_port, you can also bind
+    # to a specific network device like below.
+
+    # device = eth0
+    # port = 4242
+
+    # Assuming the eth0 device has the address
+    # 10.55.0.72/24, the above configuration would
+    # be equivalent to the following manual setup.
+    # Note that we are both listening and forwarding to
+    # the broadcast address of the network segments.
+
+    # listen_ip = 10.55.0.255
+    # listen_port = 4242
+    # forward_ip = 10.55.0.255
+    # forward_port = 4242
+
+    # You can of course also communicate only with
+    # a single IP address
+
+    # listen_ip = 10.55.0.15
+    # listen_port = 4242
+    # forward_ip = 10.55.0.16
+    # forward_port = 4242
+
+
+  # This example demonstrates a TCP server interface.
+  # It will listen for incoming connections on the
+  # specified IP address and port number.
+  
+  [[TCP Server Interface]]
+    type = TCPServerInterface
+    enabled = no
+
+    # This configuration will listen on all IP
+    # interfaces on port 4242
+    
+    listen_ip = 0.0.0.0
+    listen_port = 4242
+
+    # Alternatively you can bind to a specific IP
+    
+    # listen_ip = 10.0.0.88
+    # listen_port = 4242
+
+    # Or a specific network device
+    
+    # device = eth0
+    # port = 4242
+
+
+  # To connect to a TCP server interface, you would
+  # naturally use the TCP client interface. Here's
+  # an example. The target_host can either be an IP
+  # address or a hostname
+
+  [[TCP Client Interface]]
+    type = TCPClientInterface
+    enabled = no
+    target_host = 127.0.0.1
+    target_port = 4242
+
+
+  # This example shows how to make your Reticulum
+  # instance available over I2P, and connect to
+  # another I2P peer. Please be aware that you
+  # must have an I2P router running on your system
+  # with the SAMv3 API enabled for this to work.
+
+  [[I2P]]
+    type = I2PInterface
+    enabled = no
+    connectable = yes
+    peers = ykzlw5ujbaqc2xkec4cpvgyxj257wcrmmgkuxqmqcur7cq3w3lha.b32.i2p
+
+
+  # Here's an example of how to add a LoRa interface
+  # using the RNode LoRa transceiver.
+
+  [[RNode LoRa Interface]]
+    type = RNodeInterface
+
+    # Enable interface if you want use it!
+    enabled = no
+
+    # Serial port for the device
+    port = /dev/ttyUSB0
+
+    # Set frequency to 867.2 MHz
+    frequency = 867200000
+
+    # Set LoRa bandwidth to 125 KHz
+    bandwidth = 125000
+
+    # Set TX power to 7 dBm (5 mW)
+    txpower = 7
+
+    # Select spreading factor 8. Valid 
+    # range is 7 through 12, with 7
+    # being the fastest and 12 having
+    # the longest range.
+    spreadingfactor = 8
+
+    # Select coding rate 5. Valid range
+    # is 5 throough 8, with 5 being the
+    # fastest, and 8 the longest range.
+    codingrate = 5
+
+    # You can configure the RNode to send
+    # out identification on the channel with
+    # a set interval by configuring the
+    # following two parameters. The trans-
+    # ceiver will only ID if the set
+    # interval has elapsed since it's last
+    # actual transmission. The interval is
+    # configured in seconds.
+    # This option is commented out and not
+    # used by default.
+    # id_callsign = MYCALL-0
+    # id_interval = 600
+
+    # For certain homebrew RNode interfaces
+    # with low amounts of RAM, using packet
+    # flow control can be useful. By default
+    # it is disabled.
+    flow_control = False
+    
+    
+  # An example KISS modem interface. Useful for running
+  # Reticulum over packet radio hardware.
+
+  [[Packet Radio KISS Interface]]
+    type = KISSInterface
+
+    # Enable interface if you want use it!
+    enabled = no
+
+    # Serial port for the device
+    port = /dev/ttyUSB1
+
+    # Set the serial baud-rate and other
+    # configuration parameters.
+    speed = 115200    
+    databits = 8
+    parity = none
+    stopbits = 1
+
+    # Set the modem preamble. A 150ms
+    # preamble should be a reasonable
+    # default, but may need to be
+    # increased for radios with slow-
+    # opening squelch and long TX/RX
+    # turnaround
+    preamble = 150
+
+    # Set the modem TX tail. In most
+    # cases this should be kept as low
+    # as possible to not waste airtime.
+    txtail = 10
+
+    # Configure CDMA parameters. These
+    # settings are reasonable defaults.
+    persistence = 200
+    slottime = 20
+
+    # You can configure the interface to send
+    # out identification on the channel with
+    # a set interval by configuring the
+    # following two parameters. The KISS
+    # interface will only ID if the set
+    # interval has elapsed since it's last
+    # actual transmission. The interval is
+    # configured in seconds.
+    # This option is commented out and not
+    # used by default.
+    # id_callsign = MYCALL-0
+    # id_interval = 600
+
+    # Whether to use KISS flow-control.
+    # This is useful for modems that have
+    # a small internal packet buffer, but
+    # support packet flow control instead.
+    flow_control = false
+
+
+  # If you're using Reticulum on amateur radio spectrum,
+  # you might want to use the AX.25 KISS interface. This
+  # way, Reticulum will automatically encapsulate it's
+  # traffic in AX.25 and also identify your stations
+  # transmissions with your callsign and SSID.
+  # 
+  # Only do this if you really need to! Reticulum doesn't
+  # need the AX.25 layer for anything, and it incurs extra
+  # overhead on every packet to encapsulate in AX.25.
+  #
+  # A more efficient way is to use the plain KISS interface
+  # with the beaconing functionality described above.
+
+  [[Packet Radio AX.25 KISS Interface]]
+    type = AX25KISSInterface
+
+    # Set the station callsign and SSID
+    callsign = NO1CLL
+    ssid = 0
+
+    # Enable interface if you want use it!
+    enabled = no
+
+    # Serial port for the device
+    port = /dev/ttyUSB2
+
+    # Set the serial baud-rate and other
+    # configuration parameters.
+    speed = 115200    
+    databits = 8
+    parity = none
+    stopbits = 1
+
+    # Whether to use KISS flow-control.
+    # This is useful for modems with a
+    # small internal packet buffer.
+    flow_control = false
+
+    # Set the modem preamble. A 150ms
+    # preamble should be a reasonable
+    # default, but may need to be
+    # increased for radios with slow-
+    # opening squelch and long TX/RX
+    # turnaround
+    preamble = 150
+
+    # Set the modem TX tail. In most
+    # cases this should be kept as low
+    # as possible to not waste airtime.
+    txtail = 10
+
+    # Configure CDMA parameters. These
+    # settings are reasonable defaults.
+    persistence = 200
+    slottime = 20
+
+'''
+
+if __name__ == "__main__":
+    main()

RNS/Utilities/rnstatus.py

@@ -0,0 +1,210 @@
+#!/usr/bin/env python3
+
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import RNS
+import argparse
+
+from RNS._version import __version__
+
+def size_str(num, suffix='B'):
+    units = ['','K','M','G','T','P','E','Z']
+    last_unit = 'Y'
+
+    if suffix == 'b':
+        num *= 8
+        units = ['','K','M','G','T','P','E','Z']
+        last_unit = 'Y'
+
+    for unit in units:
+        if abs(num) < 1000.0:
+            if unit == "":
+                return "%.0f %s%s" % (num, unit, suffix)
+            else:
+                return "%.2f %s%s" % (num, unit, suffix)
+        num /= 1000.0
+
+    return "%.2f%s%s" % (num, last_unit, suffix)
+
+def program_setup(configdir, dispall=False, verbosity=0, name_filter=None):
+    reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+
+    stats = None
+    try:
+        stats = reticulum.get_interface_stats()
+    except Exception as e:
+        pass
+
+    if stats != None:
+        for ifstat in stats["interfaces"]:
+            name = ifstat["name"]
+
+            if dispall or not (
+                name.startswith("LocalInterface[") or
+                name.startswith("TCPInterface[Client") or
+                name.startswith("I2PInterfacePeer[Connected peer") or
+                (name.startswith("I2PInterface[") and ("i2p_connectable" in ifstat and ifstat["i2p_connectable"] == False))
+                ):
+
+                if not (name.startswith("I2PInterface[") and ("i2p_connectable" in ifstat and ifstat["i2p_connectable"] == False)):
+                    if name_filter == None or name_filter.lower() in name.lower():
+                        print("")
+
+                        if ifstat["status"]:
+                            ss = "Up"
+                        else:
+                            ss = "Down"
+
+                        if ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_ACCESS_POINT:
+                            modestr = "Access Point"
+                        elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_POINT_TO_POINT:
+                            modestr = "Point-to-Point"
+                        elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_ROAMING:
+                            modestr = "Roaming"
+                        elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_BOUNDARY:
+                            modestr = "Boundary"
+                        elif ifstat["mode"] == RNS.Interfaces.Interface.Interface.MODE_GATEWAY:
+                            modestr = "Gateway"
+                        else:
+                            modestr = "Full"
+
+
+                        if ifstat["clients"] != None:
+                            clients = ifstat["clients"]
+                            if name.startswith("Shared Instance["):
+                                cnum = max(clients-1,0)
+                                if cnum == 1:
+                                    spec_str = " program"
+                                else:
+                                    spec_str = " programs"
+
+                                clients_string = "Serving : "+str(cnum)+spec_str
+                            elif name.startswith("I2PInterface["):
+                                if "i2p_connectable" in ifstat and ifstat["i2p_connectable"] == True:
+                                    cnum = clients
+                                    if cnum == 1:
+                                        spec_str = " connected I2P endpoint"
+                                    else:
+                                        spec_str = " connected I2P endpoints"
+
+                                    clients_string = "Peers   : "+str(cnum)+spec_str
+                                else:
+                                    clients_string = ""
+                            else:
+                                clients_string = "Clients : "+str(clients)
+
+                        else:
+                            clients = None
+
+                        print(" {n}".format(n=ifstat["name"]))
+
+                        if "ifac_netname" in ifstat and ifstat["ifac_netname"] != None:
+                            print("    Network : {nn}".format(nn=ifstat["ifac_netname"]))
+
+                        print("    Status  : {ss}".format(ss=ss))
+
+                        if clients != None and clients_string != "":
+                            print("    "+clients_string)
+
+                        if not (name.startswith("Shared Instance[") or name.startswith("TCPInterface[Client") or name.startswith("LocalInterface[")):
+                            print("    Mode    : {mode}".format(mode=modestr))
+
+                        if "bitrate" in ifstat and ifstat["bitrate"] != None:
+                            print("    Rate    : {ss}".format(ss=speed_str(ifstat["bitrate"])))
+                        
+                        if "peers" in ifstat and ifstat["peers"] != None:
+                            print("    Peers   : {np} reachable".format(np=ifstat["peers"]))
+
+                        if "ifac_signature" in ifstat and ifstat["ifac_signature"] != None:
+                            sigstr = "<…"+RNS.hexrep(ifstat["ifac_signature"][-5:], delimit=False)+">"
+                            print("    Access  : {nb}-bit IFAC by {sig}".format(nb=ifstat["ifac_size"]*8, sig=sigstr))
+                        
+                        if "i2p_b32" in ifstat and ifstat["i2p_b32"] != None:
+                            print("    I2P B32 : {ep}".format(ep=str(ifstat["i2p_b32"])))
+
+                        if "announce_queue" in ifstat and ifstat["announce_queue"] != None and ifstat["announce_queue"] > 0:
+                            aqn = ifstat["announce_queue"]
+                            if aqn == 1:
+                                print("    Queued  : {np} announce".format(np=aqn))
+                            else:
+                                print("    Queued  : {np} announces".format(np=aqn))
+                        
+                        print("    Traffic : {txb}↑\n              {rxb}↓".format(rxb=size_str(ifstat["rxb"]), txb=size_str(ifstat["txb"])))
+
+        if "transport_id" in stats and stats["transport_id"] != None:
+            print("\n Reticulum Transport Instance "+RNS.prettyhexrep(stats["transport_id"])+" is running")
+
+        print("")
+                
+    else:
+        print("Could not get RNS status")
+
+def main():
+    try:
+        parser = argparse.ArgumentParser(description="Reticulum Network Stack Status")
+        parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
+        parser.add_argument("--version", action="version", version="rnstatus {version}".format(version=__version__))
+
+        parser.add_argument(
+            "-a",
+            "--all",
+            action="store_true",
+            help="show all interfaces",
+            default=False
+        )
+
+        parser.add_argument('-v', '--verbose', action='count', default=0)
+
+        parser.add_argument("filter", nargs="?", default=None, help="only display interfaces with names including filter", type=str)
+        
+        args = parser.parse_args()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        program_setup(configdir = configarg, dispall = args.all, verbosity=args.verbose, name_filter=args.filter)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()
+
+def speed_str(num, suffix='bps'):
+    units = ['','k','M','G','T','P','E','Z']
+    last_unit = 'Y'
+
+    if suffix == 'Bps':
+        num /= 8
+        units = ['','K','M','G','T','P','E','Z']
+        last_unit = 'Y'
+
+    for unit in units:
+        if abs(num) < 1000.0:
+            return "%3.2f %s%s" % (num, unit, suffix)
+        num /= 1000.0
+
+    return "%.2f %s%s" % (num, last_unit, suffix)
+
+if __name__ == "__main__":
+    main()

RNS/Utilities/rnx.py

@@ -0,0 +1,714 @@
+#!/usr/bin/env python3
+
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import RNS
+import subprocess
+import argparse
+import shlex
+import time
+import sys
+import tty
+import os
+
+from RNS._version import __version__
+
+APP_NAME = "rnx"
+identity = None
+reticulum = None
+allow_all = False
+allowed_identity_hashes = []
+
+def prepare_identity(identity_path):
+    global identity
+    if identity_path == None:
+        identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
+
+    if os.path.isfile(identity_path):
+        identity = RNS.Identity.from_file(identity_path)                
+
+    if identity == None:
+        RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
+        identity = RNS.Identity()
+        identity.to_file(identity_path)
+
+def listen(configdir, identitypath = None, verbosity = 0, quietness = 0, allowed = [], print_identity = False, disable_auth = None, disable_announce=False):
+    global identity, allow_all, allowed_identity_hashes, reticulum
+
+    targetloglevel = 3+verbosity-quietness
+    reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
+    
+    prepare_identity(identitypath)
+    destination = RNS.Destination(identity, RNS.Destination.IN, RNS.Destination.SINGLE, APP_NAME, "execute")
+
+    if print_identity:
+        print("Identity     : "+str(identity))
+        print("Listening on : "+RNS.prettyhexrep(destination.hash))
+        exit(0)
+
+    if disable_auth:
+        allow_all = True
+    else:
+        if allowed != None:
+            for a in allowed:
+                try:
+                    dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+                    if len(a) != dest_len:
+                        raise ValueError("Allowed destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+                    try:
+                        destination_hash = bytes.fromhex(a)
+                        allowed_identity_hashes.append(destination_hash)
+                    except Exception as e:
+                        raise ValueError("Invalid destination entered. Check your input.")
+                except Exception as e:
+                    print(str(e))
+                    exit(1)
+
+    if len(allowed_identity_hashes) < 1 and not disable_auth:
+        print("Warning: No allowed identities configured, rncx will not accept any commands!")
+
+    destination.set_link_established_callback(command_link_established)
+
+    if not allow_all:
+        destination.register_request_handler(
+            path = "command",
+            response_generator = execute_received_command,
+            allow = RNS.Destination.ALLOW_LIST,
+            allowed_list = allowed_identity_hashes
+        )
+    else:
+        destination.register_request_handler(
+            path = "command",
+            response_generator = execute_received_command,
+            allow = RNS.Destination.ALLOW_ALL,
+        )
+
+    RNS.log("rnx listening for commands on "+RNS.prettyhexrep(destination.hash))
+
+    if not disable_announce:
+        destination.announce()
+    
+    while True:
+        time.sleep(1)
+
+def command_link_established(link):
+    link.set_remote_identified_callback(initiator_identified)
+    link.set_link_closed_callback(command_link_closed)
+    RNS.log("Command link "+str(link)+" established")
+
+def command_link_closed(link):
+    RNS.log("Command link "+str(link)+" closed")
+
+def initiator_identified(link, identity):
+    global allow_all
+    RNS.log("Initiator of link "+str(link)+" identified as "+RNS.prettyhexrep(identity.hash))
+    if not allow_all and not identity.hash in allowed_identity_hashes:
+        RNS.log("Identity "+RNS.prettyhexrep(identity.hash)+" not allowed, tearing down link")
+        link.teardown()
+
+def execute_received_command(path, data, request_id, remote_identity, requested_at):
+    command = data[0].decode("utf-8")  # Command to execute
+    timeout = data[1]                  # Timeout in seconds
+    o_limit = data[2]                  # Size limit for stdout
+    e_limit = data[3]                  # Size limit for stderr
+    stdin   = data[4]                  # Data passed to stdin
+
+    if remote_identity != None:
+        RNS.log("Executing command ["+command+"] for "+RNS.prettyhexrep(remote_identity.hash))
+    else:
+        RNS.log("Executing command ["+command+"] for unknown requestor")
+
+    result    = [
+        False,                         # 0: Command was executed
+        None,                          # 1: Return value
+        None,                          # 2: Stdout
+        None,                          # 3: Stderr
+        None,                          # 4: Total stdout length
+        None,                          # 5: Total stderr length
+        time.time(),                   # 6: Started
+        None,                          # 7: Concluded
+    ]
+
+    try:
+        process = subprocess.Popen(shlex.split(command), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        result[0] = True
+
+    except Exception as e:
+        result[0] = False
+        return result
+
+    stdout = b""
+    stderr = b""
+    timed_out = False
+
+    if stdin != None:
+        process.stdin.write(stdin)
+
+    while True:
+        try:
+            stdout, stderr = process.communicate(timeout=1)
+            if process.poll() != None:
+                break
+
+            if len(stdout) > 0:
+                print(str(stdout))
+                sys.stdout.flush()
+
+        except subprocess.TimeoutExpired:
+            pass
+
+        if timeout != None and time.time() > result[6]+timeout:
+            RNS.log("Command ["+command+"] timed out and is being killed...")
+            process.terminate()
+            process.wait()
+            if process.poll() != None:
+                stdout, stderr = process.communicate()
+            else:
+                stdout = None
+                stderr = None
+
+            break
+
+    if timeout != None and time.time() < result[6]+timeout:
+        result[7] = time.time()
+
+    # Deliver result
+    result[1] = process.returncode
+
+    if o_limit != None and len(stdout) > o_limit:
+        if o_limit == 0:
+            result[2] = b""
+        else:
+            result[2] = stdout[0:o_limit]
+    else:
+        result[2] = stdout
+
+    if e_limit != None and len(stderr) > e_limit:
+        if e_limit == 0:
+            result[3] = b""
+        else:
+            result[3] = stderr[0:e_limit]
+    else:
+        result[3] = stderr
+
+    result[4] = len(stdout)
+    result[5] = len(stderr)
+
+    if timed_out:
+        RNS.log("Command timed out")
+        return result
+
+    if remote_identity != None:
+        RNS.log("Delivering result of command ["+str(command)+"] to "+RNS.prettyhexrep(remote_identity.hash))
+    else:
+        RNS.log("Delivering result of command ["+str(command)+"] to unknown requestor")
+
+    return result
+
+def spin(until=None, msg=None, timeout=None):
+    i = 0
+    syms = "⢄⢂⢁⡁⡈⡐⡠"
+    if timeout != None:
+        timeout = time.time()+timeout
+
+    print(msg+"  ", end=" ")
+    while (timeout == None or time.time()<timeout) and not until():
+        time.sleep(0.1)
+        print(("\b\b"+syms[i]+" "), end="")
+        sys.stdout.flush()
+        i = (i+1)%len(syms)
+
+    print("\r"+" "*len(msg)+"  \r", end="")
+
+    if timeout != None and time.time() > timeout:
+        return False
+    else:
+        return True
+
+current_progress = 0.0
+stats = []
+speed = 0.0
+def spin_stat(until=None, timeout=None):
+    global current_progress, response_transfer_size, speed
+    i = 0
+    syms = "⢄⢂⢁⡁⡈⡐⡠"
+    if timeout != None:
+        timeout = time.time()+timeout
+
+    while (timeout == None or time.time()<timeout) and not until():
+        time.sleep(0.1)
+        prg = current_progress
+        percent = round(prg * 100.0, 1)
+        stat_str = str(percent)+"% - " + size_str(int(prg*response_transfer_size)) + " of " + size_str(response_transfer_size) + " - " +size_str(speed, "b")+"ps"
+        print("\r                                                                                  \rReceiving result "+syms[i]+" "+stat_str, end=" ")
+
+        sys.stdout.flush()
+        i = (i+1)%len(syms)
+
+    print("\r                                                                                  \r", end="")
+
+    if timeout != None and time.time() > timeout:
+        return False
+    else:
+        return True
+
+def remote_execution_done(request_receipt):
+    pass
+
+def remote_execution_progress(request_receipt):
+    stats_max = 32
+    global current_progress, response_transfer_size, speed
+    current_progress = request_receipt.progress
+    response_transfer_size = request_receipt.response_transfer_size
+    now = time.time()
+    got = current_progress*response_transfer_size
+    entry = [now, got]
+    stats.append(entry)
+    while len(stats) > stats_max:
+        stats.pop(0)
+
+    span = now - stats[0][0]
+    if span == 0:
+        speed = 0
+    else:
+        diff = got - stats[0][1]
+        speed = diff/span
+
+link = None
+listener_destination = None
+remote_exec_grace = 2.0
+def execute(configdir, identitypath = None, verbosity = 0, quietness = 0, detailed = False, mirror = False, noid = False, destination = None, command = None, stdin = None, stdoutl = None, stderrl = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, result_timeout = None, interactive = False):
+    global identity, reticulum, link, listener_destination, remote_exec_grace
+
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination) != dest_len:
+            raise ValueError("Allowed destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+        try:
+            destination_hash = bytes.fromhex(destination)
+        except Exception as e:
+            raise ValueError("Invalid destination entered. Check your input.")
+    except Exception as e:
+        print(str(e))
+        exit(241)
+
+    if reticulum == None:
+        targetloglevel = 3+verbosity-quietness
+        reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
+
+    if identity == None:
+        prepare_identity(identitypath)
+
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.Transport.request_path(destination_hash)
+        if not spin(until=lambda: RNS.Transport.has_path(destination_hash), msg="Path to "+RNS.prettyhexrep(destination_hash)+" requested", timeout=timeout):
+            print("Path not found")
+            exit(242)
+
+    if listener_destination == None:
+        listener_identity = RNS.Identity.recall(destination_hash)
+        listener_destination = RNS.Destination(
+            listener_identity,
+            RNS.Destination.OUT,
+            RNS.Destination.SINGLE,
+            APP_NAME,
+            "execute"
+        )
+
+    if link == None or link.status == RNS.Link.CLOSED or link.status == RNS.Link.PENDING:
+        link = RNS.Link(listener_destination)
+        link.did_identify = False
+    
+    if not spin(until=lambda: link.status == RNS.Link.ACTIVE, msg="Establishing link with "+RNS.prettyhexrep(destination_hash), timeout=timeout):
+        print("Could not establish link with "+RNS.prettyhexrep(destination_hash))
+        exit(243)
+
+    if not noid and not link.did_identify:
+        link.identify(identity)
+        link.did_identify = True
+
+    if stdin != None:
+        stdin = stdin.encode("utf-8")
+
+    request_data = [
+        command.encode("utf-8"),  # Command to execute
+        timeout,                  # Timeout in seconds
+        stdoutl,                  # Size limit for stdout
+        stderrl,                  # Size limit for stderr
+        stdin,                    # Data passed to stdin
+    ]
+
+    # TODO: Tune
+    rexec_timeout = timeout+link.rtt*4+remote_exec_grace
+
+    request_receipt = link.request(
+        path="command",
+        data=request_data,
+        response_callback=remote_execution_done,
+        failed_callback=remote_execution_done,
+        progress_callback=remote_execution_progress,
+        timeout=rexec_timeout
+    )
+
+    spin(
+        until=lambda:link.status == RNS.Link.CLOSED or (request_receipt.status != RNS.RequestReceipt.FAILED and request_receipt.status != RNS.RequestReceipt.SENT),
+        msg="Sending execution request",
+        timeout=rexec_timeout+0.5
+    )
+
+    if link.status == RNS.Link.CLOSED:
+        print("Could not request remote execution, link was closed")
+        exit(244)
+
+    if request_receipt.status == RNS.RequestReceipt.FAILED:
+        print("Could not request remote execution")
+        if interactive:
+            return
+        else:
+            exit(244)
+
+    spin(
+        until=lambda:request_receipt.status != RNS.RequestReceipt.DELIVERED,
+        msg="Command delivered, awaiting result",
+        timeout=timeout
+    )
+
+    if request_receipt.status == RNS.RequestReceipt.FAILED:
+        print("No result was received")
+        if interactive:
+            return
+        else:
+            exit(245)
+
+    spin_stat(
+        until=lambda:request_receipt.status != RNS.RequestReceipt.RECEIVING,
+        timeout=result_timeout
+    )
+
+    if request_receipt.status == RNS.RequestReceipt.FAILED:
+        print("Receiving result failed")
+        if interactive:
+            return
+        else:
+            exit(246)
+
+    if request_receipt.response != None:
+        try:
+            executed = request_receipt.response[0]
+            retval = request_receipt.response[1]
+            stdout = request_receipt.response[2]
+            stderr = request_receipt.response[3]
+            outlen = request_receipt.response[4]
+            errlen = request_receipt.response[5]
+            started = request_receipt.response[6]
+            concluded = request_receipt.response[7]
+
+        except Exception as e:
+            print("Received invalid result")
+            if interactive:
+                return
+            else:
+                exit(247)
+
+        if executed:
+            if detailed:
+                if stdout != None and len(stdout) > 0:
+                    print(stdout.decode("utf-8"), end="")
+                if stderr != None and len(stderr) > 0:
+                    print(stderr.decode("utf-8"), file=sys.stderr, end="")
+
+                sys.stdout.flush()
+                sys.stderr.flush()
+
+                print("\n--- End of remote output, rnx done ---")
+                if started != None and concluded != None:
+                    cmd_duration = round(concluded - started, 3)
+                    print("Remote command execution took "+str(cmd_duration)+" seconds")
+
+                    total_size = request_receipt.response_size
+                    if request_receipt.request_size != None:
+                        total_size += request_receipt.request_size
+
+                    transfer_duration = round(request_receipt.response_concluded_at - request_receipt.sent_at - cmd_duration, 3)
+                    if transfer_duration == 1:
+                        tdstr = " in 1 second"
+                    elif transfer_duration < 10:
+                        tdstr = " in "+str(transfer_duration)+" seconds"
+                    else:
+                        tdstr = " in "+pretty_time(transfer_duration)
+
+                    spdstr = ", effective rate "+size_str(total_size/transfer_duration, "b")+"ps"
+
+                    print("Transferred "+size_str(total_size)+tdstr+spdstr)
+
+                if outlen != None and stdout != None:
+                    if len(stdout) < outlen:
+                        tstr = ", "+str(len(stdout))+" bytes displayed"
+                    else:
+                        tstr = ""
+                    print("Remote wrote "+str(outlen)+" bytes to stdout"+tstr)
+                
+                if errlen != None and stderr != None:
+                    if len(stderr) < errlen:
+                        tstr = ", "+str(len(stderr))+" bytes displayed"
+                    else:
+                        tstr = ""
+                    print("Remote wrote "+str(errlen)+" bytes to stderr"+tstr)
+
+            else:
+                if stdout != None and len(stdout) > 0:
+                    print(stdout.decode("utf-8"), end="")
+                if stderr != None and len(stderr) > 0:
+                    print(stderr.decode("utf-8"), file=sys.stderr, end="")
+
+
+                if (stdoutl != 0 and len(stdout) < outlen) or (stderrl != 0 and len(stderr) < errlen):
+                    sys.stdout.flush()
+                    sys.stderr.flush()
+                    print("\nOutput truncated before being returned:")
+                    if len(stdout) != 0 and len(stdout) < outlen:
+                        print("  stdout truncated to "+str(len(stdout))+" bytes")
+                    if len(stderr) != 0 and len(stderr) < errlen:
+                        print("  stderr truncated to "+str(len(stderr))+" bytes")
+        else:
+            print("Remote could not execute command")
+            if interactive:
+                return
+            else:
+                exit(248)
+    else:
+        print("No response")
+        if interactive:
+            return
+        else:
+            exit(249)
+
+    try:
+        if not interactive:
+            link.teardown()
+
+    except Exception as e:
+        pass
+
+    if not interactive and mirror:
+        if request_receipt.response[1] != None:
+            exit(request_receipt.response[1])
+        else:
+            exit(240)
+    else:
+        if interactive:
+            if mirror:
+                return request_receipt.response[1]
+            else:
+                return None
+        else:
+            exit(0)
+
+def main():
+    try:
+        parser = argparse.ArgumentParser(description="Reticulum Remote Execution Utility")
+        parser.add_argument("destination", nargs="?", default=None, help="hexadecimal hash of the listener", type=str)
+        parser.add_argument("command", nargs="?", default=None, help="command to be execute", type=str)
+        parser.add_argument("--config", metavar="path", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
+        parser.add_argument('-v', '--verbose', action='count', default=0, help="increase verbosity")
+        parser.add_argument('-q', '--quiet', action='count', default=0, help="decrease verbosity")
+        parser.add_argument('-p', '--print-identity', action='store_true', default=False, help="print identity and destination info and exit")
+        parser.add_argument("-l", '--listen', action='store_true', default=False, help="listen for incoming commands")
+        parser.add_argument('-i', metavar="identity", action='store', dest="identity", default=None, help="path to identity to use", type=str)
+        parser.add_argument("-x", '--interactive', action='store_true', default=False, help="enter interactive mode")
+        parser.add_argument("-b", '--no-announce', action='store_true', default=False, help="don't announce at program start")
+        parser.add_argument('-a', metavar="allowed_hash", dest="allowed", action='append', help="accept from this identity", type=str)
+        parser.add_argument('-n', '--noauth', action='store_true', default=False, help="accept files from anyone")
+        parser.add_argument('-N', '--noid', action='store_true', default=False, help="don't identify to listener")
+        parser.add_argument("-d", '--detailed', action='store_true', default=False, help="show detailed result output")
+        parser.add_argument("-m", action='store_true', dest="mirror", default=False, help="mirror exit code of remote command")
+        parser.add_argument("-w", action="store", metavar="seconds", type=float, help="connect and request timeout before giving up", default=RNS.Transport.PATH_REQUEST_TIMEOUT)
+        parser.add_argument("-W", action="store", metavar="seconds", type=float, help="max result download time", default=None)
+        parser.add_argument("--stdin", action='store', default=None, help="pass input to stdin", type=str)
+        parser.add_argument("--stdout", action='store', default=None, help="max size in bytes of returned stdout", type=int)
+        parser.add_argument("--stderr", action='store', default=None, help="max size in bytes of returned stderr", type=int)
+        parser.add_argument("--version", action="version", version="rnx {version}".format(version=__version__))
+        
+        args = parser.parse_args()
+
+        if args.listen or args.print_identity:
+            listen(
+                configdir = args.config,
+                identitypath = args.identity,
+                verbosity=args.verbose,
+                quietness=args.quiet,
+                allowed = args.allowed,
+                print_identity=args.print_identity,
+                disable_auth=args.noauth,
+                disable_announce=args.no_announce,
+            )
+
+        elif args.destination != None and args.command != None:
+            execute(
+                configdir = args.config,
+                identitypath = args.identity,
+                verbosity = args.verbose,
+                quietness = args.quiet,
+                detailed = args.detailed,
+                mirror = args.mirror,
+                noid = args.noid,
+                destination = args.destination,
+                command = args.command,
+                stdin = args.stdin,
+                stdoutl = args.stdout,
+                stderrl = args.stderr,
+                timeout = args.w,
+                result_timeout = args.W,
+                interactive = args.interactive,
+            )
+
+        if args.destination != None and args.interactive:
+            # command_history_max = 5000
+            # command_history = []
+            # command_current = ""
+            # history_idx = 0
+            # tty.setcbreak(sys.stdin.fileno())
+
+            code = None
+            while True:
+                try:
+                    cstr = str(code) if code and code != 0 else ""
+                    prompt = cstr+"> "
+                    print(prompt,end="")
+
+                    # cmdbuf = b""
+                    # while True:
+                    #     ch = sys.stdin.read(1)
+                    #     cmdbuf += ch.encode("utf-8")
+                    #     print("\r"+prompt+cmdbuf.decode("utf-8"), end="")    
+                    
+                    command = input()
+                    if command.lower() == "exit" or command.lower() == "quit":
+                        exit(0)
+
+                except KeyboardInterrupt:
+                    exit(0)
+                except EOFError:
+                    exit(0)
+
+                if command.lower() == "clear":
+                    print('\033c', end='')
+
+                # command_history.append(command)
+                # while len(command_history) > command_history_max:
+                #     command_history.pop(0)
+
+                else:
+                    code = execute(
+                        configdir = args.config,
+                        identitypath = args.identity,
+                        verbosity = args.verbose,
+                        quietness = args.quiet,
+                        detailed = args.detailed,
+                        mirror = args.mirror,
+                        noid = args.noid,
+                        destination = args.destination,
+                        command = command,
+                        stdin = None,
+                        stdoutl = args.stdout,
+                        stderrl = args.stderr,
+                        timeout = args.w,
+                        result_timeout = args.W,
+                        interactive = True,
+                    )
+
+        else:
+            print("")
+            parser.print_help()
+            print("")
+
+    except KeyboardInterrupt:
+        # tty.setnocbreak(sys.stdin.fileno())
+        print("")
+        if link != None:
+            link.teardown()
+        exit()
+
+def size_str(num, suffix='B'):
+    units = ['','K','M','G','T','P','E','Z']
+    last_unit = 'Y'
+
+    if suffix == 'b':
+        num *= 8
+        units = ['','K','M','G','T','P','E','Z']
+        last_unit = 'Y'
+
+    for unit in units:
+        if abs(num) < 1000.0:
+            if unit == "":
+                return "%.0f %s%s" % (num, unit, suffix)
+            else:
+                return "%.2f %s%s" % (num, unit, suffix)
+        num /= 1000.0
+
+    return "%.2f%s%s" % (num, last_unit, suffix)
+
+def pretty_time(time, verbose=False):
+    days = int(time // (24 * 3600))
+    time = time % (24 * 3600)
+    hours = int(time // 3600)
+    time %= 3600
+    minutes = int(time // 60)
+    time %= 60
+    seconds = round(time, 2)
+    
+    ss = "" if seconds == 1 else "s"
+    sm = "" if minutes == 1 else "s"
+    sh = "" if hours == 1 else "s"
+    sd = "" if days == 1 else "s"
+
+    components = []
+    if days > 0:
+        components.append(str(days)+" day"+sd if verbose else str(days)+"d")
+
+    if hours > 0:
+        components.append(str(hours)+" hour"+sh if verbose else str(hours)+"h")
+
+    if minutes > 0:
+        components.append(str(minutes)+" minute"+sm if verbose else str(minutes)+"m")
+
+    if seconds > 0:
+        components.append(str(seconds)+" second"+ss if verbose else str(seconds)+"s")
+
+    i = 0
+    tstr = ""
+    for c in components:
+        i += 1
+        if i == 1:
+            pass
+        elif i < len(components):
+            tstr += ", "
+        elif i == len(components):
+            tstr += " and "
+
+        tstr += c
+
+    return tstr
+
+if __name__ == "__main__":
+    main()

RNS/__init__.py

@@ -1,17 +1,44 @@
+# MIT License
+#
+# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 import os
 import sys
 import glob
 import time
 import random
+import threading
+
+from ._version import __version__
 
 from .Reticulum import Reticulum
 from .Identity import Identity
-from .Link import Link
+from .Link import Link, RequestReceipt
 from .Transport import Transport
 from .Destination import Destination
 from .Packet import Packet
 from .Packet import PacketReceipt
-from .Resource import Resource
+from .Resource import Resource, ResourceAdvertisement
+from .Cryptography import HKDF
+from .Cryptography import Hashes
 
 modules = glob.glob(os.path.dirname(__file__)+"/*.py")
 __all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]
@@ -25,65 +52,180 @@ LOG_VERBOSE  = 5
 LOG_DEBUG    = 6
 LOG_EXTREME  = 7
 
-LOG_STDOUT	 = 0x91
+LOG_STDOUT   = 0x91
 LOG_FILE     = 0x92
 
-loglevel 	 = LOG_NOTICE
+LOG_MAXSIZE  = 5*1024*1024
+
+loglevel     = LOG_NOTICE
 logfile      = None
 logdest      = LOG_STDOUT
 logtimefmt   = "%Y-%m-%d %H:%M:%S"
 
-random.seed(os.urandom(10))
+instance_random = random.Random()
+instance_random.seed(os.urandom(10))
+
+_always_override_destination = False
+
+logging_lock = threading.Lock()
 
 def loglevelname(level):
-	if (level == LOG_CRITICAL):
-		return "Critical"
-	if (level == LOG_ERROR):
-		return "Error"
-	if (level == LOG_WARNING):
-		return "Warning"
-	if (level == LOG_NOTICE):
-		return "Notice"
-	if (level == LOG_INFO):
-		return "Info"
-	if (level == LOG_VERBOSE):
-		return "Verbose"
-	if (level == LOG_DEBUG):
-		return "Debug"
-	if (level == LOG_EXTREME):
-		return "Extra"
-	
-	return "Unknown"
+    if (level == LOG_CRITICAL):
+        return "Critical"
+    if (level == LOG_ERROR):
+        return "Error"
+    if (level == LOG_WARNING):
+        return "Warning"
+    if (level == LOG_NOTICE):
+        return "Notice"
+    if (level == LOG_INFO):
+        return "Info"
+    if (level == LOG_VERBOSE):
+        return "Verbose"
+    if (level == LOG_DEBUG):
+        return "Debug"
+    if (level == LOG_EXTREME):
+        return "Extra"
+    
+    return "Unknown"
 
-def log(msg, level=3):
-	# TODO: not thread safe
-	if loglevel >= level:
-		timestamp = time.time()
-		logstring = "["+time.strftime(logtimefmt)+"] ["+loglevelname(level)+"] "+msg
+def version():
+    return __version__
 
-		if (logdest == LOG_STDOUT):
-			print(logstring)
+def host_os():
+    from .vendor.platformutils import get_platform
+    return get_platform()
 
-		if (logdest == LOG_FILE and logfile != None):
-			file = open(logfile, "a")
-			file.write(logstring+"\n")
-			file.close()
+def timestamp_str(time_s):
+    timestamp = time.localtime(time_s)
+    return time.strftime(logtimefmt, timestamp)
+
+def log(msg, level=3, _override_destination = False):
+    global _always_override_destination
+    
+    if loglevel >= level:
+        logstring = "["+timestamp_str(time.time())+"] ["+loglevelname(level)+"] "+msg
+        logging_lock.acquire()
+
+        if (logdest == LOG_STDOUT or _always_override_destination or _override_destination):
+            print(logstring)
+            logging_lock.release()
+
+        elif (logdest == LOG_FILE and logfile != None):
+            try:
+                file = open(logfile, "a")
+                file.write(logstring+"\n")
+                file.close()
+                
+                if os.path.getsize(logfile) > LOG_MAXSIZE:
+                    prevfile = logfile+".1"
+                    if os.path.isfile(prevfile):
+                        os.unlink(prevfile)
+                    os.rename(logfile, prevfile)
+
+                logging_lock.release()
+            except Exception as e:
+                logging_lock.release()
+                _always_override_destination = True
+                log("Exception occurred while writing log message to log file: "+str(e), LOG_CRITICAL)
+                log("Dumping future log events to console!", LOG_CRITICAL)
+                log(msg, level)
+                
 
 def rand():
-	result = random.random()
-	return result
+    result = instance_random.random()
+    return result
 
 def hexrep(data, delimit=True):
-	delimiter = ":"
-	if not delimit:
-		delimiter = ""
-	hexrep = delimiter.join("{:02x}".format(c) for c in data)
-	return hexrep
+    try:
+        iter(data)
+    except TypeError:
+        data = [data]
+        
+    delimiter = ":"
+    if not delimit:
+        delimiter = ""
+    hexrep = delimiter.join("{:02x}".format(c) for c in data)
+    return hexrep
 
 def prettyhexrep(data):
-	delimiter = ""
-	hexrep = "<"+delimiter.join("{:02x}".format(c) for c in data)+">"
-	return hexrep
+    delimiter = ""
+    hexrep = "<"+delimiter.join("{:02x}".format(c) for c in data)+">"
+    return hexrep
+
+def prettysize(num, suffix='B'):
+    units = ['','K','M','G','T','P','E','Z']
+    last_unit = 'Y'
+
+    if suffix == 'b':
+        num *= 8
+        units = ['','K','M','G','T','P','E','Z']
+        last_unit = 'Y'
+
+    for unit in units:
+        if abs(num) < 1000.0:
+            if unit == "":
+                return "%.0f %s%s" % (num, unit, suffix)
+            else:
+                return "%.2f %s%s" % (num, unit, suffix)
+        num /= 1000.0
+
+    return "%.2f%s%s" % (num, last_unit, suffix)
+
+def prettytime(time, verbose=False):
+    days = int(time // (24 * 3600))
+    time = time % (24 * 3600)
+    hours = int(time // 3600)
+    time %= 3600
+    minutes = int(time // 60)
+    time %= 60
+    seconds = round(time, 2)
+    
+    ss = "" if seconds == 1 else "s"
+    sm = "" if minutes == 1 else "s"
+    sh = "" if hours == 1 else "s"
+    sd = "" if days == 1 else "s"
+
+    components = []
+    if days > 0:
+        components.append(str(days)+" day"+sd if verbose else str(days)+"d")
+
+    if hours > 0:
+        components.append(str(hours)+" hour"+sh if verbose else str(hours)+"h")
+
+    if minutes > 0:
+        components.append(str(minutes)+" minute"+sm if verbose else str(minutes)+"m")
+
+    if seconds > 0:
+        components.append(str(seconds)+" second"+ss if verbose else str(seconds)+"s")
+
+    i = 0
+    tstr = ""
+    for c in components:
+        i += 1
+        if i == 1:
+            pass
+        elif i < len(components):
+            tstr += ", "
+        elif i == len(components):
+            tstr += " and "
+
+        tstr += c
+
+    return tstr
+
+def phyparams():
+    print("Required Physical Layer MTU : "+str(Reticulum.MTU)+" bytes")
+    print("Plaintext Packet MDU        : "+str(Packet.PLAIN_MDU)+" bytes")
+    print("Encrypted Packet MDU        : "+str(Packet.ENCRYPTED_MDU)+" bytes")
+    print("Link Curve                  : "+str(Link.CURVE))
+    print("Link Packet MDU             : "+str(Packet.ENCRYPTED_MDU)+" bytes")
+    print("Link Public Key Size        : "+str(Link.ECPUBSIZE*8)+" bits")
+    print("Link Private Key Size       : "+str(Link.KEYSIZE*8)+" bits")
 
 def panic():
-	os._exit(255)
+    os._exit(255)
+
+def exit():
+    print("")
+    sys.exit(0)

RNS/_version.py

@@ -0,0 +1 @@
+__version__ = "0.3.13"

RNS/vendor/configobj.py

@@ -19,7 +19,7 @@ import sys
 
 from codecs import BOM_UTF8, BOM_UTF16, BOM_UTF16_BE, BOM_UTF16_LE
 
-import six
+import RNS.vendor.six as six
 __version__ = '5.0.6'
 
 # imported lazily to avoid startup performance hit if it isn't used

RNS/vendor/i2plib/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2018 Viktor Villainov
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

RNS/vendor/i2plib/__init__.py

@@ -0,0 +1,25 @@
+"""
+A modern asynchronous library for building I2P applications. 
+"""
+
+from .__version__ import (
+    __title__, __description__, __url__, __version__,
+    __author__, __author_email__, __license__, __copyright__
+)
+
+from .sam import Destination, PrivateKey
+
+from .aiosam import (
+    get_sam_socket, dest_lookup, new_destination, 
+    create_session, stream_connect, stream_accept,
+    Session, StreamConnection, StreamAcceptor
+)
+
+from .tunnel import ClientTunnel, ServerTunnel
+
+from .utils import get_sam_address
+
+from .exceptions import (
+     CantReachPeer, DuplicatedDest, DuplicatedId, I2PError,
+     InvalidId, InvalidKey, KeyNotFound, PeerNotFound, Timeout,
+)

RNS/vendor/i2plib/__version__.py

@@ -0,0 +1,8 @@
+__title__ = 'i2plib'
+__description__ = 'A modern asynchronous library for building I2P applications.'
+__url__ = 'https://github.com/l-n-s/i2plib'
+__version__ = '0.0.14'
+__author__ = 'Viktor Villainov'
+__author_email__ = 'supervillain@riseup.net'
+__license__ = 'MIT'
+__copyright__ = 'Copyright 2018 Viktor Villainov'

RNS/vendor/i2plib/aiosam.py

@@ -0,0 +1,258 @@
+import asyncio
+
+from . import sam
+from . import exceptions
+from . import utils
+from .log import logger
+
+def parse_reply(data):
+    if not data:
+        raise ConnectionAbortedError("Empty response: SAM API went offline")
+
+    try:
+        msg = sam.Message(data.decode().strip())
+        logger.debug("SAM reply: "+str(msg))
+    except:
+        raise ConnectionAbortedError("Invalid SAM response")
+
+    return msg
+
+
+async def get_sam_socket(sam_address=sam.DEFAULT_ADDRESS, loop=None):
+    """A couroutine used to create a new SAM socket.
+
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) event loop instance
+    :return: A (reader, writer) pair
+    """
+    reader, writer = await asyncio.open_connection(*sam_address)
+    writer.write(sam.hello("3.1", "3.1"))
+    reply = parse_reply(await reader.readline())
+    if reply.ok:
+        return (reader, writer)
+    else:
+        writer.close()
+        raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
+
+async def dest_lookup(domain, sam_address=sam.DEFAULT_ADDRESS, 
+                      loop=None):
+    """A coroutine used to lookup a full I2P destination by .i2p domain or 
+    .b32.i2p address.
+
+    :param domain: Address to be resolved, can be a .i2p domain or a .b32.i2p 
+                   address.
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) Event loop instance
+    :return: An instance of :class:`Destination`
+    """
+    reader, writer = await get_sam_socket(sam_address, loop)
+    writer.write(sam.naming_lookup(domain))
+    reply = parse_reply(await reader.readline())
+    writer.close()
+    if reply.ok:
+        return sam.Destination(reply["VALUE"])
+    else:
+        raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
+
+async def new_destination(sam_address=sam.DEFAULT_ADDRESS, loop=None,
+                      sig_type=sam.Destination.default_sig_type):
+    """A coroutine used to generate a new destination with a private key of a 
+    chosen signature type.
+
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) Event loop instance
+    :param sig_type: (optional) Signature type
+    :return: An instance of :class:`Destination`
+    """
+    reader, writer = await get_sam_socket(sam_address, loop)
+    writer.write(sam.dest_generate(sig_type))
+    reply = parse_reply(await reader.readline())
+    writer.close()
+    return sam.Destination(reply["PRIV"], has_private_key=True)
+
+async def create_session(session_name, sam_address=sam.DEFAULT_ADDRESS, 
+                         loop=None, style="STREAM",
+                         signature_type=sam.Destination.default_sig_type,
+                         destination=None, options={}):
+    """A coroutine used to create a new SAM session.
+
+    :param session_name: Session nick name
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) Event loop instance
+    :param style: (optional) Session style, can be STREAM, DATAGRAM, RAW
+    :param signature_type: (optional) If the destination is TRANSIENT, this 
+                        signature type is used
+    :param destination: (optional) Destination to use in this session. Can be 
+                        a base64 encoded string, :class:`Destination` 
+                        instance or None. TRANSIENT destination is used when it
+                        is None.
+    :param options: (optional) A dict object with i2cp options
+    :return: A (reader, writer) pair
+    """
+    logger.debug("Creating session {}".format(session_name))
+    if destination:
+        if type(destination) == sam.Destination:
+            destination = destination
+        else:
+            destination = sam.Destination(
+                    destination, has_private_key=True)
+
+        dest_string = destination.private_key.base64
+    else:
+        dest_string = sam.TRANSIENT_DESTINATION
+
+    options = " ".join(["{}={}".format(k, v) for k, v in options.items()])
+
+    reader, writer = await get_sam_socket(sam_address, loop)
+    writer.write(sam.session_create(
+            style, session_name, dest_string, options))
+
+    reply = parse_reply(await reader.readline())
+    if reply.ok:
+        if not destination:
+            destination = sam.Destination(
+                    reply["DESTINATION"], has_private_key=True) 
+        logger.debug(destination.base32)
+        logger.debug("Session created {}".format(session_name))
+        return (reader, writer)
+    else:
+        writer.close()
+        raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
+
+async def stream_connect(session_name, destination, 
+                         sam_address=sam.DEFAULT_ADDRESS, loop=None):
+    """A coroutine used to connect to a remote I2P destination.
+
+    :param session_name: Session nick name
+    :param destination: I2P destination to connect to
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) Event loop instance
+    :return: A (reader, writer) pair
+    """
+    logger.debug("Connecting stream {}".format(session_name))
+    if isinstance(destination, str) and not destination.endswith(".i2p"):
+        destination = sam.Destination(destination)
+    elif isinstance(destination, str):
+        destination = await dest_lookup(destination, sam_address, loop)
+
+    reader, writer = await get_sam_socket(sam_address, loop)
+    writer.write(sam.stream_connect(session_name, destination.base64,
+                                           silent="false"))
+    reply = parse_reply(await reader.readline())
+    if reply.ok:
+        logger.debug("Stream connected {}".format(session_name))
+        return (reader, writer)
+    else:
+        writer.close()
+        raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
+
+async def stream_accept(session_name, sam_address=sam.DEFAULT_ADDRESS,
+                        loop=None):
+    """A coroutine used to accept a connection from the I2P network.
+
+    :param session_name: Session nick name
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) Event loop instance
+    :return: A (reader, writer) pair
+    """
+    reader, writer = await get_sam_socket(sam_address, loop)
+    writer.write(sam.stream_accept(session_name, silent="false"))
+    reply = parse_reply(await reader.readline())
+    if reply.ok:
+        return (reader, writer)
+    else:
+        writer.close()
+        raise exceptions.SAM_EXCEPTIONS[reply["RESULT"]]()
+
+### Context managers
+
+class Session:
+    """Async SAM session context manager.
+
+    :param session_name: Session nick name
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) Event loop instance
+    :param style: (optional) Session style, can be STREAM, DATAGRAM, RAW
+    :param signature_type: (optional) If the destination is TRANSIENT, this 
+                        signature type is used
+    :param destination: (optional) Destination to use in this session. Can be 
+                        a base64 encoded string, :class:`Destination` 
+                        instance or None. TRANSIENT destination is used when it
+                        is None.
+    :param options: (optional) A dict object with i2cp options
+    :return: :class:`Session` object
+    """
+    def __init__(self, session_name, sam_address=sam.DEFAULT_ADDRESS, 
+                         loop=None, style="STREAM",
+                         signature_type=sam.Destination.default_sig_type,
+                         destination=None, options={}):
+        self.session_name = session_name
+        self.sam_address = sam_address
+        self.loop = loop
+        self.style = style
+        self.signature_type = signature_type
+        self.destination = destination
+        self.options = options
+
+    async def __aenter__(self):
+        self.reader, self.writer = await create_session(self.session_name, 
+                sam_address=self.sam_address, loop=self.loop, style=self.style, 
+                signature_type=self.signature_type, 
+                destination=self.destination, options=self.options)
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        ### TODO handle exceptions
+        self.writer.close()
+
+class StreamConnection:
+    """Async stream connection context manager.
+
+    :param session_name: Session nick name
+    :param destination: I2P destination to connect to
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) Event loop instance
+    :return: :class:`StreamConnection` object
+    """
+    def __init__(self, session_name, destination, 
+                 sam_address=sam.DEFAULT_ADDRESS, loop=None):
+        self.session_name = session_name
+        self.sam_address = sam_address
+        self.loop = loop
+        self.destination = destination
+
+    async def __aenter__(self):
+        self.reader, self.writer = await stream_connect(self.session_name, 
+                self.destination, sam_address=self.sam_address, loop=self.loop)
+        self.read = self.reader.read
+        self.write = self.writer.write
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        ### TODO handle exceptions
+        self.writer.close()
+
+class StreamAcceptor:
+    """Async stream acceptor context manager.
+
+    :param session_name: Session nick name
+    :param sam_address: (optional) SAM API address
+    :param loop: (optional) Event loop instance
+    :return: :class:`StreamAcceptor` object
+    """
+    def __init__(self, session_name, sam_address=sam.DEFAULT_ADDRESS, 
+                 loop=None):
+        self.session_name = session_name
+        self.sam_address = sam_address
+        self.loop = loop
+
+    async def __aenter__(self):
+        self.reader, self.writer = await stream_accept(self.session_name, 
+                        sam_address=self.sam_address, loop=self.loop)
+        self.read = self.reader.read
+        self.write = self.writer.write
+        return self
+
+    async def __aexit__(self, exc_type, exc, tb):
+        ### TODO handle exceptions
+        self.writer.close()

RNS/vendor/i2plib/exceptions.py

@@ -0,0 +1,44 @@
+# SAM exceptions
+
+class SAMException(IOError):
+    """Base class for SAM exceptions"""
+
+class CantReachPeer(SAMException):
+    """The peer exists, but cannot be reached"""
+
+class DuplicatedDest(SAMException):
+    """The specified Destination is already in use"""
+
+class DuplicatedId(SAMException):
+    """The nickname is already associated with a session"""
+
+class I2PError(SAMException):
+    """A generic I2P error"""
+
+class InvalidId(SAMException):
+    """STREAM SESSION ID doesn't exist"""
+
+class InvalidKey(SAMException):
+    """The specified key is not valid (bad format, etc.)"""
+
+class KeyNotFound(SAMException):
+    """The naming system can't resolve the given name"""
+
+class PeerNotFound(SAMException):
+    """The peer cannot be found on the network"""
+
+class Timeout(SAMException):
+    """The peer cannot be found on the network"""
+
+SAM_EXCEPTIONS = {
+    "CANT_REACH_PEER": CantReachPeer,
+    "DUPLICATED_DEST": DuplicatedDest,
+    "DUPLICATED_ID": DuplicatedId,
+    "I2P_ERROR": I2PError,
+    "INVALID_ID": InvalidId,
+    "INVALID_KEY": InvalidKey,
+    "KEY_NOT_FOUND": KeyNotFound,
+    "PEER_NOT_FOUND": PeerNotFound,
+    "TIMEOUT": Timeout,
+}
+

RNS/vendor/i2plib/log.py

@@ -0,0 +1,5 @@
+"""Logging configuration."""
+import logging
+
+# Name the logger after the package.
+logger = logging.getLogger(__package__)

RNS/vendor/i2plib/sam.py

@@ -0,0 +1,147 @@
+from base64 import b64decode, b64encode, b32encode
+from hashlib import sha256
+import struct
+import re
+
+
+I2P_B64_CHARS = "-~"
+
+def i2p_b64encode(x):
+    """Encode I2P destination"""
+    return b64encode(x, altchars=I2P_B64_CHARS.encode()).decode() 
+
+def i2p_b64decode(x):
+    """Decode I2P destination"""
+    return b64decode(x, altchars=I2P_B64_CHARS, validate=True)
+
+SAM_BUFSIZE = 4096
+DEFAULT_ADDRESS = ("127.0.0.1", 7656)
+DEFAULT_MIN_VER = "3.1"
+DEFAULT_MAX_VER = "3.1"
+TRANSIENT_DESTINATION = "TRANSIENT"
+
+VALID_BASE32_ADDRESS = re.compile(r"^([a-zA-Z0-9]{52}).b32.i2p$")
+VALID_BASE64_ADDRESS = re.compile(r"^([a-zA-Z0-9-~=]{516,528})$")
+
+class Message(object):
+    """Parse SAM message to an object"""
+    def __init__(self, s):
+        self.opts = {}
+        if type(s) != str:
+            self._reply_string = s.decode().strip()
+        else:
+            self._reply_string = s
+
+        self.cmd, self.action, opts = self._reply_string.split(" ", 2)
+        for v in opts.split(" "):
+            data = v.split("=", 1) if "=" in v else (v, True)
+            self.opts[data[0]] = data[1]
+
+    def __getitem__(self, key):
+        return self.opts[key]
+
+    @property
+    def ok(self):
+        return self["RESULT"] == "OK"
+
+    def __repr__(self):
+        return self._reply_string
+
+
+# SAM request messages
+
+def hello(min_version, max_version):
+    return "HELLO VERSION MIN={} MAX={}\n".format(min_version, 
+            max_version).encode()
+
+def session_create(style, session_id, destination, options=""):
+    return "SESSION CREATE STYLE={} ID={} DESTINATION={} {}\n".format(
+            style, session_id, destination, options).encode()
+
+
+def stream_connect(session_id, destination, silent="false"):
+    return "STREAM CONNECT ID={} DESTINATION={} SILENT={}\n".format(
+            session_id, destination, silent).encode()
+
+def stream_accept(session_id, silent="false"):
+    return "STREAM ACCEPT ID={} SILENT={}\n".format(session_id, silent).encode()
+
+def stream_forward(session_id, port, options=""):
+    return "STREAM FORWARD ID={} PORT={} {}\n".format(
+            session_id, port, options).encode()
+
+
+
+def naming_lookup(name):
+    return "NAMING LOOKUP NAME={}\n".format(name).encode()
+
+def dest_generate(signature_type):
+    return "DEST GENERATE SIGNATURE_TYPE={}\n".format(signature_type).encode()
+
+class Destination(object):
+    """I2P destination
+
+    https://geti2p.net/spec/common-structures#destination
+
+    :param data: (optional) Base64 encoded data or binary data 
+    :param path: (optional) A path to a file with binary data 
+    :param has_private_key: (optional) Does data have a private key? 
+    """
+
+    ECDSA_SHA256_P256 = 1
+    ECDSA_SHA384_P384 = 2
+    ECDSA_SHA512_P521 = 3
+    EdDSA_SHA512_Ed25519 = 7
+
+    default_sig_type = EdDSA_SHA512_Ed25519
+
+    _pubkey_size = 256
+    _signkey_size = 128
+    _min_cert_size = 3
+
+    def __init__(self, data=None, path=None, has_private_key=False):
+        #: Binary destination
+        self.data = bytes() 
+        #: Base64 encoded destination
+        self.base64 = ""    
+        #: :class:`RNS.vendor.i2plib.PrivateKey` instance or None
+        self.private_key = None    
+        
+        if path:
+            with open(path, "rb") as f: data = f.read()
+
+        if data and has_private_key:
+            self.private_key = PrivateKey(data)
+
+            cert_len = struct.unpack("!H", self.private_key.data[385:387])[0]
+            data = self.private_key.data[:387+cert_len]
+
+        if not data:
+            raise Exception("Can't create a destination with no data")
+
+        self.data = data if type(data) == bytes else i2p_b64decode(data)
+        self.base64 = data if type(data) == str else i2p_b64encode(data)
+
+    def __repr__(self):
+        return "<Destination: {}>".format(self.base32)
+
+    @property
+    def base32(self):
+        """Base32 destination hash of this destination"""
+        desthash = sha256(self.data).digest()
+        return b32encode(desthash).decode()[:52].lower()
+    
+class PrivateKey(object):
+    """I2P private key
+
+    https://geti2p.net/spec/common-structures#keysandcert
+
+    :param data: Base64 encoded data or binary data 
+    """
+
+    def __init__(self, data):
+        #: Binary private key
+        self.data = data if type(data) == bytes else i2p_b64decode(data)
+        #: Base64 encoded private key
+        self.base64 = data if type(data) == str else i2p_b64encode(data)
+

RNS/vendor/i2plib/tunnel.py

@@ -0,0 +1,234 @@
+import logging
+import asyncio
+import argparse
+
+from . import sam 
+from . import aiosam
+from . import utils
+from .log import logger
+
+BUFFER_SIZE = 65536
+
+async def proxy_data(reader, writer):
+    """Proxy data from reader to writer"""
+    try:
+        while True:
+            data = await reader.read(BUFFER_SIZE)
+            if not data:
+                break
+            writer.write(data)
+    except Exception as e:
+        logger.debug('proxy_data_task exception {}'.format(e))
+    finally:
+        try:
+            writer.close()
+        except RuntimeError:
+            pass
+        logger.debug('close connection')
+
+class I2PTunnel(object):
+    """Base I2P Tunnel object, not to be used directly
+
+    :param local_address: A local address to use for a tunnel. 
+                          E.g. ("127.0.0.1", 6668)
+    :param destination: (optional) Destination to use for this tunnel. Can be 
+                        a base64 encoded string, :class:`Destination`
+                        instance or None. A new destination is created when it
+                        is None.
+    :param session_name: (optional) Session nick name. A new session nickname is
+                        generated if not specified.
+    :param options: (optional) A dict object with i2cp options
+    :param loop: (optional) Event loop instance
+    :param sam_address: (optional) SAM API address
+    """
+
+    def __init__(self, local_address, destination=None, session_name=None, 
+                 options={}, loop=None, sam_address=sam.DEFAULT_ADDRESS):
+        self.local_address = local_address
+        self.destination = destination
+        self.session_name = session_name or utils.generate_session_id()
+        self.options = options
+        self.loop = loop
+        self.sam_address = sam_address
+
+    async def _pre_run(self):
+        if not self.destination:
+            self.destination = await aiosam.new_destination(
+                sam_address=self.sam_address, loop=self.loop)
+        _, self.session_writer = await aiosam.create_session(
+                self.session_name, style=self.style, options=self.options,
+                sam_address=self.sam_address, 
+                loop=self.loop, destination=self.destination)
+
+    def stop(self):
+        """Stop the tunnel"""
+        self.session_writer.close()
+
+class ClientTunnel(I2PTunnel):
+    """Client tunnel, a subclass of tunnel.I2PTunnel
+
+    If you run a client tunnel with a local address ("127.0.0.1", 6668) and
+    a remote destination "irc.echelon.i2p", all connections to 127.0.0.1:6668 
+    will be proxied to irc.echelon.i2p.
+
+    :param remote_destination: Remote I2P destination, can be either .i2p 
+                        domain, .b32.i2p address, base64 destination or 
+                        :class:`Destination` instance
+    """
+
+    def __init__(self, remote_destination, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.style = "STREAM"
+        self.remote_destination = remote_destination
+
+    async def run(self):
+        """A coroutine used to run the tunnel"""
+        await self._pre_run()
+
+        self.status = { "setup_ran": False, "setup_failed": False, "exception": None, "connect_tasks": [] }
+        async def handle_client(client_reader, client_writer):
+            """Handle local client connection"""
+            try:
+                sc_task = aiosam.stream_connect(
+                        self.session_name, self.remote_destination, 
+                        sam_address=self.sam_address, loop=self.loop)
+                self.status["connect_tasks"].append(sc_task)
+                
+                remote_reader, remote_writer = await sc_task
+                asyncio.ensure_future(proxy_data(remote_reader, client_writer), 
+                                      loop=self.loop)
+                asyncio.ensure_future(proxy_data(client_reader, remote_writer),
+                                      loop=self.loop)
+
+            except Exception as e:
+                self.status["setup_ran"] = True
+                self.status["setup_failed"] = True
+                self.status["exception"] = e
+
+
+        try:
+            self.server = await asyncio.start_server(handle_client, *self.local_address)
+            self.status["setup_ran"] = True
+
+        except Exception as e:
+            self.status["setup_ran"] = True
+            self.status["setup_failed"] = True
+            self.status["exception"] = e
+
+    def stop(self):
+        super().stop()
+        self.server.close()
+
+class ServerTunnel(I2PTunnel):
+    """Server tunnel, a subclass of tunnel.I2PTunnel
+
+    If you want to expose a local service 127.0.0.1:80 to the I2P network, run
+    a server tunnel with a local address ("127.0.0.1", 80). If you don't 
+    provide a private key or a session name, it will use a TRANSIENT 
+    destination.
+    """
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.style = "STREAM"
+
+    async def run(self):
+        """A coroutine used to run the tunnel"""
+        await self._pre_run()
+
+        self.status = { "setup_ran": False, "setup_failed": False, "exception": None, "connect_tasks": [] }
+        async def handle_client(incoming, client_reader, client_writer):
+            try:
+                # data and dest may come in one chunk
+                dest, data = incoming.split(b"\n", 1) 
+                remote_destination = sam.Destination(dest.decode())
+                logger.debug("{} client connected: {}.b32.i2p".format(
+                    self.session_name, remote_destination.base32))
+            
+            except Exception as e:
+                self.status["exception"] = e
+                self.status["setup_failed"] = True
+                data = None
+
+            try:
+                sc_task = asyncio.wait_for(
+                        asyncio.open_connection(
+                           host=self.local_address[0], 
+                           port=self.local_address[1]),
+                        timeout=5)
+                self.status["connect_tasks"].append(sc_task)
+
+                remote_reader, remote_writer = await sc_task
+                if data: remote_writer.write(data)
+                asyncio.ensure_future(proxy_data(remote_reader, client_writer),
+                                      loop=self.loop)
+                asyncio.ensure_future(proxy_data(client_reader, remote_writer),
+                                      loop=self.loop)
+
+            except ConnectionRefusedError:
+                client_writer.close()
+                self.status["exception"] = e
+                self.status["setup_failed"] = True
+
+        async def server_loop():
+            try:
+                while True:
+                    client_reader, client_writer = await aiosam.stream_accept(
+                            self.session_name, sam_address=self.sam_address, 
+                            loop=self.loop)
+                    incoming = await client_reader.read(BUFFER_SIZE)
+                    asyncio.ensure_future(handle_client(
+                        incoming, client_reader, client_writer), loop=self.loop)
+            except asyncio.CancelledError:
+                pass
+
+        self.server_loop = asyncio.ensure_future(server_loop(), loop=self.loop)
+        self.status["setup_ran"] = True
+
+    def stop(self):
+        super().stop()
+        self.server_loop.cancel()
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('type', metavar="TYPE", choices=('server', 'client'),
+                        help="Tunnel type (server or client)")
+    parser.add_argument('address', metavar="ADDRESS", 
+                        help="Local address (e.g. 127.0.0.1:8000)")
+    parser.add_argument('--debug', '-d', action='store_true',
+                        help='Debugging')
+    parser.add_argument('--key', '-k', default='', metavar='PRIVATE_KEY',
+                        help='Path to private key file')
+    parser.add_argument('--destination', '-D', default='', 
+                        metavar='DESTINATION', help='Remote destination')
+    args = parser.parse_args()
+
+    SAM_ADDRESS = utils.get_sam_address()
+
+    logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)
+    loop = asyncio.get_event_loop()
+    loop.set_debug(args.debug)
+
+    if args.key:
+        destination = sam.Destination(path=args.key, has_private_key=True)
+    else:
+        destination = None
+
+    local_address = utils.address_from_string(args.address)
+
+    if args.type == "client":
+        tunnel = ClientTunnel(args.destination, local_address, loop=loop, 
+                destination=destination, sam_address=SAM_ADDRESS)
+    elif args.type == "server":
+        tunnel = ServerTunnel(local_address, loop=loop, destination=destination, 
+                sam_address=SAM_ADDRESS)
+
+    asyncio.ensure_future(tunnel.run(), loop=loop)
+
+    try:
+        loop.run_forever()
+    except KeyboardInterrupt:
+        tunnel.stop()
+    finally:
+        loop.stop()
+        loop.close()

RNS/vendor/i2plib/utils.py

@@ -0,0 +1,42 @@
+import socket
+import os
+import random
+import string
+
+from . import sam
+
+def get_free_port():
+    """Get a free port on your local host"""
+    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+    s.bind(('', 0))
+    free_port = s.getsockname()[1]
+    s.close()
+    return free_port
+
+def is_address_accessible(address):
+    """Check if address is accessible or down"""
+    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    is_accessible = s.connect_ex(address) == 0
+    s.close()
+    return is_accessible
+
+def address_from_string(address_string):
+    """Address tuple from host:port string"""
+    address = address_string.split(":")
+    return (address[0], int(address[1]))
+
+def get_sam_address():
+    """
+    Get SAM address from environment variable I2P_SAM_ADDRESS, or use a default
+    value
+    """
+    value = os.getenv("I2P_SAM_ADDRESS")
+    return address_from_string(value) if value else sam.DEFAULT_ADDRESS
+
+def generate_session_id(length=6):
+    """Generate random session id"""
+    rand = random.SystemRandom()
+    sid = [rand.choice(string.ascii_letters) for _ in range(length)]
+    return "reticulum-" + "".join(sid)
+

RNS/vendor/platformutils.py

@@ -0,0 +1,45 @@
+def get_platform():
+    from os import environ
+    if "ANDROID_ARGUMENT" in environ:
+        return "android"
+    elif "ANDROID_ROOT" in environ:
+        return "android"
+    else:
+        import sys
+        return sys.platform
+
+def is_darwin():
+    if get_platform() == "darwin":
+        return True
+    else:
+        return False
+
+def is_android():
+    if get_platform() == "android":
+        return True
+    else:
+        return False
+
+def is_windows():
+    if str(get_platform()).startswith("win"):
+        return True
+    else:
+        return False
+
+def platform_checks():
+    if is_windows():
+        import sys
+        if sys.version_info.major >= 3 and sys.version_info.minor >= 8:
+            pass
+        else:
+            import RNS
+            RNS.log("On Windows, Reticulum requires Python 3.8 or higher.", RNS.LOG_ERROR)
+            RNS.log("Please update Python to run Reticulum.", RNS.LOG_ERROR)
+            RNS.panic()
+
+def cryptography_old_api():
+    import cryptography
+    if cryptography.__version__ == "2.8":
+        return True
+    else:
+        return False

RNS/vendor/six.py

@@ -0,0 +1,998 @@
+# Copyright (c) 2010-2020 Benjamin Peterson
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+"""Utilities for writing code that runs on Python 2 and 3"""
+
+from __future__ import absolute_import
+
+import functools
+import itertools
+import operator
+import sys
+import types
+
+__author__ = "Benjamin Peterson <benjamin@python.org>"
+__version__ = "1.16.0"
+
+
+# Useful for very coarse version differentiation.
+PY2 = sys.version_info[0] == 2
+PY3 = sys.version_info[0] == 3
+PY34 = sys.version_info[0:2] >= (3, 4)
+
+if PY3:
+    string_types = str,
+    integer_types = int,
+    class_types = type,
+    text_type = str
+    binary_type = bytes
+
+    MAXSIZE = sys.maxsize
+else:
+    string_types = basestring,
+    integer_types = (int, long)
+    class_types = (type, types.ClassType)
+    text_type = unicode
+    binary_type = str
+
+    if sys.platform.startswith("java"):
+        # Jython always uses 32 bits.
+        MAXSIZE = int((1 << 31) - 1)
+    else:
+        # It's possible to have sizeof(long) != sizeof(Py_ssize_t).
+        class X(object):
+
+            def __len__(self):
+                return 1 << 31
+        try:
+            len(X())
+        except OverflowError:
+            # 32-bit
+            MAXSIZE = int((1 << 31) - 1)
+        else:
+            # 64-bit
+            MAXSIZE = int((1 << 63) - 1)
+        del X
+
+if PY34:
+    from importlib.util import spec_from_loader
+else:
+    spec_from_loader = None
+
+
+def _add_doc(func, doc):
+    """Add documentation to a function."""
+    func.__doc__ = doc
+
+
+def _import_module(name):
+    """Import module, returning the module after the last dot."""
+    __import__(name)
+    return sys.modules[name]
+
+
+class _LazyDescr(object):
+
+    def __init__(self, name):
+        self.name = name
+
+    def __get__(self, obj, tp):
+        result = self._resolve()
+        setattr(obj, self.name, result)  # Invokes __set__.
+        try:
+            # This is a bit ugly, but it avoids running this again by
+            # removing this descriptor.
+            delattr(obj.__class__, self.name)
+        except AttributeError:
+            pass
+        return result
+
+
+class MovedModule(_LazyDescr):
+
+    def __init__(self, name, old, new=None):
+        super(MovedModule, self).__init__(name)
+        if PY3:
+            if new is None:
+                new = name
+            self.mod = new
+        else:
+            self.mod = old
+
+    def _resolve(self):
+        return _import_module(self.mod)
+
+    def __getattr__(self, attr):
+        _module = self._resolve()
+        value = getattr(_module, attr)
+        setattr(self, attr, value)
+        return value
+
+
+class _LazyModule(types.ModuleType):
+
+    def __init__(self, name):
+        super(_LazyModule, self).__init__(name)
+        self.__doc__ = self.__class__.__doc__
+
+    def __dir__(self):
+        attrs = ["__doc__", "__name__"]
+        attrs += [attr.name for attr in self._moved_attributes]
+        return attrs
+
+    # Subclasses should override this
+    _moved_attributes = []
+
+
+class MovedAttribute(_LazyDescr):
+
+    def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
+        super(MovedAttribute, self).__init__(name)
+        if PY3:
+            if new_mod is None:
+                new_mod = name
+            self.mod = new_mod
+            if new_attr is None:
+                if old_attr is None:
+                    new_attr = name
+                else:
+                    new_attr = old_attr
+            self.attr = new_attr
+        else:
+            self.mod = old_mod
+            if old_attr is None:
+                old_attr = name
+            self.attr = old_attr
+
+    def _resolve(self):
+        module = _import_module(self.mod)
+        return getattr(module, self.attr)
+
+
+class _SixMetaPathImporter(object):
+
+    """
+    A meta path importer to import six.moves and its submodules.
+
+    This class implements a PEP302 finder and loader. It should be compatible
+    with Python 2.5 and all existing versions of Python3
+    """
+
+    def __init__(self, six_module_name):
+        self.name = six_module_name
+        self.known_modules = {}
+
+    def _add_module(self, mod, *fullnames):
+        for fullname in fullnames:
+            self.known_modules[self.name + "." + fullname] = mod
+
+    def _get_module(self, fullname):
+        return self.known_modules[self.name + "." + fullname]
+
+    def find_module(self, fullname, path=None):
+        if fullname in self.known_modules:
+            return self
+        return None
+
+    def find_spec(self, fullname, path, target=None):
+        if fullname in self.known_modules:
+            return spec_from_loader(fullname, self)
+        return None
+
+    def __get_module(self, fullname):
+        try:
+            return self.known_modules[fullname]
+        except KeyError:
+            raise ImportError("This loader does not know module " + fullname)
+
+    def load_module(self, fullname):
+        try:
+            # in case of a reload
+            return sys.modules[fullname]
+        except KeyError:
+            pass
+        mod = self.__get_module(fullname)
+        if isinstance(mod, MovedModule):
+            mod = mod._resolve()
+        else:
+            mod.__loader__ = self
+        sys.modules[fullname] = mod
+        return mod
+
+    def is_package(self, fullname):
+        """
+        Return true, if the named module is a package.
+
+        We need this method to get correct spec objects with
+        Python 3.4 (see PEP451)
+        """
+        return hasattr(self.__get_module(fullname), "__path__")
+
+    def get_code(self, fullname):
+        """Return None
+
+        Required, if is_package is implemented"""
+        self.__get_module(fullname)  # eventually raises ImportError
+        return None
+    get_source = get_code  # same as get_code
+
+    def create_module(self, spec):
+        return self.load_module(spec.name)
+
+    def exec_module(self, module):
+        pass
+
+_importer = _SixMetaPathImporter(__name__)
+
+
+class _MovedItems(_LazyModule):
+
+    """Lazy loading of moved objects"""
+    __path__ = []  # mark as package
+
+
+_moved_attributes = [
+    MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
+    MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
+    MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"),
+    MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
+    MovedAttribute("intern", "__builtin__", "sys"),
+    MovedAttribute("map", "itertools", "builtins", "imap", "map"),
+    MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"),
+    MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"),
+    MovedAttribute("getoutput", "commands", "subprocess"),
+    MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"),
+    MovedAttribute("reload_module", "__builtin__", "importlib" if PY34 else "imp", "reload"),
+    MovedAttribute("reduce", "__builtin__", "functools"),
+    MovedAttribute("shlex_quote", "pipes", "shlex", "quote"),
+    MovedAttribute("StringIO", "StringIO", "io"),
+    MovedAttribute("UserDict", "UserDict", "collections"),
+    MovedAttribute("UserList", "UserList", "collections"),
+    MovedAttribute("UserString", "UserString", "collections"),
+    MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
+    MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
+    MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"),
+    MovedModule("builtins", "__builtin__"),
+    MovedModule("configparser", "ConfigParser"),
+    MovedModule("collections_abc", "collections", "collections.abc" if sys.version_info >= (3, 3) else "collections"),
+    MovedModule("copyreg", "copy_reg"),
+    MovedModule("dbm_gnu", "gdbm", "dbm.gnu"),
+    MovedModule("dbm_ndbm", "dbm", "dbm.ndbm"),
+    MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread" if sys.version_info < (3, 9) else "_thread"),
+    MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
+    MovedModule("http_cookies", "Cookie", "http.cookies"),
+    MovedModule("html_entities", "htmlentitydefs", "html.entities"),
+    MovedModule("html_parser", "HTMLParser", "html.parser"),
+    MovedModule("http_client", "httplib", "http.client"),
+    MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"),
+    MovedModule("email_mime_image", "email.MIMEImage", "email.mime.image"),
+    MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"),
+    MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"),
+    MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"),
+    MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
+    MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
+    MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
+    MovedModule("cPickle", "cPickle", "pickle"),
+    MovedModule("queue", "Queue"),
+    MovedModule("reprlib", "repr"),
+    MovedModule("socketserver", "SocketServer"),
+    MovedModule("_thread", "thread", "_thread"),
+    MovedModule("tkinter", "Tkinter"),
+    MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
+    MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
+    MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
+    MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
+    MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
+    MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"),
+    MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
+    MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
+    MovedModule("tkinter_colorchooser", "tkColorChooser",
+                "tkinter.colorchooser"),
+    MovedModule("tkinter_commondialog", "tkCommonDialog",
+                "tkinter.commondialog"),
+    MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
+    MovedModule("tkinter_font", "tkFont", "tkinter.font"),
+    MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
+    MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
+                "tkinter.simpledialog"),
+    MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"),
+    MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"),
+    MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"),
+    MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
+    MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
+    MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
+]
+# Add windows specific modules.
+if sys.platform == "win32":
+    _moved_attributes += [
+        MovedModule("winreg", "_winreg"),
+    ]
+
+for attr in _moved_attributes:
+    setattr(_MovedItems, attr.name, attr)
+    if isinstance(attr, MovedModule):
+        _importer._add_module(attr, "moves." + attr.name)
+del attr
+
+_MovedItems._moved_attributes = _moved_attributes
+
+moves = _MovedItems(__name__ + ".moves")
+_importer._add_module(moves, "moves")
+
+
+class Module_six_moves_urllib_parse(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_parse"""
+
+
+_urllib_parse_moved_attributes = [
+    MovedAttribute("ParseResult", "urlparse", "urllib.parse"),
+    MovedAttribute("SplitResult", "urlparse", "urllib.parse"),
+    MovedAttribute("parse_qs", "urlparse", "urllib.parse"),
+    MovedAttribute("parse_qsl", "urlparse", "urllib.parse"),
+    MovedAttribute("urldefrag", "urlparse", "urllib.parse"),
+    MovedAttribute("urljoin", "urlparse", "urllib.parse"),
+    MovedAttribute("urlparse", "urlparse", "urllib.parse"),
+    MovedAttribute("urlsplit", "urlparse", "urllib.parse"),
+    MovedAttribute("urlunparse", "urlparse", "urllib.parse"),
+    MovedAttribute("urlunsplit", "urlparse", "urllib.parse"),
+    MovedAttribute("quote", "urllib", "urllib.parse"),
+    MovedAttribute("quote_plus", "urllib", "urllib.parse"),
+    MovedAttribute("unquote", "urllib", "urllib.parse"),
+    MovedAttribute("unquote_plus", "urllib", "urllib.parse"),
+    MovedAttribute("unquote_to_bytes", "urllib", "urllib.parse", "unquote", "unquote_to_bytes"),
+    MovedAttribute("urlencode", "urllib", "urllib.parse"),
+    MovedAttribute("splitquery", "urllib", "urllib.parse"),
+    MovedAttribute("splittag", "urllib", "urllib.parse"),
+    MovedAttribute("splituser", "urllib", "urllib.parse"),
+    MovedAttribute("splitvalue", "urllib", "urllib.parse"),
+    MovedAttribute("uses_fragment", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_netloc", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_params", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_query", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_relative", "urlparse", "urllib.parse"),
+]
+for attr in _urllib_parse_moved_attributes:
+    setattr(Module_six_moves_urllib_parse, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"),
+                      "moves.urllib_parse", "moves.urllib.parse")
+
+
+class Module_six_moves_urllib_error(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_error"""
+
+
+_urllib_error_moved_attributes = [
+    MovedAttribute("URLError", "urllib2", "urllib.error"),
+    MovedAttribute("HTTPError", "urllib2", "urllib.error"),
+    MovedAttribute("ContentTooShortError", "urllib", "urllib.error"),
+]
+for attr in _urllib_error_moved_attributes:
+    setattr(Module_six_moves_urllib_error, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"),
+                      "moves.urllib_error", "moves.urllib.error")
+
+
+class Module_six_moves_urllib_request(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_request"""
+
+
+_urllib_request_moved_attributes = [
+    MovedAttribute("urlopen", "urllib2", "urllib.request"),
+    MovedAttribute("install_opener", "urllib2", "urllib.request"),
+    MovedAttribute("build_opener", "urllib2", "urllib.request"),
+    MovedAttribute("pathname2url", "urllib", "urllib.request"),
+    MovedAttribute("url2pathname", "urllib", "urllib.request"),
+    MovedAttribute("getproxies", "urllib", "urllib.request"),
+    MovedAttribute("Request", "urllib2", "urllib.request"),
+    MovedAttribute("OpenerDirector", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyHandler", "urllib2", "urllib.request"),
+    MovedAttribute("BaseHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"),
+    MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"),
+    MovedAttribute("FileHandler", "urllib2", "urllib.request"),
+    MovedAttribute("FTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"),
+    MovedAttribute("UnknownHandler", "urllib2", "urllib.request"),
+    MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"),
+    MovedAttribute("urlretrieve", "urllib", "urllib.request"),
+    MovedAttribute("urlcleanup", "urllib", "urllib.request"),
+    MovedAttribute("URLopener", "urllib", "urllib.request"),
+    MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
+    MovedAttribute("proxy_bypass", "urllib", "urllib.request"),
+    MovedAttribute("parse_http_list", "urllib2", "urllib.request"),
+    MovedAttribute("parse_keqv_list", "urllib2", "urllib.request"),
+]
+for attr in _urllib_request_moved_attributes:
+    setattr(Module_six_moves_urllib_request, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"),
+                      "moves.urllib_request", "moves.urllib.request")
+
+
+class Module_six_moves_urllib_response(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_response"""
+
+
+_urllib_response_moved_attributes = [
+    MovedAttribute("addbase", "urllib", "urllib.response"),
+    MovedAttribute("addclosehook", "urllib", "urllib.response"),
+    MovedAttribute("addinfo", "urllib", "urllib.response"),
+    MovedAttribute("addinfourl", "urllib", "urllib.response"),
+]
+for attr in _urllib_response_moved_attributes:
+    setattr(Module_six_moves_urllib_response, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"),
+                      "moves.urllib_response", "moves.urllib.response")
+
+
+class Module_six_moves_urllib_robotparser(_LazyModule):
+
+    """Lazy loading of moved objects in six.moves.urllib_robotparser"""
+
+
+_urllib_robotparser_moved_attributes = [
+    MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"),
+]
+for attr in _urllib_robotparser_moved_attributes:
+    setattr(Module_six_moves_urllib_robotparser, attr.name, attr)
+del attr
+
+Module_six_moves_urllib_robotparser._moved_attributes = _urllib_robotparser_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"),
+                      "moves.urllib_robotparser", "moves.urllib.robotparser")
+
+
+class Module_six_moves_urllib(types.ModuleType):
+
+    """Create a six.moves.urllib namespace that resembles the Python 3 namespace"""
+    __path__ = []  # mark as package
+    parse = _importer._get_module("moves.urllib_parse")
+    error = _importer._get_module("moves.urllib_error")
+    request = _importer._get_module("moves.urllib_request")
+    response = _importer._get_module("moves.urllib_response")
+    robotparser = _importer._get_module("moves.urllib_robotparser")
+
+    def __dir__(self):
+        return ['parse', 'error', 'request', 'response', 'robotparser']
+
+_importer._add_module(Module_six_moves_urllib(__name__ + ".moves.urllib"),
+                      "moves.urllib")
+
+
+def add_move(move):
+    """Add an item to six.moves."""
+    setattr(_MovedItems, move.name, move)
+
+
+def remove_move(name):
+    """Remove item from six.moves."""
+    try:
+        delattr(_MovedItems, name)
+    except AttributeError:
+        try:
+            del moves.__dict__[name]
+        except KeyError:
+            raise AttributeError("no such move, %r" % (name,))
+
+
+if PY3:
+    _meth_func = "__func__"
+    _meth_self = "__self__"
+
+    _func_closure = "__closure__"
+    _func_code = "__code__"
+    _func_defaults = "__defaults__"
+    _func_globals = "__globals__"
+else:
+    _meth_func = "im_func"
+    _meth_self = "im_self"
+
+    _func_closure = "func_closure"
+    _func_code = "func_code"
+    _func_defaults = "func_defaults"
+    _func_globals = "func_globals"
+
+
+try:
+    advance_iterator = next
+except NameError:
+    def advance_iterator(it):
+        return it.next()
+next = advance_iterator
+
+
+try:
+    callable = callable
+except NameError:
+    def callable(obj):
+        return any("__call__" in klass.__dict__ for klass in type(obj).__mro__)
+
+
+if PY3:
+    def get_unbound_function(unbound):
+        return unbound
+
+    create_bound_method = types.MethodType
+
+    def create_unbound_method(func, cls):
+        return func
+
+    Iterator = object
+else:
+    def get_unbound_function(unbound):
+        return unbound.im_func
+
+    def create_bound_method(func, obj):
+        return types.MethodType(func, obj, obj.__class__)
+
+    def create_unbound_method(func, cls):
+        return types.MethodType(func, None, cls)
+
+    class Iterator(object):
+
+        def next(self):
+            return type(self).__next__(self)
+
+    callable = callable
+_add_doc(get_unbound_function,
+         """Get the function out of a possibly unbound function""")
+
+
+get_method_function = operator.attrgetter(_meth_func)
+get_method_self = operator.attrgetter(_meth_self)
+get_function_closure = operator.attrgetter(_func_closure)
+get_function_code = operator.attrgetter(_func_code)
+get_function_defaults = operator.attrgetter(_func_defaults)
+get_function_globals = operator.attrgetter(_func_globals)
+
+
+if PY3:
+    def iterkeys(d, **kw):
+        return iter(d.keys(**kw))
+
+    def itervalues(d, **kw):
+        return iter(d.values(**kw))
+
+    def iteritems(d, **kw):
+        return iter(d.items(**kw))
+
+    def iterlists(d, **kw):
+        return iter(d.lists(**kw))
+
+    viewkeys = operator.methodcaller("keys")
+
+    viewvalues = operator.methodcaller("values")
+
+    viewitems = operator.methodcaller("items")
+else:
+    def iterkeys(d, **kw):
+        return d.iterkeys(**kw)
+
+    def itervalues(d, **kw):
+        return d.itervalues(**kw)
+
+    def iteritems(d, **kw):
+        return d.iteritems(**kw)
+
+    def iterlists(d, **kw):
+        return d.iterlists(**kw)
+
+    viewkeys = operator.methodcaller("viewkeys")
+
+    viewvalues = operator.methodcaller("viewvalues")
+
+    viewitems = operator.methodcaller("viewitems")
+
+_add_doc(iterkeys, "Return an iterator over the keys of a dictionary.")
+_add_doc(itervalues, "Return an iterator over the values of a dictionary.")
+_add_doc(iteritems,
+         "Return an iterator over the (key, value) pairs of a dictionary.")
+_add_doc(iterlists,
+         "Return an iterator over the (key, [values]) pairs of a dictionary.")
+
+
+if PY3:
+    def b(s):
+        return s.encode("latin-1")
+
+    def u(s):
+        return s
+    unichr = chr
+    import struct
+    int2byte = struct.Struct(">B").pack
+    del struct
+    byte2int = operator.itemgetter(0)
+    indexbytes = operator.getitem
+    iterbytes = iter
+    import io
+    StringIO = io.StringIO
+    BytesIO = io.BytesIO
+    del io
+    _assertCountEqual = "assertCountEqual"
+    if sys.version_info[1] <= 1:
+        _assertRaisesRegex = "assertRaisesRegexp"
+        _assertRegex = "assertRegexpMatches"
+        _assertNotRegex = "assertNotRegexpMatches"
+    else:
+        _assertRaisesRegex = "assertRaisesRegex"
+        _assertRegex = "assertRegex"
+        _assertNotRegex = "assertNotRegex"
+else:
+    def b(s):
+        return s
+    # Workaround for standalone backslash
+
+    def u(s):
+        return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
+    unichr = unichr
+    int2byte = chr
+
+    def byte2int(bs):
+        return ord(bs[0])
+
+    def indexbytes(buf, i):
+        return ord(buf[i])
+    iterbytes = functools.partial(itertools.imap, ord)
+    import StringIO
+    StringIO = BytesIO = StringIO.StringIO
+    _assertCountEqual = "assertItemsEqual"
+    _assertRaisesRegex = "assertRaisesRegexp"
+    _assertRegex = "assertRegexpMatches"
+    _assertNotRegex = "assertNotRegexpMatches"
+_add_doc(b, """Byte literal""")
+_add_doc(u, """Text literal""")
+
+
+def assertCountEqual(self, *args, **kwargs):
+    return getattr(self, _assertCountEqual)(*args, **kwargs)
+
+
+def assertRaisesRegex(self, *args, **kwargs):
+    return getattr(self, _assertRaisesRegex)(*args, **kwargs)
+
+
+def assertRegex(self, *args, **kwargs):
+    return getattr(self, _assertRegex)(*args, **kwargs)
+
+
+def assertNotRegex(self, *args, **kwargs):
+    return getattr(self, _assertNotRegex)(*args, **kwargs)
+
+
+if PY3:
+    exec_ = getattr(moves.builtins, "exec")
+
+    def reraise(tp, value, tb=None):
+        try:
+            if value is None:
+                value = tp()
+            if value.__traceback__ is not tb:
+                raise value.with_traceback(tb)
+            raise value
+        finally:
+            value = None
+            tb = None
+
+else:
+    def exec_(_code_, _globs_=None, _locs_=None):
+        """Execute code in a namespace."""
+        if _globs_ is None:
+            frame = sys._getframe(1)
+            _globs_ = frame.f_globals
+            if _locs_ is None:
+                _locs_ = frame.f_locals
+            del frame
+        elif _locs_ is None:
+            _locs_ = _globs_
+        exec("""exec _code_ in _globs_, _locs_""")
+
+    exec_("""def reraise(tp, value, tb=None):
+    try:
+        raise tp, value, tb
+    finally:
+        tb = None
+""")
+
+
+if sys.version_info[:2] > (3,):
+    exec_("""def raise_from(value, from_value):
+    try:
+        raise value from from_value
+    finally:
+        value = None
+""")
+else:
+    def raise_from(value, from_value):
+        raise value
+
+
+print_ = getattr(moves.builtins, "print", None)
+if print_ is None:
+    def print_(*args, **kwargs):
+        """The new-style print function for Python 2.4 and 2.5."""
+        fp = kwargs.pop("file", sys.stdout)
+        if fp is None:
+            return
+
+        def write(data):
+            if not isinstance(data, basestring):
+                data = str(data)
+            # If the file has an encoding, encode unicode with it.
+            if (isinstance(fp, file) and
+                    isinstance(data, unicode) and
+                    fp.encoding is not None):
+                errors = getattr(fp, "errors", None)
+                if errors is None:
+                    errors = "strict"
+                data = data.encode(fp.encoding, errors)
+            fp.write(data)
+        want_unicode = False
+        sep = kwargs.pop("sep", None)
+        if sep is not None:
+            if isinstance(sep, unicode):
+                want_unicode = True
+            elif not isinstance(sep, str):
+                raise TypeError("sep must be None or a string")
+        end = kwargs.pop("end", None)
+        if end is not None:
+            if isinstance(end, unicode):
+                want_unicode = True
+            elif not isinstance(end, str):
+                raise TypeError("end must be None or a string")
+        if kwargs:
+            raise TypeError("invalid keyword arguments to print()")
+        if not want_unicode:
+            for arg in args:
+                if isinstance(arg, unicode):
+                    want_unicode = True
+                    break
+        if want_unicode:
+            newline = unicode("\n")
+            space = unicode(" ")
+        else:
+            newline = "\n"
+            space = " "
+        if sep is None:
+            sep = space
+        if end is None:
+            end = newline
+        for i, arg in enumerate(args):
+            if i:
+                write(sep)
+            write(arg)
+        write(end)
+if sys.version_info[:2] < (3, 3):
+    _print = print_
+
+    def print_(*args, **kwargs):
+        fp = kwargs.get("file", sys.stdout)
+        flush = kwargs.pop("flush", False)
+        _print(*args, **kwargs)
+        if flush and fp is not None:
+            fp.flush()
+
+_add_doc(reraise, """Reraise an exception.""")
+
+if sys.version_info[0:2] < (3, 4):
+    # This does exactly the same what the :func:`py3:functools.update_wrapper`
+    # function does on Python versions after 3.2. It sets the ``__wrapped__``
+    # attribute on ``wrapper`` object and it doesn't raise an error if any of
+    # the attributes mentioned in ``assigned`` and ``updated`` are missing on
+    # ``wrapped`` object.
+    def _update_wrapper(wrapper, wrapped,
+                        assigned=functools.WRAPPER_ASSIGNMENTS,
+                        updated=functools.WRAPPER_UPDATES):
+        for attr in assigned:
+            try:
+                value = getattr(wrapped, attr)
+            except AttributeError:
+                continue
+            else:
+                setattr(wrapper, attr, value)
+        for attr in updated:
+            getattr(wrapper, attr).update(getattr(wrapped, attr, {}))
+        wrapper.__wrapped__ = wrapped
+        return wrapper
+    _update_wrapper.__doc__ = functools.update_wrapper.__doc__
+
+    def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS,
+              updated=functools.WRAPPER_UPDATES):
+        return functools.partial(_update_wrapper, wrapped=wrapped,
+                                 assigned=assigned, updated=updated)
+    wraps.__doc__ = functools.wraps.__doc__
+
+else:
+    wraps = functools.wraps
+
+
+def with_metaclass(meta, *bases):
+    """Create a base class with a metaclass."""
+    # This requires a bit of explanation: the basic idea is to make a dummy
+    # metaclass for one level of class instantiation that replaces itself with
+    # the actual metaclass.
+    class metaclass(type):
+
+        def __new__(cls, name, this_bases, d):
+            if sys.version_info[:2] >= (3, 7):
+                # This version introduced PEP 560 that requires a bit
+                # of extra care (we mimic what is done by __build_class__).
+                resolved_bases = types.resolve_bases(bases)
+                if resolved_bases is not bases:
+                    d['__orig_bases__'] = bases
+            else:
+                resolved_bases = bases
+            return meta(name, resolved_bases, d)
+
+        @classmethod
+        def __prepare__(cls, name, this_bases):
+            return meta.__prepare__(name, bases)
+    return type.__new__(metaclass, 'temporary_class', (), {})
+
+
+def add_metaclass(metaclass):
+    """Class decorator for creating a class with a metaclass."""
+    def wrapper(cls):
+        orig_vars = cls.__dict__.copy()
+        slots = orig_vars.get('__slots__')
+        if slots is not None:
+            if isinstance(slots, str):
+                slots = [slots]
+            for slots_var in slots:
+                orig_vars.pop(slots_var)
+        orig_vars.pop('__dict__', None)
+        orig_vars.pop('__weakref__', None)
+        if hasattr(cls, '__qualname__'):
+            orig_vars['__qualname__'] = cls.__qualname__
+        return metaclass(cls.__name__, cls.__bases__, orig_vars)
+    return wrapper
+
+
+def ensure_binary(s, encoding='utf-8', errors='strict'):
+    """Coerce **s** to six.binary_type.
+
+    For Python 2:
+      - `unicode` -> encoded to `str`
+      - `str` -> `str`
+
+    For Python 3:
+      - `str` -> encoded to `bytes`
+      - `bytes` -> `bytes`
+    """
+    if isinstance(s, binary_type):
+        return s
+    if isinstance(s, text_type):
+        return s.encode(encoding, errors)
+    raise TypeError("not expecting type '%s'" % type(s))
+
+
+def ensure_str(s, encoding='utf-8', errors='strict'):
+    """Coerce *s* to `str`.
+
+    For Python 2:
+      - `unicode` -> encoded to `str`
+      - `str` -> `str`
+
+    For Python 3:
+      - `str` -> `str`
+      - `bytes` -> decoded to `str`
+    """
+    # Optimization: Fast return for the common case.
+    if type(s) is str:
+        return s
+    if PY2 and isinstance(s, text_type):
+        return s.encode(encoding, errors)
+    elif PY3 and isinstance(s, binary_type):
+        return s.decode(encoding, errors)
+    elif not isinstance(s, (text_type, binary_type)):
+        raise TypeError("not expecting type '%s'" % type(s))
+    return s
+
+
+def ensure_text(s, encoding='utf-8', errors='strict'):
+    """Coerce *s* to six.text_type.
+
+    For Python 2:
+      - `unicode` -> `unicode`
+      - `str` -> `unicode`
+
+    For Python 3:
+      - `str` -> `str`
+      - `bytes` -> decoded to `str`
+    """
+    if isinstance(s, binary_type):
+        return s.decode(encoding, errors)
+    elif isinstance(s, text_type):
+        return s
+    else:
+        raise TypeError("not expecting type '%s'" % type(s))
+
+
+def python_2_unicode_compatible(klass):
+    """
+    A class decorator that defines __unicode__ and __str__ methods under Python 2.
+    Under Python 3 it does nothing.
+
+    To support Python 2 and 3 with a single code base, define a __str__ method
+    returning text and apply this decorator to the class.
+    """
+    if PY2:
+        if '__str__' not in klass.__dict__:
+            raise ValueError("@python_2_unicode_compatible cannot be applied "
+                             "to %s because it doesn't define __str__()." %
+                             klass.__name__)
+        klass.__unicode__ = klass.__str__
+        klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
+    return klass
+
+
+# Complete the moves implementation.
+# This code is at the end of this module to speed up module loading.
+# Turn this module into a package.
+__path__ = []  # required for PEP 302 and PEP 451
+__package__ = __name__  # see PEP 366 @ReservedAssignment
+if globals().get("__spec__") is not None:
+    __spec__.submodule_search_locations = []  # PEP 451 @UndefinedVariable
+# Remove other six meta path importers, since they cause problems. This can
+# happen if six is removed from sys.modules and then reloaded. (Setuptools does
+# this for some reason.)
+if sys.meta_path:
+    for i, importer in enumerate(sys.meta_path):
+        # Here's some real nastiness: Another "instance" of the six module might
+        # be floating around. Therefore, we can't use isinstance() to check for
+        # the six meta path importer, since the other six instance will have
+        # inserted an importer with different class.
+        if (type(importer).__name__ == "_SixMetaPathImporter" and
+                importer.name == __name__):
+            del sys.meta_path[i]
+            break
+    del i, importer
+# Finally, add the importer to the meta path import hook.
+sys.meta_path.append(_importer)

RNS/vendor/umsgpack.py

@@ -1,4 +1,4 @@
-# u-msgpack-python v2.5.0 - v at sergeev.io
+# u-msgpack-python v2.7.1 - v at sergeev.io
 # https://github.com/vsergeev/u-msgpack-python
 #
 # u-msgpack-python is a lightweight MessagePack serializer and deserializer
@@ -10,7 +10,7 @@
 #
 # MIT License
 #
-# Copyright (c) 2013-2016 vsergeev / Ivan (Vanya) A. Sergeev
+# Copyright (c) 2013-2020 vsergeev / Ivan (Vanya) A. Sergeev
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -31,7 +31,7 @@
 # THE SOFTWARE.
 #
 """
-u-msgpack-python v2.5.0 - v at sergeev.io
+u-msgpack-python v2.7.1 - v at sergeev.io
 https://github.com/vsergeev/u-msgpack-python
 
 u-msgpack-python is a lightweight MessagePack serializer and deserializer
@@ -49,10 +49,15 @@ import datetime
 import sys
 import io
 
-__version__ = "2.5.0"
+if sys.version_info[0:2] >= (3, 3):
+    from collections.abc import Hashable
+else:
+    from collections import Hashable
+
+__version__ = "2.7.1"
 "Module version string"
 
-version = (2, 5, 0)
+version = (2, 7, 1)
 "Module version tuple"
 
 
@@ -61,7 +66,7 @@ version = (2, 5, 0)
 ##############################################################################
 
 # Extension type for application-defined types and data
-class Ext:
+class Ext(object):
     """
     The Ext class facilitates creating a serializable extension object to store
     an application-defined type and data byte array.
@@ -75,23 +80,33 @@ class Ext:
             type: application-defined type integer
             data: application-defined data byte array
 
+        TypeError:
+            Type is not an integer.
+        ValueError:
+            Type is out of range of -128 to 127.
+        TypeError::
+            Data is not type 'bytes' (Python 3) or not type 'str' (Python 2).
+
         Example:
-        >>> foo = umsgpack.Ext(0x05, b"\x01\x02\x03")
+        >>> foo = umsgpack.Ext(5, b"\x01\x02\x03")
         >>> umsgpack.packb({u"special stuff": foo, u"awesome": True})
         '\x82\xa7awesome\xc3\xadspecial stuff\xc7\x03\x05\x01\x02\x03'
         >>> bar = umsgpack.unpackb(_)
         >>> print(bar["special stuff"])
-        Ext Object (Type: 0x05, Data: 01 02 03)
+        Ext Object (Type: 5, Data: 01 02 03)
         >>>
         """
-        # Check type is type int
+        # Check type is type int and in range
         if not isinstance(type, int):
             raise TypeError("ext type is not type integer")
-        # Check data is type bytes
+        elif not (-2**7 <= type <= 2**7 - 1):
+            raise ValueError("ext type value {:d} is out of range (-128 to 127)".format(type))
+        # Check data is type bytes or str
         elif sys.version_info[0] == 3 and not isinstance(data, bytes):
             raise TypeError("ext data is not type \'bytes\'")
         elif sys.version_info[0] == 2 and not isinstance(data, str):
             raise TypeError("ext data is not type \'str\'")
+
         self.type = type
         self.data = data
 
@@ -99,9 +114,8 @@ class Ext:
         """
         Compare this Ext object with another for equality.
         """
-        return (isinstance(other, self.__class__) and
-                self.type == other.type and
-                self.data == other.data)
+        return isinstance(other, self.__class__) \
+            and self.type == other.type and self.data == other.data
 
     def __ne__(self, other):
         """
@@ -113,8 +127,8 @@ class Ext:
         """
         String representation of this Ext object.
         """
-        s = "Ext Object (Type: 0x%02x, Data: " % self.type
-        s += " ".join(["0x%02x" % ord(self.data[i:i + 1])
+        s = "Ext Object (Type: {:d}, Data: ".format(self.type)
+        s += " ".join(["0x{:02}".format(ord(self.data[i:i + 1]))
                        for i in xrange(min(len(self.data), 8))])
         if len(self.data) > 8:
             s += " ..."
@@ -130,7 +144,52 @@ class Ext:
 
 class InvalidString(bytes):
     """Subclass of bytes to hold invalid UTF-8 strings."""
-    pass
+
+
+##############################################################################
+# Ext Serializable Decorator
+##############################################################################
+
+_ext_class_to_type = {}
+_ext_type_to_class = {}
+
+
+def ext_serializable(ext_type):
+    """
+    Return a decorator to register a class for automatic packing and unpacking
+    with the specified Ext type code. The application class should implement a
+    `packb()` method that returns serialized bytes, and an `unpackb()` class
+    method or static method that accepts serialized bytes and returns an
+    instance of the application class.
+
+    Args:
+        ext_type: application-defined Ext type code
+
+    Raises:
+        TypeError:
+            Ext type is not an integer.
+        ValueError:
+            Ext type is out of range of -128 to 127.
+        ValueError:
+            Ext type or class already registered.
+    """
+    def wrapper(cls):
+        if not isinstance(ext_type, int):
+            raise TypeError("Ext type is not type integer")
+        elif not (-2**7 <= ext_type <= 2**7 - 1):
+            raise ValueError("Ext type value {:d} is out of range of -128 to 127".format(ext_type))
+        elif ext_type in _ext_type_to_class:
+            raise ValueError("Ext type {:d} already registered with class {:s}".format(ext_type, repr(_ext_type_to_class[ext_type])))
+        elif cls in _ext_class_to_type:
+            raise ValueError("Class {:s} already registered with Ext type {:d}".format(repr(cls), ext_type))
+
+        _ext_type_to_class[ext_type] = cls
+        _ext_class_to_type[cls] = ext_type
+
+        return cls
+
+    return wrapper
+
 
 ##############################################################################
 # Exceptions
@@ -140,39 +199,32 @@ class InvalidString(bytes):
 # Base Exception classes
 class PackException(Exception):
     "Base class for exceptions encountered during packing."
-    pass
 
 
 class UnpackException(Exception):
     "Base class for exceptions encountered during unpacking."
-    pass
 
 
 # Packing error
 class UnsupportedTypeException(PackException):
     "Object type not supported for packing."
-    pass
 
 
 # Unpacking error
 class InsufficientDataException(UnpackException):
     "Insufficient data to unpack the serialized object."
-    pass
 
 
 class InvalidStringException(UnpackException):
     "Invalid UTF-8 string encountered during unpacking."
-    pass
 
 
 class UnsupportedTimestampException(UnpackException):
     "Unsupported timestamp format encountered during unpacking."
-    pass
 
 
 class ReservedCodeException(UnpackException):
     "Reserved code encountered during unpacking."
-    pass
 
 
 class UnhashableKeyException(UnpackException):
@@ -180,12 +232,10 @@ class UnhashableKeyException(UnpackException):
     Unhashable key encountered during map unpacking.
     The serialized map cannot be deserialized into a Python dictionary.
     """
-    pass
 
 
 class DuplicateKeyException(UnpackException):
     "Duplicate key encountered during map unpacking."
-    pass
 
 
 # Backwards compatibility
@@ -250,15 +300,15 @@ def _pack_integer(obj, fp, options):
         else:
             raise UnsupportedTypeException("huge signed int")
     else:
-        if obj <= 127:
+        if obj < 128:
             fp.write(struct.pack("B", obj))
-        elif obj <= 2**8 - 1:
+        elif obj < 2**8:
             fp.write(b"\xcc" + struct.pack("B", obj))
-        elif obj <= 2**16 - 1:
+        elif obj < 2**16:
             fp.write(b"\xcd" + struct.pack(">H", obj))
-        elif obj <= 2**32 - 1:
+        elif obj < 2**32:
             fp.write(b"\xce" + struct.pack(">I", obj))
-        elif obj <= 2**64 - 1:
+        elif obj < 2**64:
             fp.write(b"\xcf" + struct.pack(">Q", obj))
         else:
             raise UnsupportedTypeException("huge unsigned int")
@@ -285,94 +335,99 @@ def _pack_float(obj, fp, options):
 
 def _pack_string(obj, fp, options):
     obj = obj.encode('utf-8')
-    if len(obj) <= 31:
-        fp.write(struct.pack("B", 0xa0 | len(obj)) + obj)
-    elif len(obj) <= 2**8 - 1:
-        fp.write(b"\xd9" + struct.pack("B", len(obj)) + obj)
-    elif len(obj) <= 2**16 - 1:
-        fp.write(b"\xda" + struct.pack(">H", len(obj)) + obj)
-    elif len(obj) <= 2**32 - 1:
-        fp.write(b"\xdb" + struct.pack(">I", len(obj)) + obj)
+    obj_len = len(obj)
+    if obj_len < 32:
+        fp.write(struct.pack("B", 0xa0 | obj_len) + obj)
+    elif obj_len < 2**8:
+        fp.write(b"\xd9" + struct.pack("B", obj_len) + obj)
+    elif obj_len < 2**16:
+        fp.write(b"\xda" + struct.pack(">H", obj_len) + obj)
+    elif obj_len < 2**32:
+        fp.write(b"\xdb" + struct.pack(">I", obj_len) + obj)
     else:
         raise UnsupportedTypeException("huge string")
 
 
 def _pack_binary(obj, fp, options):
-    if len(obj) <= 2**8 - 1:
-        fp.write(b"\xc4" + struct.pack("B", len(obj)) + obj)
-    elif len(obj) <= 2**16 - 1:
-        fp.write(b"\xc5" + struct.pack(">H", len(obj)) + obj)
-    elif len(obj) <= 2**32 - 1:
-        fp.write(b"\xc6" + struct.pack(">I", len(obj)) + obj)
+    obj_len = len(obj)
+    if obj_len < 2**8:
+        fp.write(b"\xc4" + struct.pack("B", obj_len) + obj)
+    elif obj_len < 2**16:
+        fp.write(b"\xc5" + struct.pack(">H", obj_len) + obj)
+    elif obj_len < 2**32:
+        fp.write(b"\xc6" + struct.pack(">I", obj_len) + obj)
     else:
         raise UnsupportedTypeException("huge binary string")
 
 
 def _pack_oldspec_raw(obj, fp, options):
-    if len(obj) <= 31:
-        fp.write(struct.pack("B", 0xa0 | len(obj)) + obj)
-    elif len(obj) <= 2**16 - 1:
-        fp.write(b"\xda" + struct.pack(">H", len(obj)) + obj)
-    elif len(obj) <= 2**32 - 1:
-        fp.write(b"\xdb" + struct.pack(">I", len(obj)) + obj)
+    obj_len = len(obj)
+    if obj_len < 32:
+        fp.write(struct.pack("B", 0xa0 | obj_len) + obj)
+    elif obj_len < 2**16:
+        fp.write(b"\xda" + struct.pack(">H", obj_len) + obj)
+    elif obj_len < 2**32:
+        fp.write(b"\xdb" + struct.pack(">I", obj_len) + obj)
     else:
         raise UnsupportedTypeException("huge raw string")
 
 
 def _pack_ext(obj, fp, options):
-    if len(obj.data) == 1:
+    obj_len = len(obj.data)
+    if obj_len == 1:
         fp.write(b"\xd4" + struct.pack("B", obj.type & 0xff) + obj.data)
-    elif len(obj.data) == 2:
+    elif obj_len == 2:
         fp.write(b"\xd5" + struct.pack("B", obj.type & 0xff) + obj.data)
-    elif len(obj.data) == 4:
+    elif obj_len == 4:
         fp.write(b"\xd6" + struct.pack("B", obj.type & 0xff) + obj.data)
-    elif len(obj.data) == 8:
+    elif obj_len == 8:
         fp.write(b"\xd7" + struct.pack("B", obj.type & 0xff) + obj.data)
-    elif len(obj.data) == 16:
+    elif obj_len == 16:
         fp.write(b"\xd8" + struct.pack("B", obj.type & 0xff) + obj.data)
-    elif len(obj.data) <= 2**8 - 1:
-        fp.write(b"\xc7" +
-                 struct.pack("BB", len(obj.data), obj.type & 0xff) + obj.data)
-    elif len(obj.data) <= 2**16 - 1:
-        fp.write(b"\xc8" +
-                 struct.pack(">HB", len(obj.data), obj.type & 0xff) + obj.data)
-    elif len(obj.data) <= 2**32 - 1:
-        fp.write(b"\xc9" +
-                 struct.pack(">IB", len(obj.data), obj.type & 0xff) + obj.data)
+    elif obj_len < 2**8:
+        fp.write(b"\xc7" + struct.pack("BB", obj_len, obj.type & 0xff) + obj.data)
+    elif obj_len < 2**16:
+        fp.write(b"\xc8" + struct.pack(">HB", obj_len, obj.type & 0xff) + obj.data)
+    elif obj_len < 2**32:
+        fp.write(b"\xc9" + struct.pack(">IB", obj_len, obj.type & 0xff) + obj.data)
     else:
         raise UnsupportedTypeException("huge ext data")
 
 
 def _pack_ext_timestamp(obj, fp, options):
-    delta = obj - _epoch
+    if not obj.tzinfo:
+        # Object is naive datetime, convert to aware date time,
+        # assuming UTC timezone
+        delta = obj.replace(tzinfo=_utc_tzinfo) - _epoch
+    else:
+        # Object is aware datetime
+        delta = obj - _epoch
+
     seconds = delta.seconds + delta.days * 86400
     microseconds = delta.microseconds
 
     if microseconds == 0 and 0 <= seconds <= 2**32 - 1:
         # 32-bit timestamp
-        fp.write(b"\xd6\xff" +
-                 struct.pack(">I", seconds))
+        fp.write(b"\xd6\xff" + struct.pack(">I", seconds))
     elif 0 <= seconds <= 2**34 - 1:
         # 64-bit timestamp
         value = ((microseconds * 1000) << 34) | seconds
-        fp.write(b"\xd7\xff" +
-                 struct.pack(">Q", value))
+        fp.write(b"\xd7\xff" + struct.pack(">Q", value))
     elif -2**63 <= abs(seconds) <= 2**63 - 1:
         # 96-bit timestamp
-        fp.write(b"\xc7\x0c\xff" +
-                 struct.pack(">I", microseconds * 1000) +
-                 struct.pack(">q", seconds))
+        fp.write(b"\xc7\x0c\xff" + struct.pack(">Iq", microseconds * 1000, seconds))
     else:
         raise UnsupportedTypeException("huge timestamp")
 
 
 def _pack_array(obj, fp, options):
-    if len(obj) <= 15:
-        fp.write(struct.pack("B", 0x90 | len(obj)))
-    elif len(obj) <= 2**16 - 1:
-        fp.write(b"\xdc" + struct.pack(">H", len(obj)))
-    elif len(obj) <= 2**32 - 1:
-        fp.write(b"\xdd" + struct.pack(">I", len(obj)))
+    obj_len = len(obj)
+    if obj_len < 16:
+        fp.write(struct.pack("B", 0x90 | obj_len))
+    elif obj_len < 2**16:
+        fp.write(b"\xdc" + struct.pack(">H", obj_len))
+    elif obj_len < 2**32:
+        fp.write(b"\xdd" + struct.pack(">I", obj_len))
     else:
         raise UnsupportedTypeException("huge array")
 
@@ -381,12 +436,13 @@ def _pack_array(obj, fp, options):
 
 
 def _pack_map(obj, fp, options):
-    if len(obj) <= 15:
-        fp.write(struct.pack("B", 0x80 | len(obj)))
-    elif len(obj) <= 2**16 - 1:
-        fp.write(b"\xde" + struct.pack(">H", len(obj)))
-    elif len(obj) <= 2**32 - 1:
-        fp.write(b"\xdf" + struct.pack(">I", len(obj)))
+    obj_len = len(obj)
+    if obj_len < 16:
+        fp.write(struct.pack("B", 0x80 | obj_len))
+    elif obj_len < 2**16:
+        fp.write(b"\xde" + struct.pack(">H", obj_len))
+    elif obj_len < 2**32:
+        fp.write(b"\xdf" + struct.pack(">I", obj_len))
     else:
         raise UnsupportedTypeException("huge array")
 
@@ -435,9 +491,14 @@ def _pack2(obj, fp, **options):
         _pack_nil(obj, fp, options)
     elif ext_handlers and obj.__class__ in ext_handlers:
         _pack_ext(ext_handlers[obj.__class__](obj), fp, options)
+    elif obj.__class__ in _ext_class_to_type:
+        try:
+            _pack_ext(Ext(_ext_class_to_type[obj.__class__], obj.packb()), fp, options)
+        except AttributeError:
+            raise NotImplementedError("Ext serializable class {:s} is missing implementation of packb()".format(repr(obj.__class__)))
     elif isinstance(obj, bool):
         _pack_boolean(obj, fp, options)
-    elif isinstance(obj, int) or isinstance(obj, long):
+    elif isinstance(obj, (int, long)):
         _pack_integer(obj, fp, options)
     elif isinstance(obj, float):
         _pack_float(obj, fp, options)
@@ -449,7 +510,7 @@ def _pack2(obj, fp, **options):
         _pack_string(obj, fp, options)
     elif isinstance(obj, str):
         _pack_binary(obj, fp, options)
-    elif isinstance(obj, list) or isinstance(obj, tuple):
+    elif isinstance(obj, (list, tuple)):
         _pack_array(obj, fp, options)
     elif isinstance(obj, dict):
         _pack_map(obj, fp, options)
@@ -464,9 +525,19 @@ def _pack2(obj, fp, **options):
             _pack_ext(ext_handlers[t](obj), fp, options)
         else:
             raise UnsupportedTypeException(
-                "unsupported type: %s" % str(type(obj)))
+                "unsupported type: {:s}".format(str(type(obj))))
+    elif _ext_class_to_type:
+        # Linear search for superclass
+        t = next((t for t in _ext_class_to_type if isinstance(obj, t)), None)
+        if t:
+            try:
+                _pack_ext(Ext(_ext_class_to_type[t], obj.packb()), fp, options)
+            except AttributeError:
+                raise NotImplementedError("Ext serializable class {:s} is missing implementation of packb()".format(repr(t)))
+        else:
+            raise UnsupportedTypeException("unsupported type: {:s}".format(str(type(obj))))
     else:
-        raise UnsupportedTypeException("unsupported type: %s" % str(type(obj)))
+        raise UnsupportedTypeException("unsupported type: {:s}".format(str(type(obj))))
 
 
 # Pack for Python 3, with unicode 'str' type, 'bytes' type, and no 'long' type
@@ -507,6 +578,11 @@ def _pack3(obj, fp, **options):
         _pack_nil(obj, fp, options)
     elif ext_handlers and obj.__class__ in ext_handlers:
         _pack_ext(ext_handlers[obj.__class__](obj), fp, options)
+    elif obj.__class__ in _ext_class_to_type:
+        try:
+            _pack_ext(Ext(_ext_class_to_type[obj.__class__], obj.packb()), fp, options)
+        except AttributeError:
+            raise NotImplementedError("Ext serializable class {:s} is missing implementation of packb()".format(repr(obj.__class__)))
     elif isinstance(obj, bool):
         _pack_boolean(obj, fp, options)
     elif isinstance(obj, int):
@@ -521,7 +597,7 @@ def _pack3(obj, fp, **options):
         _pack_string(obj, fp, options)
     elif isinstance(obj, bytes):
         _pack_binary(obj, fp, options)
-    elif isinstance(obj, list) or isinstance(obj, tuple):
+    elif isinstance(obj, (list, tuple)):
         _pack_array(obj, fp, options)
     elif isinstance(obj, dict):
         _pack_map(obj, fp, options)
@@ -536,10 +612,20 @@ def _pack3(obj, fp, **options):
             _pack_ext(ext_handlers[t](obj), fp, options)
         else:
             raise UnsupportedTypeException(
-                "unsupported type: %s" % str(type(obj)))
+                "unsupported type: {:s}".format(str(type(obj))))
+    elif _ext_class_to_type:
+        # Linear search for superclass
+        t = next((t for t in _ext_class_to_type if isinstance(obj, t)), None)
+        if t:
+            try:
+                _pack_ext(Ext(_ext_class_to_type[t], obj.packb()), fp, options)
+            except AttributeError:
+                raise NotImplementedError("Ext serializable class {:s} is missing implementation of packb()".format(repr(t)))
+        else:
+            raise UnsupportedTypeException("unsupported type: {:s}".format(str(type(obj))))
     else:
         raise UnsupportedTypeException(
-            "unsupported type: %s" % str(type(obj)))
+            "unsupported type: {:s}".format(str(type(obj))))
 
 
 def _packb2(obj, **options):
@@ -613,9 +699,20 @@ def _packb3(obj, **options):
 
 
 def _read_except(fp, n):
+    if n == 0:
+        return b""
+
     data = fp.read(n)
-    if len(data) < n:
+    if len(data) == 0:
         raise InsufficientDataException()
+
+    while len(data) < n:
+        chunk = fp.read(n - len(data))
+        if len(chunk) == 0:
+            raise InsufficientDataException()
+
+        data += chunk
+
     return data
 
 
@@ -640,21 +737,21 @@ def _unpack_integer(code, fp, options):
         return struct.unpack(">I", _read_except(fp, 4))[0]
     elif code == b'\xcf':
         return struct.unpack(">Q", _read_except(fp, 8))[0]
-    raise Exception("logic error, not int: 0x%02x" % ord(code))
+    raise Exception("logic error, not int: 0x{:02x}".format(ord(code)))
 
 
 def _unpack_reserved(code, fp, options):
     if code == b'\xc1':
         raise ReservedCodeException(
-            "encountered reserved code: 0x%02x" % ord(code))
+            "encountered reserved code: 0x{:02x}".format(ord(code)))
     raise Exception(
-        "logic error, not reserved code: 0x%02x" % ord(code))
+        "logic error, not reserved code: 0x{:02x}".format(ord(code)))
 
 
 def _unpack_nil(code, fp, options):
     if code == b'\xc0':
         return None
-    raise Exception("logic error, not nil: 0x%02x" % ord(code))
+    raise Exception("logic error, not nil: 0x{:02x}".format(ord(code)))
 
 
 def _unpack_boolean(code, fp, options):
@@ -662,7 +759,7 @@ def _unpack_boolean(code, fp, options):
         return False
     elif code == b'\xc3':
         return True
-    raise Exception("logic error, not boolean: 0x%02x" % ord(code))
+    raise Exception("logic error, not boolean: 0x{:02x}".format(ord(code)))
 
 
 def _unpack_float(code, fp, options):
@@ -670,7 +767,7 @@ def _unpack_float(code, fp, options):
         return struct.unpack(">f", _read_except(fp, 4))[0]
     elif code == b'\xcb':
         return struct.unpack(">d", _read_except(fp, 8))[0]
-    raise Exception("logic error, not float: 0x%02x" % ord(code))
+    raise Exception("logic error, not float: 0x{:02x}".format(ord(code)))
 
 
 def _unpack_string(code, fp, options):
@@ -683,7 +780,7 @@ def _unpack_string(code, fp, options):
     elif code == b'\xdb':
         length = struct.unpack(">I", _read_except(fp, 4))[0]
     else:
-        raise Exception("logic error, not string: 0x%02x" % ord(code))
+        raise Exception("logic error, not string: 0x{:02x}".format(ord(code)))
 
     # Always return raw bytes in compatibility mode
     global compatibility
@@ -707,7 +804,7 @@ def _unpack_binary(code, fp, options):
     elif code == b'\xc6':
         length = struct.unpack(">I", _read_except(fp, 4))[0]
     else:
-        raise Exception("logic error, not binary: 0x%02x" % ord(code))
+        raise Exception("logic error, not binary: 0x{:02x}".format(ord(code)))
 
     return _read_except(fp, length)
 
@@ -730,43 +827,48 @@ def _unpack_ext(code, fp, options):
     elif code == b'\xc9':
         length = struct.unpack(">I", _read_except(fp, 4))[0]
     else:
-        raise Exception("logic error, not ext: 0x%02x" % ord(code))
+        raise Exception("logic error, not ext: 0x{:02x}".format(ord(code)))
 
     ext_type = struct.unpack("b", _read_except(fp, 1))[0]
     ext_data = _read_except(fp, length)
 
-    # Create extension object
-    ext = Ext(ext_type, ext_data)
-
     # Unpack with ext handler, if we have one
     ext_handlers = options.get("ext_handlers")
-    if ext_handlers and ext.type in ext_handlers:
-        return ext_handlers[ext.type](ext)
+    if ext_handlers and ext_type in ext_handlers:
+        return ext_handlers[ext_type](Ext(ext_type, ext_data))
+
+    # Unpack with ext classes, if type is registered
+    if ext_type in _ext_type_to_class:
+        try:
+            return _ext_type_to_class[ext_type].unpackb(ext_data)
+        except AttributeError:
+            raise NotImplementedError("Ext serializable class {:s} is missing implementation of unpackb()".format(repr(_ext_type_to_class[ext_type])))
 
     # Timestamp extension
-    if ext.type == -1:
-        return _unpack_ext_timestamp(ext, options)
+    if ext_type == -1:
+        return _unpack_ext_timestamp(ext_data, options)
 
-    return ext
+    return Ext(ext_type, ext_data)
 
 
-def _unpack_ext_timestamp(ext, options):
-    if len(ext.data) == 4:
+def _unpack_ext_timestamp(ext_data, options):
+    obj_len = len(ext_data)
+    if obj_len == 4:
         # 32-bit timestamp
-        seconds = struct.unpack(">I", ext.data)[0]
+        seconds = struct.unpack(">I", ext_data)[0]
         microseconds = 0
-    elif len(ext.data) == 8:
+    elif obj_len == 8:
         # 64-bit timestamp
-        value = struct.unpack(">Q", ext.data)[0]
+        value = struct.unpack(">Q", ext_data)[0]
         seconds = value & 0x3ffffffff
         microseconds = (value >> 34) // 1000
-    elif len(ext.data) == 12:
+    elif obj_len == 12:
         # 96-bit timestamp
-        seconds = struct.unpack(">q", ext.data[4:12])[0]
-        microseconds = struct.unpack(">I", ext.data[0:4])[0] // 1000
+        seconds = struct.unpack(">q", ext_data[4:12])[0]
+        microseconds = struct.unpack(">I", ext_data[0:4])[0] // 1000
     else:
         raise UnsupportedTimestampException(
-            "unsupported timestamp with data length %d" % len(ext.data))
+            "unsupported timestamp with data length {:d}".format(len(ext_data)))
 
     return _epoch + datetime.timedelta(seconds=seconds,
                                        microseconds=microseconds)
@@ -780,7 +882,10 @@ def _unpack_array(code, fp, options):
     elif code == b'\xdd':
         length = struct.unpack(">I", _read_except(fp, 4))[0]
     else:
-        raise Exception("logic error, not array: 0x%02x" % ord(code))
+        raise Exception("logic error, not array: 0x{:02x}".format(ord(code)))
+
+    if options.get('use_tuple'):
+        return tuple((_unpack(fp, options) for i in xrange(length)))
 
     return [_unpack(fp, options) for i in xrange(length)]
 
@@ -799,10 +904,9 @@ def _unpack_map(code, fp, options):
     elif code == b'\xdf':
         length = struct.unpack(">I", _read_except(fp, 4))[0]
     else:
-        raise Exception("logic error, not map: 0x%02x" % ord(code))
+        raise Exception("logic error, not map: 0x{:02x}".format(ord(code)))
 
-    d = {} if not options.get('use_ordered_dict') \
-        else collections.OrderedDict()
+    d = {} if not options.get('use_ordered_dict') else collections.OrderedDict()
     for _ in xrange(length):
         # Unpack key
         k = _unpack(fp, options)
@@ -810,12 +914,12 @@ def _unpack_map(code, fp, options):
         if isinstance(k, list):
             # Attempt to convert list into a hashable tuple
             k = _deep_list_to_tuple(k)
-        elif not isinstance(k, collections.Hashable):
+        elif not isinstance(k, Hashable):
             raise UnhashableKeyException(
-                "encountered unhashable key: %s, %s" % (str(k), str(type(k))))
+                "encountered unhashable key: \"{:s}\" ({:s})".format(str(k), str(type(k))))
         elif k in d:
             raise DuplicateKeyException(
-                "encountered duplicate key: %s, %s" % (str(k), str(type(k))))
+                "encountered duplicate key: \"{:s}\" ({:s})".format(str(k), str(type(k))))
 
         # Unpack value
         v = _unpack(fp, options)
@@ -824,7 +928,7 @@ def _unpack_map(code, fp, options):
             d[k] = v
         except TypeError:
             raise UnhashableKeyException(
-                "encountered unhashable key: %s" % str(k))
+                "encountered unhashable key: \"{:s}\"".format(str(k)))
     return d
 
 
@@ -848,6 +952,8 @@ def _unpack2(fp, **options):
                              Ext into an object
         use_ordered_dict (bool): unpack maps into OrderedDict, instead of
                                  unordered dict (default False)
+        use_tuple (bool): unpacks arrays into tuples, instead of lists (default
+                          False)
         allow_invalid_utf8 (bool): unpack invalid strings into instances of
                                    InvalidString, for access to the bytes
                                    (default False)
@@ -892,6 +998,8 @@ def _unpack3(fp, **options):
                              Ext into an object
         use_ordered_dict (bool): unpack maps into OrderedDict, instead of
                                  unordered dict (default False)
+        use_tuple (bool): unpacks arrays into tuples, instead of lists (default
+                          False)
         allow_invalid_utf8 (bool): unpack invalid strings into instances of
                                    InvalidString, for access to the bytes
                                    (default False)
@@ -937,6 +1045,8 @@ def _unpackb2(s, **options):
                              Ext into an object
         use_ordered_dict (bool): unpack maps into OrderedDict, instead of
                                  unordered dict (default False)
+        use_tuple (bool): unpacks arrays into tuples, instead of lists (default
+                          False)
         allow_invalid_utf8 (bool): unpack invalid strings into instances of
                                    InvalidString, for access to the bytes
                                    (default False)
@@ -985,6 +1095,8 @@ def _unpackb3(s, **options):
                              Ext into an object
         use_ordered_dict (bool): unpack maps into OrderedDict, instead of
                                  unordered dict (default False)
+        use_tuple (bool): unpacks arrays into tuples, instead of lists (default
+                          False)
         allow_invalid_utf8 (bool): unpack invalid strings into instances of
                                    InvalidString, for access to the bytes
                                    (default False)
@@ -1045,9 +1157,21 @@ def __init():
     if sys.version_info[0] == 3:
         _utc_tzinfo = datetime.timezone.utc
     else:
-        _utc_tzinfo = None
+        class UTC(datetime.tzinfo):
+            ZERO = datetime.timedelta(0)
 
-    # Calculate epoch datetime
+            def utcoffset(self, dt):
+                return UTC.ZERO
+
+            def tzname(self, dt):
+                return "UTC"
+
+            def dst(self, dt):
+                return UTC.ZERO
+
+        _utc_tzinfo = UTC()
+
+    # Calculate an aware epoch datetime
     _epoch = datetime.datetime(1970, 1, 1, tzinfo=_utc_tzinfo)
 
     # Auto-detect system float precision

docs/Makefile

@@ -0,0 +1,32 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	@if [ $@ = "html" ]; then \
+		rm -rf html; \
+		rm -rf manual; \
+		cp -r build/html ./; \
+		mv html manual; \
+		echo "HTML Manual Generated"; \
+	fi
+
+	@if [ $@ = "latexpdf" ]; then \
+		cp -r build/latex/reticulumnetworkstack.pdf ./Reticulum\ Manual.pdf; \
+		echo "PDF Manual Generated"; \
+	fi

docs/index.html

@@ -0,0 +1,9 @@
+<!DOCTYPE html>
+<html>
+  <head>
+    <meta charset="utf-8" />
+    <meta http-equiv="refresh" content="0;URL='./manual/'" />
+  </head>
+  <body>
+  </body>
+</html>

docs/manual/.buildinfo

@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 573300611576ca64adbecf60b740aea5
+tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/manual/_sources/examples.rst.txt

@@ -0,0 +1,106 @@
+.. _examples-main:
+
+*************
+Code Examples
+*************
+
+A number of examples are included in the source distribution of Reticulum.
+You can use these examples to learn how to write your own programs.
+
+.. _example-minimal:
+
+Minimal
+=======
+
+The *Minimal* example demonstrates the bare-minimum setup required to connect to
+a Reticulum network from your program. In about five lines of code, you will
+have the Reticulum Network Stack initialised, and ready to pass traffic in your
+program.
+
+.. literalinclude:: ../../Examples/Minimal.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Minimal.py>`_.
+
+.. _example-announce:
+
+Announce
+========
+
+The *Announce* example builds upon the previous example by exploring how to
+announce a destination on the network, and how to let your program receive
+notifications about announces from relevant destinations.
+
+.. literalinclude:: ../../Examples/Announce.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Announce.py>`_.
+
+.. _example-broadcast:
+
+Broadcast
+=========
+The *Broadcast* example explores how to transmit plaintext broadcast messages
+over the network.
+
+.. literalinclude:: ../../Examples/Broadcast.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Broadcast.py>`_.
+
+.. _example-echo:
+
+Echo
+====
+
+The *Echo* example demonstrates communication between two destinations using
+the Packet interface.
+
+.. literalinclude:: ../../Examples/Echo.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Echo.py>`_.
+
+.. _example-link:
+
+Link
+====
+
+The *Link* example explores establishing an encrypted link to a remote
+destination, and passing traffic back and forth over the link.
+
+.. literalinclude:: ../../Examples/Link.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Link.py>`_.
+
+.. _example-identify:
+
+Identification
+==============
+
+The *Identify* example explores identifying an intiator of a link, once
+the link has been established.
+
+.. literalinclude:: ../../Examples/Identify.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Identify.py>`_.
+
+.. _example-request:
+
+Requests & Responses
+====================
+
+The *Request* example explores sendig requests and receiving responses.
+
+.. literalinclude:: ../../Examples/Request.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Request.py>`_.
+
+.. _example-filetransfer:
+
+Filetransfer
+============
+
+The *Filetransfer* example implements a basic file-server program that
+allow clients to connect and download files. The program uses the Resource
+interface to efficiently pass files of any size over a Reticulum :ref:`Link<api-link>`.
+
+.. literalinclude:: ../../Examples/Filetransfer.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Filetransfer.py>`_.

docs/manual/_sources/gettingstartedfast.rst.txt

@@ -0,0 +1,370 @@
+********************
+Getting Started Fast
+********************
+
+The best way to get started with the Reticulum Network Stack depends on what
+you want to do. This guide will outline sensible starting paths for different
+scenarios.
+
+
+Try Using a Reticulum-based Program
+=============================================
+
+If you simply want to try using a program built with Reticulum, a few different
+programs exist that allow basic communication and a range of other useful functions
+over even extremely low-bandwidth Reticulum networks.
+
+These programs will let you get a feel for how Reticulum works. They have been designed
+to run well over networks based on LoRa or packet radio, but can also be used completely
+over local WiFi, wired Ethernet, the Internet, or any combination.
+
+As such, it is easy to get started experimenting, without having to set up any radio
+transceivers or infrastructure just to try it out. Launching the programs on separate
+devices connected to the same WiFi network is enough to get started, and physical
+radio interfaces can then be added later.
+
+Nomad Network
+^^^^^^^^^^^^^
+
+The terminal-based program `Nomad Network <https://github.com/markqvist/nomadnet>`_
+provides a complete encrypted communications suite built with Reticulum. It features
+encrypted messaging (both direct and delayed-delivery for offline users), file sharing,
+and has a built-in text-browser and page server with support for dynamically rendered pages,
+user authentication and more.
+
+.. image:: screenshots/nomadnet_3.png
+    :target: _images/nomadnet_3.png
+
+`Nomad Network <https://github.com/markqvist/nomadnet>`_ is a user-facing client
+for the messaging and information-sharing protocol
+`LXMF <https://github.com/markqvist/lxmf>`_, another project built with Reticulum.
+
+You can install Nomad Network via pip:
+
+.. code::
+
+   # Install ...
+   pip3 install nomadnet
+
+   # ... and run
+   nomadnet
+
+**Please Note**: If this is the very first time you use pip to install a program
+on your system, you might need to reboot your system for your program to become
+available. If you get a "command not found" error or similar when running the
+program, reboot your system and try again.
+
+Sideband
+^^^^^^^^
+
+If you would rather use a program with a graphical user interface, you can take
+a look at `Sideband <https://unsigned.io/sideband>`_, which is available for Android,
+Linux and macOS.
+
+.. image:: screenshots/sideband_1.png
+    :align: center
+    :target: _images/sideband_1.png
+
+Sideband is currently in the early stages of development, but already provides basic
+communication features, and interoperates with Nomad Network, or any other LXMF client.
+
+Using the Included Utilities
+=============================================
+Reticulum comes with a range of included utilities that make it easier to
+manage your network, check connectivity and make Reticulum available to other
+programs on your system.
+
+You can use ``rnsd`` to run Reticulum as a background or foreground service,
+and the ``rnstatus``, ``rnpath`` and ``rnprobe`` utilities to view and query
+network status and connectivity.
+
+To learn more about these utility programs, have a look at the
+:ref:`Using Reticulum on Your System<using-main>` chapter of this manual.
+
+
+Creating a Network With Reticulum
+=============================================
+To create a network, you will need to specify one or more *interfaces* for
+Reticulum to use. This is done in the Reticulum configuration file, which by
+default is located at ``~/.reticulum/config``. You can edit this file by hand,
+or use the interactive ``rnsconfig`` utility. 
+
+When Reticulum is started for the first time, it will create a default
+configuration file, with one active interface. This default interface uses
+your existing Ethernet and WiFi networks (if any), and only allows you to
+communicate with other Reticulum peers within your local broadcast domains.
+
+To communicate further, you will have to add one or more interfaces. The default
+configuration includes a number of examples, ranging from using TCP over the
+internet, to LoRa and Packet Radio interfaces.
+
+With Reticulum, you only need to configure what interfaces you want to communicate
+over. There is no need to configure address spaces, subnets, routing tables,
+or other things you might be used to from other network types.
+
+Once Reticulum knows which interfaces it should use, it will automatically
+discover topography and configure transport of data to any destinations it
+knows about.
+
+In situations where you already have an established WiFi or Ethernet network, and
+many devices that want to utilise the same external Reticulum network paths (for example over
+LoRa), it will often be sufficient to let one system act as a Reticulum gateway, by
+adding any external interfaces to the configuration of this system, and then enabling transport on it. Any
+other device on your local WiFi will then be able to connect to this wider Reticulum
+network just using the default (:ref:`AutoInterface<interfaces-auto>`) configuration.
+
+Possibly, the examples in the config file are enough to get you started. If
+you want more information, you can read the :ref:`Building Networks<networks-main>`
+and :ref:`Interfaces<interfaces-main>` chapters of this manual.
+
+Connecting Reticulum Instances Over the Internet
+================================================
+Reticulum currently offers two interfaces suitable for connecting instances over the Internet: :ref:`TCP<interfaces-tcps>`
+and :ref:`I2P<interfaces-i2p>`. Each interface offers a different set of features, and Reticulum 
+users should carefully choose the interface which best suites their needs. 
+
+The ``TCPServerInterface`` allows users to host an instance accessible over TCP/IP. This
+method is generally faster, lower latency, and more energy efficient than using ``I2PInterface``,
+however it also leaks more data about the server host.
+
+TCP connections reveal the IP address of both your instance and the server to anyone who can
+inspect the connection. Someone could use this information to determine your location or identity. Adversaries 
+inspecting your packets may be able to record packet metadata like time of transmission and packet size.
+Even though Reticulum encrypts traffic, TCP does not, so an adversary may be able to use
+packet inspection to learn that a system is running Reticulum, and what other IP addresses connect to it.
+Hosting a publicly reachable instance over TCP also requires a publicly reachable IP address,
+which most Internet connections don't offer anymore.
+
+The ``I2PInterface`` routes messages through the `Invisible Internet Protocol 
+(I2P) <https://geti2p.net/en/>`_. To use this interface, users must also run an I2P daemon in
+parallel to ``rnsd``. For always-on I2P nodes it is recommended to use `i2pd <https://i2pd.website/>`_. 
+
+By default, I2P will encrypt and mix all traffic sent over the Internet, and 
+hide both the sender and receiver Reticulum instance IP addresses. Running an I2P node 
+will also relay other I2P user's encrypted packets, which will use extra
+bandwidth and compute power, but also makes timing attacks and other forms of 
+deep-packet-inspection much more difficult.
+
+I2P also allows users to host globally available Reticulum instances from non-public IP's and behind firewalls and NAT.
+
+In general it is recommended to use an I2P node if you want to host a publicly accessible
+instance, while preserving anonymity. If you care more about performance, and a slightly
+easier setup, use TCP.
+
+
+Connect to the Public Testnet
+===========================================
+
+An experimental public testnet has been made accessible over both I2P and TCP. You can join it
+by adding one of the following interfaces to your ``.reticulum/config`` file:
+
+.. code::
+
+  # TCP/IP interface to the Dublin hub
+  [[RNS Testnet Dublin]]
+    type = TCPClientInterface
+    enabled = yes
+    target_host = dublin.connect.reticulum.network
+    target_port = 4965
+
+  # TCP/IP interface to the Frankfurt hub
+  [[RNS Testnet Dublin]]
+    type = TCPClientInterface
+    enabled = yes
+    target_host = frankfurt.connect.reticulum.network
+    target_port = 5377
+
+  # Interface to I2P hub A
+  [[RNS Testnet I2P Hub A]]
+    type = I2PInterface
+    enabled = yes
+    peers = uxg5kubabakh3jtnvsipingbr5574dle7bubvip7llfvwx2tgrua.b32.i2p
+
+Many other Reticulum instances are connecting to this testnet, and you can also join it
+via other entry points if you know them. There is absolutely no control over the network
+topography, usage or what types of instances connect. It will also occasionally be used
+to test various failure scenarios, and there are no availability or service guarantees.
+
+
+Adding Radio Interfaces
+==============================================
+Once you have Reticulum installed and working, you can add radio interfaces with
+any compatible hardware you have available. Reticulum supports a wide range of radio
+hardware, and if you already have any available, it is very likely that it will
+work with Reticulum. For information on how to configure this, see the
+:ref:`Interfaces<interfaces-main>` section of this manual.
+
+If you do not already have transceiver hardware available, you can easily and
+cheaply build an :ref:`RNode<rnode-main>`, which is a general-purpose long-range
+digital radio transceiver, that integrates easily with Reticulum.
+
+To build one yourself requires installing a custom firmware on a supported LoRa
+development board with an auto-install script. Please see the :ref:`Communications Hardware<hardware-main>`
+chapter for a guide. If you prefer purchasing a ready-made unit, you can refer to the
+:ref:`list of suppliers<rnode-suppliers>`. For more information on RNode, you can also
+refer to these additional external resources:
+
+* `How To Make Your Own RNodes <https://unsigned.io/how-to-make-your-own-rnodes/>`_
+* `Installing RNode Firmware on Compatible LoRa Devices <https://unsigned.io/installing-rnode-firmware-on-t-beam-and-lora32-devices/>`_
+* `Private, Secure and Uncensorable Messaging Over a LoRa Mesh <https://unsigned.io/private-messaging-over-lora/>`_
+* `RNode Firmware <https://github.com/markqvist/RNode_Firmware/>`_
+
+If you have communications hardware that is not already supported by any of the
+:ref:`existing interface types<interfaces-main>`, but you think would be suitable for use with Reticulum,
+you are welcome to head over to the `GitHub discussion pages <https://github.com/markqvist/Reticulum/discussions>`_
+and propose adding an interface for the hardware.
+
+
+Develop a Program with Reticulum
+===========================================
+If you want to develop programs that use Reticulum, the easiest way to get
+started is to install the latest release of Reticulum via pip:
+
+.. code::
+
+   pip3 install rns
+
+The above command will install Reticulum and dependencies, and you will be
+ready to import and use RNS in your own programs. The next step will most
+likely be to look at some :ref:`Example Programs<examples-main>`.
+
+For extended functionality, you can install optional dependencies:
+
+.. code::
+
+   pip3 install pyserial netifaces
+
+
+Further information can be found in the :ref:`API Reference<api-main>`.
+
+
+Participate in Reticulum Development
+==============================================
+If you want to participate in the development of Reticulum and associated
+utilities, you'll want to get the latest source from GitHub. In that case,
+don't use pip, but try this recipe:
+
+.. code::
+
+    # Install dependencies
+    pip3 install cryptography pyserial netifaces
+
+    # Clone repository
+    git clone https://github.com/markqvist/Reticulum.git
+
+    # Move into Reticulum folder and symlink library to examples folder
+    cd Reticulum
+    ln -s ../RNS ./Examples/
+
+    # Run an example
+    python3 Examples/Echo.py -s
+
+    # Unless you've manually created a config file, Reticulum will do so now,
+    # and immediately exit. Make any necessary changes to the file:
+    nano ~/.reticulum/config
+
+    # ... and launch the example again.
+    python3 Examples/Echo.py -s
+
+    # You can now repeat the process on another computer,
+    # and run the same example with -h to get command line options.
+    python3 Examples/Echo.py -h
+
+    # Run the example in client mode to "ping" the server.
+    # Replace the hash below with the actual destination hash of your server.
+    python3 Examples/Echo.py 3e12fc71692f8ec47bc5
+
+    # Have a look at another example
+    python3 Examples/Filetransfer.py -h
+
+When you have experimented with the basic examples, it's time to go read the
+:ref:`Understanding Reticulum<understanding-main>` chapter.
+
+
+Reticulum on ARM64
+==============================================
+On some architectures, including ARM64, not all dependencies have precompiled
+binaries. On such systems, you will need to install ``python3-dev`` before
+installing Reticulum or programs that depend on Reticulum.
+
+.. code::
+
+   # Install Python and development packages
+   sudo apt update
+   sudo apt install python3 python3-pip python3-dev
+
+   # Install Reticulum
+   python3 -m pip install rns
+
+
+Reticulum on Android
+==============================================
+Reticulum can be used on Android in different ways. The easiest way to get
+started is using an app like `Sideband <https://unsigned.io/sideband>`_.
+
+For more control and features, you can use Reticulum and related programs via
+the `Termux app <https://termux.com/>`_, at the time of writing available on
+`F-droid <https://f-droid.org>`_.
+
+Termux is a terminal emulator and Linux environment for Android based devices,
+which includes the ability to use many different programs and libraries,
+including Reticulum.
+
+Since the Python cryptography.io module does not offer pre-built wheels for
+Android, the standard one-line install of Reticulum does not work on Android,
+and a few extra commands are required.
+
+From within Termux, execute the following:
+
+.. code::
+
+    # First, make sure indexes and packages are up to date.
+    pkg update
+    pkg upgrade
+
+    # Then install dependencies for the cryptography library.
+    pkg install python build-essential openssl libffi rust
+
+    # Make sure pip is up to date, and install the wheel module.
+    pip3 install wheel pip --upgrade
+
+    # To allow the installer to build the cryptography module,
+    # we need to let it know what platform we are compiling for:
+    export CARGO_BUILD_TARGET="aarch64-linux-android"
+
+    # Start the install process for the cryptography module.
+    # Depending on your device, this can take several minutes,
+    # since the module must be compiled locally on your device.
+    pip3 install cryptography
+
+    # If the above installation succeeds, you can now install
+    # Reticulum and any related software
+    pip3 install rns
+
+It is also possible to include Reticulum in apps compiled and distributed as
+Android APKs. A detailed tutorial and example source code will be included
+here at a later point.
+
+Pure-Python Reticulum
+==============================================
+In some rare cases, and on more obscure system types, it is not possible to
+install one or more dependencies
+
+On more unusual systems, and in some rare cases, it might not be possible to
+install or even compile one or more of the above modules. In such situations,
+you can use the ``rnspure`` package instead of the ``rns`` package. The ``rnspure``
+package requires no external dependencies for installation. Please note that the
+actual contents of the ``rns`` and ``rnspure`` packages are *completely identical*.
+The only difference is that the ``rnspure`` package lists no dependencies required
+for installation.
+
+No matter how Reticulum is installed and started, it will load external dependencies
+only if they are *needed* and *available*. If for example you want to use Reticulum
+on a system that cannot support ``pyserial``, it is perfectly possible to do so using
+the `rnspure` package, but Reticulum will not be able to use serial-based interfaces.
+All other available modules will still be loaded when needed.
+
+**Please Note!** If you use the `rnspure` package to run Reticulum on systems that
+do not support `PyCA/cryptography <https://github.com/pyca/cryptography>`_, it is
+important that you read and understand the :ref:`Cryptographic Primitives <understanding-primitives>`
+section of this manual.

docs/manual/_sources/hardware.rst.txt

@@ -0,0 +1,245 @@
+.. _hardware-main:
+
+***********************
+Communications Hardware
+***********************
+
+One of the truly valuable aspects of Reticulum is the ability to use it over
+almost any conceivable kind of communications medium. The :ref:`interface types<interfaces-main>`
+available for configuration in Reticulum are flexible enough to cover the use
+of most wired and wireless communications hardware available, from decades-old
+packet radio modems to modern millimeter-wave backhaul systems.
+
+If you already have or operate some kind of communications hardware, there is a
+very good chance that it will work with Reticulum out of the box. In case it does
+not, it is possible to provide the necessary glue with very little effort using
+for example the :ref:`PipeInterface<interfaces-pipe>` or the :ref:`TCPClientInterface<interfaces-tcpc>`
+in combination with code like `TCP KISS Server <https://github.com/simplyequipped/tcpkissserver>`_
+by `simplyequipped <https://github.com/simplyequipped>`_.
+
+While this broad support and flexibility is very useful, an abundance of options
+can sometimes make it difficult to know where to begin, especially when you are
+starting from scratch.
+
+This chapter will outline a few different sensible starting paths to get
+real-world functional wireless communications up and running with minimal cost
+and effort. Two fundamental devices categories will be covered, *RNodes* and
+*WiFi-based radios*.
+
+While there are many other device categories that are useful in building Reticulum
+networks, knowing how to employ just these two will make it possible to build
+a wide range of useful networks with little effort.
+
+.. _rnode-main:
+
+RNode
+=====
+
+Reliable and general-purpose long-range digital radio transceiver systems are
+commonly either very expensive, difficult to set up and operate, hard to source,
+power-hungry, or all of the above at the same time. In an attempt to alleviate
+this situation, the transceiver system *RNode* was designed. It is important to
+note that RNode is not one specific device, from one particular vendor, but
+*an open plaform* that anyone can use to build interoperable digital transceivers
+suited to their needs and particular situations.
+
+An RNode is a general purpose, interoperable, low-power and long-range, reliable,
+open and flexible radio communications device. Depending on its components, it can
+operate on many different frequency bands, and use many different modulation
+schemes, but most commonly, and for the purposes of this chapter, we will limit
+the discussion to RNodes using *LoRa* modulation in common ISM bands.
+
+**Avoid Confusion!** RNodes can use LoRa as a *physical-layer modulation*, but it
+does not use, and has nothing to do with the *LoRaWAN* protocol and standard, commonly
+used for centrally controlled IoT devices. RNodes use *raw LoRa modulation*, without
+any additional protocol overhead. All high-level protocol functionality is handled
+directly by Reticulum.
+
+.. _rnode-creating:
+
+Creating RNodes
+^^^^^^^^^^^^^^^
+RNode has been designed as a system that is easy to replicate across time and
+space. You can put together a functioning transceiver using commonly available
+components, and a few open source software tools. While you can design and build RNodes
+completely from scratch, to your exact desired specifications, this chapter
+will explain the easiest possible approach to creating RNodes: Using common
+LoRa development boards. This approach can be boiled down to two simple steps:
+
+1. Obtain one or more supported development boards
+2. Install the RNode firmware with the automated installer
+
+Once the firmware has been installed and provisioned by the install script, it
+is ready to use with any software that supports RNodes, including Reticulum.
+The device can be used with Reticulum by adding an :ref:`RNodeInterface<interfaces-rnode>`
+to the configuration.
+
+.. _rnode-supported:
+
+Supported Boards
+^^^^^^^^^^^^^^^^
+To create one or more RNodes, you will need to obtain supported development
+boards. The following boards are supported by the auto-installer.
+
+LilyGO LoRa32 v2.1
+""""""""""""""""""
+.. image:: graphics/board_t3v21.png
+    :width: 46%
+    :align: center
+
+- **Supported Firmware Lines** v1.x & v2.x
+- **Transceiver IC** Semtech SX1276
+- **Device Platform** ESP32
+- **Manufacturer** `LilyGO <https://lilygo.cn>`_
+
+
+LilyGO LoRa32 v2.0
+""""""""""""""""""
+.. image:: graphics/board_t3v20.png
+    :width: 46%
+    :align: center
+
+- **Supported Firmware Lines** v1.x & v2.x
+- **Transceiver IC** Semtech SX1276
+- **Device Platform** ESP32
+- **Manufacturer** `LilyGO <https://lilygo.cn>`_
+
+
+LilyGO T-Beam
+"""""""""""""
+.. image:: graphics/board_tbeam.png
+    :width: 75%
+    :align: center
+
+- **Supported Firmware Lines** v1.x & v2.x
+- **Transceiver IC** Semtech SX1276
+- **Device Platform** ESP32
+- **Manufacturer** `LilyGO <https://lilygo.cn>`_
+
+
+Heltec LoRa32 v2.0
+""""""""""""""""""
+.. image:: graphics/board_heltec32.png
+    :width: 58%
+    :align: center
+
+- **Supported Firmware Lines** v1.x & v2.x
+- **Transceiver IC** Semtech SX1276
+- **Device Platform** ESP32
+- **Manufacturer** `Heltec Automation <https://heltec.org>`_
+
+
+Unsigned RNode v2.x
+"""""""""""""""""""
+.. image:: graphics/board_rnodev2.png
+    :width: 58%
+    :align: center
+
+- **Supported Firmware Lines** v1.x & v2.x
+- **Transceiver IC** Semtech SX1276
+- **Device Platform** ESP32
+- **Manufacturer** `unsigned.io <https://unsigned.io>`_
+
+
+Unsigned RNode v1.x
+"""""""""""""""""""
+.. image:: graphics/board_rnode.png
+    :width: 50%
+    :align: center
+
+- **Supported Firmware Lines** v1.x
+- **Transceiver IC** Semtech SX1276
+- **Device Platform** AVR ATmega1284p
+- **Manufacturer** `unsigned.io <https://unsigned.io>`_
+
+
+.. _rnode-installation:
+
+Installation
+^^^^^^^^^^^^
+
+Once you have obtained compatible boards, you can install the `RNode Firmware <https://github.com/markqvist/RNode_Firmware>`_
+using the `RNode Configuration Utility <https://github.com/markqvist/rnodeconfigutil>`_.
+Make sure that ``Python3`` and ``pip`` is installed on your system, and then install
+the config utility with ``pip``:
+
+.. code::
+
+   pip3 install rnodeconf
+
+Once installation has completed, it is time to start installing the firmware on your
+devices. Run ``rnodeconf`` in auto-install mode like so:
+
+.. code::
+
+   rnodeconf --autoinstall
+
+The utility will guide you through the installation process by asking a series of
+questions about your hardware. Simply follow the guide, and the utility will
+auto-install and configure your devices
+
+**Important Note!** It is currently recommended to use the v1.x line of the RNode firmware,
+even though the v2.x line is available for early testing. The v2.x line should still be
+considered an experimental pre-release. Only use the v2.x firmware line if you want to test
+out the absolutely newest version, and don't care about stability.
+
+.. _rnode-usage:
+
+Usage with Reticulum
+^^^^^^^^^^^^^^^^^^^^
+When the devices have been installed and provisioned, you can use them with Reticulum
+by adding the :ref:`relevant interface section<interfaces-rnode>` to the configuration
+file of Reticulum. For v1.x firmwares, you will have to specify all interface parameters,
+such as serial port and on-air parameters. For v2.x firmwares, you just need to specify
+the Connection ID of the RNode, and Reticulum will automatically locate and connect to the
+RNode, using the parameters stored in the RNode itself.
+
+.. _rnode-suppliers:
+
+Suppliers
+^^^^^^^^^
+Get in touch if you want to have your RNode supplier listed here, or if you want help to
+get started with producing RNodes.
+
+
+WiFi-based Hardware
+===================
+
+It is possible to use all kinds of both short- and long-range WiFi-based hardware
+with Reticulum. Any kind of hardware that fully supports bridged Ethernet over the
+WiFi interface will work with the :ref:`AutoInterface<interfaces-auto>` in Reticulum.
+Most devices will behave like this by default, or allow it via configuration options.
+
+This means that you can simply configure the physical links of the WiFi based devices,
+and start communicating over them using Reticulum. It is not necessary to enable any IP
+infrastructure such as DHCP servers, DNS or similar, as long as at least Ethernet is
+available, and packets are passed transparently over the physical WiFi-based devices.
+
+.. only:: html
+
+    .. image:: graphics/radio_rblhg5.png
+        :width: 49%
+
+    .. image:: graphics/radio_is5ac.png
+        :width: 49%
+
+Below is a list of example WiFi (and similar) radios that work well for high capacity
+Reticulum links over long distances:
+
+- `Ubiquiti airMAX radios <https://store.ui.com/collections/operator-airmax-devices>`_
+- `Ubiquiti LTU radios <https://store.ui.com/collections/operator-ltu>`_
+- `MikroTik radios <https://mikrotik.com/products/group/wireless-systems>`_
+
+This list is by no means exhaustive, and only serves as a few examples of radio hardware
+that is relatively cheap while providing long range and high capacity for Reticulum
+networks. As in all other cases, it is also possible for Reticulum to co-exist with IP
+networks running concurrently on such devices.
+
+Combining Hardware Types
+========================
+
+It is useful to combine different link and hardware types when designing and
+building a network. One useful design pattern is to employ high-capacity point-to-point
+links based on WiFi or millimeter-wave radios (with high-gain directional antennas)
+for the network backbone, and using LoRa-based RNodes for covering large areas with
+connectivity for client devices.

docs/manual/_sources/index.rst.txt

@@ -0,0 +1,38 @@
+******************************
+Reticulum Network Stack Manual
+******************************
+This manual aims to provide you with all the information you need to
+understand Reticulum, build networks or develop programs using it, or
+to participate in the development of Reticulum itself.
+
+.. only:: html
+
+   Table Of Contents
+   =================
+
+.. toctree::
+   :maxdepth: 3
+
+   whatis
+   gettingstartedfast
+   using
+   understanding
+   hardware
+   interfaces
+   networks
+   examples
+   support
+
+.. toctree::
+   :maxdepth: 2
+
+   reference
+
+
+.. only:: html
+
+   Indices and Tables
+   ==================
+
+   * :ref:`genindex`
+   * :ref:`search`