Updated changelog

Extend RAK4631 support

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,11 @@
+blank_issues_enabled: false
+contact_links:
+  - name: ✨ Feature Request or Idea
+    url: https://github.com/markqvist/Reticulum/discussions/new?category=ideas
+    about: Propose and discuss features and ideas
+  - name: 💬 Questions, Help & Discussion
+    about: Ask anything, or get help
+    url: https://github.com/markqvist/Reticulum/discussions/new/choose
+  - name: 📖 Read the Reticulum Manual
+    url: https://markqvist.github.io/Reticulum/manual/
+    about: The complete documentation for Reticulum

.github/ISSUE_TEMPLATE/🐛-bug-report.md

@@ -0,0 +1,35 @@
+---
+name: "\U0001F41B Bug Report"
+about: Report a reproducible bug
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Read the Contribution Guidelines**
+Before creating a bug report on this issue tracker, you **must** read the [Contribution Guidelines](https://github.com/markqvist/Reticulum/blob/master/Contributing.md). Issues that do not follow the contribution guidelines **will be deleted without comment**.
+
+- The issue tracker is used by developers of this project. **Do not use it to ask general questions, or for support requests**.
+- Ideas and feature requests can be made on the [Discussions](https://github.com/markqvist/Reticulum/discussions). **Only** feature requests accepted by maintainers and developers are tracked and included on the issue tracker. **Do not post feature requests here**.
+- After reading the [Contribution Guidelines](https://github.com/markqvist/Reticulum/blob/master/Contributing.md), delete this section from your bug report.
+
+**Describe the Bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Describe in detail how to reproduce the bug.
+
+**Expected Behavior**
+A clear and concise description of what you expected to happen.
+
+**Logs & Screenshots**
+Please include any relevant log output. If applicable, also add screenshots to help explain your problem.
+
+**System Information**
+- OS and version
+- Python version
+- Program version
+
+**Additional context**
+Add any other context about the problem here.

.gitignore

@@ -10,6 +10,6 @@ docs/build
 rns*.egg-info
 profile.data
 tests/rnsconfig/storage
-tests/rnsconfig/logfile
+tests/rnsconfig/logfile*
 *.data
 *.result

Changelog.md

@@ -1,3 +1,403 @@
+### 2024-05-18: RNS β 0.7.5
+
+This release adds support for AutoInterface on Windows platforms, fixes a number of bugs and adds several new supported boards to `rnodeconf`. Thanks to @faragher, @jacobeva and @liamcottle who contributed to this release!
+
+**Changes**
+- Added support for AutoInterface on Windows
+- Added support for recursive path resolution for clients on roaming-mode interfaces
+- Added RAK4631 support to `rnodeconf`
+- Added LilyGO T3S3 support to `rnodeconf`
+- Added ability to get target and calculated hashes via `rnodeconf`
+- Fixed DTR timing making flashing fail on Windows in `rnodeconf`
+- Fixed various output and menu bugs in `rnodeconf`
+
+**Release Hashes**
+```
+99ec876966afdea45fcf164242c8e76c284f9e3edf09fb907638fba76e1324b1 rns-0.7.5-py3-none-any.whl
+11156f6301707e4d17ff2ca6d58059bc8ba6fe1bbc4dc3de165dd96dc41ee75f rnspure-0.7.5-py3-none-any.whl
+```
+
+### 2024-05-05: RNS β 0.7.4
+
+This maintenance release fixes a number of bugs, improves path requests and responses, and adds several useful features and capabilities. Thanks to @cobraPA, @jschulthess, @thiaguetz and @nothingbutlucas who contributed to this release!
+
+**Changes**
+- Added support for flashing and autoinstalling Heltec V3 boards to `rnodeconf`
+- Added custom EEPROM bootstrapping capabilities to `rnodeconf`
+- Added ability to load identities from file to Echo and Link examples
+- Added ability to specify multicast address type in AutoInterface configuration
+- Added link getter to resource advertisement class
+- Improved path response logic and timing
+- Improved path request timing
+- Fixed a bug in Link Request proof delivery on unknown hop count paths
+- Fixed broken link packet routing in topologies where transport packets leak to non-intended instances in the link chain
+- Fixed typos in documentation
+
+**Release Hashes**
+```
+f5c35f1b8720778eb508b687d66334d01b4ab266b2d8c2bc186702220dcaae29  rns-0.7.4-py3-none-any.whl
+9eaa7170f97dad49551136965d3fcc971b56b1c2eda48c24b9ffd58d71daa016  rnspure-0.7.4-py3-none-any.whl
+```
+
+### 2024-03-09: RNS β 0.7.3
+
+This release adds the ability to specify custom firmware URLs for flashing boards with `rnodeconf`. Thanks to @attermann who contributed to this release!
+
+**Changes**
+- Added ability to specify custom firmware URLs for flashing boards with `rnodeconf`
+
+**Release Hashes**
+```
+bb24445ae9a3a63d348e4d7fe80b750608f257851b97b38fadab929b7a774bc9 rns-0.7.3-py3-none-any.whl
+1b148d013103c35ba9a8e105082ef50686c130676d0a560ed709cb546129287e rnspure-0.7.3-py3-none-any.whl
+```
+
+### 2024-03-02: RNS β 0.7.2
+
+This maintenance release improves memory consumption, fixes a few bugs, and adds ability to flash new boards with `rnodeconf`.
+
+**Changes**
+- Added ability to flash new boards with `rnodeconf`, including T3 boards with TCXOs
+- Improved memory consumption on Transport Instances with many interfaces
+- Fixed a bug that could cause the on-disk known destinations store to become corrupted
+
+**Release Hashes**
+```
+3ce3ba80d5ae8d19c6b55bd51f44bd4beccbcea31554cb1f0d65428e4587b3d6 rns-0.7.2-py3-none-any.whl
+83f914aaba2a8929a8cee95830a847e190197232a0cca4e7b906b15c6bbf8296 rnspure-0.7.2-py3-none-any.whl
+```
+
+### 2024-02-14: RNS β 0.7.1
+
+This release adds support for RNodes based on SX1262, SX1268 and SX1280 modems, and fixes a number of bugs. Thanks to @jacobeva, who contributed to this release!
+
+**Changes**
+- Added support for SX1262, SX1268 and SX1280 based RNodes
+- Updated `rnodeconf` to allow flashing T-Beam devices with SX126x chips
+- Fixed an invalid RSSI offset reference
+
+**Release Hashes**
+```
+8ecfbb42b6a699fd4ac5374ab5640e4bb164e80bb9ab4401ea82da132e497877 rns-0.7.1-py3-none-any.whl
+e0ab487305ba1aee2d16044640e7eb72d332bbf51aeb0b8bf984d037a64cb493 rnspure-0.7.1-py3-none-any.whl
+```
+
+### 2024-01-17: RNS β 0.7.0
+
+This maintenance release fixes a number of bugs. Thanks to @jooray and @jacobeva, who contributed to this release!
+
+**Changes**
+- Fixed large resource transfers failing under some conditions
+- Fixed a potential division by zero
+- Fixed a missing check on malformed advertisement packets
+- Fixed a formatting issue in `rnprobe`
+- Improved resource timeout calculations
+
+**Release Hashes**
+```
+0dc2abe5373b9afadfba7ec05bf7ddeff659c004aa339a94001ebed5b46f5b47 rns-0.7.0-py3-none-any.whl
+97f6e65a20b53bbdccd54b4d2bdaa36dc1712e144a55f40800c63fe7113819a5 rnspure-0.7.0-py3-none-any.whl
+```
+
+### 2023-12-07: RNS β 0.6.9
+
+This release adds a few convenience functions to the `rnid` utility, and improves roaming support on Android.
+
+**Changes**
+- Added identity import and export in hex, base32 and base64 formats to the `rnid` utility.
+- Added better carrier change detection for AutoInterface on Android.
+
+**Release Hashes**
+```
+258daf22cb6e72c6cd04fe94447daedf51dfd968eb2f3370eab9c71ad0898dd0 rns-0.6.9-py3-none-any.whl
+3644b64af5b4efd3969172bf0cf95ae1afba6c8ea99ce47d8e49e31a832bbaf8 rnspure-0.6.9-py3-none-any.whl
+```
+
+### 2023-11-14: RNS β 0.6.8
+
+This maintenance release fixes a single bug.
+
+**Bugfixes**
+- Fixed packet receipts not being initialised in time for arriving proofs on fast interfaces
+
+**Release Hashes**
+```
+3ffb01f3f45e35105ea30e60e5e493ba50528df38b4ea62672c9e1c093073b1c rns-0.6.8-py3-none-any.whl
+de372814082ef7db59f4b2745b1f22b2ef9d97815190ec16c0596ba20406e0fb rnspure-0.6.8-py3-none-any.whl
+```
+
+### 2023-11-06: RNS β 0.6.7
+
+This maintenance release improves tranport performance and fixes a logging bug.
+
+**Changes**
+- Improved local and remote transport performance by approximately 6x on faster links
+- Significantly decreased latency over faster links
+
+**Bugfixes**
+- Fixed logging an error message when local clients connect while shared instance is still starting up
+
+**Release Hashes**
+```
+c37dd1f59e037841f69ec518deecdae6719f978947de2473f04e7d95247805ac rns-0.6.7-py3-none-any.whl
+1e2dcb44ec7271a4d26180db138fc54dce6d0d3cf3f816432d4d6a4b1cf83868 rnspure-0.6.7-py3-none-any.whl
+```
+
+### 2023-11-04: RNS β 0.6.6
+
+This maintenance release improves transfers over unreliable links and fixes a bug in requests.
+
+**Changes**
+- Improved reliability of resource transfers over very slow and unreliable links
+
+**Bugfixes**
+- Fixed a bug that could cause requests to timeout prematurely
+
+**Release Hashes**
+```
+b1127745750a43cd7389212d31aa09ccc735ab2d69e3b80bd28874f10082c322 rns-0.6.6-py3-none-any.whl
+bf5ba5da4f37b93c14817367952cda63787ec88bbe601e41c13fcbb3fc22b6b6 rnspure-0.6.6-py3-none-any.whl
+```
+
+### 2023-11-02: RNS β 0.6.5
+
+This release fixes a bug in path rediscovery for shared instance clients.
+
+**Bugfixes**
+- Fixed a bug in path rediscovery for shared instance clients
+
+**Release Hashes**
+```
+5d54a5cfebe907c759351357a8f7d771670c895ff57f1325bf7fec42bcb46ba3 rns-0.6.5-py3-none-any.whl
+accd2855e18ff06455b9454957388089e293073ec7093c64dee0dc7aa46ecb46 rnspure-0.6.5-py3-none-any.whl
+```
+
+### 2023-11-02: RNS β 0.6.4
+
+This release fixes a number of bugs that had crept in while adding the new ultra low bandwidth link timing and faster path rediscovery mechanisms.
+
+**Changes**
+- Adjusted link timings for better support of very slow mediums
+- Adjusted bluetooth read timeouts to account for occasional high latency in congested 2.4GHz environments
+- Added a probe count option to the `rnprobe` utility.
+
+**Bugfixes**
+- Fixed a missing timeout calculation
+- Fixed a redundant path request on path rediscovery
+- Fixed missing path state resetting on stale path rediscovery
+- Fixed a bug that could cause an attribute to be uninitialised
+
+**Release Hashes**
+```
+566c725f68aa154eaca0880c894a39503027bf91714f17691e51d047800444c0 rns-0.6.4-py3-none-any.whl
+a3a447fd40bf02fdb982523de0e4e9933e8e4cd4d4bd478254ea7dcac29e3fc1 rnspure-0.6.4-py3-none-any.whl
+```
+
+### 2023-10-31: RNS β 0.6.3
+
+This release brings a series of under-the-hood reliability improvements and bugfixes. But most notably, Reticulum can now establish links over even ultra low bandwidth mediums, all the way down to 5 bits per second.
+
+Thanks to @jschulthess, who contributed to this release!
+
+**Changes**
+- Implemented link establishment on ultra low bandwidth links
+- Added link quality calculations to RNode interfaces
+- Added physical layer link stats to Link and Packet classes
+- Added userspace service documentation to the manual
+- Improved path rediscovery in quickly changing topographies
+- Improved shared interface reconnection on service restart
+- Improved exception handling on interface detachment
+- Updated formatted print functions
+
+**Bugfixes**
+- Fixed a missing USB command definition in the RNode interface driver
+- Fixed a bug in link error handling that could cause an interface to detach
+
+**Release Hashes**
+```
+1f54d4c6ff7ab7721089cbee6630783765f65efd51312879c0d3e5bee3ceab2f rns-0.6.3-py3-none-any.whl
+5a90840f0fc9f1a62a3c37b514fb6222fd701a30024275dae8bcc27e29d40f25 rnspure-0.6.3-py3-none-any.whl
+```
+
+### 2023-10-07: RNS β 0.6.2
+
+This maintenance release adds the ability to specify the shared instance RPC key in the Reticulum config file, making it possible to use all Reticulum functionality in the terminal on Android.
+
+**Changes**
+- Added configuration option to specify shared instance RPC key
+- Reordered airtime stats in `rnstatus`
+- Updated log levels on Android
+
+**Bugfixes**
+- Adding missing superclass init on Android interfaces
+
+**Release Hashes**
+```
+a9958ad90f34f344003e18077f7abd3fa85666a39dc0cae8580071820dee13f9 rns-0.6.2-py3-none-any.whl
+e68e8837d35d1a07a82c4b0e9db50ceace737a650e6e7e9ce2d9a013fd28f529 rnspure-0.6.2-py3-none-any.whl
+```
+
+### 2023-10-01: RNS β 0.6.1
+
+This release brings a number of bugfixes, along with useful new mechanisms for ensuring network stability under high, non-constructive and unusual announce load situation.
+
+**Changes**
+- Added announce ingress rate control for new and unknown destinations
+- Added per-interface announce frequency monitoring to the transport engine
+- Added per-interface announce burst hold queues
+- Added announce frequency statistics to `rnstatus`
+- Added option to sort `rnstatus` output according to various metrics
+- Added timeout options to `rnprobe`
+- Added ability to drop all paths via a specific transport instance to `rnpath`
+- Added new options and features to documentation and manual
+
+**Bugfixes**
+- Fixed announce queue not clearing all announces with exceeded retry limit at the same time
+- Fixed a bug that caused local packet RSSI and SNR cache to get stuck
+- Fixed output formatting in `rncp`
+- Fixed `rnid` not allowing single-aspect destination names
+- Fixed a number of typos in the documentation
+
+**Release Hashes**
+```
+461e5cafa7560dcd3ec047141d10f0f48f151c36e1af1d65ec6c65f732cea46a rns-0.6.1-py3-none-any.whl
+be6a4a6069f2d050e21582f2cf9d3bb59ed4040a0f07761a540bd752d90ea591 rnspure-0.6.1-py3-none-any.whl
+```
+
+#### 2023-09-21: RNS β 0.6.0
+
+This release brings a few performance improvements, additions to the included utilities, and fixes a number of bugs.
+
+**Changes**
+- Added ability to run automatic probe responder on Transport Instances
+- Improved `rnprobe` utility
+- Improved AutoInterface peering on Android devices
+- Improved Transport performance
+- Improved path re-discovery when local nodes roam to other network segments
+- Updated various parts of the documentation
+
+**Bugfixes**
+- Fixed missing timeout check in `rncp`
+- Fixed missing link status check on `Identify()` call, which could lead to an unnecessary exception
+
+**Release Hashes**
+```
+88a26b1593e82a628dab96dbe8820548aea0159235f730fa992bf1833db59246 rns-0.6.0-py3-none-any.whl
+bcee416e4fb52346d01f6e0c46b1cebf84b127cc516603367fc2ae00a4149fa2 rnspure-0.6.0-py3-none-any.whl
+```
+
+### 2023-09-19: RNS β 0.5.9
+
+This release brings major efficiency improvements to `Channel` and `Buffer` classes, adds a range of usability improvements to the included utilities and fixes a number of bugs.
+
+**Changes**
+- Improved `Channel` sequencing, retries and transfer efficiency
+- Added adaptive compression to `Buffer` class
+- Added `rnid` examples and documentation to manual
+- Added silent mode to `rncp`
+- Added remote fetch mode to `rncp`
+- Added allowed_identities file support to `rncp`
+- Added Transport Instance uptime to `rnstatus` output
+- Added channel CSMA parameter stats to RNode Interface `rnstatus` output
+- Added ability to set custom RNode OLED display address with rnodeconf
+
+**Bugfixes**
+- Fixed inadverdent AutoInterface multi-IF deque hit for resource transfer retries
+- Fixed invalid path for firmware hash generation while using extracted firmware to autoinstall in `rnodeconf`
+- Fixed various minor missing error checks
+- Fixed `rnid` status output bug
+
+**Release Hashes**
+```
+207ab20bd68bab16b417fbd41a4ecdbcf1e2f6fa553d48df6c8fc181b6e84dac rns-0.5.9-py3-none-any.whl
+93f0965567dfc2c43f3d703481fe1a7d7b1b8d0b3837ad41c37f28a8af5c1acc rnspure-0.5.9-py3-none-any.whl
+```
+
+### 2023-09-14: RNS β 0.5.8
+
+This maintenance release contains a number of usability improvements to Reticulum and related tools.
+
+**Changes**
+- Various documentation updates
+- Improved path-resolution in mixed networks with roaming-mode nodes
+- Added channel load and airtime stats to `rnstatus` output
+
+**Release Hashes**
+```
+27ba5cdc4724fc8c7211c3b504f097f6adf47f7b80775e6297e4c4e621ef6348 rns-0.5.8-py3-none-any.whl
+1ea1c949763c9478ec48f064f7f7864d9f859101ab91b44400879371f490800f rnspure-0.5.8-py3-none-any.whl
+```
+
+### 2023-08-14: RNS β 0.5.7
+
+This maintenance release contains a number of bugfixes and quality improvements to Reticulum and related tools.
+
+**Changes**
+- Added bytes input to destination hash convenience functions
+- Fixed possible invalid comparison in link watchdog job
+- Add option to `rnodeconf` to set baud rate when flashing
+- Added better explanation in `rnodeconf` when flashing fails
+- Fixed EEPROM dump directory in `rnodeconf`
+
+**Release Hashes**
+```
+867fbb5c73c2a49a75e1f8f3e9f376b507b683328e26c64d4387acd0cc1dbbc7 rns-0.5.7-py3-none-any.whl
+7bab2865264b32208e023b5c4bbe88c37f51e3176ca4a8cf332d95f59a6d7f2c rnspure-0.5.7-py3-none-any.whl
+```
+
+### 2023-07-09: RNS β 0.5.6
+
+This maintenance release contains a few bugfixes.
+
+**Changes**
+- Fixed an issue in `rnodeconf` that prevented Heltec LoRa32 v2 boards from being flashed.
+- Fixed a typo in the `rnid` utility.
+
+**Release Hashes**
+```
+255a5b4bac28326c6b2cc85f43b26dcb0606404a4abd2dfa8244937155838973 rns-0.5.6-py3-none-any.whl
+1510b6da4641ceaa4c599a142e498c7e2c1ae12035868f9db1c111e5600161e9 rnspure-0.5.6-py3-none-any.whl
+```
+
+### 2023-06-13: RNS β 0.5.5
+
+This maintenance release brings a single bugfix.
+
+**Changes**
+- Fixed a race condition for link initiators on timed out link establishments.
+
+**Release Hashes**
+```
+4ae61d28bf981a7cb853c179e9de3b56b350d2dc984fb671a21d38c4ce5b449e rns-0.5.5-py3-none-any.whl
+ed417cbd3c90e9f1b68565a3411ca5c9bc936b495300fd1ace3c4a6414aabd5a rnspure-0.5.5-py3-none-any.whl
+```
+
+### 2023-05-19: RNS β 0.5.4
+
+This maintenance release brings a single bugfix.
+
+**Changes**
+- Fixed a potential race condition when timed-out link receives a late establishment proof a few milliseconds after it has timed out.
+
+**Release Hashes**
+```
+71b42fe737da97a4b63bb227c29bb67854a7f003c9585f085b0ff68c8f460815 rns-0.5.4-py3-none-any.whl
+af6949d581445444f57cfca75756200e7c509a6fc66483d859716ce6a06064db rnspure-0.5.4-py3-none-any.whl
+```
+
+### 2023-05-19: RNS β 0.5.3
+
+This maintenance release brings a single, but important bugfix.
+
+**Changes**
+- Fixed a bug that could cause data corruption to occur over when using `Buffer` instances.
+
+**Release Hashes**
+```
+f23c8d655c9e80a12a6728495aec56f19f27184d3d8e6b6ed6184b0e89d4be35 rns-0.5.3-py3-none-any.whl
+2c692a2153bb766a9dc2391340a06f429c13a75b86b746b69c6fcd5a4fe5ee33 rnspure-0.5.3-py3-none-any.whl
+```
+
 ### 2023-05-12: RNS β 0.5.2
 
 This maintenance release brings a number of bugfixes and improvements.
@@ -764,4 +1164,4 @@ This was the first publicly available pre-release alpha of Reticulum.
 
 ### 2016-05-29: Inintial Repository Commit
 
-The first commit to the Reticulum reference implementation was 9a9630cfd29e11ace3f12716ddb4dff0e5419b4b, which occurred on Sunday, the 22nd of May 2016.
+The first commit to the Reticulum reference implementation was 9a9630cfd29e11ace3f12716ddb4dff0e5419b4b, which occurred on Sunday, the 29th of May 2016.

Contributing.md

@@ -2,22 +2,42 @@
 
 Welcome, and thank you for your interest in contributing to Reticulum!
 
-Apart from writing code, there are many ways in which you can contribute. Before getting started, please read these guidelines.
+Apart from writing code, there are many ways in which you can contribute. Before interacting with this community, read these short and simple guidelines.
+
+## Expected Conduct
+
+First and foremost, there is one simple requirement for taking part in this community: While we primarily interact virtually, your actions matter and have real consequences. Therefore: **Act like a responsible, civilized person** - also in the face of disputes and heated disagreements. Speak your mind here, discussions are welcome. Just do so in the spirit of being face-to-face with everyone else. Thank you.
 
 ## Asking Questions
 
-If you want to ask a question, do not open an issue.
+If you want to ask a question, **do not open an issue**. The issue tracker is used by people *working on Reticulum* to track bugs, issues and improvements.
 
-Instead, ask away on the [discussions](https://github.com/markqvist/Reticulum/discussions) or on the [Reticulum Matrix channel](https://unsigned.io/contact.html#reticulum:matrix.org) at `#reticulum:matrix.org`
+Instead, ask away on the [discussions](https://github.com/markqvist/Reticulum/discussions) or on the [Reticulum Matrix channel](https://matrix.to/#/#reticulum:matrix.org) at `#reticulum:matrix.org`
 
-## Providing Feedback
+## Providing Feedback & Ideas
 
-Likewise, feedback, ideas and feature requests are a very welcome way to contribute, and should also be posted on the [discussions](https://github.com/markqvist/Reticulum/discussions), or on the [Reticulum Matrix channel](https://unsigned.io/contact.html#reticulum:matrix.org) at `#reticulum:matrix.org`
+Likewise, feedback, ideas and feature requests are a very welcome way to contribute, and should also be posted on the [discussions](https://github.com/markqvist/Reticulum/discussions), or on the [Reticulum Matrix channel](https://matrix.to/#/#reticulum:matrix.org) at `#reticulum:matrix.org`.
+
+Please do not post feature requests or general ideas on the issue tracker, or in direct messages to the primary developers. You are much more likely to get a response and start a constructive discussion by posting your ideas in the public channels created for these purposes.
 
 ## Reporting Issues
 
-If you have found a bug or issue in Reticulum, please report it on the [issue tracker](https://github.com/markqvist/Reticulum/issues).
+If you have found a bug or issue in this project, please report it using the [issue tracker](https://github.com/markqvist/Reticulum/issues). If at all possible, be sure to include details on how to reproduce the bug.
+
+Anything submitted to the issue tracker that does not follow these guidelines will be closed and removed without comments or explanation.
 
 ## Writing Code
 
-If you are interested in contributing code, fixing open issues or adding features, please coordinate the effort with the maintainer or one of the main developers first, to ensure your efforts are in alignment with the [Roadmap](./Roadmap.md) and current development focus.
+If you are interested in contributing code, fixing open issues or adding features, please coordinate the effort with the maintainer or one of the main developers **before** submitting a pull request. Before deciding to contribute, it is also a good idea to ensure your efforts are in alignment with the [Roadmap](./Roadmap.md) and current development focus.
+
+Pull requests have a high chance of being accepted if they are:
+
+- In alignment with the [Roadmap](./Roadmap.md) or solve an open issue or feature request
+- Sufficiently tested to work with all API functions, and pass the standard test suite
+- Functionally and conceptually complete and well-designed
+- Not simply formatting or code style changes
+- Well-documented
+
+Even new ideas and proposals that have not been approved by a maintainer, or fall outside the established roadmap, are *occasionally* accepted - if they possess the remaining of the above qualities. If not, they will be closed and removed without comments or explanation.
+
+By contributing code to this project, you agree that copyright for the code is transferred to the Reticulum maintainers and that the code is irrevocably placed under the [MIT license](./LICENSE).

Examples/Announce.py

@@ -32,7 +32,7 @@ def program_setup(configpath):
     # Destinations are endpoints in Reticulum, that can be addressed
     # and communicated with. Destinations can also announce their
     # existence, which will let the network know they are reachable
-    # and autoomatically create paths to them, from anywhere else
+    # and automatically create paths to them, from anywhere else
     # in the network.
     destination_1 = RNS.Destination(
         identity,
@@ -53,7 +53,7 @@ def program_setup(configpath):
     )
 
     # We configure the destinations to automatically prove all
-    # packets adressed to it. By doing this, RNS will automatically
+    # packets addressed to it. By doing this, RNS will automatically
     # generate a proof for each incoming packet and transmit it
     # back to the sender of that packet. This will let anyone that
     # tries to communicate with the destination know whether their

Examples/Echo.py

@@ -5,6 +5,7 @@
 # of the packet.                                         #
 ##########################################################
 
+import os
 import argparse
 import RNS
 
@@ -27,8 +28,19 @@ def server(configpath):
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
     
-    # Randomly create a new identity for our echo server
-    server_identity = RNS.Identity()
+    # Load identity from file if it exist or randomley create
+    if configpath:
+        ifilepath = "%s/storage/identitiesy/%s" % (configpath,APP_NAME)
+    else:
+        ifilepath = "%s/storage/identities/%s" % (RNS.Reticulum.configdir,APP_NAME)
+    if os.path.exists(ifilepath):
+        # Load identity from file
+        server_identity = RNS.Identity.from_file(ifilepath)
+        RNS.log("loaded identity from file: "+ifilepath, RNS.LOG_VERBOSE)
+    else:
+        # Randomly create a new identity for our echo example
+        server_identity = RNS.Identity()
+        RNS.log("created new identity", RNS.LOG_VERBOSE)
 
     # We create a destination that clients can query. We want
     # to be able to verify echo replies to our clients, so we
@@ -46,7 +58,7 @@ def server(configpath):
     )
 
     # We configure the destination to automatically prove all
-    # packets adressed to it. By doing this, RNS will automatically
+    # packets addressed to it. By doing this, RNS will automatically
     # generate a proof for each incoming packet and transmit it
     # back to the sender of that packet.
     echo_destination.set_proof_strategy(RNS.Destination.PROVE_ALL)
@@ -328,4 +340,4 @@ if __name__ == "__main__":
                 client(args.destination, configarg, timeout=timeoutarg)
     except KeyboardInterrupt:
         print("")
-        exit()
+        exit()

Examples/Filetransfer.py

@@ -449,8 +449,7 @@ def link_established(link):
     # And set up a small job to check for
     # a potential timeout in receiving the
     # file list
-    thread = threading.Thread(target=filelist_timeout_job)
-    thread.setDaemon(True)
+    thread = threading.Thread(target=filelist_timeout_job, daemon=True)
     thread.start()
 
 # This job just sleeps for the specified

Examples/Link.py

@@ -28,8 +28,20 @@ def server(configpath):
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
     
-    # Randomly create a new identity for our link example
-    server_identity = RNS.Identity()
+    # Load identity from file if it exist or randomley create
+    if configpath:
+        ifilepath = "%s/storage/identitiesy/%s" % (configpath,APP_NAME)
+    else:
+        ifilepath = "%s/storage/identities/%s" % (RNS.Reticulum.configdir,APP_NAME)
+    RNS.log("ifilepath: %s" % ifilepath)
+    if os.path.exists(ifilepath):
+        # Load identity from file
+        server_identity = RNS.Identity.from_file(ifilepath)
+        RNS.log("loaded identity from file: "+ifilepath, RNS.LOG_VERBOSE)
+    else:
+        # Randomly create a new identity for our link example
+        server_identity = RNS.Identity()
+        RNS.log("created new identity", RNS.LOG_VERBOSE)
 
     # We create a destination that clients can connect to. We
     # want clients to create links to this destination, so we
@@ -288,4 +300,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        exit()

Examples/Minimal.py

@@ -25,7 +25,7 @@ def program_setup(configpath):
     # Destinations are endpoints in Reticulum, that can be addressed
     # and communicated with. Destinations can also announce their
     # existence, which will let the network know they are reachable
-    # and autoomatically create paths to them, from anywhere else
+    # and automatically create paths to them, from anywhere else
     # in the network.
     destination = RNS.Destination(
         identity,
@@ -36,7 +36,7 @@ def program_setup(configpath):
     )
 
     # We configure the destination to automatically prove all
-    # packets adressed to it. By doing this, RNS will automatically
+    # packets addressed to it. By doing this, RNS will automatically
     # generate a proof for each incoming packet and transmit it
     # back to the sender of that packet. This will let anyone that
     # tries to communicate with the destination know whether their

FUNDING.yml

@@ -0,0 +1,2 @@
+ko_fi: markqvist
+custom: "https://unsigned.io/donate"

LICENSE

@@ -1,6 +1,6 @@
 MIT License, unless otherwise noted
 
-Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+Copyright (c) 2016-2024 Mark Qvist / unsigned.io
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -35,7 +35,7 @@ userland, and can run on practically any system that runs Python 3.
 ## Read The Manual
 The full documentation for Reticulum is available at [markqvist.github.io/Reticulum/manual/](https://markqvist.github.io/Reticulum/manual/).
 
-You can also [download the Reticulum manual as a PDF](https://github.com/markqvist/Reticulum/raw/master/docs/Reticulum%20Manual.pdf)
+You can also download the [Reticulum manual as a PDF](https://github.com/markqvist/Reticulum/raw/master/docs/Reticulum%20Manual.pdf) or [as an e-book in EPUB format](https://github.com/markqvist/Reticulum/raw/master/docs/Reticulum%20Manual.epub).
 
 For more info, see [reticulum.network](https://reticulum.network/)
 
@@ -45,7 +45,7 @@ For more info, see [reticulum.network](https://reticulum.network/)
 - Initiator anonymity, communicate without revealing your identity
 - Asymmetric X25519 encryption and Ed25519 signatures as a basis for all communication
 - Forward Secrecy with ephemeral Elliptic Curve Diffie-Hellman keys on Curve25519
-- Reticulum uses the [Fernet](https://github.com/fernet/spec/blob/master/Spec.md) specification for on-the-wire / over-the-air encryption
+- Reticulum uses the following format for encrypted tokens:
   - Keys are ephemeral and derived from an ECDH key exchange on Curve25519
   - AES-128 in CBC mode with PKCS7 padding
   - HMAC using SHA256 for authentication
@@ -59,11 +59,13 @@ For more info, see [reticulum.network](https://reticulum.network/)
   - The API is very easy to use, and provides transfer progress
 - Lightweight, flexible and expandable Request/Response mechanism
 - Efficient link establishment
-  - Total bandwidth cost of setting up an encrypted link is 3 packets totaling 297 bytes
+  - Total cost of setting up an encrypted and verified link is only 3 packets, totalling 297 bytes
   - Low cost of keeping links open at only 0.44 bits per second
+- Reliable sequential delivery with Channel and Buffer mechanisms
 
 ## Roadmap
-While Reticulum is already a fully featured and functional networking stack, many improvements and additions are actively being worked on, and planned for the future.
+While Reticulum is already a fully featured and functional networking stack,
+many improvements and additions are actively being worked on, and planned for the future.
 
 To learn more about the direction and future of Reticulum, please see the [Development Roadmap](./Roadmap.md).
 
@@ -78,7 +80,7 @@ following resources.
 
 ## Where can Reticulum be used?
 Over practically any medium that can support at least a half-duplex channel
-with 500 bits per second throughput, and an MTU of 500 bytes. Data radios,
+with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios,
 modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes,
 WiFi and Ethernet devices, free-space optical links, and similar systems are
 all examples of the types of physical devices Reticulum can use.
@@ -108,14 +110,28 @@ you want to do. For full details and examples, have a look at the
 [Getting Started Fast](https://markqvist.github.io/Reticulum/manual/gettingstartedfast.html) 
 section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/).
 
-To simply install Reticulum and related utilities on your system, the easiest way is via pip:
+To simply install Reticulum and related utilities on your system, the easiest way is via `pip`.
+You can then start any program that uses Reticulum, or start Reticulum as a system service with
+[the rnsd utility](https://markqvist.github.io/Reticulum/manual/using.html#the-rnsd-utility).
 
 ```bash
 pip install rns
 ```
 
-You can then start any program that uses Reticulum, or start Reticulum as a
-system service with [the rnsd utility](https://markqvist.github.io/Reticulum/manual/using.html#the-rnsd-utility).
+If you are using an operating system that blocks normal user package installation via `pip`,
+you can return `pip` to normal behaviour by editing the `~/.config/pip/pip.conf` file,
+and adding the following directive in the `[global]` section:
+
+```text
+[global]
+break-system-packages = true
+```
+
+Alternatively, you can use the `pipx` tool to install Reticulum in an isolated environment:
+
+```bash
+pipx install rns
+```
 
 When first started, Reticulum will create a default configuration file,
 providing basic connectivity to other Reticulum peers that might be locally
@@ -139,12 +155,15 @@ section of the [Reticulum Manual](https://markqvist.github.io/Reticulum/manual/)
 - An interface status utility called `rnstatus`, that displays information about interfaces
 - The path lookup and management tool `rnpath` letting you view and modify path tables
 - A diagnostics tool called `rnprobe` for checking connectivity to destinations
-- A simple file transfer program called `rncp` making it easy to copy files to remote systems
+- A simple file transfer program called `rncp` making it easy to transfer files between systems
+- The identity management and encryption utility `rnid` let's you manage Identities and encrypt/decrypt files
 - The remote command execution program `rnx` let's you run commands and
   programs and retrieve output from remote systems
 
 All tools, including `rnx` and `rncp`, work reliably and well even over very
-low-bandwidth links like LoRa or Packet Radio.
+low-bandwidth links like LoRa or Packet Radio. For full-featured remote shells
+over Reticulum, also have a look at the [rnsh](https://github.com/acehoss/rnsh)
+program.
 
 ## Supported interface types and devices
 
@@ -172,8 +191,8 @@ functionality and performance on low-bandwidth mediums. The goal is to
 provide a dynamic performance envelope from 250 bits per second, to 1 gigabit
 per second on normal hardware.
 
-Currently, the usable performance envelope is approximately 500 bits per second
-to 20 megabits per second, with physical mediums faster than that not being
+Currently, the usable performance envelope is approximately 150 bits per second
+to 40 megabits per second, with physical mediums faster than that not being
 saturated. Performance beyond the current level is intended for future
 upgrades, but not highly prioritised at this point in time.
 
@@ -227,11 +246,11 @@ I2P. Just add one of the following interfaces to your Reticulum configuration
 file:
 
 ```
-# TCP/IP interface to the RNS Dublin Hub
-  [[RNS Testnet Dublin]]
+# TCP/IP interface to the RNS Amsterdam Hub
+  [[RNS Testnet Amsterdam]]
     type = TCPClientInterface
     enabled = yes
-    target_host = dublin.connect.reticulum.network
+    target_host = amsterdam.connect.reticulum.network
     target_port = 4965
 
 # TCP/IP interface to the BetweenTheBorders Hub (community-provided)
@@ -245,7 +264,7 @@ file:
   [[RNS Testnet I2P Hub]]
     type = I2PInterface
     enabled = yes
-    peers = pmlm3l5rpympihoy2o5ago43kluei2jjjzsalcuiuylbve3mwi2a.b32.i2p
+    peers = g3br23bvx3lq5uddcsjii74xgmn6y5q325ovrkq2zw2wbzbqgbuq.b32.i2p
 ```
 
 The testnet also contains a number of [Nomad Network](https://github.com/markqvist/nomadnet) nodes, and LXMF propagation nodes.
@@ -254,16 +273,16 @@ The testnet also contains a number of [Nomad Network](https://github.com/markqvi
 You can help support the continued development of open, free and private communications systems by donating via one of the following channels:
 
 - Monero:
- ```
+  ```
   84FpY1QbxHcgdseePYNmhTHcrgMX4nFfBYtz2GKYToqHVVhJp8Eaw1Z1EedRnKD19b3B8NiLCGVxzKV17UMmmeEsCrPyA5w
   ```
 - Ethereum
   ```
-  0x81F7B979fEa6134bA9FD5c701b3501A2e61E897a
+  0xFDabC71AC4c0C78C95aDDDe3B4FA19d6273c5E73
   ```
 - Bitcoin
   ```
-  3CPmacGm34qYvR6XWLVEJmi2aNe3PZqUuq
+  35G9uWVzrpJJibzUwpNUQGQNFzLirhrYAH
   ```
 - Ko-Fi: https://ko-fi.com/markqvist
 

RNS/Buffer.py

@@ -1,6 +1,29 @@
+# MIT License
+#
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 from __future__ import annotations
 import bz2
 import sys
+import time
 import threading
 from threading import RLock
 import struct
@@ -28,7 +51,7 @@ class StreamDataMessage(MessageBase):
     calculcated based on the value of OVERHEAD
     """
 
-    def __init__(self, stream_id: int = None, data: bytes = None, eof: bool = False):
+    def __init__(self, stream_id: int = None, data: bytes = None, eof: bool = False, compressed: bool = False):
         """
         This class is used to encapsulate binary stream
         data to be sent over a ``Channel``.
@@ -41,7 +64,7 @@ class StreamDataMessage(MessageBase):
         if stream_id is not None and stream_id > self.STREAM_ID_MAX:
             raise ValueError("stream_id must be 0-16383")
         self.stream_id = stream_id
-        self.compressed = False
+        self.compressed = compressed
         self.data = data or bytes()
         self.eof = eof
 
@@ -49,13 +72,6 @@ class StreamDataMessage(MessageBase):
         if self.stream_id is None:
             raise ValueError("stream_id")
 
-        compressed_data   = bz2.compress(self.data)
-        saved = len(self.data)-len(compressed_data)
-
-        if saved > 0:
-            self.data = compressed_data
-            self.compressed = True
-
         header_val = (0x3fff & self.stream_id) | (0x8000 if self.eof else 0x0000) | (0x4000 if self.compressed > 0 else 0x0000)
         return bytes(struct.pack(">H", header_val) + (self.data if self.data else bytes()))
 
@@ -65,6 +81,7 @@ class StreamDataMessage(MessageBase):
         self.compressed = (0x4000 & self.stream_id) > 0
         self.stream_id = self.stream_id & 0x3fff
         self.data = raw[2:]
+
         if self.compressed:
             self.data = bz2.decompress(self.data)
 
@@ -129,9 +146,9 @@ class RawChannelReader(RawIOBase, AbstractContextManager):
                         self._eof = True
                     for listener in self._listeners:
                         try:
-                            listener(len(self._buffer))
+                            threading.Thread(target=listener, name="Message Callback", args=[len(self._buffer)], daemon=True).start()
                         except Exception as ex:
-                            RNS.log("Error calling RawChannelReader(" + str(self._stream_id) + ") callback: " + str(ex))
+                            RNS.log("Error calling RawChannelReader(" + str(self._stream_id) + ") callback: " + str(ex), RNS.LOG_ERROR)
                     return True
         return False
 
@@ -184,6 +201,10 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
       object, see the Python documentation for
       ``RawIOBase``.
     """
+
+    MAX_CHUNK_LEN     = 1024*16
+    COMPRESSION_TRIES = 4
+
     def __init__(self, stream_id: int, channel: Channel):
         """
         Create a raw channel writer.
@@ -197,16 +218,51 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
 
     def write(self, __b: bytes) -> int | None:
         try:
-            chunk = bytes(__b[:StreamDataMessage.MAX_DATA_LEN])
-            message = StreamDataMessage(self._stream_id, chunk, self._eof)
+            comp_tries = RawChannelWriter.COMPRESSION_TRIES
+            comp_try = 1
+            comp_success = False
+            chunk_len = len(__b)
+            if chunk_len > RawChannelWriter.MAX_CHUNK_LEN:
+                chunk_len = RawChannelWriter.MAX_CHUNK_LEN
+                __b = __b[:RawChannelWriter.MAX_CHUNK_LEN]
+            chunk_segment = None
+            while chunk_len > 32 and comp_try < comp_tries:
+                chunk_segment_length = int(chunk_len/comp_try)
+                compressed_chunk = bz2.compress(__b[:chunk_segment_length])
+                compressed_length = len(compressed_chunk)
+                if compressed_length < StreamDataMessage.MAX_DATA_LEN and compressed_length < chunk_segment_length:
+                    comp_success = True
+                    break
+                else:
+                    comp_try += 1
+
+            if comp_success:
+                chunk = compressed_chunk
+                processed_length = chunk_segment_length
+            else:
+                chunk = bytes(__b[:StreamDataMessage.MAX_DATA_LEN])
+                processed_length = len(chunk)
+
+            message = StreamDataMessage(self._stream_id, chunk, self._eof, comp_success)
+            
             self._channel.send(message)
-            return len(chunk)
+            return processed_length
+
         except RNS.Channel.ChannelException as cex:
             if cex.type != RNS.Channel.CEType.ME_LINK_NOT_READY:
                 raise
         return 0
 
     def close(self):
+        try:
+            link_rtt = self._channel._outlet.link.rtt
+            timeout = time.time() + (link_rtt * len(self._channel._tx_ring) * 1)
+        except Exception as e:
+            timeout = time.time() + 15
+
+        while time.time() < timeout and not self._channel.is_ready_to_send():
+            time.sleep(0.05)
+
         self._eof = True
         self.write(bytes())
 

RNS/Channel.py

@@ -176,6 +176,9 @@ class Envelope:
             raise ChannelException(CEType.ME_NOT_REGISTERED, f"Unable to find constructor for Channel MSGTYPE {hex(msgtype)}")
         message = ctor()
         message.unpack(raw)
+        self.unpacked = True
+        self.message = message
+
         return message
 
     def pack(self) -> bytes:
@@ -183,6 +186,7 @@ class Envelope:
             raise ChannelException(CEType.ME_NO_MSG_TYPE, f"{self.message.__class__} lacks MSGTYPE")
         data = self.message.pack()
         self.raw = struct.pack(">HHH", self.message.MSGTYPE, self.sequence, len(data)) + data
+        self.packed = True
         return self.raw
 
     def __init__(self, outlet: ChannelOutletBase, message: MessageBase = None, raw: bytes = None, sequence: int = None):
@@ -194,6 +198,8 @@ class Envelope:
         self.sequence = sequence
         self.outlet = outlet
         self.tries = 0
+        self.unpacked = False
+        self.packed = False
         self.tracked = False
 
 
@@ -228,13 +234,16 @@ class Channel(contextlib.AbstractContextManager):
     WINDOW     = 2
 
     # Absolute minimum window size
-    WINDOW_MIN = 1
+    WINDOW_MIN = 2
+    WINDOW_MIN_LIMIT_SLOW    = 2
+    WINDOW_MIN_LIMIT_MEDIUM  = 5
+    WINDOW_MIN_LIMIT_FAST    = 16
 
     # The maximum window size for transfers on slow links
     WINDOW_MAX_SLOW      = 5
 
     # The maximum window size for transfers on mid-speed links
-    WINDOW_MAX_MEDIUM    = 16
+    WINDOW_MAX_MEDIUM    = 12
 
     # The maximum window size for transfers on fast links
     WINDOW_MAX_FAST      = 48
@@ -249,7 +258,7 @@ class Channel(contextlib.AbstractContextManager):
 
     # If the RTT rate is higher than this value,
     # the max window size for fast links will be used.
-    RTT_FAST            = 0.25
+    RTT_FAST            = 0.18
     RTT_MEDIUM          = 0.75
     RTT_SLOW            = 1.45
 
@@ -371,22 +380,28 @@ class Channel(contextlib.AbstractContextManager):
     def _emplace_envelope(self, envelope: Envelope, ring: collections.deque[Envelope]) -> bool:
         with self._lock:
             i = 0
+            
             for existing in ring:
-                if existing.sequence > envelope.sequence \
-                   and not existing.sequence // 2 > envelope.sequence:  # account for overflow
-                    ring.insert(i, envelope)
-                    return True
-                if existing.sequence == envelope.sequence:
+
+                if envelope.sequence == existing.sequence:
                     RNS.log(f"Envelope: Emplacement of duplicate envelope with sequence "+str(envelope.sequence), RNS.LOG_EXTREME)
                     return False
+                
+                if envelope.sequence < existing.sequence and not (self._next_rx_sequence - envelope.sequence) > (Channel.SEQ_MAX//2):
+                    ring.insert(i, envelope)
+
+                    envelope.tracked = True
+                    return True
+                
                 i += 1
+            
             envelope.tracked = True
             ring.append(envelope)
+
             return True
 
     def _run_callbacks(self, message: MessageBase):
-        with self._lock:
-            cbs = self._message_callbacks.copy()
+        cbs = self._message_callbacks.copy()
 
         for cb in cbs:
             try:
@@ -405,30 +420,38 @@ class Channel(contextlib.AbstractContextManager):
                     window_overflow = (self._next_rx_sequence+Channel.WINDOW_MAX) % Channel.SEQ_MODULUS
                     if window_overflow < self._next_rx_sequence:
                         if envelope.sequence > window_overflow:
-                            RNS.log("Invalid packet sequence ("+str(envelope.sequence)+") received on channel "+str(self), RNS.LOG_DEBUG)
+                            RNS.log("Invalid packet sequence ("+str(envelope.sequence)+") received on channel "+str(self), RNS.LOG_EXTREME)
                             return
                     else:
-                        if envelope.sequence < self._next_rx_sequence:
-                            RNS.log("Invalid packet sequence ("+str(envelope.sequence)+") received on channel "+str(self), RNS.LOG_DEBUG)
-                            return
+                        RNS.log("Invalid packet sequence ("+str(envelope.sequence)+") received on channel "+str(self), RNS.LOG_EXTREME)
+                        return
 
                 is_new = self._emplace_envelope(envelope, self._rx_ring)
 
             if not is_new:
                 RNS.log("Duplicate message received on channel "+str(self), RNS.LOG_EXTREME)
                 return
-            else:            
+            else:
                 with self._lock:
                     contigous = []
                     for e in self._rx_ring:
                         if e.sequence == self._next_rx_sequence:
                             contigous.append(e)
                             self._next_rx_sequence = (self._next_rx_sequence + 1) % Channel.SEQ_MODULUS
+                            if self._next_rx_sequence == 0:
+                                for e in self._rx_ring:
+                                    if e.sequence == self._next_rx_sequence:
+                                        contigous.append(e)
+                                        self._next_rx_sequence = (self._next_rx_sequence + 1) % Channel.SEQ_MODULUS
 
                     for e in contigous:
-                        m = e.unpack(self._message_factories)
+                        if not e.unpacked:
+                            m = e.unpack(self._message_factories)
+                        else:
+                            m = e.message
+                            
                         self._rx_ring.remove(e)
-                        threading.Thread(target=self._run_callbacks, name="Message Callback", args=[m], daemon=True).start()
+                        self._run_callbacks(m)
 
         except Exception as e:
             RNS.log("An error ocurred while receiving data on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
@@ -458,6 +481,7 @@ class Channel(contextlib.AbstractContextManager):
         with self._lock:
             envelope = next(filter(lambda e: self._outlet.get_packet_id(e.packet) == self._outlet.get_packet_id(packet),
                                    self._tx_ring), None)
+
             if envelope and op(envelope):
                 envelope.tracked = False
                 if envelope in self._tx_ring:
@@ -465,11 +489,9 @@ class Channel(contextlib.AbstractContextManager):
 
                     if self.window < self.window_max:
                         self.window += 1
-                        if (self.window - self.window_min) > (self.window_flexibility-1):
-                            self.window_min += 1
 
                         # TODO: Remove at some point
-                        RNS.log("Increased "+str(self)+" window to "+str(self.window), RNS.LOG_DEBUG)
+                        # RNS.log("Increased "+str(self)+" window to "+str(self.window), RNS.LOG_DEBUG)
 
                     if self._outlet.rtt != 0:
                         if self._outlet.rtt > Channel.RTT_FAST:
@@ -482,28 +504,39 @@ class Channel(contextlib.AbstractContextManager):
                                 self.medium_rate_rounds += 1
                                 if self.window_max < Channel.WINDOW_MAX_MEDIUM and self.medium_rate_rounds == Channel.FAST_RATE_THRESHOLD:
                                     self.window_max = Channel.WINDOW_MAX_MEDIUM
+                                    self.window_min = Channel.WINDOW_MIN_LIMIT_MEDIUM
                                     # TODO: Remove at some point
-                                    RNS.log("Increased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_EXTREME)
+                                    # RNS.log("Increased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
+                                    # RNS.log("Increased "+str(self)+" min window to "+str(self.window_min), RNS.LOG_DEBUG)
                             
                         else:
                             self.fast_rate_rounds += 1
                             if self.window_max < Channel.WINDOW_MAX_FAST and self.fast_rate_rounds == Channel.FAST_RATE_THRESHOLD:
-                                self.window_max = Channel.WINDOW_MAX_FAST                                
+                                self.window_max = Channel.WINDOW_MAX_FAST
+                                self.window_min = Channel.WINDOW_MIN_LIMIT_FAST
                                 # TODO: Remove at some point
-                                RNS.log("Increased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_EXTREME)
-
+                                # RNS.log("Increased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
+                                # RNS.log("Increased "+str(self)+" min window to "+str(self.window_min), RNS.LOG_DEBUG)
 
 
                 else:
-                    RNS.log("Envelope not found in TX ring for "+str(self), RNS.LOG_DEBUG)
+                    RNS.log("Envelope not found in TX ring for "+str(self), RNS.LOG_EXTREME)
         if not envelope:
             RNS.log("Spurious message received on "+str(self), RNS.LOG_EXTREME)
 
     def _packet_delivered(self, packet: TPacket):
         self._packet_tx_op(packet, lambda env: True)
 
+    def _update_packet_timeouts(self):
+        for envelope in self._tx_ring:
+            updated_timeout = self._get_packet_timeout_time(envelope.tries)
+            if envelope.packet and hasattr(envelope.packet, "receipt") and envelope.packet.receipt and envelope.packet.receipt.timeout:
+                if updated_timeout > envelope.packet.receipt.timeout:
+                    envelope.packet.receipt.set_timeout(updated_timeout)
+
     def _get_packet_timeout_time(self, tries: int) -> float:
-        return pow(2, tries - 1) * max(self._outlet.rtt, 0.01) * 5
+        to = pow(1.5, tries - 1) * max(self._outlet.rtt*2.5, 0.025) * (len(self._tx_ring)+1.5)
+        return to
 
     def _packet_timeout(self, packet: TPacket):
         def retry_envelope(envelope: Envelope) -> bool:
@@ -512,20 +545,25 @@ class Channel(contextlib.AbstractContextManager):
                 self._shutdown()  # start on separate thread?
                 self._outlet.timed_out()
                 return True
+
             envelope.tries += 1
             self._outlet.resend(envelope.packet)
             self._outlet.set_packet_delivered_callback(envelope.packet, self._packet_delivered)
             self._outlet.set_packet_timeout_callback(envelope.packet, self._packet_timeout, self._get_packet_timeout_time(envelope.tries))
+            self._update_packet_timeouts()
 
             if self.window > self.window_min:
                 self.window -= 1
-                if self.window_max > self.window_min:
+                # TODO: Remove at some point
+                # RNS.log("Decreased "+str(self)+" window to "+str(self.window), RNS.LOG_DEBUG)
+
+                if self.window_max > (self.window_min+self.window_flexibility):
                     self.window_max -= 1
-                    if (self.window_max - self.window) > (self.window_flexibility-1):
-                        self.window_max -= 1
+                    # TODO: Remove at some point
+                    # RNS.log("Decreased "+str(self)+" max window to "+str(self.window_max), RNS.LOG_DEBUG)
 
                 # TODO: Remove at some point
-                RNS.log("Decreased "+str(self)+" window to "+str(self.window), RNS.LOG_EXTREME)
+                # RNS.log("Decreased "+str(self)+" window to "+str(self.window), RNS.LOG_EXTREME)
 
             return False
 
@@ -543,20 +581,23 @@ class Channel(contextlib.AbstractContextManager):
         with self._lock:
             if not self.is_ready_to_send():
                 raise ChannelException(CEType.ME_LINK_NOT_READY, f"Link is not ready")
+        
             envelope = Envelope(self._outlet, message=message, sequence=self._next_sequence)
             self._next_sequence = (self._next_sequence + 1) % Channel.SEQ_MODULUS
-
             self._emplace_envelope(envelope, self._tx_ring)
+
         if envelope is None:
             raise BlockingIOError()
 
         envelope.pack()
         if len(envelope.raw) > self._outlet.mdu:
             raise ChannelException(CEType.ME_TOO_BIG, f"Packed message too big for packet: {len(envelope.raw)} > {self._outlet.mdu}")
+        
         envelope.packet = self._outlet.send(envelope.raw)
         envelope.tries += 1
         self._outlet.set_packet_delivered_callback(envelope.packet, self._packet_delivered)
         self._outlet.set_packet_timeout_callback(envelope.packet, self._packet_timeout, self._get_packet_timeout_time(envelope.tries))
+        self._update_packet_timeouts()
 
         return envelope
 
@@ -591,7 +632,6 @@ class LinkChannelOutlet(ChannelOutletBase):
         return packet
 
     def resend(self, packet: RNS.Packet) -> RNS.Packet:
-        RNS.log("Resending packet " + RNS.prettyhexrep(packet.packet_hash), RNS.LOG_DEBUG)
         receipt = packet.resend()
         if not receipt:
             RNS.log("Failed to resend packet", RNS.LOG_ERROR)

RNS/Cryptography/Fernet.py

@@ -45,10 +45,10 @@ class Fernet():
 
     def __init__(self, key = None):
         if key == None:
-            raise ValueError("Fernet key cannot be None")
+            raise ValueError("Token key cannot be None")
 
         if len(key) != 32:
-            raise ValueError("Fernet key must be 32 bytes, not "+str(len(key)))
+            raise ValueError("Token key must be 32 bytes, not "+str(len(key)))
             
         self._signing_key = key[:16]
         self._encryption_key = key[16:]
@@ -72,7 +72,7 @@ class Fernet():
         current_time = int(time.time())
 
         if not isinstance(data, bytes):
-            raise TypeError("Fernet token plaintext input must be bytes")
+            raise TypeError("Token plaintext input must be bytes")
 
         ciphertext = AES_128_CBC.encrypt(
             plaintext = PKCS7.pad(data),
@@ -87,10 +87,10 @@ class Fernet():
 
     def decrypt(self, token = None):
         if not isinstance(token, bytes):
-            raise TypeError("Fernet token must be bytes")
+            raise TypeError("Token must be bytes")
 
         if not self.verify_hmac(token):
-            raise ValueError("Fernet token HMAC was invalid")
+            raise ValueError("Token HMAC was invalid")
 
         iv = token[:16]
         ciphertext = token[16:-32]
@@ -107,4 +107,4 @@ class Fernet():
             return plaintext
 
         except Exception as e:
-            raise ValueError("Could not decrypt Fernet token")
+            raise ValueError("Could not decrypt token")

RNS/Destination.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -99,7 +99,12 @@ class Destination:
         name_hash = RNS.Identity.full_hash(Destination.expand_name(None, app_name, *aspects).encode("utf-8"))[:(RNS.Identity.NAME_HASH_LENGTH//8)]
         addr_hash_material = name_hash
         if identity != None:
-            addr_hash_material += identity.hash
+            if isinstance(identity, RNS.Identity):
+                addr_hash_material += identity.hash
+            elif isinstance(identity, bytes) and len(identity) == RNS.Reticulum.TRUNCATED_HASHLENGTH//8:
+                addr_hash_material += identity
+            else:
+                raise TypeError("Invalid material supplied for destination hash calculation")
 
         return RNS.Identity.full_hash(addr_hash_material)[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8]
 
@@ -176,6 +181,9 @@ class Destination:
         """
         if self.type != Destination.SINGLE:
             raise TypeError("Only SINGLE destination types can be announced")
+
+        if self.direction != Destination.IN:
+            raise TypeError("Only IN destination types can be announced")
         
         now = time.time()
         stale_responses = []

RNS/Identity.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -145,9 +145,12 @@ class Identity:
                 except:
                     pass
 
-            for destination_hash in storage_known_destinations:
-                if not destination_hash in Identity.known_destinations:
-                    Identity.known_destinations[destination_hash] = storage_known_destinations[destination_hash]
+            try:
+                for destination_hash in storage_known_destinations:
+                    if not destination_hash in Identity.known_destinations:
+                        Identity.known_destinations[destination_hash] = storage_known_destinations[destination_hash]
+            except Exception as e:
+                RNS.log("Skipped recombining known destinations from disk, since an error occurred: "+str(e), RNS.LOG_WARNING)
 
             RNS.log("Saving "+str(len(Identity.known_destinations))+" known destinations to storage...", RNS.LOG_DEBUG)
             file = open(RNS.Reticulum.storagepath+"/known_destinations","wb")
@@ -164,6 +167,7 @@ class Identity:
 
         except Exception as e:
             RNS.log("Error while saving known destinations to disk, the contained exception was: "+str(e), RNS.LOG_ERROR)
+            RNS.trace_exception(e)
 
         Identity.saving_known_destinations = False
 
@@ -181,7 +185,8 @@ class Identity:
                         Identity.known_destinations[known_destination] = loaded_known_destinations[known_destination]
 
                 RNS.log("Loaded "+str(len(Identity.known_destinations))+" known destination from storage", RNS.LOG_VERBOSE)
-            except:
+
+            except Exception as e:
                 RNS.log("Error loading known destinations from disk, file will be recreated on exit", RNS.LOG_ERROR)
         else:
             RNS.log("Destinations file does not exist, no known destinations loaded", RNS.LOG_VERBOSE)
@@ -217,7 +222,7 @@ class Identity:
         return Identity.truncated_hash(os.urandom(Identity.TRUNCATED_HASHLENGTH//8))
 
     @staticmethod
-    def validate_announce(packet):
+    def validate_announce(packet, only_validate_signature=False):
         try:
             if packet.packet_type == RNS.Packet.ANNOUNCE:
                 destination_hash = packet.destination_hash
@@ -238,6 +243,10 @@ class Identity:
                 announced_identity.load_public_key(public_key)
 
                 if announced_identity.pub != None and announced_identity.validate(signature, signed_data):
+                    if only_validate_signature:
+                        del announced_identity
+                        return True
+
                     hash_material = name_hash+announced_identity.hash
                     expected_hash = RNS.Identity.full_hash(hash_material)[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8]
 
@@ -452,7 +461,7 @@ class Identity:
             return False
         except Exception as e:
             RNS.log("Error while loading identity from "+str(path), RNS.LOG_ERROR)
-            RNS.log("The contained exception was: "+str(e))
+            RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
 
     def get_salt(self):
         return self.hash

RNS/Interfaces/AX25KISSInterface.py

@@ -77,8 +77,7 @@ class AX25KISSInterface(Interface):
             RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
             RNS.panic()
 
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 564
         
@@ -367,5 +366,8 @@ class AX25KISSInterface(Interface):
 
         RNS.log("Reconnected serial port for "+str(self))
 
+    def should_ingress_limit(self):
+        return False
+
     def __str__(self):
         return "AX25KISSInterface["+self.name+"]"

RNS/Interfaces/Android/KISSInterface.py

@@ -82,8 +82,7 @@ class KISSInterface(Interface):
         else:
             raise SystemError("Android-specific interface was used on non-Android OS")
 
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
         
         self.HW_MTU = 564
         
@@ -403,5 +402,8 @@ class KISSInterface(Interface):
 
         RNS.log("Reconnected serial port for "+str(self))
 
+    def should_ingress_limit(self):
+        return False
+
     def __str__(self):
         return "KISSInterface["+self.name+"]"

RNS/Interfaces/Android/RNodeInterface.py

@@ -43,20 +43,22 @@ class KISS():
     CMD_CR          = 0x05
     CMD_RADIO_STATE = 0x06
     CMD_RADIO_LOCK  = 0x07
+    CMD_ST_ALOCK    = 0x0B
+    CMD_LT_ALOCK    = 0x0C
     CMD_DETECT      = 0x08
-    CMD_IMPLICIT    = 0x09
     CMD_LEAVE       = 0x0A
     CMD_READY       = 0x0F
     CMD_STAT_RX     = 0x21
     CMD_STAT_TX     = 0x22
     CMD_STAT_RSSI   = 0x23
     CMD_STAT_SNR    = 0x24
+    CMD_STAT_CHTM   = 0x25
+    CMD_STAT_PHYPRM = 0x26
     CMD_BLINK       = 0x30
     CMD_RANDOM      = 0x40
     CMD_FB_EXT      = 0x41
     CMD_FB_READ     = 0x42
     CMD_FB_WRITE    = 0x43
-    CMD_FB_READL    = 0x44
     CMD_BT_CTRL     = 0x46
     CMD_PLATFORM    = 0x48
     CMD_MCU         = 0x49
@@ -113,7 +115,7 @@ class AndroidBluetoothManager():
         if self.bt_enabled():
             return self.bt_adapter.getDefaultAdapter().getBondedDevices()
         else:
-            RNS.log("Could not query paired devices, Bluetooth is disabled", RNS.LOG_DEBUG)
+            RNS.log("Could not query paired devices, Bluetooth is disabled", RNS.LOG_EXTREME)
             return []
 
     def get_potential_devices(self):
@@ -165,8 +167,8 @@ class AndroidBluetoothManager():
                                 raise IOError("The Bluetooth RFcomm socket could not be connected: "+str(e))
 
                 except Exception as e:
-                    RNS.log("Could not create and connect Bluetooth RFcomm socket for "+str(device.getName())+" "+str(device.getAddress()), RNS.LOG_ERROR)
-                    RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+                    RNS.log("Could not create and connect Bluetooth RFcomm socket for "+str(device.getName())+" "+str(device.getAddress()), RNS.LOG_EXTREME)
+                    RNS.log("The contained exception was: "+str(e), RNS.LOG_EXTREME)
 
     def close(self):
         if self.connected:
@@ -231,6 +233,10 @@ class RNodeInterface(Interface):
     RECONNECT_WAIT = 5
     PORT_IO_TIMEOUT = 3
 
+    Q_SNR_MIN_BASE = -9
+    Q_SNR_MAX      = 6
+    Q_SNR_STEP     = 2
+
     @classmethod
     def bluetooth_control(device_serial = None, port = None, enable_bluetooth = False, disable_bluetooth = False, pairing_mode = False):
         if (port != None or device_serial != None) and (enable_bluetooth or disable_bluetooth or pairing_mode):
@@ -315,7 +321,7 @@ class RNodeInterface(Interface):
         self, owner, name, port, frequency = None, bandwidth = None, txpower = None,
         sf = None, cr = None, flow_control = False, id_interval = None,
         allow_bluetooth = False, target_device_name = None,
-        target_device_address = None, id_callsign = None):
+        target_device_address = None, id_callsign = None, st_alock = None, lt_alock = None):
         import importlib
         if RNS.vendor.platformutils.is_android():
             self.on_android  = True
@@ -348,8 +354,7 @@ class RNodeInterface(Interface):
         else:
             raise SystemError("Android-specific interface was used on non-Android OS")
 
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 508
         
@@ -373,6 +378,8 @@ class RNodeInterface(Interface):
         self.cr          = cr
         self.state       = KISS.RADIO_STATE_OFF
         self.bitrate     = 0
+        self.st_alock    = st_alock
+        self.lt_alock    = lt_alock
         self.platform    = None
         self.display     = None
         self.mcu         = None
@@ -395,7 +402,18 @@ class RNodeInterface(Interface):
         self.r_stat_rx   = None
         self.r_stat_tx   = None
         self.r_stat_rssi = None
+        self.r_stat_snr  = None
+        self.r_st_alock  = None
+        self.r_lt_alock  = None
         self.r_random    = None
+        self.r_airtime_short      = 0.0
+        self.r_airtime_long       = 0.0
+        self.r_channel_load_short = 0.0
+        self.r_channel_load_long  = 0.0
+        self.r_symbol_time_ms = None
+        self.r_symbol_rate = None
+        self.r_preamble_symbols = None
+        self.r_premable_time_ms = None
 
         self.packet_queue    = []
         self.flow_control    = flow_control
@@ -426,6 +444,14 @@ class RNodeInterface(Interface):
             RNS.log("Invalid coding rate configured for "+str(self), RNS.LOG_ERROR)
             self.validcfg = False
 
+        if (self.st_alock and (self.st_alock < 0.0 or self.st_alock > 100.0)):
+            RNS.log("Invalid short-term airtime limit configured for "+str(self), RNS.LOG_ERROR)
+            self.validcfg = False
+
+        if (self.lt_alock and (self.lt_alock < 0.0 or self.lt_alock > 100.0)):
+            RNS.log("Invalid long-term airtime limit configured for "+str(self), RNS.LOG_ERROR)
+            self.validcfg = False
+
         if id_interval != None and id_callsign != None:
             if (len(id_callsign.encode("utf-8")) <= RNodeInterface.CALLSIGN_MAX_LEN):
                 self.should_id = True
@@ -576,8 +602,11 @@ class RNodeInterface(Interface):
             raise IOError("Invalid device firmware")
 
         if self.serial != None and self.port != None:
+            self.timeout = 200
             RNS.log("Serial port "+self.port+" is now open")
+
         if self.bt_manager != None and self.bt_manager.connected:
+            self.timeout = 1500
             RNS.log("Bluetooth connection to RNode now open")
 
         RNS.log("Configuring RNode interface...", RNS.LOG_VERBOSE)
@@ -615,6 +644,12 @@ class RNodeInterface(Interface):
         
         self.setCodingRate()
         time.sleep(0.15)
+
+        self.setSTALock()
+        time.sleep(0.15)
+        
+        self.setLTALock()
+        time.sleep(0.15)
         
         self.setRadioState(KISS.RADIO_STATE_ON)
         time.sleep(0.15)
@@ -740,6 +775,30 @@ class RNodeInterface(Interface):
         if written != len(kiss_command):
             raise IOError("An IO error occurred while configuring coding rate for "+str(self))
 
+    def setSTALock(self):
+        if self.st_alock != None:
+            at = int(self.st_alock*100)
+            c1 = at >> 8 & 0xFF
+            c2 = at & 0xFF
+            data = KISS.escape(bytes([c1])+bytes([c2]))
+
+            kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_ST_ALOCK])+data+bytes([KISS.FEND])
+            written = self.write_mux(kiss_command)
+            if written != len(kiss_command):
+                raise IOError("An IO error occurred while configuring short-term airtime limit for "+str(self))
+
+    def setLTALock(self):
+        if self.lt_alock != None:
+            at = int(self.lt_alock*100)
+            c1 = at >> 8 & 0xFF
+            c2 = at & 0xFF
+            data = KISS.escape(bytes([c1])+bytes([c2]))
+
+            kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_LT_ALOCK])+data+bytes([KISS.FEND])
+            written = self.write_mux(kiss_command)
+            if written != len(kiss_command):
+                raise IOError("An IO error occurred while configuring long-term airtime limit for "+str(self))
+
     def setRadioState(self, state):
         self.state = state
         kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_RADIO_STATE])+bytes([state])+bytes([KISS.FEND])
@@ -809,9 +868,6 @@ class RNodeInterface(Interface):
             self.owner.inbound(data, self)
         threading.Thread(target=af, daemon=True).start()
 
-        self.r_stat_rssi = None
-        self.r_stat_snr = None
-
 
     def processOutgoing(self,data):
         datalen = len(data)
@@ -993,6 +1049,97 @@ class RNodeInterface(Interface):
                             self.r_stat_rssi = byte-RNodeInterface.RSSI_OFFSET
                         elif (command == KISS.CMD_STAT_SNR):
                             self.r_stat_snr = int.from_bytes(bytes([byte]), byteorder="big", signed=True) * 0.25
+                            try:
+                                sfs = self.r_sf-7
+                                snr = self.r_stat_snr
+                                q_snr_min = RNodeInterface.Q_SNR_MIN_BASE-sfs*RNodeInterface.Q_SNR_STEP
+                                q_snr_max = RNodeInterface.Q_SNR_MAX
+                                q_snr_span = q_snr_max-q_snr_min
+                                quality = round(((snr-q_snr_min)/(q_snr_span))*100,1)
+                                if quality > 100.0: quality = 100.0
+                                if quality < 0.0: quality = 0.0
+                                self.r_stat_q = quality
+                            except:
+                                pass
+
+                        elif (command == KISS.CMD_ST_ALOCK):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 2):
+                                    at = command_buffer[0] << 8 | command_buffer[1]
+                                    self.r_st_alock = at/100.0
+                                    RNS.log(str(self)+" Radio reporting short-term airtime limit is "+str(self.r_st_alock)+"%", RNS.LOG_DEBUG)
+                        elif (command == KISS.CMD_LT_ALOCK):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 2):
+                                    at = command_buffer[0] << 8 | command_buffer[1]
+                                    self.r_lt_alock = at/100.0
+                                    RNS.log(str(self)+" Radio reporting long-term airtime limit is "+str(self.r_lt_alock)+"%", RNS.LOG_DEBUG)
+                        elif (command == KISS.CMD_STAT_CHTM):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 8):
+                                    ats = command_buffer[0] << 8 | command_buffer[1]
+                                    atl = command_buffer[2] << 8 | command_buffer[3]
+                                    cus = command_buffer[4] << 8 | command_buffer[5]
+                                    cul = command_buffer[6] << 8 | command_buffer[7]
+                                    
+                                    self.r_airtime_short      = ats/100.0
+                                    self.r_airtime_long       = atl/100.0
+                                    self.r_channel_load_short = cus/100.0
+                                    self.r_channel_load_long  = cul/100.0
+                        elif (command == KISS.CMD_STAT_PHYPRM):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 10):
+                                    lst = (command_buffer[0] << 8 | command_buffer[1])/1000.0
+                                    lsr = command_buffer[2] << 8 | command_buffer[3]
+                                    prs = command_buffer[4] << 8 | command_buffer[5]
+                                    prt = command_buffer[6] << 8 | command_buffer[7]
+                                    cst = command_buffer[8] << 8 | command_buffer[9]
+
+                                    if lst != self.r_symbol_time_ms or lsr != self.r_symbol_rate or prs != self.r_preamble_symbols or prt != self.r_premable_time_ms or cst != self.r_csma_slot_time_ms:
+                                        self.r_symbol_time_ms    = lst
+                                        self.r_symbol_rate       = lsr
+                                        self.r_preamble_symbols  = prs
+                                        self.r_premable_time_ms  = prt
+                                        self.r_csma_slot_time_ms = cst
+                                        RNS.log(str(self)+" Radio reporting symbol time is "+str(round(self.r_symbol_time_ms,2))+"ms (at "+str(self.r_symbol_rate)+" baud)", RNS.LOG_DEBUG)
+                                        RNS.log(str(self)+" Radio reporting preamble is "+str(self.r_preamble_symbols)+" symbols ("+str(self.r_premable_time_ms)+"ms)", RNS.LOG_DEBUG)
+                                        RNS.log(str(self)+" Radio reporting CSMA slot time is "+str(self.r_csma_slot_time_ms)+"ms", RNS.LOG_DEBUG)
                         elif (command == KISS.CMD_RANDOM):
                             self.r_random = byte
                         elif (command == KISS.CMD_PLATFORM):
@@ -1026,7 +1173,7 @@ class RNodeInterface(Interface):
                 if got == 0:
                     time_since_last = int(time.time()*1000) - last_read_ms
                     if len(data_buffer) > 0 and time_since_last > self.timeout:
-                        RNS.log(str(self)+" serial read timeout", RNS.LOG_DEBUG)
+                        RNS.log(str(self)+" serial read timeout in command "+str(command), RNS.LOG_WARNING)
                         data_buffer = b""
                         in_frame = False
                         command = KISS.CMD_UNKNOWN
@@ -1104,6 +1251,9 @@ class RNodeInterface(Interface):
         self.setRadioState(KISS.RADIO_STATE_OFF)
         self.leave()
 
+    def should_ingress_limit(self):
+        return False
+
     def __str__(self):
         return "RNodeInterface["+str(self.name)+"]"
 

RNS/Interfaces/Android/SerialInterface.py

@@ -72,8 +72,7 @@ class SerialInterface(Interface):
         else:
             raise SystemError("Android-specific interface was used on non-Android OS")
 
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 564
         
@@ -254,5 +253,8 @@ class SerialInterface(Interface):
 
         RNS.log("Reconnected serial port for "+str(self))
 
+    def should_ingress_limit(self):
+        return False
+
     def __str__(self):
         return "SerialInterface["+self.name+"]"

RNS/Interfaces/AutoInterface.py

@@ -43,6 +43,9 @@ class AutoInterface(Interface):
     SCOPE_ORGANISATION = "8"
     SCOPE_GLOBAL       = "e"
 
+    MULTICAST_PERMANENT_ADDRESS_TYPE = "0"
+    MULTICAST_TEMPORARY_ADDRESS_TYPE = "1"
+
     PEERING_TIMEOUT    = 7.5
 
     ALL_IGNORE_IFS     = ["lo0"]
@@ -51,7 +54,8 @@ class AutoInterface(Interface):
 
     BITRATE_GUESS      = 10*1000*1000
 
-    MULTI_IF_DEQUE_LEN = 64
+    MULTI_IF_DEQUE_LEN = 48
+    MULTI_IF_DEQUE_TTL = 0.75
 
     def handler_factory(self, callback):
         def create_handler(*args, **keys):
@@ -73,12 +77,19 @@ class AutoInterface(Interface):
         ifas = self.netinfo.ifaddresses(ifname)
         return ifas
 
-    def __init__(self, owner, name, group_id=None, discovery_scope=None, discovery_port=None, data_port=None, allowed_interfaces=None, ignored_interfaces=None, configured_bitrate=None):
-        from RNS.vendor.ifaddr import niwrapper
-        self.netinfo = niwrapper
+    def interface_name_to_index(self, ifname):
 
-        self.rxb = 0
-        self.txb = 0
+        # socket.if_nametoindex doesn't work with uuid interface names on windows, it wants the ethernet_0 style
+        # we will just get the index from netinfo instead as it seems to work
+        if RNS.vendor.platformutils.is_windows():
+            return self.netinfo.interface_names_to_indexes()[ifname]
+
+        return socket.if_nametoindex(ifname)
+
+    def __init__(self, owner, name, group_id=None, discovery_scope=None, discovery_port=None, multicast_address_type=None, data_port=None, allowed_interfaces=None, ignored_interfaces=None, configured_bitrate=None):
+        from RNS.vendor.ifaddr import niwrapper
+        super().__init__()
+        self.netinfo = niwrapper
 
         self.HW_MTU = 1064
 
@@ -93,6 +104,7 @@ class AutoInterface(Interface):
         self.multicast_echoes = {}
         self.timed_out_interfaces = {}
         self.mif_deque = deque(maxlen=AutoInterface.MULTI_IF_DEQUE_LEN)
+        self.mif_deque_times = deque(maxlen=AutoInterface.MULTI_IF_DEQUE_LEN)
         self.carrier_changed = False
 
         self.outbound_udp_socket = None
@@ -103,6 +115,11 @@ class AutoInterface(Interface):
         self.peering_timeout   = AutoInterface.PEERING_TIMEOUT
         self.multicast_echo_timeout = AutoInterface.PEERING_TIMEOUT/2
 
+        # Increase peering timeout on Android, due to potential
+        # low-power modes implemented on many chipsets.
+        if RNS.vendor.platformutils.is_android():
+            self.peering_timeout *= 3
+
         if allowed_interfaces == None:
             self.allowed_interfaces = []
         else:
@@ -123,6 +140,15 @@ class AutoInterface(Interface):
         else:
             self.discovery_port = discovery_port
 
+        if multicast_address_type == None:
+            self.multicast_address_type = AutoInterface.MULTICAST_TEMPORARY_ADDRESS_TYPE
+        elif str(multicast_address_type).lower() == "temporary":
+            self.multicast_address_type = AutoInterface.MULTICAST_TEMPORARY_ADDRESS_TYPE
+        elif str(multicast_address_type).lower() == "permanent":
+            self.multicast_address_type = AutoInterface.MULTICAST_PERMANENT_ADDRESS_TYPE
+        else:
+            self.multicast_address_type = AutoInterface.MULTICAST_TEMPORARY_ADDRESS_TYPE
+
         if data_port == None:
             self.data_port = AutoInterface.DEFAULT_DATA_PORT
         else:
@@ -151,73 +177,94 @@ class AutoInterface(Interface):
         gt += ":"+"{:02x}".format(g[9]+(g[8]<<8))
         gt += ":"+"{:02x}".format(g[11]+(g[10]<<8))
         gt += ":"+"{:02x}".format(g[13]+(g[12]<<8))
-        self.mcast_discovery_address = "ff1"+self.discovery_scope+":"+gt
+        self.mcast_discovery_address = "ff"+self.multicast_address_type+self.discovery_scope+":"+gt
 
         suitable_interfaces = 0
         for ifname in self.list_interfaces():
-            if RNS.vendor.platformutils.is_darwin() and ifname in AutoInterface.DARWIN_IGNORE_IFS and not ifname in self.allowed_interfaces:
-                RNS.log(str(self)+" skipping Darwin AWDL or tethering interface "+str(ifname), RNS.LOG_EXTREME)
-            elif RNS.vendor.platformutils.is_darwin() and ifname == "lo0":
-                RNS.log(str(self)+" skipping Darwin loopback interface "+str(ifname), RNS.LOG_EXTREME)
-            elif RNS.vendor.platformutils.is_android() and ifname in AutoInterface.ANDROID_IGNORE_IFS and not ifname in self.allowed_interfaces:
-                RNS.log(str(self)+" skipping Android system interface "+str(ifname), RNS.LOG_EXTREME)
-            elif ifname in self.ignored_interfaces:
-                RNS.log(str(self)+" ignoring disallowed interface "+str(ifname), RNS.LOG_EXTREME)
-            elif ifname in AutoInterface.ALL_IGNORE_IFS:
-                RNS.log(str(self)+" skipping interface "+str(ifname), RNS.LOG_EXTREME)
-            else:
-                if len(self.allowed_interfaces) > 0 and not ifname in self.allowed_interfaces:
-                    RNS.log(str(self)+" ignoring interface "+str(ifname)+" since it was not allowed", RNS.LOG_EXTREME)
+            try:
+                if RNS.vendor.platformutils.is_darwin() and ifname in AutoInterface.DARWIN_IGNORE_IFS and not ifname in self.allowed_interfaces:
+                    RNS.log(str(self)+" skipping Darwin AWDL or tethering interface "+str(ifname), RNS.LOG_EXTREME)
+                elif RNS.vendor.platformutils.is_darwin() and ifname == "lo0":
+                    RNS.log(str(self)+" skipping Darwin loopback interface "+str(ifname), RNS.LOG_EXTREME)
+                elif RNS.vendor.platformutils.is_android() and ifname in AutoInterface.ANDROID_IGNORE_IFS and not ifname in self.allowed_interfaces:
+                    RNS.log(str(self)+" skipping Android system interface "+str(ifname), RNS.LOG_EXTREME)
+                elif ifname in self.ignored_interfaces:
+                    RNS.log(str(self)+" ignoring disallowed interface "+str(ifname), RNS.LOG_EXTREME)
+                elif ifname in AutoInterface.ALL_IGNORE_IFS:
+                    RNS.log(str(self)+" skipping interface "+str(ifname), RNS.LOG_EXTREME)
                 else:
-                    addresses = self.list_addresses(ifname)
-                    if self.netinfo.AF_INET6 in addresses:
-                        link_local_addr = None
-                        for address in addresses[self.netinfo.AF_INET6]:
-                            if "addr" in address:
-                                if address["addr"].startswith("fe80:"):
-                                    link_local_addr = self.descope_linklocal(address["addr"])
-                                    self.link_local_addresses.append(link_local_addr)
-                                    self.adopted_interfaces[ifname] = link_local_addr
-                                    self.multicast_echoes[ifname] = time.time()
-                                    RNS.log(str(self)+" Selecting link-local address "+str(link_local_addr)+" for interface "+str(ifname), RNS.LOG_EXTREME)
+                    if len(self.allowed_interfaces) > 0 and not ifname in self.allowed_interfaces:
+                        RNS.log(str(self)+" ignoring interface "+str(ifname)+" since it was not allowed", RNS.LOG_EXTREME)
+                    else:
+                        addresses = self.list_addresses(ifname)
+                        if self.netinfo.AF_INET6 in addresses:
+                            link_local_addr = None
+                            for address in addresses[self.netinfo.AF_INET6]:
+                                if "addr" in address:
+                                    if address["addr"].startswith("fe80:"):
+                                        link_local_addr = self.descope_linklocal(address["addr"])
+                                        self.link_local_addresses.append(link_local_addr)
+                                        self.adopted_interfaces[ifname] = link_local_addr
+                                        self.multicast_echoes[ifname] = time.time()
+                                        nice_name = self.netinfo.interface_name_to_nice_name(ifname)
+                                        if nice_name != None and nice_name != ifname:
+                                            RNS.log(f"{self} Selecting link-local address {link_local_addr} for interface {nice_name} / {ifname}", RNS.LOG_EXTREME)
+                                        else:
+                                            RNS.log(f"{self} Selecting link-local address {link_local_addr} for interface {ifname}", RNS.LOG_EXTREME)
 
-                        if link_local_addr == None:
-                            RNS.log(str(self)+" No link-local IPv6 address configured for "+str(ifname)+", skipping interface", RNS.LOG_EXTREME)
-                        else:
-                            mcast_addr = self.mcast_discovery_address
-                            RNS.log(str(self)+" Creating multicast discovery listener on "+str(ifname)+" with address "+str(mcast_addr), RNS.LOG_EXTREME)
-
-                            # Struct with interface index
-                            if_struct = struct.pack("I", socket.if_nametoindex(ifname))
-
-                            # Set up multicast socket
-                            discovery_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
-                            discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-                            if hasattr(socket, "SO_REUSEPORT"):
-                                discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
-                            discovery_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_IF, if_struct)
-
-                            # Join multicast group
-                            mcast_group = socket.inet_pton(socket.AF_INET6, mcast_addr) + if_struct
-                            discovery_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_JOIN_GROUP, mcast_group)
-
-                            # Bind socket
-                            if self.discovery_scope == AutoInterface.SCOPE_LINK:
-                                addr_info = socket.getaddrinfo(mcast_addr+"%"+ifname, self.discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
+                            if link_local_addr == None:
+                                RNS.log(str(self)+" No link-local IPv6 address configured for "+str(ifname)+", skipping interface", RNS.LOG_EXTREME)
                             else:
-                                addr_info = socket.getaddrinfo(mcast_addr, self.discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
+                                mcast_addr = self.mcast_discovery_address
+                                RNS.log(str(self)+" Creating multicast discovery listener on "+str(ifname)+" with address "+str(mcast_addr), RNS.LOG_EXTREME)
 
-                            discovery_socket.bind(addr_info[0][4])
+                                # Struct with interface index
+                                if_struct = struct.pack("I", self.interface_name_to_index(ifname))
 
-                            # Set up thread for discovery packets
-                            def discovery_loop():
-                                self.discovery_handler(discovery_socket, ifname)
+                                # Set up multicast socket
+                                discovery_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
+                                discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+                                if hasattr(socket, "SO_REUSEPORT"):
+                                    discovery_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+                                discovery_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_IF, if_struct)
 
-                            thread = threading.Thread(target=discovery_loop)
-                            thread.daemon = True
-                            thread.start()
+                                # Join multicast group
+                                mcast_group = socket.inet_pton(socket.AF_INET6, mcast_addr) + if_struct
+                                discovery_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_JOIN_GROUP, mcast_group)
 
-                            suitable_interfaces += 1
+                                # Bind socket
+                                if RNS.vendor.platformutils.is_windows():
+
+                                    # window throws "[WinError 10049] The requested address is not valid in its context"
+                                    # when trying to use the multicast address as host, or when providing interface index
+                                    # passing an empty host appears to work, but probably not exactly how we want it to...
+                                    discovery_socket.bind(('', self.discovery_port))
+
+                                else:
+
+                                    if self.discovery_scope == AutoInterface.SCOPE_LINK:
+                                        addr_info = socket.getaddrinfo(mcast_addr+"%"+ifname, self.discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
+                                    else:
+                                        addr_info = socket.getaddrinfo(mcast_addr, self.discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
+
+                                    discovery_socket.bind(addr_info[0][4])
+
+                                # Set up thread for discovery packets
+                                def discovery_loop():
+                                    self.discovery_handler(discovery_socket, ifname)
+
+                                thread = threading.Thread(target=discovery_loop)
+                                thread.daemon = True
+                                thread.start()
+
+                                suitable_interfaces += 1
+
+            except Exception as e:
+                nice_name = self.netinfo.interface_name_to_nice_name(ifname)
+                if nice_name != None and nice_name != ifname:
+                    RNS.log(f"Could not configure the system interface {nice_name} / {ifname} for use with {self}, skipping it. The contained exception was: {e}", RNS.LOG_ERROR)
+                else:
+                    RNS.log(f"Could not configure the system interface {ifname} for use with {self}, skipping it. The contained exception was: {e}", RNS.LOG_ERROR)
 
         if suitable_interfaces == 0:
             RNS.log(str(self)+" could not autoconfigure. This interface currently provides no connectivity.", RNS.LOG_WARNING)
@@ -236,7 +283,7 @@ class AutoInterface(Interface):
             socketserver.UDPServer.address_family = socket.AF_INET6
 
             for ifname in self.adopted_interfaces:
-                local_addr = self.adopted_interfaces[ifname]+"%"+ifname
+                local_addr = self.adopted_interfaces[ifname]+"%"+str(self.interface_name_to_index(ifname))
                 addr_info = socket.getaddrinfo(local_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
                 address = addr_info[0][4]
 
@@ -329,6 +376,8 @@ class AutoInterface(Interface):
                                         thread.daemon = True
                                         thread.start()
 
+                                        self.carrier_changed = True
+
                 except Exception as e:
                     RNS.log("Could not get device information while updating link-local addresses for "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
 
@@ -361,7 +410,7 @@ class AutoInterface(Interface):
             announce_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
             addr_info = socket.getaddrinfo(self.mcast_discovery_address, self.discovery_port, socket.AF_INET6, socket.SOCK_DGRAM)
 
-            ifis = struct.pack("I", socket.if_nametoindex(ifname))
+            ifis = struct.pack("I", self.interface_name_to_index(ifname))
             announce_socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_IF, ifis)
             announce_socket.sendto(discovery_token, addr_info[0][4])
             announce_socket.close()
@@ -396,8 +445,16 @@ class AutoInterface(Interface):
 
     def processIncoming(self, data):
         data_hash = RNS.Identity.full_hash(data)
-        if not data_hash in self.mif_deque:
+        deque_hit = False
+        if data_hash in self.mif_deque:
+            for te in self.mif_deque_times:
+                if te[0] == data_hash and time.time() < te[1]+AutoInterface.MULTI_IF_DEQUE_TTL:
+                    deque_hit = True
+                    break
+
+        if not deque_hit:
             self.mif_deque.append(data_hash)
+            self.mif_deque_times.append([data_hash, time.time()])
             self.rxb += len(data)
             self.owner.inbound(data, self)
 
@@ -406,8 +463,8 @@ class AutoInterface(Interface):
                 try:
                     if self.outbound_udp_socket == None:
                         self.outbound_udp_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
-                    
-                    peer_addr = str(peer)+"%"+str(self.peers[peer][0])
+
+                    peer_addr = str(peer)+"%"+str(self.interface_name_to_index(self.peers[peer][0]))
                     addr_info = socket.getaddrinfo(peer_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
                     self.outbound_udp_socket.sendto(data, addr_info[0][4])
 
@@ -418,6 +475,11 @@ class AutoInterface(Interface):
             self.txb += len(data)
             
 
+    # Until per-device sub-interfacing is implemented,
+    # ingress limiting should be disabled on AutoInterface
+    def should_ingress_limit(self):
+        return False
+
     def __str__(self):
         return "AutoInterface["+self.name+"]"
 

RNS/Interfaces/I2PInterface.py

@@ -390,8 +390,7 @@ class I2PInterfacePeer(Interface):
     TUNNEL_STATE_STALE   = 0x02
 
     def __init__(self, parent_interface, owner, name, target_i2p_dest=None, connected_socket=None, max_reconnect_tries=None):
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 1064
         
@@ -832,8 +831,7 @@ class I2PInterface(Interface):
     BITRATE_GUESS      = 256*1000
 
     def __init__(self, owner, name, rns_storagepath, peers, connectable = False, ifac_size = 16, ifac_netname = None, ifac_netkey = None):
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
         
         self.HW_MTU = 1064
 
@@ -964,6 +962,12 @@ class I2PInterface(Interface):
     def processOutgoing(self, data):
         pass
 
+    def received_announce(self, from_spawned=False):
+        if from_spawned: self.ia_freq_deque.append(time.time())
+
+    def sent_announce(self, from_spawned=False):
+        if from_spawned: self.oa_freq_deque.append(time.time())
+
     def detach(self):
         RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
         self.i2p.stop()

RNS/Interfaces/Interface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -23,6 +23,7 @@
 import RNS
 import time
 import threading
+from collections import deque
 
 class Interface:
     IN  = False
@@ -39,18 +40,161 @@ class Interface:
     MODE_BOUNDARY       = 0x05
     MODE_GATEWAY        = 0x06
 
-    # Which interface modes a Transport Node
-    # should actively discover paths for.
-    DISCOVER_PATHS_FOR  = [MODE_ACCESS_POINT, MODE_GATEWAY]
+    # Which interface modes a Transport Node should
+    # actively discover paths for.
+    DISCOVER_PATHS_FOR  = [MODE_ACCESS_POINT, MODE_GATEWAY, MODE_ROAMING]
+
+    # How many samples to use for announce
+    # frequency calculations
+    IA_FREQ_SAMPLES     = 6
+    OA_FREQ_SAMPLES     = 6
+
+    # Maximum amount of ingress limited announces
+    # to hold at any given time.
+    MAX_HELD_ANNOUNCES  = 256
+
+    # How long a spawned interface will be
+    # considered to be newly created. Two
+    # hours by default.
+    IC_NEW_TIME              = 2*60*60
+    IC_BURST_FREQ_NEW        = 3.5
+    IC_BURST_FREQ            = 12
+    IC_BURST_HOLD            = 1*60
+    IC_BURST_PENALTY         = 5*60
+    IC_HELD_RELEASE_INTERVAL = 30
 
     def __init__(self):
         self.rxb = 0
         self.txb = 0
+        self.created = time.time()
         self.online = False
+        self.bitrate = 1e6
+
+        self.ingress_control = True
+        self.ic_max_held_announces = Interface.MAX_HELD_ANNOUNCES
+        self.ic_burst_hold = Interface.IC_BURST_HOLD
+        self.ic_burst_active = False
+        self.ic_burst_activated = 0
+        self.ic_held_release = 0
+        self.ic_burst_freq_new = Interface.IC_BURST_FREQ_NEW
+        self.ic_burst_freq = Interface.IC_BURST_FREQ
+        self.ic_new_time = Interface.IC_NEW_TIME
+        self.ic_burst_penalty = Interface.IC_BURST_PENALTY
+        self.ic_held_release_interval = Interface.IC_HELD_RELEASE_INTERVAL
+        self.held_announces = {}
+
+        self.ia_freq_deque = deque(maxlen=Interface.IA_FREQ_SAMPLES)
+        self.oa_freq_deque = deque(maxlen=Interface.OA_FREQ_SAMPLES)
 
     def get_hash(self):
         return RNS.Identity.full_hash(str(self).encode("utf-8"))
 
+    # This is a generic function for determining when an interface
+    # should activate ingress limiting. Since this can vary for
+    # different interface types, this function should be overwritten
+    # in case a particular interface requires a different approach.
+    def should_ingress_limit(self):
+        if self.ingress_control:
+            freq_threshold = self.ic_burst_freq_new if self.age() < self.ic_new_time else self.ic_burst_freq
+            ia_freq = self.incoming_announce_frequency()
+
+            if self.ic_burst_active:
+                if ia_freq < freq_threshold and time.time() > self.ic_burst_activated+self.ic_burst_hold:
+                    self.ic_burst_active = False
+                    self.ic_held_release = time.time() + self.ic_burst_penalty
+                return True
+
+            else:
+                if ia_freq > freq_threshold:
+                    self.ic_burst_active = True
+                    self.ic_burst_activated = time.time()
+                    return True
+
+                else:
+                    return False
+
+        else:
+            return False
+
+    def age(self):
+        return time.time()-self.created
+
+    def hold_announce(self, announce_packet):
+        if announce_packet.destination_hash in self.held_announces:
+            self.held_announces[announce_packet.destination_hash] = announce_packet
+        elif not len(self.held_announces) >= self.ic_max_held_announces:
+            self.held_announces[announce_packet.destination_hash] = announce_packet
+
+    def process_held_announces(self):
+        try:
+            if not self.should_ingress_limit() and len(self.held_announces) > 0 and time.time() > self.ic_held_release:
+                freq_threshold = self.ic_burst_freq_new if self.age() < self.ic_new_time else self.ic_burst_freq
+                ia_freq = self.incoming_announce_frequency()
+                if ia_freq < freq_threshold:
+                    selected_announce_packet = None
+                    min_hops = RNS.Transport.PATHFINDER_M
+                    for destination_hash in self.held_announces:
+                        announce_packet = self.held_announces[destination_hash]
+                        if announce_packet.hops < min_hops:
+                            min_hops = announce_packet.hops
+                            selected_announce_packet = announce_packet
+
+                    if selected_announce_packet != None:
+                        RNS.log("Releasing held announce packet "+str(selected_announce_packet)+" from "+str(self), RNS.LOG_EXTREME)
+                        self.ic_held_release = time.time() + self.ic_held_release_interval
+                        self.held_announces.pop(selected_announce_packet.destination_hash)
+                        def release():
+                            RNS.Transport.inbound(selected_announce_packet.raw, selected_announce_packet.receiving_interface)
+                        threading.Thread(target=release, daemon=True).start()
+        
+        except Exception as e:
+            RNS.log("An error occurred while processing held announces for "+str(self), RNS.LOG_ERROR)
+            RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+
+    def received_announce(self):
+        self.ia_freq_deque.append(time.time())
+        if hasattr(self, "parent_interface") and self.parent_interface != None:
+            self.parent_interface.received_announce(from_spawned=True)
+
+    def sent_announce(self):
+        self.oa_freq_deque.append(time.time())
+        if hasattr(self, "parent_interface") and self.parent_interface != None:
+            self.parent_interface.sent_announce(from_spawned=True)
+
+    def incoming_announce_frequency(self):
+        if not len(self.ia_freq_deque) > 1:
+            return 0
+        else:
+            dq_len = len(self.ia_freq_deque)
+            delta_sum = 0
+            for i in range(1,dq_len):
+                delta_sum += self.ia_freq_deque[i]-self.ia_freq_deque[i-1]
+            delta_sum += time.time() - self.ia_freq_deque[dq_len-1]
+            
+            if delta_sum == 0:
+                avg = 0
+            else:
+                avg = 1/(delta_sum/(dq_len))
+
+            return avg
+
+    def outgoing_announce_frequency(self):
+        if not len(self.oa_freq_deque) > 1:
+            return 0
+        else:
+            dq_len = len(self.oa_freq_deque)
+            delta_sum = 0
+            for i in range(1,dq_len):
+                delta_sum += self.oa_freq_deque[i]-self.oa_freq_deque[i-1]
+            delta_sum += time.time() - self.oa_freq_deque[dq_len-1]
+            
+            if delta_sum == 0:
+                avg = 0
+            else:
+                avg = 1/(delta_sum/(dq_len))
+
+            return avg
+
     def process_announce_queue(self):
         if not hasattr(self, "announce_cap"):
             self.announce_cap = RNS.Reticulum.ANNOUNCE_CAP
@@ -79,6 +223,7 @@ class Interface:
                     self.announce_allowed_at = now + wait_time
 
                     self.processOutgoing(selected["raw"])
+                    self.sent_announce()
 
                     if selected in self.announce_queue:
                         self.announce_queue.remove(selected)

RNS/Interfaces/KISSInterface.py

@@ -70,8 +70,7 @@ class KISSInterface(Interface):
             RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
             RNS.panic()
 
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
         
         self.HW_MTU = 564
         
@@ -349,5 +348,8 @@ class KISSInterface(Interface):
 
         RNS.log("Reconnected serial port for "+str(self))
 
+    def should_ingress_limit(self):
+        return False
+
     def __str__(self):
         return "KISSInterface["+self.name+"]"

RNS/Interfaces/LocalInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -28,6 +28,7 @@ import time
 import sys
 import os
 import RNS
+from threading import Lock
 
 class HDLC():
     FLAG              = 0x7E
@@ -50,11 +51,10 @@ class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
         self.server_address = self.socket.getsockname()
 
 class LocalClientInterface(Interface):
-    RECONNECT_WAIT = 3
+    RECONNECT_WAIT = 8
 
     def __init__(self, owner, name, target_port = None, connected_socket=None):
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         # TODO: Remove at some point
         # self.rxptime = 0
@@ -78,6 +78,7 @@ class LocalClientInterface(Interface):
             self.target_ip   = None
             self.target_port = None
             self.socket      = connected_socket
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
 
             self.is_connected_to_shared_instance = False
 
@@ -103,8 +104,12 @@ class LocalClientInterface(Interface):
             thread.daemon = True
             thread.start()
 
+    def should_ingress_limit(self):
+        return False
+
     def connect(self):
         self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
         self.socket.connect((self.target_ip, self.target_port))
 
         self.online = True
@@ -137,7 +142,10 @@ class LocalClientInterface(Interface):
                 thread = threading.Thread(target=self.read_loop)
                 thread.daemon = True
                 thread.start()
-                RNS.Transport.shared_connection_reappeared()
+                def job():
+                    time.sleep(LocalClientInterface.RECONNECT_WAIT+2)
+                    RNS.Transport.shared_connection_reappeared()
+                threading.Thread(target=job, daemon=True).start()
         
         else:
             RNS.log("Attempt to reconnect on a non-initiator shared local interface. This should not happen.", RNS.LOG_ERROR)
@@ -145,9 +153,6 @@ class LocalClientInterface(Interface):
 
 
     def processIncoming(self, data):
-        if self._force_bitrate:
-            time.sleep(len(data) / self.bitrate * 8)
-
         self.rxb += len(data)
         if hasattr(self, "parent_interface") and self.parent_interface != None:
             self.parent_interface.rxb += len(data)
@@ -165,8 +170,16 @@ class LocalClientInterface(Interface):
         if self.online:
             try:
                 self.writing = True
+
                 if self._force_bitrate:
-                    time.sleep(len(data) / self.bitrate * 8)
+                    if not hasattr(self, "send_lock"):
+                        self.send_lock = Lock()
+
+                    with self.send_lock:
+                        s = len(data) / self.bitrate * 8
+                        RNS.log(f"Simulating latency of {RNS.prettytime(s)} for {len(data)} bytes")
+                        time.sleep(s)
+
                 data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
                 self.socket.sendall(data)
                 self.writing = False
@@ -259,7 +272,8 @@ class LocalClientInterface(Interface):
             RNS.Transport.local_client_interfaces.remove(self)
             if hasattr(self, "parent_interface") and self.parent_interface != None:
                 self.parent_interface.clients -= 1
-                RNS.Transport.owner._should_persist_data()
+                if hasattr(RNS.Transport, "owner") and RNS.Transport.owner != None:
+                    RNS.Transport.owner._should_persist_data()
 
         if nowarning == False:
             RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
@@ -280,8 +294,7 @@ class LocalClientInterface(Interface):
 class LocalServerInterface(Interface):
 
     def __init__(self, owner, bindport=None):
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
         self.online = False
         self.clients = 0
         
@@ -329,7 +342,9 @@ class LocalServerInterface(Interface):
         spawned_interface.target_port = str(handler.client_address[1])
         spawned_interface.parent_interface = self
         spawned_interface.bitrate = self.bitrate
-        RNS.log("Accepting new connection to shared instance: "+str(spawned_interface), RNS.LOG_EXTREME)
+        if hasattr(self, "_force_bitrate"):
+            spawned_interface._force_bitrate = self._force_bitrate
+        # RNS.log("Accepting new connection to shared instance: "+str(spawned_interface), RNS.LOG_EXTREME)
         RNS.Transport.interfaces.append(spawned_interface)
         RNS.Transport.local_client_interfaces.append(spawned_interface)
         self.clients += 1
@@ -338,6 +353,12 @@ class LocalServerInterface(Interface):
     def processOutgoing(self, data):
         pass
 
+    def received_announce(self, from_spawned=False):
+        if from_spawned: self.ia_freq_deque.append(time.time())
+
+    def sent_announce(self, from_spawned=False):
+        if from_spawned: self.oa_freq_deque.append(time.time())
+
     def __str__(self):
         return "Shared Instance["+str(self.bind_port)+"]"
 

RNS/Interfaces/PipeInterface.py

@@ -54,8 +54,7 @@ class PipeInterface(Interface):
         if respawn_delay == None:
             respawn_delay = 5
 
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 1064
         

RNS/Interfaces/RNodeInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -43,6 +43,8 @@ class KISS():
     CMD_CR          = 0x05
     CMD_RADIO_STATE = 0x06
     CMD_RADIO_LOCK  = 0x07
+    CMD_ST_ALOCK    = 0x0B
+    CMD_LT_ALOCK    = 0x0C
     CMD_DETECT      = 0x08
     CMD_LEAVE       = 0x0A
     CMD_READY       = 0x0F
@@ -50,11 +52,14 @@ class KISS():
     CMD_STAT_TX     = 0x22
     CMD_STAT_RSSI   = 0x23
     CMD_STAT_SNR    = 0x24
+    CMD_STAT_CHTM   = 0x25
+    CMD_STAT_PHYPRM = 0x26
     CMD_BLINK       = 0x30
     CMD_RANDOM      = 0x40
     CMD_FB_EXT      = 0x41
     CMD_FB_READ     = 0x42
     CMD_FB_WRITE    = 0x43
+    CMD_BT_CTRL     = 0x46
     CMD_PLATFORM    = 0x48
     CMD_MCU         = 0x49
     CMD_FW_VERSION  = 0x50
@@ -87,7 +92,7 @@ class RNodeInterface(Interface):
     MAX_CHUNK = 32768
 
     FREQ_MIN = 137000000
-    FREQ_MAX = 1020000000
+    FREQ_MAX = 3000000000
 
     RSSI_OFFSET = 157
 
@@ -98,9 +103,13 @@ class RNodeInterface(Interface):
 
     RECONNECT_WAIT = 5
 
-    def __init__(self, owner, name, port, frequency = None, bandwidth = None, txpower = None, sf = None, cr = None, flow_control = False, id_interval = None, id_callsign = None):
+    Q_SNR_MIN_BASE = -9
+    Q_SNR_MAX      = 6
+    Q_SNR_STEP     = 2
+
+    def __init__(self, owner, name, port, frequency = None, bandwidth = None, txpower = None, sf = None, cr = None, flow_control = False, id_interval = None, id_callsign = None, st_alock = None, lt_alock = None):
         if RNS.vendor.platformutils.is_android():
-            raise SystemError("Invlaid interface type. The Android-specific RNode interface must be used on Android")
+            raise SystemError("Invalid interface type. The Android-specific RNode interface must be used on Android")
 
         import importlib
         if importlib.util.find_spec('serial') != None:
@@ -110,8 +119,7 @@ class RNodeInterface(Interface):
             RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
             RNS.panic()
 
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 508
         
@@ -135,6 +143,8 @@ class RNodeInterface(Interface):
         self.cr          = cr
         self.state       = KISS.RADIO_STATE_OFF
         self.bitrate     = 0
+        self.st_alock    = st_alock
+        self.lt_alock    = lt_alock
         self.platform    = None
         self.display     = None
         self.mcu         = None
@@ -157,7 +167,18 @@ class RNodeInterface(Interface):
         self.r_stat_rx   = None
         self.r_stat_tx   = None
         self.r_stat_rssi = None
+        self.r_stat_snr  = None
+        self.r_st_alock  = None
+        self.r_lt_alock  = None
         self.r_random    = None
+        self.r_airtime_short      = 0.0
+        self.r_airtime_long       = 0.0
+        self.r_channel_load_short = 0.0
+        self.r_channel_load_long  = 0.0
+        self.r_symbol_time_ms = None
+        self.r_symbol_rate = None
+        self.r_preamble_symbols = None
+        self.r_premable_time_ms = None
 
         self.packet_queue    = []
         self.flow_control    = flow_control
@@ -169,15 +190,15 @@ class RNodeInterface(Interface):
             RNS.log("Invalid frequency configured for "+str(self), RNS.LOG_ERROR)
             self.validcfg = False
 
-        if (self.txpower < 0 or self.txpower > 17):
+        if (self.txpower < 0 or self.txpower > 22):
             RNS.log("Invalid TX power configured for "+str(self), RNS.LOG_ERROR)
             self.validcfg = False
 
-        if (self.bandwidth < 7800 or self.bandwidth > 500000):
+        if (self.bandwidth < 7800 or self.bandwidth > 1625000):
             RNS.log("Invalid bandwidth configured for "+str(self), RNS.LOG_ERROR)
             self.validcfg = False
 
-        if (self.sf < 7 or self.sf > 12):
+        if (self.sf < 5 or self.sf > 12):
             RNS.log("Invalid spreading factor configured for "+str(self), RNS.LOG_ERROR)
             self.validcfg = False
 
@@ -185,6 +206,14 @@ class RNodeInterface(Interface):
             RNS.log("Invalid coding rate configured for "+str(self), RNS.LOG_ERROR)
             self.validcfg = False
 
+        if (self.st_alock and (self.st_alock < 0.0 or self.st_alock > 100.0)):
+            RNS.log("Invalid short-term airtime limit configured for "+str(self), RNS.LOG_ERROR)
+            self.validcfg = False
+
+        if (self.lt_alock and (self.lt_alock < 0.0 or self.lt_alock > 100.0)):
+            RNS.log("Invalid long-term airtime limit configured for "+str(self), RNS.LOG_ERROR)
+            self.validcfg = False
+
         if id_interval != None and id_callsign != None:
             if (len(id_callsign.encode("utf-8")) <= RNodeInterface.CALLSIGN_MAX_LEN):
                 self.should_id = True
@@ -280,6 +309,8 @@ class RNodeInterface(Interface):
         self.setTXPower()
         self.setSpreadingFactor()
         self.setCodingRate()
+        self.setSTALock()
+        self.setLTALock()
         self.setRadioState(KISS.RADIO_STATE_ON)
 
     def detect(self):
@@ -384,6 +415,30 @@ class RNodeInterface(Interface):
         if written != len(kiss_command):
             raise IOError("An IO error occurred while configuring coding rate for "+str(self))
 
+    def setSTALock(self):
+        if self.st_alock != None:
+            at = int(self.st_alock*100)
+            c1 = at >> 8 & 0xFF
+            c2 = at & 0xFF
+            data = KISS.escape(bytes([c1])+bytes([c2]))
+
+            kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_ST_ALOCK])+data+bytes([KISS.FEND])
+            written = self.serial.write(kiss_command)
+            if written != len(kiss_command):
+                raise IOError("An IO error occurred while configuring short-term airtime limit for "+str(self))
+
+    def setLTALock(self):
+        if self.lt_alock != None:
+            at = int(self.lt_alock*100)
+            c1 = at >> 8 & 0xFF
+            c2 = at & 0xFF
+            data = KISS.escape(bytes([c1])+bytes([c2]))
+
+            kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_LT_ALOCK])+data+bytes([KISS.FEND])
+            written = self.serial.write(kiss_command)
+            if written != len(kiss_command):
+                raise IOError("An IO error occurred while configuring long-term airtime limit for "+str(self))
+
     def setRadioState(self, state):
         self.state = state
         kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_RADIO_STATE])+bytes([state])+bytes([KISS.FEND])
@@ -621,6 +676,96 @@ class RNodeInterface(Interface):
                             self.r_stat_rssi = byte-RNodeInterface.RSSI_OFFSET
                         elif (command == KISS.CMD_STAT_SNR):
                             self.r_stat_snr = int.from_bytes(bytes([byte]), byteorder="big", signed=True) * 0.25
+                            try:
+                                sfs = self.r_sf-7
+                                snr = self.r_stat_snr
+                                q_snr_min = RNodeInterface.Q_SNR_MIN_BASE-sfs*RNodeInterface.Q_SNR_STEP
+                                q_snr_max = RNodeInterface.Q_SNR_MAX
+                                q_snr_span = q_snr_max-q_snr_min
+                                quality = round(((snr-q_snr_min)/(q_snr_span))*100,1)
+                                if quality > 100.0: quality = 100.0
+                                if quality < 0.0: quality = 0.0
+                                self.r_stat_q = quality
+                            except:
+                                pass
+                        elif (command == KISS.CMD_ST_ALOCK):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 2):
+                                    at = command_buffer[0] << 8 | command_buffer[1]
+                                    self.r_st_alock = at/100.0
+                                    RNS.log(str(self)+" Radio reporting short-term airtime limit is "+str(self.r_st_alock)+"%", RNS.LOG_DEBUG)
+                        elif (command == KISS.CMD_LT_ALOCK):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 2):
+                                    at = command_buffer[0] << 8 | command_buffer[1]
+                                    self.r_lt_alock = at/100.0
+                                    RNS.log(str(self)+" Radio reporting long-term airtime limit is "+str(self.r_lt_alock)+"%", RNS.LOG_DEBUG)
+                        elif (command == KISS.CMD_STAT_CHTM):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 8):
+                                    ats = command_buffer[0] << 8 | command_buffer[1]
+                                    atl = command_buffer[2] << 8 | command_buffer[3]
+                                    cus = command_buffer[4] << 8 | command_buffer[5]
+                                    cul = command_buffer[6] << 8 | command_buffer[7]
+                                    
+                                    self.r_airtime_short      = ats/100.0
+                                    self.r_airtime_long       = atl/100.0
+                                    self.r_channel_load_short = cus/100.0
+                                    self.r_channel_load_long  = cul/100.0
+                        elif (command == KISS.CMD_STAT_PHYPRM):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 10):
+                                    lst = (command_buffer[0] << 8 | command_buffer[1])/1000.0
+                                    lsr = command_buffer[2] << 8 | command_buffer[3]
+                                    prs = command_buffer[4] << 8 | command_buffer[5]
+                                    prt = command_buffer[6] << 8 | command_buffer[7]
+                                    cst = command_buffer[8] << 8 | command_buffer[9]
+
+                                    if lst != self.r_symbol_time_ms or lsr != self.r_symbol_rate or prs != self.r_preamble_symbols or prt != self.r_premable_time_ms or cst != self.r_csma_slot_time_ms:
+                                        self.r_symbol_time_ms    = lst
+                                        self.r_symbol_rate       = lsr
+                                        self.r_preamble_symbols  = prs
+                                        self.r_premable_time_ms  = prt
+                                        self.r_csma_slot_time_ms = cst
+                                        RNS.log(str(self)+" Radio reporting symbol time is "+str(round(self.r_symbol_time_ms,2))+"ms (at "+str(self.r_symbol_rate)+" baud)", RNS.LOG_DEBUG)
+                                        RNS.log(str(self)+" Radio reporting preamble is "+str(self.r_preamble_symbols)+" symbols ("+str(self.r_premable_time_ms)+"ms)", RNS.LOG_DEBUG)
+                                        RNS.log(str(self)+" Radio reporting CSMA slot time is "+str(self.r_csma_slot_time_ms)+"ms", RNS.LOG_DEBUG)
                         elif (command == KISS.CMD_RANDOM):
                             self.r_random = byte
                         elif (command == KISS.CMD_PLATFORM):
@@ -654,7 +799,7 @@ class RNodeInterface(Interface):
                 else:
                     time_since_last = int(time.time()*1000) - last_read_ms
                     if len(data_buffer) > 0 and time_since_last > self.timeout:
-                        RNS.log(str(self)+" serial read timeout", RNS.LOG_DEBUG)
+                        RNS.log(str(self)+" serial read timeout in command "+str(command), RNS.LOG_WARNING)
                         data_buffer = b""
                         in_frame = False
                         command = KISS.CMD_UNKNOWN
@@ -709,5 +854,8 @@ class RNodeInterface(Interface):
         self.setRadioState(KISS.RADIO_STATE_OFF)
         self.leave()
 
+    def should_ingress_limit(self):
+        return False
+
     def __str__(self):
         return "RNodeInterface["+str(self.name)+"]"

RNS/Interfaces/SerialInterface.py

@@ -60,8 +60,7 @@ class SerialInterface(Interface):
             RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
             RNS.panic()
 
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 564
         
@@ -201,5 +200,8 @@ class SerialInterface(Interface):
 
         RNS.log("Reconnected serial port for "+str(self))
 
+    def should_ingress_limit(self):
+        return False
+
     def __str__(self):
         return "SerialInterface["+self.name+"]"

RNS/Interfaces/TCPInterface.py

@@ -79,8 +79,7 @@ class TCPClientInterface(Interface):
     I2P_PROBES = 5
 
     def __init__(self, owner, name, target_ip=None, target_port=None, connected_socket=None, max_reconnect_tries=None, kiss_framing=False, i2p_tunneled = False, connect_timeout = None):
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
         
         self.HW_MTU = 1064
         
@@ -117,6 +116,8 @@ class TCPClientInterface(Interface):
             elif platform.system() == "Darwin":
                 self.set_timeouts_osx()
 
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+
         elif target_ip != None and target_port != None:
             self.receives    = True
             self.target_ip   = target_ip
@@ -201,6 +202,7 @@ class TCPClientInterface(Interface):
 
             self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
             self.socket.settimeout(TCPClientInterface.INITIAL_CONNECT_TIMEOUT)
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
             self.socket.connect((self.target_ip, self.target_port))
             self.socket.settimeout(None)
             self.online  = True
@@ -419,8 +421,7 @@ class TCPServerInterface(Interface):
         return ifaddr[netinfo.AF_INET][0]["broadcast"]
 
     def __init__(self, owner, name, device=None, bindip=None, bindport=None, i2p_tunneled=False):
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 1064
 
@@ -505,6 +506,12 @@ class TCPServerInterface(Interface):
         self.clients += 1
         spawned_interface.read_loop()
 
+    def received_announce(self, from_spawned=False):
+        if from_spawned: self.ia_freq_deque.append(time.time())
+
+    def sent_announce(self, from_spawned=False):
+        if from_spawned: self.oa_freq_deque.append(time.time())
+
     def processOutgoing(self, data):
         pass
 

RNS/Interfaces/UDPInterface.py

@@ -45,8 +45,7 @@ class UDPInterface(Interface):
         return ifaddr[netinfo.AF_INET][0]["broadcast"]
 
     def __init__(self, owner, name, device=None, bindip=None, bindport=None, forwardip=None, forwardport=None):
-        self.rxb = 0
-        self.txb = 0
+        super().__init__()
 
         self.HW_MTU = 1064
 

RNS/Link.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -112,9 +112,10 @@ class Link:
                 link = Link(owner = owner, peer_pub_bytes=data[:Link.ECPUBSIZE//2], peer_sig_pub_bytes=data[Link.ECPUBSIZE//2:Link.ECPUBSIZE])
                 link.set_link_id(packet)
                 link.destination = packet.destination
-                link.establishment_timeout = Link.ESTABLISHMENT_TIMEOUT_PER_HOP * max(1, packet.hops)
+                link.establishment_timeout = Link.ESTABLISHMENT_TIMEOUT_PER_HOP * max(1, packet.hops) + Link.KEEPALIVE
                 link.establishment_cost += len(packet.raw)
                 RNS.log("Validating link request "+RNS.prettyhexrep(link.link_id), RNS.LOG_VERBOSE)
+                RNS.log(f"Establishment timeout is {RNS.prettytime(link.establishment_timeout)} for incoming link request "+RNS.prettyhexrep(link.link_id), RNS.LOG_EXTREME)
                 link.handshake()
                 link.attached_interface = packet.receiving_interface
                 link.prove()
@@ -123,7 +124,7 @@ class Link:
                 link.last_inbound = time.time()
                 link.start_watchdog()
                 
-                RNS.log("Incoming link request "+str(link)+" accepted", RNS.LOG_VERBOSE)
+                RNS.log("Incoming link request "+str(link)+" accepted", RNS.LOG_DEBUG)
                 return link
 
             except Exception as e:
@@ -132,7 +133,7 @@ class Link:
                 return None
 
         else:
-            RNS.log("Invalid link request payload size, dropping request", RNS.LOG_VERBOSE)
+            RNS.log("Invalid link request payload size, dropping request", RNS.LOG_DEBUG)
             return None
 
 
@@ -149,10 +150,14 @@ class Link:
         self.last_inbound = 0
         self.last_outbound = 0
         self.last_proof = 0
+        self.last_data = 0
         self.tx = 0
         self.rx = 0
         self.txbytes = 0
         self.rxbytes = 0
+        self.rssi = None
+        self.snr = None
+        self.q = None
         self.traffic_timeout_factor = Link.TRAFFIC_TIMEOUT_FACTOR
         self.keepalive_timeout_factor = Link.KEEPALIVE_TIMEOUT_FACTOR
         self.keepalive = Link.KEEPALIVE
@@ -163,16 +168,21 @@ class Link:
         self.type = RNS.Destination.LINK
         self.owner = owner
         self.destination = destination
+        self.expected_hops = None
         self.attached_interface = None
         self.__remote_identity = None
+        self.__track_phy_stats = False
         self._channel = None
+
         if self.destination == None:
             self.initiator = False
             self.prv     = X25519PrivateKey.generate()
             self.sig_prv = self.owner.identity.sig_prv
         else:
             self.initiator = True
-            self.establishment_timeout = Link.ESTABLISHMENT_TIMEOUT_PER_HOP * max(1, RNS.Transport.hops_to(destination.hash))
+            self.expected_hops = RNS.Transport.hops_to(self.destination.hash)
+            self.establishment_timeout  = RNS.Reticulum.get_instance().get_first_hop_timeout(destination.hash)
+            self.establishment_timeout += Link.ESTABLISHMENT_TIMEOUT_PER_HOP * max(1, RNS.Transport.hops_to(destination.hash))
             self.prv     = X25519PrivateKey.generate()
             self.sig_prv = Ed25519PrivateKey.generate()
 
@@ -208,6 +218,7 @@ class Link:
             self.packet.send()
             self.had_outbound()
             RNS.log("Link request "+RNS.prettyhexrep(self.link_id)+" sent to "+str(self.destination), RNS.LOG_DEBUG)
+            RNS.log(f"Establishment timeout is {RNS.prettytime(self.establishment_timeout)} for link request "+RNS.prettyhexrep(self.link_id), RNS.LOG_EXTREME)
 
 
     def load_peer(self, peer_pub_bytes, peer_sig_pub_bytes):
@@ -225,15 +236,18 @@ class Link:
         self.hash = self.link_id
 
     def handshake(self):
-        self.status = Link.HANDSHAKE
-        self.shared_key = self.prv.exchange(self.peer_pub)
+        if self.status == Link.PENDING and self.prv != None:
+            self.status = Link.HANDSHAKE
+            self.shared_key = self.prv.exchange(self.peer_pub)
 
-        self.derived_key = RNS.Cryptography.hkdf(
-            length=32,
-            derive_from=self.shared_key,
-            salt=self.get_salt(),
-            context=self.get_context(),
-        )
+            self.derived_key = RNS.Cryptography.hkdf(
+                length=32,
+                derive_from=self.shared_key,
+                salt=self.get_salt(),
+                context=self.get_context(),
+            )
+        else:
+            RNS.log("Handshake attempt on "+str(self)+" with invalid state "+str(self.status), RNS.LOG_ERROR)
 
 
     def prove(self):
@@ -261,41 +275,50 @@ class Link:
         self.had_outbound()
 
     def validate_proof(self, packet):
-        if self.status == Link.PENDING:
-            if self.initiator and len(packet.data) == RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2:
-                peer_pub_bytes = packet.data[RNS.Identity.SIGLENGTH//8:RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2]
-                peer_sig_pub_bytes = self.destination.identity.get_public_key()[Link.ECPUBSIZE//2:Link.ECPUBSIZE]
-                self.load_peer(peer_pub_bytes, peer_sig_pub_bytes)
-                self.handshake()
+        try:
+            if self.status == Link.PENDING:
+                if self.initiator and len(packet.data) == RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2:
+                    peer_pub_bytes = packet.data[RNS.Identity.SIGLENGTH//8:RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2]
+                    peer_sig_pub_bytes = self.destination.identity.get_public_key()[Link.ECPUBSIZE//2:Link.ECPUBSIZE]
+                    self.load_peer(peer_pub_bytes, peer_sig_pub_bytes)
+                    self.handshake()
 
-                self.establishment_cost += len(packet.raw)
-                signed_data = self.link_id+self.peer_pub_bytes+self.peer_sig_pub_bytes
-                signature = packet.data[:RNS.Identity.SIGLENGTH//8]
-                
-                if self.destination.identity.validate(signature, signed_data):
-                    self.rtt = time.time() - self.request_time
-                    self.attached_interface = packet.receiving_interface
-                    self.__remote_identity = self.destination.identity
-                    self.status = Link.ACTIVE
-                    self.activated_at = time.time()
-                    self.last_proof = self.activated_at
-                    RNS.Transport.activate_link(self)
-                    RNS.log("Link "+str(self)+" established with "+str(self.destination)+", RTT is "+str(round(self.rtt, 3))+"s", RNS.LOG_VERBOSE)
+                    self.establishment_cost += len(packet.raw)
+                    signed_data = self.link_id+self.peer_pub_bytes+self.peer_sig_pub_bytes
+                    signature = packet.data[:RNS.Identity.SIGLENGTH//8]
                     
-                    if self.rtt != None and self.establishment_cost != None and self.rtt > 0 and self.establishment_cost > 0:
-                        self.establishment_rate = self.establishment_cost/self.rtt
+                    if self.destination.identity.validate(signature, signed_data):
+                        if self.status != Link.HANDSHAKE:
+                            raise IOError("Invalid link state for proof validation: "+str(self.status))
 
-                    rtt_data = umsgpack.packb(self.rtt)
-                    rtt_packet = RNS.Packet(self, rtt_data, context=RNS.Packet.LRRTT)
-                    rtt_packet.send()
-                    self.had_outbound()
+                        self.rtt = time.time() - self.request_time
+                        self.attached_interface = packet.receiving_interface
+                        self.__remote_identity = self.destination.identity
+                        self.status = Link.ACTIVE
+                        self.activated_at = time.time()
+                        self.last_proof = self.activated_at
+                        RNS.Transport.activate_link(self)
+                        RNS.log("Link "+str(self)+" established with "+str(self.destination)+", RTT is "+str(round(self.rtt, 3))+"s", RNS.LOG_VERBOSE)
+                        
+                        if self.rtt != None and self.establishment_cost != None and self.rtt > 0 and self.establishment_cost > 0:
+                            self.establishment_rate = self.establishment_cost/self.rtt
 
-                    if self.callbacks.link_established != None:
-                        thread = threading.Thread(target=self.callbacks.link_established, args=(self,))
-                        thread.daemon = True
-                        thread.start()
-                else:
-                    RNS.log("Invalid link proof signature received by "+str(self)+". Ignoring.", RNS.LOG_DEBUG)
+                        rtt_data = umsgpack.packb(self.rtt)
+                        rtt_packet = RNS.Packet(self, rtt_data, context=RNS.Packet.LRRTT)
+                        rtt_packet.send()
+                        self.had_outbound()
+
+                        if self.callbacks.link_established != None:
+                            thread = threading.Thread(target=self.callbacks.link_established, args=(self,))
+                            thread.daemon = True
+                            thread.start()
+                    else:
+                        RNS.log("Invalid link proof signature received by "+str(self)+". Ignoring.", RNS.LOG_DEBUG)
+        
+        except Exception as e:
+            self.status = Link.CLOSED
+            RNS.log("An error ocurred while validating link request proof on "+str(self)+".", RNS.LOG_ERROR)
+            RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
 
 
     def identify(self, identity):
@@ -307,7 +330,7 @@ class Link:
 
         :param identity: An RNS.Identity instance to identify as.
         """
-        if self.initiator:
+        if self.initiator and self.status == Link.ACTIVE:
             signed_data = self.link_id + identity.get_public_key()
             signature = identity.sign(signed_data)
             proof_data = identity.get_public_key() + signature
@@ -333,7 +356,7 @@ class Link:
         packed_request    = umsgpack.packb(unpacked_request)
 
         if timeout == None:
-            timeout = self.rtt * self.traffic_timeout_factor + RNS.Resource.RESPONSE_MAX_GRACE_TIME/4.0
+            timeout = self.rtt * self.traffic_timeout_factor + RNS.Resource.RESPONSE_MAX_GRACE_TIME*1.125
 
         if len(packed_request) <= Link.MDU:
             request_packet   = RNS.Packet(self, packed_request, RNS.Packet.DATA, context = RNS.Packet.REQUEST)
@@ -373,24 +396,57 @@ class Link:
         try:
             measured_rtt = time.time() - self.request_time
             plaintext = self.decrypt(packet.data)
-            rtt = umsgpack.unpackb(plaintext)
-            self.rtt = max(measured_rtt, rtt)
-            self.status = Link.ACTIVE
-            self.activated_at = time.time()
+            if plaintext != None:
+                rtt = umsgpack.unpackb(plaintext)
+                self.rtt = max(measured_rtt, rtt)
+                self.status = Link.ACTIVE
+                self.activated_at = time.time()
 
-            if self.rtt != None and self.establishment_cost != None and self.rtt > 0 and self.establishment_cost > 0:
-                self.establishment_rate = self.establishment_cost/self.rtt
+                if self.rtt != None and self.establishment_cost != None and self.rtt > 0 and self.establishment_cost > 0:
+                    self.establishment_rate = self.establishment_cost/self.rtt
 
-            try:
-                if self.owner.callbacks.link_established != None:
-                        self.owner.callbacks.link_established(self)
-            except Exception as e:
-                RNS.log("Error occurred in external link establishment callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
+                try:
+                    if self.owner.callbacks.link_established != None:
+                            self.owner.callbacks.link_established(self)
+                except Exception as e:
+                    RNS.log("Error occurred in external link establishment callback. The contained exception was: "+str(e), RNS.LOG_ERROR)
 
         except Exception as e:
             RNS.log("Error occurred while processing RTT packet, tearing down link. The contained exception was: "+str(e), RNS.LOG_ERROR)
             self.teardown()
 
+    def track_phy_stats(self, track):
+        """
+        You can enable physical layer statistics on a per-link basis. If this is enabled,
+        and the link is running over an interface that supports reporting physical layer
+        statistics, you will be able to retrieve stats such as *RSSI*, *SNR* and physical
+        *Link Quality* for the link.
+
+        :param track: Whether or not to keep track of physical layer statistics. Value must be ``True`` or ``False``.
+        """
+        if track:
+            self.__track_phy_stats = True
+        else:
+            self.__track_phy_stats = False
+
+    def get_rssi(self):
+        """
+        :returns: The physical layer *Received Signal Strength Indication* if available, otherwise ``None``. Physical layer statistics must be enabled on the link for this method to return a value.
+        """
+        return self.rssi
+
+    def get_snr(self):
+        """
+        :returns: The physical layer *Signal-to-Noise Ratio* if available, otherwise ``None``. Physical layer statistics must be enabled on the link for this method to return a value.
+        """
+        return self.rssi
+
+    def get_q(self):
+        """
+        :returns: The physical layer *Link Quality* if available, otherwise ``None``. Physical layer statistics must be enabled on the link for this method to return a value.
+        """
+        return self.rssi
+
     def get_establishment_rate(self):
         """
         :returns: The data transfer rate at which the link establishment procedure ocurred, in bits per second.
@@ -408,7 +464,7 @@ class Link:
 
     def no_inbound_for(self):
         """
-        :returns: The time in seconds since last inbound packet on the link.
+        :returns: The time in seconds since last inbound packet on the link. This includes keepalive packets.
         """
         activated_at = self.activated_at if self.activated_at != None else 0
         last_inbound = max(self.last_inbound, activated_at)
@@ -416,13 +472,19 @@ class Link:
 
     def no_outbound_for(self):
         """
-        :returns: The time in seconds since last outbound packet on the link.
+        :returns: The time in seconds since last outbound packet on the link. This includes keepalive packets.
         """
         return time.time() - self.last_outbound
 
+    def no_data_for(self):
+        """
+        :returns: The time in seconds since payload data traversed the link. This excludes keepalive packets.
+        """
+        return time.time() - self.last_data
+
     def inactive_for(self):
         """
-        :returns: The time in seconds since activity on the link.
+        :returns: The time in seconds since activity on the link. This includes keepalive packets.
         """
         return min(self.no_inbound_for(), self.no_outbound_for())
 
@@ -432,8 +494,10 @@ class Link:
         """
         return self.__remote_identity
 
-    def had_outbound(self):
+    def had_outbound(self, is_keepalive=False):
         self.last_outbound = time.time()
+        if not is_keepalive:
+            self.last_data = self.last_outbound
 
     def teardown(self):
         """
@@ -460,6 +524,7 @@ class Link:
                     self.teardown_reason = Link.DESTINATION_CLOSED
                 else:
                     self.teardown_reason = Link.INITIATOR_CLOSED
+                self.__update_phy_stats(packet)
                 self.link_closed()
         except Exception as e:
             pass
@@ -498,7 +563,11 @@ class Link:
     def __watchdog_job(self):
         while not self.status == Link.CLOSED:
             while (self.watchdog_lock):
-                sleep(max(self.rtt, 0.025))
+                rtt_wait = 0.025
+                if hasattr(self, "rtt") and self.rtt:
+                    rtt_wait = self.rtt
+
+                sleep(max(rtt_wait, 0.025))
 
             if not self.status == Link.CLOSED:
                 # Link was initiated, but no response
@@ -517,16 +586,16 @@ class Link:
                     next_check = self.request_time + self.establishment_timeout
                     sleep_time = next_check - time.time()
                     if time.time() >= self.request_time + self.establishment_timeout:
-                        if self.initiator:
-                            RNS.log("Timeout waiting for link request proof", RNS.LOG_DEBUG)
-                        else:
-                            RNS.log("Timeout waiting for RTT packet from link initiator", RNS.LOG_DEBUG)
-
                         self.status = Link.CLOSED
                         self.teardown_reason = Link.TIMEOUT
                         self.link_closed()
                         sleep_time = 0.001
 
+                        if self.initiator:
+                            RNS.log("Timeout waiting for link request proof", RNS.LOG_DEBUG)
+                        else:
+                            RNS.log("Timeout waiting for RTT packet from link initiator", RNS.LOG_DEBUG)
+
                 elif self.status == Link.ACTIVE:
                     activated_at = self.activated_at if self.activated_at != None else 0
                     last_inbound = max(max(self.last_inbound, self.last_proof), activated_at)
@@ -561,10 +630,25 @@ class Link:
                 sleep(sleep_time)
 
 
+    def __update_phy_stats(self, packet, query_shared = True):
+        if self.__track_phy_stats:
+            if query_shared:
+                reticulum = RNS.Reticulum.get_instance()
+                if packet.rssi == None: packet.rssi = reticulum.get_packet_rssi(packet.packet_hash)
+                if packet.snr  == None: packet.snr  = reticulum.get_packet_snr(packet.packet_hash)
+                if packet.q    == None: packet.q    = reticulum.get_packet_q(packet.packet_hash)
+
+            if packet.rssi != None:
+                self.rssi = packet.rssi
+            if packet.snr != None:
+                self.snr = packet.snr
+            if packet.q != None:
+                self.q = packet.q
+    
     def send_keepalive(self):
         keepalive_packet = RNS.Packet(self, bytes([0xFF]), context=RNS.Packet.KEEPALIVE)
         keepalive_packet.send()
-        self.had_outbound()
+        self.had_outbound(is_keepalive = True)
 
     def handle_request(self, request_id, unpacked_request):
         if self.status == Link.ACTIVE:
@@ -615,7 +699,9 @@ class Link:
                     remove = pending_request
                     try:
                         pending_request.response_size = response_size
-                        pending_request.response_transfer_size = response_transfer_size
+                        if pending_request.response_transfer_size == None:
+                            pending_request.response_transfer_size = 0
+                        pending_request.response_transfer_size += response_transfer_size
                         pending_request.response_received(response_data)
                     except Exception as e:
                         RNS.log("Error occurred while handling response. The contained exception was: "+str(e), RNS.LOG_ERROR)
@@ -668,135 +754,167 @@ class Link:
                 RNS.log("Link-associated packet received on unexpected interface! Someone might be trying to manipulate your communication!", RNS.LOG_ERROR)
             else:
                 self.last_inbound = time.time()
+                if packet.context != RNS.Packet.KEEPALIVE:
+                    self.last_data = self.last_inbound
                 self.rx += 1
                 self.rxbytes += len(packet.data)
                 if self.status == Link.STALE:
                     self.status = Link.ACTIVE
 
                 if packet.packet_type == RNS.Packet.DATA:
+                    should_query = False
                     if packet.context == RNS.Packet.NONE:
                         plaintext = self.decrypt(packet.data)
-                        if self.callbacks.packet != None:
-                            thread = threading.Thread(target=self.callbacks.packet, args=(plaintext, packet))
-                            thread.daemon = True
-                            thread.start()
-                        
-                        if self.destination.proof_strategy == RNS.Destination.PROVE_ALL:
-                            packet.prove()
+                        if plaintext != None:
+                            if self.callbacks.packet != None:
+                                thread = threading.Thread(target=self.callbacks.packet, args=(plaintext, packet))
+                                thread.daemon = True
+                                thread.start()
+                            
+                            if self.destination.proof_strategy == RNS.Destination.PROVE_ALL:
+                                packet.prove()
+                                should_query = True
 
-                        elif self.destination.proof_strategy == RNS.Destination.PROVE_APP:
-                            if self.destination.callbacks.proof_requested:
-                                try:
-                                    self.destination.callbacks.proof_requested(packet)
-                                except Exception as e:
-                                    RNS.log("Error while executing proof request callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+                            elif self.destination.proof_strategy == RNS.Destination.PROVE_APP:
+                                if self.destination.callbacks.proof_requested:
+                                    try:
+                                        if self.destination.callbacks.proof_requested(packet):
+                                            packet.prove()
+                                            should_query = True
+                                    except Exception as e:
+                                        RNS.log("Error while executing proof request callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+
+                            self.__update_phy_stats(packet, query_shared=should_query)
 
                     elif packet.context == RNS.Packet.LINKIDENTIFY:
                         plaintext = self.decrypt(packet.data)
+                        if plaintext != None:
+                            if not self.initiator and len(plaintext) == RNS.Identity.KEYSIZE//8 + RNS.Identity.SIGLENGTH//8:
+                                public_key   = plaintext[:RNS.Identity.KEYSIZE//8]
+                                signed_data  = self.link_id+public_key
+                                signature    = plaintext[RNS.Identity.KEYSIZE//8:RNS.Identity.KEYSIZE//8+RNS.Identity.SIGLENGTH//8]
+                                identity     = RNS.Identity(create_keys=False)
+                                identity.load_public_key(public_key)
 
-                        if not self.initiator and len(plaintext) == RNS.Identity.KEYSIZE//8 + RNS.Identity.SIGLENGTH//8:
-                            public_key   = plaintext[:RNS.Identity.KEYSIZE//8]
-                            signed_data  = self.link_id+public_key
-                            signature    = plaintext[RNS.Identity.KEYSIZE//8:RNS.Identity.KEYSIZE//8+RNS.Identity.SIGLENGTH//8]
-                            identity     = RNS.Identity(create_keys=False)
-                            identity.load_public_key(public_key)
-
-                            if identity.validate(signature, signed_data):
-                                self.__remote_identity = identity
-                                if self.callbacks.remote_identified != None:
-                                    try:
-                                        self.callbacks.remote_identified(self, self.__remote_identity)
-                                    except Exception as e:
-                                        RNS.log("Error while executing remote identified callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+                                if identity.validate(signature, signed_data):
+                                    self.__remote_identity = identity
+                                    if self.callbacks.remote_identified != None:
+                                        try:
+                                            self.callbacks.remote_identified(self, self.__remote_identity)
+                                        except Exception as e:
+                                            RNS.log("Error while executing remote identified callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+                                
+                                    self.__update_phy_stats(packet, query_shared=True)
 
                     elif packet.context == RNS.Packet.REQUEST:
                         try:
                             request_id = packet.getTruncatedHash()
                             packed_request = self.decrypt(packet.data)
-                            unpacked_request = umsgpack.unpackb(packed_request)
-                            self.handle_request(request_id, unpacked_request)
+                            if packed_request != None:
+                                unpacked_request = umsgpack.unpackb(packed_request)
+                                self.handle_request(request_id, unpacked_request)
+                                self.__update_phy_stats(packet, query_shared=True)
                         except Exception as e:
                             RNS.log("Error occurred while handling request. The contained exception was: "+str(e), RNS.LOG_ERROR)
 
                     elif packet.context == RNS.Packet.RESPONSE:
                         try:
                             packed_response = self.decrypt(packet.data)
-                            unpacked_response = umsgpack.unpackb(packed_response)
-                            request_id = unpacked_response[0]
-                            response_data = unpacked_response[1]
-                            transfer_size = len(umsgpack.packb(response_data))-2
-                            self.handle_response(request_id, response_data, transfer_size, transfer_size)
+                            if packed_response != None:
+                                unpacked_response = umsgpack.unpackb(packed_response)
+                                request_id = unpacked_response[0]
+                                response_data = unpacked_response[1]
+                                transfer_size = len(umsgpack.packb(response_data))-2
+                                self.handle_response(request_id, response_data, transfer_size, transfer_size)
+                                self.__update_phy_stats(packet, query_shared=True)
                         except Exception as e:
                             RNS.log("Error occurred while handling response. The contained exception was: "+str(e), RNS.LOG_ERROR)
 
                     elif packet.context == RNS.Packet.LRRTT:
                         if not self.initiator:
                             self.rtt_packet(packet)
+                            self.__update_phy_stats(packet, query_shared=True)
 
                     elif packet.context == RNS.Packet.LINKCLOSE:
                         self.teardown_packet(packet)
+                        self.__update_phy_stats(packet, query_shared=True)
 
                     elif packet.context == RNS.Packet.RESOURCE_ADV:
                         packet.plaintext = self.decrypt(packet.data)
+                        if packet.plaintext != None:
+                            self.__update_phy_stats(packet, query_shared=True)
 
-                        if RNS.ResourceAdvertisement.is_request(packet):
-                            RNS.Resource.accept(packet, callback=self.request_resource_concluded)
-                        elif RNS.ResourceAdvertisement.is_response(packet):
-                            request_id = RNS.ResourceAdvertisement.read_request_id(packet)
-                            for pending_request in self.pending_requests:
-                                if pending_request.request_id == request_id:
-                                    RNS.Resource.accept(packet, callback=self.response_resource_concluded, progress_callback=pending_request.response_resource_progress, request_id = request_id)
-                                    pending_request.response_size = RNS.ResourceAdvertisement.read_size(packet)
-                                    pending_request.response_transfer_size = RNS.ResourceAdvertisement.read_transfer_size(packet)
-                                    pending_request.started_at = time.time()
-                        elif self.resource_strategy == Link.ACCEPT_NONE:
-                            pass
-                        elif self.resource_strategy == Link.ACCEPT_APP:
-                            if self.callbacks.resource != None:
-                                try:
-                                    resource_advertisement = RNS.ResourceAdvertisement.unpack(packet.plaintext)
-                                    resource_advertisement.link = self
-                                    if self.callbacks.resource(resource_advertisement):
-                                        RNS.Resource.accept(packet, self.callbacks.resource_concluded)
-                                except Exception as e:
-                                    RNS.log("Error while executing resource accept callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
-                        elif self.resource_strategy == Link.ACCEPT_ALL:
-                            RNS.Resource.accept(packet, self.callbacks.resource_concluded)
+                            if RNS.ResourceAdvertisement.is_request(packet):
+                                RNS.Resource.accept(packet, callback=self.request_resource_concluded)
+                            elif RNS.ResourceAdvertisement.is_response(packet):
+                                request_id = RNS.ResourceAdvertisement.read_request_id(packet)
+                                for pending_request in self.pending_requests:
+                                    if pending_request.request_id == request_id:
+                                        response_resource = RNS.Resource.accept(packet, callback=self.response_resource_concluded, progress_callback=pending_request.response_resource_progress, request_id = request_id)
+                                        if response_resource != None:
+                                            if pending_request.response_size == None:
+                                                pending_request.response_size = RNS.ResourceAdvertisement.read_size(packet)
+                                            if pending_request.response_transfer_size == None:
+                                                pending_request.response_transfer_size = 0
+                                            pending_request.response_transfer_size += RNS.ResourceAdvertisement.read_transfer_size(packet)
+                                            if pending_request.started_at == None:
+                                                pending_request.started_at = time.time()
+                                            pending_request.response_resource_progress(response_resource)
+
+                            elif self.resource_strategy == Link.ACCEPT_NONE:
+                                pass
+                            elif self.resource_strategy == Link.ACCEPT_APP:
+                                if self.callbacks.resource != None:
+                                    try:
+                                        resource_advertisement = RNS.ResourceAdvertisement.unpack(packet.plaintext)
+                                        resource_advertisement.link = self
+                                        if self.callbacks.resource(resource_advertisement):
+                                            RNS.Resource.accept(packet, self.callbacks.resource_concluded)
+                                    except Exception as e:
+                                        RNS.log("Error while executing resource accept callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+                            elif self.resource_strategy == Link.ACCEPT_ALL:
+                                RNS.Resource.accept(packet, self.callbacks.resource_concluded)
 
                     elif packet.context == RNS.Packet.RESOURCE_REQ:
                         plaintext = self.decrypt(packet.data)
-                        if ord(plaintext[:1]) == RNS.Resource.HASHMAP_IS_EXHAUSTED:
-                            resource_hash = plaintext[1+RNS.Resource.MAPHASH_LEN:RNS.Identity.HASHLENGTH//8+1+RNS.Resource.MAPHASH_LEN]
-                        else:
-                            resource_hash = plaintext[1:RNS.Identity.HASHLENGTH//8+1]
+                        if plaintext != None:
+                            self.__update_phy_stats(packet, query_shared=True)
+                            if ord(plaintext[:1]) == RNS.Resource.HASHMAP_IS_EXHAUSTED:
+                                resource_hash = plaintext[1+RNS.Resource.MAPHASH_LEN:RNS.Identity.HASHLENGTH//8+1+RNS.Resource.MAPHASH_LEN]
+                            else:
+                                resource_hash = plaintext[1:RNS.Identity.HASHLENGTH//8+1]
 
-                        for resource in self.outgoing_resources:
-                            if resource.hash == resource_hash:
-                                # We need to check that this request has not been
-                                # received before in order to avoid sequencing errors.
-                                if not packet.packet_hash in resource.req_hashlist:
-                                    resource.req_hashlist.append(packet.packet_hash)
-                                    resource.request(plaintext)
+                            for resource in self.outgoing_resources:
+                                if resource.hash == resource_hash:
+                                    # We need to check that this request has not been
+                                    # received before in order to avoid sequencing errors.
+                                    if not packet.packet_hash in resource.req_hashlist:
+                                        resource.req_hashlist.append(packet.packet_hash)
+                                        resource.request(plaintext)
 
                     elif packet.context == RNS.Packet.RESOURCE_HMU:
                         plaintext = self.decrypt(packet.data)
-                        resource_hash = plaintext[:RNS.Identity.HASHLENGTH//8]
-                        for resource in self.incoming_resources:
-                            if resource_hash == resource.hash:
-                                resource.hashmap_update_packet(plaintext)
+                        if plaintext != None:
+                            self.__update_phy_stats(packet, query_shared=True)
+                            resource_hash = plaintext[:RNS.Identity.HASHLENGTH//8]
+                            for resource in self.incoming_resources:
+                                if resource_hash == resource.hash:
+                                    resource.hashmap_update_packet(plaintext)
 
                     elif packet.context == RNS.Packet.RESOURCE_ICL:
                         plaintext = self.decrypt(packet.data)
-                        resource_hash = plaintext[:RNS.Identity.HASHLENGTH//8]
-                        for resource in self.incoming_resources:
-                            if resource_hash == resource.hash:
-                                resource.cancel()
+                        if plaintext != None:
+                            self.__update_phy_stats(packet)
+                            resource_hash = plaintext[:RNS.Identity.HASHLENGTH//8]
+                            for resource in self.incoming_resources:
+                                if resource_hash == resource.hash:
+                                    resource.cancel()
 
                     elif packet.context == RNS.Packet.KEEPALIVE:
                         if not self.initiator and packet.data == bytes([0xFF]):
                             keepalive_packet = RNS.Packet(self, bytes([0xFE]), context=RNS.Packet.KEEPALIVE)
                             keepalive_packet.send()
-                            self.had_outbound()
+                            self.had_outbound(is_keepalive = True)
 
 
                     # TODO: find the most efficient way to allow multiple
@@ -806,27 +924,17 @@ class Link:
                     elif packet.context == RNS.Packet.RESOURCE:
                         for resource in self.incoming_resources:
                             resource.receive_part(packet)
+                            self.__update_phy_stats(packet)
 
                     elif packet.context == RNS.Packet.CHANNEL:
                         if not self._channel:
                             RNS.log(f"Channel data received without open channel", RNS.LOG_DEBUG)
                         else:
-                            # TODO: Remove packet loss simulator ######
-                            # if not hasattr(self, "drop_counter"):
-                            #     self.drop_counter = 0
-                            # self.drop_counter += 1
-
-                            # if self.drop_counter%6 == 0:
-                            #     RNS.log("Dropping channel packet for testing", RNS.LOG_DEBUG)
-                            # else:
-                            #     packet.prove()
-                            #     plaintext = self.decrypt(packet.data)
-                            #     self._channel._receive(plaintext)
-                            ############################################
-
                             packet.prove()
                             plaintext = self.decrypt(packet.data)
-                            self._channel._receive(plaintext)
+                            if plaintext != None:
+                                self.__update_phy_stats(packet)
+                                self._channel._receive(plaintext)
 
                 elif packet.packet_type == RNS.Packet.PROOF:
                     if packet.context == RNS.Packet.RESOURCE_PRF:
@@ -834,6 +942,7 @@ class Link:
                         for resource in self.outgoing_resources:
                             if resource_hash == resource.hash:
                                 resource.validate_proof(packet.data)
+                                self.__update_phy_stats(packet, query_shared=True)
 
         self.watchdog_lock = False
 
@@ -844,7 +953,7 @@ class Link:
                 try:
                     self.fernet = Fernet(self.derived_key)
                 except Exception as e:
-                    RNS.log("Could not "+str(self)+" instantiate Fernet while performin encryption on link. The contained exception was: "+str(e), RNS.LOG_ERROR)
+                    RNS.log("Could not instantiate Fernet while performin encryption on link "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
                     raise e
 
             return self.fernet.encrypt(plaintext)
@@ -863,6 +972,7 @@ class Link:
 
         except Exception as e:
             RNS.log("Decryption failed on link "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+            return None
 
 
     def sign(self, message):
@@ -1043,7 +1153,8 @@ class RequestReceipt():
     def request_resource_concluded(self, resource):
         if resource.status == RNS.Resource.COMPLETE:
             RNS.log("Request "+RNS.prettyhexrep(self.request_id)+" successfully sent as resource.", RNS.LOG_DEBUG)
-            self.started_at = time.time()
+            if self.started_at == None:
+                self.started_at = time.time()
             self.status = RequestReceipt.DELIVERED
             self.__resource_response_timeout = time.time()+self.timeout
             response_timeout_thread = threading.Thread(target=self.__response_timeout_job)
@@ -1084,24 +1195,26 @@ class RequestReceipt():
 
 
     def response_resource_progress(self, resource):
-        if not self.status == RequestReceipt.FAILED:
-            self.status = RequestReceipt.RECEIVING
-            if self.packet_receipt != None:
-                self.packet_receipt.status = RNS.PacketReceipt.DELIVERED
-                self.packet_receipt.proved = True
-                self.packet_receipt.concluded_at = time.time()
-                if self.packet_receipt.callbacks.delivery != None:
-                    self.packet_receipt.callbacks.delivery(self.packet_receipt)
+        if resource != None:
+            if not self.status == RequestReceipt.FAILED:
+                self.status = RequestReceipt.RECEIVING
+                if self.packet_receipt != None:
+                    if self.packet_receipt.status != RNS.PacketReceipt.DELIVERED:
+                        self.packet_receipt.status = RNS.PacketReceipt.DELIVERED
+                        self.packet_receipt.proved = True
+                        self.packet_receipt.concluded_at = time.time()
+                        if self.packet_receipt.callbacks.delivery != None:
+                            self.packet_receipt.callbacks.delivery(self.packet_receipt)
 
-            self.progress = resource.get_progress()
-            
-            if self.callbacks.progress != None:
-                try:
-                    self.callbacks.progress(self)
-                except Exception as e:
-                    RNS.log("Error while executing response progress callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
-        else:
-            resource.cancel()
+                self.progress = resource.get_progress()
+                
+                if self.callbacks.progress != None:
+                    try:
+                        self.callbacks.progress(self)
+                    except Exception as e:
+                        RNS.log("Error while executing response progress callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+            else:
+                resource.cancel()
 
     
     def response_received(self, response):

RNS/Packet.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -30,7 +30,7 @@ class Packet:
     """
     The Packet class is used to create packet instances that can be sent
     over a Reticulum network. Packets will automatically be encrypted if
-    they are adressed to a ``RNS.Destination.SINGLE`` destination,
+    they are addressed to a ``RNS.Destination.SINGLE`` destination,
     ``RNS.Destination.GROUP`` destination or a :ref:`RNS.Link<api-link>`.
 
     For ``RNS.Destination.GROUP`` destinations, Reticulum will use the
@@ -138,6 +138,7 @@ class Packet:
         self.receiving_interface = None
         self.rssi = None
         self.snr = None
+        self.q = None
 
     def get_packed_flags(self):
         if self.context == Packet.LRPROOF:
@@ -370,8 +371,8 @@ class PacketReceipt:
         if packet.destination.type == RNS.Destination.LINK:
             self.timeout    = packet.destination.rtt * packet.destination.traffic_timeout_factor
         else:
-            self.timeout    = Packet.TIMEOUT_PER_HOP * RNS.Transport.hops_to(self.destination.hash)
-
+            self.timeout    = RNS.Reticulum.get_instance().get_first_hop_timeout(self.destination.hash)
+            self.timeout   += Packet.TIMEOUT_PER_HOP * RNS.Transport.hops_to(self.destination.hash)
 
     def get_status(self):
         """

RNS/Resolver.py

@@ -0,0 +1,27 @@
+# MIT License
+#
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+class Resolver:
+  
+  @staticmethod
+  def resolve_identity(full_name):
+    pass

RNS/Resource.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -25,7 +25,9 @@ import os
 import bz2
 import math
 import time
+import tempfile
 import threading
+from threading import Lock
 from .vendor import umsgpack as umsgpack
 from time import sleep
 
@@ -47,7 +49,7 @@ class Resource:
     WINDOW               = 4
 
     # Absolute minimum window size during transfer
-    WINDOW_MIN           = 1
+    WINDOW_MIN           = 2
 
     # The maximum window size for transfers on slow links
     WINDOW_MAX_SLOW      = 10
@@ -103,9 +105,10 @@ class Resource:
 
     PART_TIMEOUT_FACTOR           = 4
     PART_TIMEOUT_FACTOR_AFTER_RTT = 2
-    MAX_RETRIES                   = 8
+    MAX_RETRIES                   = 16
     MAX_ADV_RETRIES               = 4
-    SENDER_GRACE_TIME             = 10
+    SENDER_GRACE_TIME             = 10.0
+    PROCESSING_GRACE              = 1.0
     RETRY_GRACE_TIME              = 0.25
     PER_RETRY_DELAY               = 0.5
 
@@ -167,15 +170,13 @@ class Resource:
             resource.hashmap = [None] * resource.total_parts
             resource.hashmap_height = 0
             resource.waiting_for_hmu = False
-
             resource.receiving_part = False
-
-            resource.consecutive_completed_height = 0
+            resource.consecutive_completed_height = -1
             
             if not resource.link.has_incoming_resource(resource):
                 resource.link.register_incoming_resource(resource)
 
-                RNS.log("Accepting resource advertisement for "+RNS.prettyhexrep(resource.hash), RNS.LOG_DEBUG)
+                RNS.log(f"Accepting resource advertisement for {RNS.prettyhexrep(resource.hash)}. Transfer size is {RNS.prettysize(resource.size)} in {resource.total_parts} parts.", RNS.LOG_DEBUG)
                 if resource.link.callbacks.resource_started != None:
                     try:
                         resource.link.callbacks.resource_started(resource)
@@ -204,9 +205,19 @@ class Resource:
         resource_data = None
         self.assembly_lock = False
 
+        if data != None:
+            if not hasattr(data, "read") and len(data) > Resource.MAX_EFFICIENT_SIZE:
+                original_data = data
+                data_size = len(original_data)
+                data = tempfile.TemporaryFile()
+                data.write(original_data)
+                del original_data
+
         if hasattr(data, "read"):
-            data_size = os.stat(data.name).st_size
-            self.total_size  = data_size
+            if data_size == None:
+                data_size = os.stat(data.name).st_size
+
+            self.total_size = data_size
             self.grand_total_parts = math.ceil(data_size/Resource.SDU)
 
             if data_size <= Resource.MAX_EFFICIENT_SIZE:
@@ -279,7 +290,7 @@ class Resource:
             self.uncompressed_data = data
 
             compression_began = time.time()
-            if (auto_compress and len(self.uncompressed_data) < Resource.AUTO_COMPRESS_MAX_SIZE):
+            if (auto_compress and len(self.uncompressed_data) <= Resource.AUTO_COMPRESS_MAX_SIZE):
                 RNS.log("Compressing resource data...", RNS.LOG_DEBUG)
                 self.compressed_data   = bz2.compress(self.uncompressed_data)
                 RNS.log("Compression completed in "+str(round(time.time()-compression_began, 3))+" seconds", RNS.LOG_DEBUG)
@@ -366,7 +377,8 @@ class Resource:
             if advertise:
                 self.advertise()
         else:
-            pass
+            self.receive_lock = Lock()
+            
 
     def hashmap_update_packet(self, plaintext):
         if not self.status == Resource.FAILED:
@@ -440,7 +452,7 @@ class Resource:
             sleep_time = None
 
             if self.status == Resource.ADVERTISED:
-                sleep_time = (self.adv_sent+self.timeout)-time.time()
+                sleep_time = (self.adv_sent+self.timeout+Resource.PROCESSING_GRACE)-time.time()
                 if sleep_time < 0:
                     if self.retries_left <= 0:
                         RNS.log("Resource transfer timeout after sending advertisement", RNS.LOG_DEBUG)
@@ -456,7 +468,7 @@ class Resource:
                             self.adv_sent = self.last_activity
                             sleep_time = 0.001
                         except Exception as e:
-                            RNS.log("Could not resend advertisement packet, cancelling resource", RNS.LOG_VERBOSE)
+                            RNS.log("Could not resend advertisement packet, cancelling resource. The contained exception was: "+str(e), RNS.LOG_VERBOSE)
                             self.cancel()
                     
 
@@ -476,7 +488,8 @@ class Resource:
                     
                     if sleep_time < 0:
                         if self.retries_left > 0:
-                            RNS.log("Timed out waiting for parts, requesting retry", RNS.LOG_DEBUG)
+                            ms = "" if self.outstanding_parts == 1 else "s"
+                            RNS.log("Timed out waiting for "+str(self.outstanding_parts)+" part"+ms+", requesting retry", RNS.LOG_DEBUG)
                             if self.window > self.window_min:
                                 self.window -= 1
                                 if self.window_max > self.window_min:
@@ -611,10 +624,26 @@ class Resource:
                                 self.callback(self)
                             except Exception as e:
                                 RNS.log("Error while executing resource concluded callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+                            finally:
+                                try:
+                                    if hasattr(self, "input_file"):
+                                        if hasattr(self.input_file, "close") and callable(self.input_file.close):
+                                            self.input_file.close()
+
+                                except Exception as e:
+                                    RNS.log("Error while closing resource input file: "+str(e), RNS.LOG_ERROR)
                     else:
                         # Otherwise we'll recursively create the
                         # next segment of the resource
-                        Resource(self.input_file, self.link, callback = self.callback, segment_index = self.segment_index+1, original_hash=self.original_hash, progress_callback = self.__progress_callback)
+                        Resource(
+                            self.input_file, self.link,
+                            callback = self.callback,
+                            segment_index = self.segment_index+1,
+                            original_hash=self.original_hash,
+                            progress_callback = self.__progress_callback,
+                            request_id = self.request_id,
+                            is_response = self.is_response,
+                        )
                 else:
                     pass
             else:
@@ -622,98 +651,99 @@ class Resource:
 
 
     def receive_part(self, packet):
-        while self.receiving_part:
-            sleep(0.001)
+        with self.receive_lock:
 
-        self.receiving_part = True
-        self.last_activity = time.time()
-        self.retries_left = self.max_retries
+            self.receiving_part = True
+            self.last_activity = time.time()
+            self.retries_left = self.max_retries
 
-        if self.req_resp == None:
-            self.req_resp = self.last_activity
-            rtt = self.req_resp-self.req_sent
-            
-            self.part_timeout_factor = Resource.PART_TIMEOUT_FACTOR_AFTER_RTT
-            if self.rtt == None:
-                self.rtt = self.link.rtt
-                self.watchdog_job()
-            elif rtt < self.rtt:
-                self.rtt = max(self.rtt - self.rtt*0.05, rtt)
-            elif rtt > self.rtt:
-                self.rtt = min(self.rtt + self.rtt*0.05, rtt)
+            if self.req_resp == None:
+                self.req_resp = self.last_activity
+                rtt = self.req_resp-self.req_sent
+                
+                self.part_timeout_factor = Resource.PART_TIMEOUT_FACTOR_AFTER_RTT
+                if self.rtt == None:
+                    self.rtt = self.link.rtt
+                    self.watchdog_job()
+                elif rtt < self.rtt:
+                    self.rtt = max(self.rtt - self.rtt*0.05, rtt)
+                elif rtt > self.rtt:
+                    self.rtt = min(self.rtt + self.rtt*0.05, rtt)
 
-            if rtt > 0:
-                req_resp_cost = len(packet.raw)+self.req_sent_bytes
-                self.req_resp_rtt_rate = req_resp_cost / rtt
+                if rtt > 0:
+                    req_resp_cost = len(packet.raw)+self.req_sent_bytes
+                    self.req_resp_rtt_rate = req_resp_cost / rtt
 
-                if self.req_resp_rtt_rate > Resource.RATE_FAST and self.fast_rate_rounds < Resource.FAST_RATE_THRESHOLD:
-                    self.fast_rate_rounds += 1
+                    if self.req_resp_rtt_rate > Resource.RATE_FAST and self.fast_rate_rounds < Resource.FAST_RATE_THRESHOLD:
+                        self.fast_rate_rounds += 1
 
-                    if self.fast_rate_rounds == Resource.FAST_RATE_THRESHOLD:
-                        self.window_max = Resource.WINDOW_MAX_FAST
+                        if self.fast_rate_rounds == Resource.FAST_RATE_THRESHOLD:
+                            self.window_max = Resource.WINDOW_MAX_FAST
 
-        if not self.status == Resource.FAILED:
-            self.status = Resource.TRANSFERRING
-            part_data = packet.data
-            part_hash = self.get_map_hash(part_data)
+            if not self.status == Resource.FAILED:
+                self.status = Resource.TRANSFERRING
+                part_data = packet.data
+                part_hash = self.get_map_hash(part_data)
 
-            i = self.consecutive_completed_height
-            for map_hash in self.hashmap[self.consecutive_completed_height:self.consecutive_completed_height+self.window]:
-                if map_hash == part_hash:
-                    if self.parts[i] == None:
-                        # Insert data into parts list
-                        self.parts[i] = part_data
-                        self.rtt_rxd_bytes += len(part_data)
-                        self.received_count += 1
-                        self.outstanding_parts -= 1
+                consecutive_index = self.consecutive_completed_height if self.consecutive_completed_height >= 0 else 0
+                i = consecutive_index
+                for map_hash in self.hashmap[consecutive_index:consecutive_index+self.window]:
+                    if map_hash == part_hash:
+                        if self.parts[i] == None:
 
-                        # Update consecutive completed pointer
-                        if i == self.consecutive_completed_height + 1:
-                            self.consecutive_completed_height = i
-                        
-                        cp = self.consecutive_completed_height + 1
-                        while cp < len(self.parts) and self.parts[cp] != None:
-                            self.consecutive_completed_height = cp
-                            cp += 1
+                            # Insert data into parts list
+                            self.parts[i] = part_data
+                            self.rtt_rxd_bytes += len(part_data)
+                            self.received_count += 1
+                            self.outstanding_parts -= 1
 
-                        if self.__progress_callback != None:
-                            try:
-                                self.__progress_callback(self)
-                            except Exception as e:
-                                RNS.log("Error while executing progress callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+                            # Update consecutive completed pointer
+                            if i == self.consecutive_completed_height + 1:
+                                self.consecutive_completed_height = i
+                            
+                            cp = self.consecutive_completed_height + 1
+                            while cp < len(self.parts) and self.parts[cp] != None:
+                                self.consecutive_completed_height = cp
+                                cp += 1
 
-                i += 1
+                            if self.__progress_callback != None:
+                                try:
+                                    self.__progress_callback(self)
+                                except Exception as e:
+                                    RNS.log("Error while executing progress callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
 
-            self.receiving_part = False
+                    i += 1
 
-            if self.received_count == self.total_parts and not self.assembly_lock:
-                self.assembly_lock = True
-                self.assemble()
-            elif self.outstanding_parts == 0:
-                # TODO: Figure out if there is a mathematically
-                # optimal way to adjust windows
-                if self.window < self.window_max:
-                    self.window += 1
-                    if (self.window - self.window_min) > (self.window_flexibility-1):
-                        self.window_min += 1
+                self.receiving_part = False
 
-                if self.req_sent != 0:
-                    rtt = time.time()-self.req_sent
-                    req_transferred = self.rtt_rxd_bytes - self.rtt_rxd_bytes_at_part_req
+                if self.received_count == self.total_parts and not self.assembly_lock:
+                    self.assembly_lock = True
+                    self.assemble()
+                elif self.outstanding_parts == 0:
+                    # TODO: Figure out if there is a mathematically
+                    # optimal way to adjust windows
+                    if self.window < self.window_max:
+                        self.window += 1
+                        if (self.window - self.window_min) > (self.window_flexibility-1):
+                            self.window_min += 1
 
-                    if rtt != 0:
-                        self.req_data_rtt_rate = req_transferred/rtt
-                        self.rtt_rxd_bytes_at_part_req = self.rtt_rxd_bytes
+                    if self.req_sent != 0:
+                        rtt = time.time()-self.req_sent
+                        req_transferred = self.rtt_rxd_bytes - self.rtt_rxd_bytes_at_part_req
 
-                        if self.req_data_rtt_rate > Resource.RATE_FAST and self.fast_rate_rounds < Resource.FAST_RATE_THRESHOLD:
-                            self.fast_rate_rounds += 1
+                        if rtt != 0:
+                            self.req_data_rtt_rate = req_transferred/rtt
+                            self.rtt_rxd_bytes_at_part_req = self.rtt_rxd_bytes
 
-                            if self.fast_rate_rounds == Resource.FAST_RATE_THRESHOLD:
-                                self.window_max = Resource.WINDOW_MAX_FAST
+                            if self.req_data_rtt_rate > Resource.RATE_FAST and self.fast_rate_rounds < Resource.FAST_RATE_THRESHOLD:
+                                self.fast_rate_rounds += 1
 
-                self.request_next()
-        else:
-            self.receiving_part = False
+                                if self.fast_rate_rounds == Resource.FAST_RATE_THRESHOLD:
+                                    self.window_max = Resource.WINDOW_MAX_FAST
+
+                    self.request_next()
+            else:
+                self.receiving_part = False
 
     # Called on incoming resource to send a request for more data
     def request_next(self):
@@ -726,11 +756,11 @@ class Resource:
                 hashmap_exhausted = Resource.HASHMAP_IS_NOT_EXHAUSTED
                 requested_hashes = b""
 
-                offset = (1 if self.consecutive_completed_height > 0 else 0)
-                i = 0; pn = self.consecutive_completed_height+offset
+                i = 0; pn = self.consecutive_completed_height+1
                 search_start = pn
-
-                for part in self.parts[search_start:search_start+self.window]:
+                search_size = self.window
+                
+                for part in self.parts[search_start:search_start+search_size]:
                     if part == None:
                         part_hash = self.hashmap[pn]
                         if part_hash != None:
@@ -750,7 +780,6 @@ class Resource:
                     hmu_part += last_map_hash
                     self.waiting_for_hmu = True
 
-                requested_data = b""
                 request_data = hmu_part + self.hash + requested_hashes
                 request_packet = RNS.Packet(self.link, request_data, context = RNS.Packet.RESOURCE_REQ)
 
@@ -760,6 +789,7 @@ class Resource:
                     self.req_sent = self.last_activity
                     self.req_sent_bytes = len(request_packet.raw)
                     self.req_resp = None
+
                 except Exception as e:
                     RNS.log("Could not send resource request packet, cancelling resource", RNS.LOG_DEBUG)
                     RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG)
@@ -905,8 +935,7 @@ class Resource:
             else:
                 self.progress_total_parts = float(self.total_parts)
 
-        
-        progress = self.processed_parts / self.progress_total_parts
+        progress = min(1.0, self.processed_parts / self.progress_total_parts)
         return progress
 
     def get_transfer_size(self):
@@ -994,6 +1023,7 @@ class ResourceAdvertisement:
 
 
     def __init__(self, resource=None, request_id=None, is_response=False):
+        self.link = None
         if resource != None:
             self.t = resource.size              # Transfer size
             self.d = resource.total_size        # Total uncompressed data size
@@ -1040,6 +1070,9 @@ class ResourceAdvertisement:
     def is_compressed(self):
         return self.c
 
+    def get_link(self):
+        return self.link
+
     def pack(self, segment=0):
         hashmap_start = segment*ResourceAdvertisement.HASHMAP_MAX_LEN
         hashmap_end   = min((segment+1)*(ResourceAdvertisement.HASHMAP_MAX_LEN), self.n)

RNS/Reticulum.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -79,7 +79,7 @@ class Reticulum:
     MTU            = 500
     """
     The MTU that Reticulum adheres to, and will expect other peers to
-    adhere to. By default, the MTU is 507 bytes. In custom RNS network
+    adhere to. By default, the MTU is 500 bytes. In custom RNS network
     implementations, it is possible to change this value, but doing so will
     completely break compatibility with all other RNS networks. An identical
     MTU is a prerequisite for peers to communicate in the same network.
@@ -106,17 +106,19 @@ class Reticulum:
     it will eventually be dropped.
 
     This value will be applied by default to all created interfaces,
-    but it can be configured individually on a per-interface basis.
+    but it can be configured individually on a per-interface basis. In
+    general, the global default setting should not be changed, and any
+    alterations should be made on a per-interface basis instead.
     """
 
-    MINIMUM_BITRATE = 500
+    MINIMUM_BITRATE = 5
+    """
+    Minimum bitrate required across a medium for Reticulum to be able
+    to successfully establish links. Currently 5 bits per second.
+    """
 
-    # TODO: To reach the 300bps level without unreasonably impacting
-    # performance on faster links, we need a mechanism for setting
-    # this value more intelligently. One option could be inferring it
-    # from interface speed, but a better general approach would most
-    # probably be to let Reticulum somehow continously build a map of
-    # per-hop latencies and use this map for the timeout calculation. 
+    # TODO: Let Reticulum somehow continously build a map of per-hop
+    # latencies and use this map for global timeout calculation.
     DEFAULT_PER_HOP_TIMEOUT = 6
 
     # Length of truncated hashes in bits.
@@ -133,6 +135,7 @@ class Reticulum:
     JOB_INTERVAL     = 5*60
     CLEAN_INTERVAL   = 15*60
     PERSIST_INTERVAL = 60*60*12
+    GRACIOUS_PERSIST_INTERVAL = 60*5
 
     router           = None
     config           = None
@@ -144,6 +147,8 @@ class Reticulum:
     configpath       = ""
     storagepath      = ""
     cachepath        = ""
+
+    __instance       = None
     
     @staticmethod
     def exit_handler():
@@ -167,6 +172,13 @@ class Reticulum:
         RNS.exit()
 
 
+    @staticmethod
+    def get_instance():
+        """
+        Return the currently running Reticulum instance
+        """
+        return Reticulum.__instance
+
     def __init__(self,configdir=None, loglevel=None, logdest=None, verbosity=None):
         """
         Initialises and starts a Reticulum instance. This must be
@@ -176,6 +188,11 @@ class Reticulum:
         :param configdir: Full path to a Reticulum configuration directory.
         """
 
+        if Reticulum.__instance != None:
+            raise OSError("Attempt to reinitialise Reticulum, when it was already running")
+        else:
+            Reticulum.__instance = self
+
         RNS.vendor.platformutils.platform_checks()
 
         if configdir != None:
@@ -200,6 +217,7 @@ class Reticulum:
 
         Reticulum.__transport_enabled = False
         Reticulum.__use_implicit_proof = True
+        Reticulum.__allow_probes = False
 
         Reticulum.panic_on_interface_error = False
 
@@ -207,6 +225,7 @@ class Reticulum:
         self.local_control_port   = 37429
         self.share_instance       = True
         self.rpc_listener         = None
+        self.rpc_key              = None
 
         self.ifac_salt = Reticulum.IFAC_SALT
 
@@ -260,7 +279,8 @@ class Reticulum:
         RNS.Transport.start(self)
 
         self.rpc_addr = ("127.0.0.1", self.local_control_port)
-        self.rpc_key  = RNS.Identity.full_hash(RNS.Transport.identity.get_private_key())
+        if self.rpc_key == None:
+            self.rpc_key  = RNS.Identity.full_hash(RNS.Transport.identity.get_private_key())
         
         if self.is_shared_instance:
             self.rpc_listener = multiprocessing.connection.Listener(self.rpc_addr, authkey=self.rpc_key)
@@ -288,7 +308,6 @@ class Reticulum:
 
             if now > self.last_data_persist+Reticulum.PERSIST_INTERVAL:
                 self.__persist_data()
-                self.last_data_persist = time.time()
             
             time.sleep(Reticulum.JOB_INTERVAL)
 
@@ -300,6 +319,10 @@ class Reticulum:
                     self.local_interface_port
                 )
                 interface.OUT = True
+                if hasattr(Reticulum, "_force_shared_instance_bitrate"):
+                    interface.bitrate = Reticulum._force_shared_instance_bitrate
+                    interface._force_bitrate = Reticulum._force_shared_instance_bitrate
+                    RNS.log(f"Forcing shared instance bitrate of {RNS.prettyspeed(interface.bitrate)}", RNS.LOG_WARNING)
                 RNS.Transport.interfaces.append(interface)
                 
                 self.is_shared_instance = True
@@ -314,11 +337,16 @@ class Reticulum:
                         self.local_interface_port)
                     interface.target_port = self.local_interface_port
                     interface.OUT = True
+                    if hasattr(Reticulum, "_force_shared_instance_bitrate"):
+                        interface.bitrate = Reticulum._force_shared_instance_bitrate
+                        interface._force_bitrate = True
+                        RNS.log(f"Forcing shared instance bitrate of {RNS.prettyspeed(interface.bitrate)}", RNS.LOG_WARNING)
                     RNS.Transport.interfaces.append(interface)
                     self.is_shared_instance = False
                     self.is_standalone_instance = False
                     self.is_connected_to_shared_instance = True
                     Reticulum.__transport_enabled = False
+                    Reticulum.__allow_probes = False
                     RNS.log("Connected to locally available Reticulum instance via: "+str(interface), RNS.LOG_DEBUG)
                 except Exception as e:
                     RNS.log("Local shared instance appears to be running, but it could not be connected", RNS.LOG_ERROR)
@@ -357,10 +385,24 @@ class Reticulum:
                 if option == "instance_control_port":
                     value = int(self.config["reticulum"][option])
                     self.local_control_port = value
+                if option == "rpc_key":
+                    try:
+                        value = bytes.fromhex(self.config["reticulum"][option])
+                        self.rpc_key = value
+                    except Exception as e:
+                        RNS.log("Invalid shared instance RPC key specified, falling back to default key", RNS.LOG_ERROR)
+                        self.rpc_key = None
                 if option == "enable_transport":
                     v = self.config["reticulum"].as_bool(option)
                     if v == True:
                         Reticulum.__transport_enabled = True
+                if option == "respond_to_probes":
+                    v = self.config["reticulum"].as_bool(option)
+                    if v == True:
+                        Reticulum.__allow_probes = True
+                if option == "force_shared_instance_bitrate":
+                    v = self.config["reticulum"].as_int(option)
+                    Reticulum._force_shared_instance_bitrate = v
                 if option == "panic_on_interface_error":
                     v = self.config["reticulum"].as_bool(option)
                     if v == True:
@@ -441,6 +483,23 @@ class Reticulum:
                             if c["pass_phrase"] != "":
                                 ifac_netkey = c["pass_phrase"]
                                 
+                        ingress_control = True
+                        if "ingress_control" in c: ingress_control = c.as_bool("ingress_control")
+                        ic_max_held_announces = None
+                        if "ic_max_held_announces" in c: ic_max_held_announces = c.as_int("ic_max_held_announces")
+                        ic_burst_hold = None
+                        if "ic_burst_hold" in c: ic_burst_hold = c.as_float("ic_burst_hold")
+                        ic_burst_freq_new = None
+                        if "ic_burst_freq_new" in c: ic_burst_freq_new = c.as_float("ic_burst_freq_new")
+                        ic_burst_freq = None
+                        if "ic_burst_freq" in c: ic_burst_freq = c.as_float("ic_burst_freq")
+                        ic_new_time = None
+                        if "ic_new_time" in c: ic_new_time = c.as_float("ic_new_time")
+                        ic_burst_penalty = None
+                        if "ic_burst_penalty" in c: ic_burst_penalty = c.as_float("ic_burst_penalty")
+                        ic_held_release_interval = None
+                        if "ic_held_release_interval" in c: ic_held_release_interval = c.as_float("ic_held_release_interval")
+
                         configured_bitrate = None
                         if "bitrate" in c:
                             if c.as_int("bitrate") >= Reticulum.MINIMUM_BITRATE:
@@ -477,44 +536,40 @@ class Reticulum:
 
                             if (("interface_enabled" in c) and c.as_bool("interface_enabled") == True) or (("enabled" in c) and c.as_bool("enabled") == True):
                                 if c["type"] == "AutoInterface":
-                                    if not RNS.vendor.platformutils.is_windows():
-                                        group_id        = c["group_id"] if "group_id" in c else None
-                                        discovery_scope = c["discovery_scope"] if "discovery_scope" in c else None
-                                        discovery_port  = int(c["discovery_port"]) if "discovery_port" in c else None
-                                        data_port  = int(c["data_port"]) if "data_port" in c else None
-                                        allowed_interfaces = c.as_list("devices") if "devices" in c else None
-                                        ignored_interfaces = c.as_list("ignored_devices") if "ignored_devices" in c else None
+                                    group_id        = c["group_id"] if "group_id" in c else None
+                                    discovery_scope = c["discovery_scope"] if "discovery_scope" in c else None
+                                    discovery_port  = int(c["discovery_port"]) if "discovery_port" in c else None
+                                    multicast_address_type = c["multicast_address_type"] if "multicast_address_type" in c else None
+                                    data_port  = int(c["data_port"]) if "data_port" in c else None
+                                    allowed_interfaces = c.as_list("devices") if "devices" in c else None
+                                    ignored_interfaces = c.as_list("ignored_devices") if "ignored_devices" in c else None
 
-                                        interface = AutoInterface.AutoInterface(
-                                            RNS.Transport,
-                                            name,
-                                            group_id,
-                                            discovery_scope,
-                                            discovery_port,
-                                            data_port,
-                                            allowed_interfaces,
-                                            ignored_interfaces
-                                        )
-
-                                        if "outgoing" in c and c.as_bool("outgoing") == False:
-                                            interface.OUT = False
-                                        else:
-                                            interface.OUT = True
-
-                                        interface.mode = interface_mode
-
-                                        interface.announce_cap = announce_cap
-                                        if configured_bitrate:
-                                            interface.bitrate = configured_bitrate
-                                        if ifac_size != None:
-                                            interface.ifac_size = ifac_size
-                                        else:
-                                            interface.ifac_size = 16
+                                    interface = AutoInterface.AutoInterface(
+                                        RNS.Transport,
+                                        name,
+                                        group_id,
+                                        discovery_scope,
+                                        discovery_port,
+                                        multicast_address_type,
+                                        data_port,
+                                        allowed_interfaces,
+                                        ignored_interfaces
+                                    )
 
+                                    if "outgoing" in c and c.as_bool("outgoing") == False:
+                                        interface.OUT = False
                                     else:
-                                        RNS.log("AutoInterface is not currently supported on Windows, disabling interface.", RNS.LOG_ERROR);
-                                        RNS.log("Please remove this AutoInterface instance from your configuration file.", RNS.LOG_ERROR);
-                                        RNS.log("You will have to manually configure other interfaces for connectivity.", RNS.LOG_ERROR);
+                                        interface.OUT = True
+
+                                    interface.mode = interface_mode
+
+                                    interface.announce_cap = announce_cap
+                                    if configured_bitrate:
+                                        interface.bitrate = configured_bitrate
+                                    if ifac_size != None:
+                                        interface.ifac_size = ifac_size
+                                    else:
+                                        interface.ifac_size = 16
 
                                 if c["type"] == "UDPInterface":
                                     device       = c["device"] if "device" in c else None
@@ -835,6 +890,8 @@ class Reticulum:
                                     flow_control = c.as_bool("flow_control") if "flow_control" in c else False
                                     id_interval = int(c["id_interval"]) if "id_interval" in c else None
                                     id_callsign = c["id_callsign"] if "id_callsign" in c else None
+                                    st_alock = float(c["airtime_limit_short"]) if "airtime_limit_short" in c else None
+                                    lt_alock = float(c["airtime_limit_long"]) if "airtime_limit_long" in c else None
 
                                     port = c["port"] if "port" in c else None
                                     
@@ -852,7 +909,9 @@ class Reticulum:
                                         cr = codingrate,
                                         flow_control = flow_control,
                                         id_interval = id_interval,
-                                        id_callsign = id_callsign
+                                        id_callsign = id_callsign,
+                                        st_alock = st_alock,
+                                        lt_alock = lt_alock
                                     )
 
                                     if "outgoing" in c and c.as_bool("outgoing") == False:
@@ -874,6 +933,14 @@ class Reticulum:
                                     interface.announce_rate_target = announce_rate_target
                                     interface.announce_rate_grace = announce_rate_grace
                                     interface.announce_rate_penalty = announce_rate_penalty
+                                    interface.ingress_control = ingress_control
+                                    if ic_max_held_announces != None: interface.ic_max_held_announces = ic_max_held_announces
+                                    if ic_burst_hold != None: interface.ic_burst_hold = ic_burst_hold
+                                    if ic_burst_freq_new != None: interface.ic_burst_freq_new = ic_burst_freq_new
+                                    if ic_burst_freq != None: interface.ic_burst_freq = ic_burst_freq
+                                    if ic_new_time != None: interface.ic_new_time = ic_new_time
+                                    if ic_burst_penalty != None: interface.ic_burst_penalty = ic_burst_penalty
+                                    if ic_held_release_interval != None: interface.ic_held_release_interval = ic_held_release_interval
 
                                     interface.ifac_netname = ifac_netname
                                     interface.ifac_netkey = ifac_netkey
@@ -960,11 +1027,13 @@ class Reticulum:
                 RNS.Transport.interfaces.append(interface)
 
     def _should_persist_data(self):
-        self.__persist_data()
+        if time.time() > self.last_data_persist+Reticulum.GRACIOUS_PERSIST_INTERVAL:
+            self.__persist_data()
 
     def __persist_data(self):
         RNS.Transport.persist_data()
         RNS.Identity.persist_data()
+        self.last_data_persist = time.time()
 
     def __clean_caches(self):
         RNS.log("Cleaning resource and packet caches...", RNS.LOG_EXTREME)
@@ -1028,18 +1097,27 @@ class Reticulum:
                     if path == "next_hop":
                         rpc_connection.send(self.get_next_hop(call["destination_hash"]))
 
+                    if path == "first_hop_timeout":
+                        rpc_connection.send(self.get_first_hop_timeout(call["destination_hash"]))
+
                     if path == "packet_rssi":
                         rpc_connection.send(self.get_packet_rssi(call["packet_hash"]))
 
                     if path == "packet_snr":
                         rpc_connection.send(self.get_packet_snr(call["packet_hash"]))
 
+                    if path == "packet_q":
+                        rpc_connection.send(self.get_packet_q(call["packet_hash"]))
+
                 if "drop" in call:
                     path = call["drop"]
 
                     if path == "path":
                         rpc_connection.send(self.drop_path(call["destination_hash"]))
 
+                    if path == "all_via":
+                        rpc_connection.send(self.drop_all_via(call["destination_hash"]))
+
                     if path == "announce_queues":
                         rpc_connection.send(self.drop_announce_queues())
 
@@ -1089,6 +1167,18 @@ class Reticulum:
                     else:
                         ifstats["tunnelstate"] = None
 
+                if hasattr(interface, "r_airtime_short"):
+                    ifstats["airtime_short"] = interface.r_airtime_short
+
+                if hasattr(interface, "r_airtime_long"):
+                    ifstats["airtime_long"] = interface.r_airtime_long
+
+                if hasattr(interface, "r_channel_load_short"):
+                    ifstats["channel_load_short"] = interface.r_channel_load_short
+
+                if hasattr(interface, "r_channel_load_long"):
+                    ifstats["channel_load_long"] = interface.r_channel_load_long
+
                 if hasattr(interface, "bitrate"):
                     if interface.bitrate != None:
                         ifstats["bitrate"] = interface.bitrate
@@ -1119,6 +1209,9 @@ class Reticulum:
                 ifstats["name"] = str(interface)
                 ifstats["rxb"] = interface.rxb
                 ifstats["txb"] = interface.txb
+                ifstats["incoming_announce_frequency"] = interface.incoming_announce_frequency()
+                ifstats["outgoing_announce_frequency"] = interface.outgoing_announce_frequency()
+                ifstats["held_announces"] = len(interface.held_announces)
                 ifstats["status"] = interface.online
                 ifstats["mode"] = interface.mode
 
@@ -1128,6 +1221,11 @@ class Reticulum:
             stats["interfaces"] = interfaces
             if Reticulum.transport_enabled():
                 stats["transport_id"] = RNS.Transport.identity.hash
+                stats["transport_uptime"] = time.time()-RNS.Transport.start_time
+                if Reticulum.probe_destination_enabled():
+                    stats["probe_responder"] = RNS.Transport.probe_destination.hash
+                else:
+                    stats["probe_responder"] = None
 
             return stats
 
@@ -1184,6 +1282,22 @@ class Reticulum:
         else:
             return RNS.Transport.expire_path(destination)
 
+    def drop_all_via(self, transport_hash):
+        if self.is_connected_to_shared_instance:
+            rpc_connection = multiprocessing.connection.Client(self.rpc_addr, authkey=self.rpc_key)
+            rpc_connection.send({"drop": "all_via", "destination_hash": transport_hash})
+            response = rpc_connection.recv()
+            return response
+
+        else:
+            dropped_count = 0
+            for destination_hash in RNS.Transport.destination_table:
+                if RNS.Transport.destination_table[destination_hash][1] == transport_hash:
+                    RNS.Transport.expire_path(destination_hash)
+                    dropped_count += 1
+
+            return dropped_count
+
     def drop_announce_queues(self):
         if self.is_connected_to_shared_instance:
             rpc_connection = multiprocessing.connection.Client(self.rpc_addr, authkey=self.rpc_key)
@@ -1204,6 +1318,26 @@ class Reticulum:
         else:
             return str(RNS.Transport.next_hop_interface(destination))
 
+    def get_first_hop_timeout(self, destination):
+        if self.is_connected_to_shared_instance:
+            try:
+                rpc_connection = multiprocessing.connection.Client(self.rpc_addr, authkey=self.rpc_key)
+                rpc_connection.send({"get": "first_hop_timeout", "destination_hash": destination})
+                response = rpc_connection.recv()
+
+                if self.is_connected_to_shared_instance and hasattr(self, "_force_shared_instance_bitrate") and self._force_shared_instance_bitrate:
+                    simulated_latency = ((1/self._force_shared_instance_bitrate)*8)*RNS.Reticulum.MTU
+                    RNS.log("Adding simulated latency of "+RNS.prettytime(simulated_latency)+" to first hop timeout", RNS.LOG_DEBUG)
+                    response += simulated_latency
+
+                return response
+            except Exception as e:
+                RNS.log("An error occurred while getting first hop timeout from shared instance: "+str(e), RNS.LOG_ERROR)
+                return RNS.Reticulum.DEFAULT_PER_HOP_TIMEOUT
+
+        else:
+            return RNS.Transport.first_hop_timeout(destination)
+
     def get_next_hop(self, destination):
         if self.is_connected_to_shared_instance:
             rpc_connection = multiprocessing.connection.Client(self.rpc_addr, authkey=self.rpc_key)
@@ -1243,6 +1377,21 @@ class Reticulum:
             return None
 
 
+    def get_packet_q(self, packet_hash):
+        if self.is_connected_to_shared_instance:
+            rpc_connection = multiprocessing.connection.Client(self.rpc_addr, authkey=self.rpc_key)
+            rpc_connection.send({"get": "packet_q", "packet_hash": packet_hash})
+            response = rpc_connection.recv()
+            return response
+
+        else:
+            for entry in RNS.Transport.local_client_q_cache:
+                if entry[0] == packet_hash:
+                    return entry[1]
+
+            return None
+
+
     @staticmethod
     def should_use_implicit_proof():
         """
@@ -1266,6 +1415,10 @@ class Reticulum:
         """
         return Reticulum.__transport_enabled
 
+    @staticmethod
+    def probe_destination_enabled():
+        return Reticulum.__allow_probes
+
 # Default configuration file:
 __default_rns_config__ = '''# This is the default Reticulum config file.
 # You should probably edit it to include any additional,

RNS/Transport.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -64,15 +64,21 @@ class Transport:
     LOCAL_REBROADCASTS_MAX = 2          # How many local rebroadcasts of an announce is allowed
 
     PATH_REQUEST_TIMEOUT = 15           # Default timuout for client path requests in seconds
-    PATH_REQUEST_GRACE   = 0.35         # Grace time before a path announcement is made, allows directly reachable peers to respond first
-    PATH_REQUEST_RW      = 2            # Path request random window
-    PATH_REQUEST_MI      = 5            # Minimum interval in seconds for automated path requests
+    PATH_REQUEST_GRACE   = 0.4          # Grace time before a path announcement is made, allows directly reachable peers to respond first
+    PATH_REQUEST_RG      = 0.6          # Extra grace time for roaming-mode interfaces to allow more suitable peers to respond first
+    PATH_REQUEST_MI      = 20           # Minimum interval in seconds for automated path requests
+
+    STATE_UNKNOWN        = 0x00
+    STATE_UNRESPONSIVE   = 0x01
+    STATE_RESPONSIVE     = 0x02
 
     LINK_TIMEOUT         = RNS.Link.STALE_TIME * 1.25
     REVERSE_TIMEOUT      = 30*60        # Reverse table entries are removed after 30 minutes
     DESTINATION_TIMEOUT  = 60*60*24*7   # Destination table entries are removed if unused for one week
     MAX_RECEIPTS         = 1024         # Maximum number of receipts to keep track of
     MAX_RATE_TIMESTAMPS  = 16           # Maximum number of announce timestamps to keep per destination
+    PERSIST_RANDOM_BLOBS = 32           # Maximum number of random blobs per destination to persist to disk
+    MAX_RANDOM_BLOBS     = 64           # Maximum number of random blobs per destination to keep in memory
 
     interfaces           = []           # All active interfaces
     destinations         = []           # All active destinations
@@ -94,6 +100,7 @@ class Transport:
     tunnels              = {}           # A table storing tunnels to other transport instances
     announce_rate_table  = {}           # A table for keeping track of announce rates
     path_requests        = {}           # A table for storing path request timestamps
+    path_states          = {}           # A table for keeping track of path states
     
     discovery_path_requests  = {}       # A table for keeping track of path requests on behalf of other nodes
     discovery_pr_tags        = []       # A table for keeping track of tagged path requests
@@ -111,13 +118,15 @@ class Transport:
 
     local_client_rssi_cache    = []
     local_client_snr_cache     = []
+    local_client_q_cache       = []
     LOCAL_CLIENT_CACHE_MAXSIZE = 512
 
     pending_local_path_requests = {}
 
-    jobs_locked = False
-    jobs_running = False
-    job_interval = 0.250
+    start_time               = None
+    jobs_locked              = False
+    jobs_running             = False
+    job_interval             = 0.250
     links_last_checked       = 0.0
     links_check_interval     = 1.0
     receipts_last_checked    = 0.0
@@ -127,6 +136,8 @@ class Transport:
     hashlist_maxsize         = 1000000
     tables_last_culled       = 0.0
     tables_cull_interval     = 5.0
+    interface_last_jobs      = 0.0
+    interface_jobs_interval  = 5.0
 
     identity = None
 
@@ -269,7 +280,17 @@ class Transport:
                 except Exception as e:
                     RNS.log("Could not load tunnel table from storage, the contained exception was: "+str(e), RNS.LOG_ERROR)
 
+            if RNS.Reticulum.probe_destination_enabled():
+                Transport.probe_destination = RNS.Destination(Transport.identity, RNS.Destination.IN, RNS.Destination.SINGLE, Transport.APP_NAME, "probe")
+                Transport.probe_destination.accepts_links(False)
+                Transport.probe_destination.set_proof_strategy(RNS.Destination.PROVE_ALL)
+                Transport.probe_destination.announce()
+                RNS.log("Transport Instance will respond to probe requests on "+str(Transport.probe_destination), RNS.LOG_NOTICE)
+            else:
+                Transport.probe_destination = None
+
             RNS.log("Transport instance "+str(Transport.identity)+" started", RNS.LOG_VERBOSE)
+            Transport.start_time = time.time()
 
         # Synthesize tunnels for any interfaces wanting it
         for interface in Transport.interfaces:
@@ -286,7 +307,8 @@ class Transport:
     @staticmethod
     def jobs():
         outgoing = []
-        path_requests = []
+        path_requests = {}
+        blocked_if = None
         Transport.jobs_running = True
 
         try:
@@ -314,7 +336,8 @@ class Transport:
                                     if time.time() - last_path_request > Transport.PATH_REQUEST_MI:
                                         RNS.log("Trying to rediscover path for "+RNS.prettyhexrep(link.destination.hash)+" since an attempted link was never established", RNS.LOG_DEBUG)
                                         if not link.destination.hash in path_requests:
-                                            path_requests.append(link.destination.hash)
+                                            blocked_if = None
+                                            path_requests[link.destination.hash] = blocked_if
 
                             Transport.pending_links.remove(link)
 
@@ -334,18 +357,19 @@ class Transport:
                     for receipt in Transport.receipts:
                         receipt.check_timeout()
                         if receipt.status != RNS.PacketReceipt.SENT:
-                            Transport.receipts.remove(receipt)
+                            if receipt in Transport.receipts:
+                                Transport.receipts.remove(receipt)
 
                     Transport.receipts_last_checked = time.time()
 
                 # Process announces needing retransmission
                 if time.time() > Transport.announces_last_checked+Transport.announces_check_interval:
+                    completed_announces = []
                     for destination_hash in Transport.announce_table:
                         announce_entry = Transport.announce_table[destination_hash]
                         if announce_entry[2] > Transport.PATHFINDER_R:
                             RNS.log("Completed announce processing for "+RNS.prettyhexrep(destination_hash)+", retry limit reached", RNS.LOG_EXTREME)
-                            Transport.announce_table.pop(destination_hash)
-                            break
+                            completed_announces.append(destination_hash)
                         else:
                             if time.time() > announce_entry[1]:
                                 announce_entry[1] = time.time() + Transport.PATHFINDER_G + Transport.PATHFINDER_RW
@@ -392,6 +416,10 @@ class Transport:
                                     Transport.announce_table[destination_hash] = held_entry
                                     RNS.log("Reinserting held announce into table", RNS.LOG_DEBUG)
 
+                    for destination_hash in completed_announces:
+                        if destination_hash in Transport.announce_table:
+                            Transport.announce_table.pop(destination_hash)
+
                     Transport.announces_last_checked = time.time()
 
 
@@ -404,6 +432,12 @@ class Transport:
                     Transport.discovery_pr_tags = Transport.discovery_pr_tags[len(Transport.discovery_pr_tags)-Transport.max_pr_tags:len(Transport.discovery_pr_tags)-1]
 
                 if time.time() > Transport.tables_last_culled + Transport.tables_cull_interval:
+                    # Remove unneeded path state entries
+                    stale_path_states = []
+                    for destination_hash in Transport.path_states:
+                        if not destination_hash in Transport.destination_table:
+                            stale_path_states.append(destination_hash)
+
                     # Cull the reverse table according to timeout
                     stale_reverse_entries = []
                     for truncated_packet_hash in Transport.reverse_table:
@@ -427,14 +461,49 @@ class Transport:
                                 if link_entry[6] in Transport.path_requests:
                                     last_path_request = Transport.path_requests[link_entry[6]]
 
+                                lr_taken_hops = link_entry[5]
+
+                                path_request_throttle = time.time() - last_path_request < Transport.PATH_REQUEST_MI
+                                path_request_conditions = False
+                                
+                                # If the path has been invalidated between the time of
+                                # making the link request and now, try to rediscover it
+                                if not Transport.has_path(link_entry[6]):
+                                    RNS.log("Trying to rediscover path for "+RNS.prettyhexrep(link_entry[6])+" since an attempted link was never established, and path is now missing", RNS.LOG_DEBUG)
+                                    path_request_conditions =True
+
                                 # If this link request was originated from a local client
                                 # attempt to rediscover a path to the destination, if this
                                 # has not already happened recently.
-                                lr_taken_hops = link_entry[5]
-                                if lr_taken_hops == 0 and time.time() - last_path_request > Transport.PATH_REQUEST_MI:
+                                elif not path_request_throttle and lr_taken_hops == 0:
                                     RNS.log("Trying to rediscover path for "+RNS.prettyhexrep(link_entry[6])+" since an attempted local client link was never established", RNS.LOG_DEBUG)
+                                    path_request_conditions = True
+
+                                # If the link destination was previously only 1 hop
+                                # away, this likely means that it was local to one
+                                # of our interfaces, and that it roamed somewhere else.
+                                # In that case, try to discover a new path.
+                                elif not path_request_throttle and Transport.hops_to(link_entry[6]) == 1:
+                                    RNS.log("Trying to rediscover path for "+RNS.prettyhexrep(link_entry[6])+" since an attempted link was never established, and destination was previously local to an interface on this instance", RNS.LOG_DEBUG)
+                                    path_request_conditions = True
+                                    blocked_if = link_entry[4]
+
+                                # If the link initiator is only 1 hop away,
+                                # this likely means that network topology has
+                                # changed. In that case, we try to discover a new path,
+                                # and mark the old one as potentially unresponsive.
+                                elif not path_request_throttle and lr_taken_hops == 1:
+                                    RNS.log("Trying to rediscover path for "+RNS.prettyhexrep(link_entry[6])+" since an attempted link was never established, and link initiator is local to an interface on this instance", RNS.LOG_DEBUG)
+                                    path_request_conditions = True
+                                    blocked_if = link_entry[4]
+
+                                    if RNS.Reticulum.transport_enabled():
+                                        if hasattr(link_entry[4], "mode") and link_entry[4].mode != RNS.Interfaces.Interface.Interface.MODE_BOUNDARY:
+                                            Transport.mark_path_unresponsive(link_entry[6])
+
+                                if path_request_conditions:
                                     if not link_entry[6] in path_requests:
-                                        path_requests.append(link_entry[6])
+                                        path_requests[link_entry[6]] = blocked_if
 
                                     if not RNS.Reticulum.transport_enabled():
                                         # Drop current path if we are not a transport instance, to
@@ -502,8 +571,6 @@ class Transport:
                         else:
                             RNS.log("Removed "+str(ti)+" tunnel paths", RNS.LOG_EXTREME)
 
-
-                    
                     i = 0
                     for truncated_packet_hash in stale_reverse_entries:
                         Transport.reverse_table.pop(truncated_packet_hash)
@@ -515,8 +582,6 @@ class Transport:
                         else:
                             RNS.log("Released "+str(i)+" reverse table entries", RNS.LOG_EXTREME)
 
-                    
-
                     i = 0
                     for link_id in stale_links:
                         Transport.link_table.pop(link_id)
@@ -561,8 +626,24 @@ class Transport:
                         else:
                             RNS.log("Removed "+str(i)+" tunnels", RNS.LOG_EXTREME)
 
+                    i = 0
+                    for destination_hash in stale_path_states:
+                        Transport.path_states.pop(destination_hash)
+                        i += 1
+
+                    if i > 0:
+                        if i == 1:
+                            RNS.log("Removed "+str(i)+" path state entry", RNS.LOG_EXTREME)
+                        else:
+                            RNS.log("Removed "+str(i)+" path state entries", RNS.LOG_EXTREME)
+
                     Transport.tables_last_culled = time.time()
 
+                if time.time() > Transport.interface_last_jobs + Transport.interface_jobs_interval:
+                    for interface in Transport.interfaces:
+                        interface.process_held_announces()
+                    Transport.interface_last_jobs = time.time()
+
             else:
                 # Transport jobs were locked, do nothing
                 pass
@@ -577,7 +658,17 @@ class Transport:
             packet.send()
 
         for destination_hash in path_requests:
-            Transport.request_path(destination_hash)
+            blocked_if = path_requests[destination_hash]
+            if blocked_if == None:
+                Transport.request_path(destination_hash)
+            else:
+                for interface in Transport.interfaces:
+                    if interface != blocked_if:
+                        # RNS.log("Transmitting path request on "+str(interface), RNS.LOG_DEBUG)
+                        Transport.request_path(destination_hash, on_interface=interface)
+                    else:
+                        pass
+                        # RNS.log("Blocking path request on "+str(interface), RNS.LOG_DEBUG)
 
     @staticmethod
     def transmit(interface, raw):
@@ -634,6 +725,30 @@ class Transport:
         sent = False
         outbound_time = time.time()
 
+        generate_receipt = False
+        if (packet.create_receipt == True and
+            # Only generate receipts for DATA packets
+            packet.packet_type == RNS.Packet.DATA and
+            # Don't generate receipts for PLAIN destinations
+            packet.destination.type != RNS.Destination.PLAIN and
+            # Don't generate receipts for link-related packets
+            not (packet.context >= RNS.Packet.KEEPALIVE and packet.context <= RNS.Packet.LRPROOF) and
+            # Don't generate receipts for resource packets
+            not (packet.context >= RNS.Packet.RESOURCE and packet.context <= RNS.Packet.RESOURCE_RCL)):
+
+            generate_receipt = True
+
+        def packet_sent(packet):
+            packet.sent = True
+            packet.sent_at = time.time()
+
+            if generate_receipt:
+                packet.receipt = RNS.PacketReceipt(packet)
+                Transport.receipts.append(packet.receipt)
+            
+            # TODO: Enable when caching has been redesigned
+            # Transport.cache(packet)
+
         # Check if we have a known path for the destination in the path table
         if packet.packet_type != RNS.Packet.ANNOUNCE and packet.destination.type != RNS.Destination.PLAIN and packet.destination.type != RNS.Destination.GROUP and packet.destination_hash in Transport.destination_table:
             outbound_interface = Transport.destination_table[packet.destination_hash][5]
@@ -651,6 +766,7 @@ class Transport:
                     new_raw += packet.raw[1:2]
                     new_raw += Transport.destination_table[packet.destination_hash][1]
                     new_raw += packet.raw[2:]
+                    packet_sent(packet)
                     Transport.transmit(outbound_interface, new_raw)
                     Transport.destination_table[packet.destination_hash][0] = time.time()
                     sent = True
@@ -670,6 +786,7 @@ class Transport:
                     new_raw += packet.raw[1:2]
                     new_raw += Transport.destination_table[packet.destination_hash][1]
                     new_raw += packet.raw[2:]
+                    packet_sent(packet)
                     Transport.transmit(outbound_interface, new_raw)
                     Transport.destination_table[packet.destination_hash][0] = time.time()
                     sent = True
@@ -678,6 +795,7 @@ class Transport:
             # directly reachable, and also on which interface, so we
             # simply transmit the packet directly on that one.
             else:
+                packet_sent(packet)
                 Transport.transmit(outbound_interface, packet.raw)
                 sent = True
 
@@ -761,7 +879,7 @@ class Transport:
                                             interface.announce_queue = []
 
                                     queued_announces = True if len(interface.announce_queue) > 0 else False
-                                    if not queued_announces and outbound_time > interface.announce_allowed_at:
+                                    if not queued_announces and outbound_time > interface.announce_allowed_at and interface.bitrate != None and interface.bitrate != 0:
                                         tx_time   = (len(packet.raw)*8) / interface.bitrate
                                         wait_time = (tx_time / interface.announce_cap)
                                         interface.announce_allowed_at = outbound_time + wait_time
@@ -842,28 +960,11 @@ class Transport:
                         # thread.start()
 
                         Transport.transmit(interface, packet.raw)
+                        if packet.packet_type == RNS.Packet.ANNOUNCE:
+                            interface.sent_announce()
+                        packet_sent(packet)
                         sent = True
 
-        if sent:
-            packet.sent = True
-            packet.sent_at = time.time()
-
-            # Don't generate receipt if it has been explicitly disabled
-            if (packet.create_receipt == True and
-                # Only generate receipts for DATA packets
-                packet.packet_type == RNS.Packet.DATA and
-                # Don't generate receipts for PLAIN destinations
-                packet.destination.type != RNS.Destination.PLAIN and
-                # Don't generate receipts for link-related packets
-                not (packet.context >= RNS.Packet.KEEPALIVE and packet.context <= RNS.Packet.LRPROOF) and
-                # Don't generate receipts for resource packets
-                not (packet.context >= RNS.Packet.RESOURCE and packet.context <= RNS.Packet.RESOURCE_RCL)):
-
-                packet.receipt = RNS.PacketReceipt(packet)
-                Transport.receipts.append(packet.receipt)
-            
-            Transport.cache(packet)
-
         Transport.jobs_locked = False
         return sent
 
@@ -872,6 +973,13 @@ class Transport:
         # TODO: Think long and hard about this.
         # Is it even strictly necessary with the current
         # transport rules?
+
+        # Filter packets intended for other transport instances
+        if packet.transport_id != None and packet.packet_type != RNS.Packet.ANNOUNCE:
+            if packet.transport_id != Transport.identity.hash:
+                RNS.log("Ignored packet "+RNS.prettyhexrep(packet.packet_hash)+" in transport for other transport instance", RNS.LOG_EXTREME)
+                return False
+
         if packet.context == RNS.Packet.KEEPALIVE:
             return True
         if packet.context == RNS.Packet.RESOURCE_REQ:
@@ -995,6 +1103,7 @@ class Transport:
         
         packet = RNS.Packet(None, raw)
         if not packet.unpack():
+            Transport.jobs_locked = False
             return
             
         packet.receiving_interface = interface
@@ -1008,7 +1117,7 @@ class Transport:
                         Transport.local_client_rssi_cache.append([packet.packet_hash, packet.rssi])
 
                         while len(Transport.local_client_rssi_cache) > Transport.LOCAL_CLIENT_CACHE_MAXSIZE:
-                            Transport.local_client_rssi_cache.pop()
+                            Transport.local_client_rssi_cache.pop(0)
 
             if hasattr(interface, "r_stat_snr"):
                 if interface.r_stat_rssi != None:
@@ -1017,7 +1126,16 @@ class Transport:
                         Transport.local_client_snr_cache.append([packet.packet_hash, packet.snr])
 
                         while len(Transport.local_client_snr_cache) > Transport.LOCAL_CLIENT_CACHE_MAXSIZE:
-                            Transport.local_client_snr_cache.pop()
+                            Transport.local_client_snr_cache.pop(0)
+
+            if hasattr(interface, "r_stat_q"):
+                if interface.r_stat_q != None:
+                    packet.q = interface.r_stat_q
+                    if len(Transport.local_client_interfaces) > 0:
+                        Transport.local_client_q_cache.append([packet.packet_hash, packet.q])
+
+                        while len(Transport.local_client_q_cache) > Transport.LOCAL_CLIENT_CACHE_MAXSIZE:
+                            Transport.local_client_q_cache.pop(0)
 
         if len(Transport.local_client_interfaces) > 0:
             if Transport.is_local_client_interface(interface):
@@ -1026,10 +1144,31 @@ class Transport:
         elif Transport.interface_to_shared_instance(interface):
             packet.hops -= 1
 
-
         if Transport.packet_filter(packet):
-            Transport.packet_hashlist.append(packet.packet_hash)
-            Transport.cache(packet)
+            # By default, remember packet hashes to avoid routing
+            # loops in the network, using the packet filter.
+            remember_packet_hash = True
+
+            # If this packet belongs to a link in our link table,
+            # we'll have to defer adding it to the filter list.
+            # In some cases, we might see a packet over a shared-
+            # medium interface, belonging to a link that transports
+            # or terminates with this instance, but before it would
+            # normally reach us. If the packet is appended to the
+            # filter list at this point, link transport will break.
+            if packet.destination_hash in Transport.link_table:
+                remember_packet_hash = False
+
+            # If this is a link request proof, don't add it until
+            # we are sure it's not actually somewhere else in the
+            # routing chain.
+            if packet.packet_type == RNS.Packet.PROOF and packet.context == RNS.Packet.LRPROOF:
+                remember_packet_hash = False
+
+            if remember_packet_hash:
+                Transport.packet_hashlist.append(packet.packet_hash)
+                # TODO: Enable when caching has been redesigned
+                # Transport.cache(packet)
             
             # Check special conditions for local clients connected
             # through a shared Reticulum instance
@@ -1076,6 +1215,7 @@ class Transport:
                 # normal processing.
                 if packet.context == RNS.Packet.CACHE_REQUEST:
                     if Transport.cache_request_packet(packet):
+                        Transport.jobs_locked = False
                         return
 
                 # If the packet is in transport, check whether we
@@ -1109,7 +1249,8 @@ class Transport:
 
                             if packet.packet_type == RNS.Packet.LINKREQUEST:
                                 now = time.time()
-                                proof_timeout = now + RNS.Link.ESTABLISHMENT_TIMEOUT_PER_HOP * max(1, remaining_hops)
+                                proof_timeout  = Transport.extra_link_proof_timeout(packet.receiving_interface)
+                                proof_timeout += now + RNS.Link.ESTABLISHMENT_TIMEOUT_PER_HOP * max(1, remaining_hops)
 
                                 # Entry format is
                                 link_entry = [  now,                            # 0: Timestamp,
@@ -1148,7 +1289,7 @@ class Transport:
                         link_entry = Transport.link_table[packet.destination_hash]
                         # If receiving and outbound interface is
                         # the same for this link, direction doesn't
-                        # matter, and we simply send the packet on.
+                        # matter, and we simply repeat the packet.
                         outbound_interface = None
                         if link_entry[2] == link_entry[4]:
                             # But check that taken hops matches one
@@ -1167,8 +1308,13 @@ class Transport:
                                 # Also check that expected hop count matches
                                 if packet.hops == link_entry[5]:
                                     outbound_interface = link_entry[2]
-                            
+
                         if outbound_interface != None:
+                            # Add this packet to the filter hashlist if we
+                            # have determined that it's actually our turn
+                            # to process it.
+                            Transport.packet_hashlist.append(packet.packet_hash)
+
                             new_raw = packet.raw[0:1]
                             new_raw += struct.pack("!B", packet.hops)
                             new_raw += packet.raw[2:]
@@ -1182,8 +1328,21 @@ class Transport:
             # announces, queueing rebroadcasts of these, and removal
             # of queued announce rebroadcasts once handed to the next node.
             if packet.packet_type == RNS.Packet.ANNOUNCE:
+                if interface != None and RNS.Identity.validate_announce(packet, only_validate_signature=True):
+                    interface.received_announce()
+
+                if not packet.destination_hash in Transport.destination_table:
+                    # This is an unknown destination, and we'll apply
+                    # potential ingress limiting. Already known
+                    # destinations will have re-announces controlled
+                    # by normal announce rate limiting.
+                    if interface.should_ingress_limit():
+                        interface.hold_announce(packet)
+                        Transport.jobs_locked = False
+                        return
+
                 local_destination = next((d for d in Transport.destinations if d.hash == packet.destination_hash), None)
-                if local_destination == None and RNS.Identity.validate_announce(packet): 
+                if local_destination == None and RNS.Identity.validate_announce(packet):
                     if packet.transport_id != None:
                         received_from = packet.transport_id
                         
@@ -1198,7 +1357,8 @@ class Transport:
                                 announce_entry[6] += 1
                                 if announce_entry[6] >= Transport.LOCAL_REBROADCASTS_MAX:
                                     RNS.log("Max local rebroadcasts of announce for "+RNS.prettyhexrep(packet.destination_hash)+" reached, dropping announce from our table", RNS.LOG_DEBUG)
-                                    Transport.announce_table.pop(packet.destination_hash)
+                                    if packet.destination_hash in Transport.announce_table:
+                                        Transport.announce_table.pop(packet.destination_hash)
 
                             if packet.hops-1 == announce_entry[4]+1 and announce_entry[2] > 0:
                                 now = time.time()
@@ -1233,6 +1393,7 @@ class Transport:
                                 # TODO: Check whether this approach works
                                 # under all circumstances
                                 if not random_blob in random_blobs:
+                                    Transport.mark_path_unknown_state(packet.destination_hash)
                                     should_add = True
                                 else:
                                     should_add = False
@@ -1250,21 +1411,39 @@ class Transport:
                                     if path_announce_emitted >= announce_emitted:
                                         break
 
+                                # If the path has expired, consider this
+                                # announce for adding to the path table.
                                 if (now >= path_expires):
-                                    # We also check that the announce is
+                                    # We check that the announce is
                                     # different from ones we've already heard,
                                     # to avoid loops in the network
                                     if not random_blob in random_blobs:
                                         # TODO: Check that this ^ approach actually
                                         # works under all circumstances
                                         RNS.log("Replacing destination table entry for "+str(RNS.prettyhexrep(packet.destination_hash))+" with new announce due to expired path", RNS.LOG_DEBUG)
+                                        Transport.mark_path_unknown_state(packet.destination_hash)
                                         should_add = True
                                     else:
                                         should_add = False
                                 else:
+                                    # If the path is not expired, but the emission
+                                    # is more recent, and we haven't already heard
+                                    # this announce before, update the path table.
                                     if (announce_emitted > path_announce_emitted):
                                         if not random_blob in random_blobs:
                                             RNS.log("Replacing destination table entry for "+str(RNS.prettyhexrep(packet.destination_hash))+" with new announce, since it was more recently emitted", RNS.LOG_DEBUG)
+                                            Transport.mark_path_unknown_state(packet.destination_hash)
+                                            should_add = True
+                                        else:
+                                            should_add = False
+                                    
+                                    # If we have already heard this announce before,
+                                    # but the path has been marked as unresponsive
+                                    # by a failed communications attempt or similar,
+                                    # allow updating the path table to this one.
+                                    elif announce_emitted == path_announce_emitted:
+                                        if Transport.path_is_unresponsive(packet.destination_hash):
+                                            RNS.log("Replacing destination table entry for "+str(RNS.prettyhexrep(packet.destination_hash))+" with new announce, since previously tried path was unresponsive", RNS.LOG_DEBUG)
                                             should_add = True
                                         else:
                                             should_add = False
@@ -1275,7 +1454,7 @@ class Transport:
                             should_add = True
 
                         if should_add:
-                            now                = time.time()
+                            now = time.time()
 
                             rate_blocked = False
                             if packet.context != RNS.Packet.PATH_RESPONSE and packet.receiving_interface.announce_rate_target != None:
@@ -1328,6 +1507,7 @@ class Transport:
                                 expires            = now + Transport.PATHFINDER_E
                             
                             random_blobs.append(random_blob)
+                            random_blobs = random_blobs[-Transport.MAX_RANDOM_BLOBS:]
 
                             if (RNS.Reticulum.transport_enabled() or Transport.from_local_client(packet)) and packet.context != RNS.Packet.PATH_RESPONSE:
                                 # Insert announce into announce table for retransmission
@@ -1493,7 +1673,7 @@ class Transport:
                                         RNS.log("Error while processing external announce callback.", RNS.LOG_ERROR)
                                         RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
 
-            # Handling for linkrequests to local destinations
+            # Handling for link requests to local destinations
             elif packet.packet_type == RNS.Packet.LINKREQUEST:
                 if packet.transport_id == None or packet.transport_id == Transport.identity.hash:
                     for destination in Transport.destinations:
@@ -1532,38 +1712,58 @@ class Transport:
                     # needs to be transported
                     if (RNS.Reticulum.transport_enabled() or for_local_client_link or from_local_client) and packet.destination_hash in Transport.link_table:
                         link_entry = Transport.link_table[packet.destination_hash]
-                        if packet.receiving_interface == link_entry[2]:
-                            try:
-                                if len(packet.data) == RNS.Identity.SIGLENGTH//8+RNS.Link.ECPUBSIZE//2:
-                                    peer_pub_bytes = packet.data[RNS.Identity.SIGLENGTH//8:RNS.Identity.SIGLENGTH//8+RNS.Link.ECPUBSIZE//2]
-                                    peer_identity = RNS.Identity.recall(link_entry[6])
-                                    peer_sig_pub_bytes = peer_identity.get_public_key()[RNS.Link.ECPUBSIZE//2:RNS.Link.ECPUBSIZE]
+                        if packet.hops == link_entry[3]:
+                            if packet.receiving_interface == link_entry[2]:
+                                try:
+                                    if len(packet.data) == RNS.Identity.SIGLENGTH//8+RNS.Link.ECPUBSIZE//2:
+                                        peer_pub_bytes = packet.data[RNS.Identity.SIGLENGTH//8:RNS.Identity.SIGLENGTH//8+RNS.Link.ECPUBSIZE//2]
+                                        peer_identity = RNS.Identity.recall(link_entry[6])
+                                        peer_sig_pub_bytes = peer_identity.get_public_key()[RNS.Link.ECPUBSIZE//2:RNS.Link.ECPUBSIZE]
 
-                                    signed_data = packet.destination_hash+peer_pub_bytes+peer_sig_pub_bytes
-                                    signature = packet.data[:RNS.Identity.SIGLENGTH//8]
+                                        signed_data = packet.destination_hash+peer_pub_bytes+peer_sig_pub_bytes
+                                        signature = packet.data[:RNS.Identity.SIGLENGTH//8]
 
-                                    if peer_identity.validate(signature, signed_data):
-                                        RNS.log("Link request proof validated for transport via "+str(link_entry[4]), RNS.LOG_EXTREME)
-                                        new_raw = packet.raw[0:1]
-                                        new_raw += struct.pack("!B", packet.hops)
-                                        new_raw += packet.raw[2:]
-                                        Transport.link_table[packet.destination_hash][7] = True
-                                        Transport.transmit(link_entry[4], new_raw)
+                                        if peer_identity.validate(signature, signed_data):
+                                            RNS.log("Link request proof validated for transport via "+str(link_entry[4]), RNS.LOG_EXTREME)
+                                            new_raw = packet.raw[0:1]
+                                            new_raw += struct.pack("!B", packet.hops)
+                                            new_raw += packet.raw[2:]
+                                            Transport.link_table[packet.destination_hash][7] = True
+                                            Transport.transmit(link_entry[4], new_raw)
 
-                                    else:
-                                        RNS.log("Invalid link request proof in transport for link "+RNS.prettyhexrep(packet.destination_hash)+", dropping proof.", RNS.LOG_DEBUG)
+                                        else:
+                                            RNS.log("Invalid link request proof in transport for link "+RNS.prettyhexrep(packet.destination_hash)+", dropping proof.", RNS.LOG_DEBUG)
 
-                            except Exception as e:
-                                RNS.log("Error while transporting link request proof. The contained exception was: "+str(e), RNS.LOG_ERROR)
+                                except Exception as e:
+                                    RNS.log("Error while transporting link request proof. The contained exception was: "+str(e), RNS.LOG_ERROR)
 
+                            else:
+                                RNS.log("Link request proof received on wrong interface, not transporting it.", RNS.LOG_DEBUG)
                         else:
-                            RNS.log("Link request proof received on wrong interface, not transporting it.", RNS.LOG_DEBUG)
+                            RNS.log("Received link request proof with hop mismatch, not transporting it", RNS.LOG_DEBUG)
                     else:
                         # Check if we can deliver it to a local
                         # pending link
                         for link in Transport.pending_links:
                             if link.link_id == packet.destination_hash:
-                                link.validate_proof(packet)
+                                # We need to also allow an expected hops value of
+                                # PATHFINDER_M, since in some cases, the number of hops
+                                # to the destination will be unknown at link creation
+                                # time. The real chance of this occuring is likely to be
+                                # extremely small, and this allowance could probably
+                                # be discarded without major issues, but it is kept
+                                # for now to ensure backwards compatibility.
+
+                                # TODO: Probably reset check back to
+                                # if packet.hops == link.expected_hops:
+                                # within one of the next releases
+
+                                if packet.hops == link.expected_hops or link.expected_hops == RNS.Transport.PATHFINDER_M:
+                                    # Add this packet to the filter hashlist if we
+                                    # have determined that it's actually destined
+                                    # for this system, and then validate the proof
+                                    Transport.packet_hashlist.append(packet.packet_hash)
+                                    link.validate_proof(packet)
 
                 elif packet.context == RNS.Packet.RESOURCE_PRF:
                     for link in Transport.active_links:
@@ -1580,7 +1780,7 @@ class Transport:
                     else:
                         proof_hash = None
 
-                    # Check if this proof neds to be transported
+                    # Check if this proof needs to be transported
                     if (RNS.Reticulum.transport_enabled() or from_local_client or proof_for_local_client) and packet.destination_hash in Transport.reverse_table:
                         reverse_entry = Transport.reverse_table.pop(packet.destination_hash)
                         if packet.receiving_interface == reverse_entry[1]:
@@ -1599,9 +1799,6 @@ class Transport:
                             if receipt.hash == proof_hash:
                                 receipt_validated = receipt.validate_proof_packet(packet)
                         else:
-                            # TODO: This looks like it should actually
-                            # be rewritten when implicit proofs are added.
-
                             # In case of an implicit proof, we have
                             # to check every single outstanding receipt
                             receipt_validated = receipt.validate_proof_packet(packet)
@@ -1740,6 +1937,8 @@ class Transport:
     def activate_link(link):
         RNS.log("Activating link "+str(link), RNS.LOG_EXTREME)
         if link in Transport.pending_links:
+            if link.status != RNS.Link.ACTIVE:
+                raise IOError("Invalid link state for link activation: "+str(link.status))
             Transport.pending_links.remove(link)
             Transport.active_links.append(link)
             link.status = RNS.Link.ACTIVE
@@ -1777,8 +1976,11 @@ class Transport:
 
     @staticmethod
     def should_cache(packet):
-        if packet.context == RNS.Packet.RESOURCE_PRF:
-            return True
+        # TODO: Rework the caching system. It's currently
+        # not very useful to even cache Resource proofs,
+        # disabling it for now, until redesigned.
+        # if packet.context == RNS.Packet.RESOURCE_PRF:
+        #     return True
 
         return False
 
@@ -1801,8 +2003,7 @@ class Transport:
                 file.close()
 
             except Exception as e:
-                RNS.log("Error writing packet to cache", RNS.LOG_ERROR)
-                RNS.log("The contained exception was: "+str(e))
+                RNS.log("Error writing packet to cache. The contained exception was: "+str(e), RNS.LOG_ERROR)
 
     @staticmethod
     def get_cached_packet(packet_hash):
@@ -1902,6 +2103,45 @@ class Transport:
         else:
             return None
 
+    @staticmethod
+    def next_hop_interface_bitrate(destination_hash):
+        next_hop_interface = Transport.next_hop_interface(destination_hash)
+        if next_hop_interface != None:
+            return next_hop_interface.bitrate
+        else:
+            return None
+
+    @staticmethod
+    def next_hop_per_bit_latency(destination_hash):
+        next_hop_interface_bitrate = Transport.next_hop_interface_bitrate(destination_hash)
+        if next_hop_interface_bitrate != None:
+            return (1/next_hop_interface_bitrate)
+        else:
+            return None
+
+    @staticmethod
+    def next_hop_per_byte_latency(destination_hash):
+        per_bit_latency = Transport.next_hop_per_bit_latency(destination_hash)
+        if per_bit_latency != None:
+            return per_bit_latency*8
+        else:
+            return None
+
+    @staticmethod
+    def first_hop_timeout(destination_hash):
+        latency = Transport.next_hop_per_byte_latency(destination_hash)
+        if latency != None:
+            return RNS.Reticulum.MTU * latency + RNS.Reticulum.DEFAULT_PER_HOP_TIMEOUT
+        else:
+            return RNS.Reticulum.DEFAULT_PER_HOP_TIMEOUT
+
+    @staticmethod
+    def extra_link_proof_timeout(interface):
+        if interface != None:
+            return ((1/interface.bitrate)*8)*RNS.Reticulum.MTU
+        else:
+            return 0
+
     @staticmethod
     def expire_path(destination_hash):
         if destination_hash in Transport.destination_table:
@@ -1911,6 +2151,38 @@ class Transport:
         else:
             return False
 
+    @staticmethod
+    def mark_path_unresponsive(destination_hash):
+        if destination_hash in Transport.destination_table:
+            Transport.path_states[destination_hash] = Transport.STATE_UNRESPONSIVE
+            return True
+        else:
+            return False
+
+    @staticmethod
+    def mark_path_responsive(destination_hash):
+        if destination_hash in Transport.destination_table:
+            Transport.path_states[destination_hash] = Transport.STATE_RESPONSIVE
+            return True
+        else:
+            return False
+
+    @staticmethod
+    def mark_path_unknown_state(destination_hash):
+        if destination_hash in Transport.destination_table:
+            Transport.path_states[destination_hash] = Transport.STATE_UNKNOWN
+            return True
+        else:
+            return False
+
+    @staticmethod
+    def path_is_unresponsive(destination_hash):
+        if destination_hash in Transport.path_states:
+            if Transport.path_states[destination_hash] == Transport.STATE_UNRESPONSIVE:
+                return True
+
+        return False
+
     @staticmethod
     def request_path(destination_hash, on_interface=None, tag=None, recursive=False):
         """
@@ -2043,41 +2315,55 @@ class Transport:
             packet = Transport.destination_table[destination_hash][6]
             next_hop = Transport.destination_table[destination_hash][1]
             received_from = Transport.destination_table[destination_hash][5]
-    
-            if requestor_transport_id != None and next_hop == requestor_transport_id:
-                # TODO: Find a bandwidth efficient way to invalidate our
-                # known path on this signal. The obvious way of signing
-                # path requests with transport instance keys is quite
-                # inefficient. There is probably a better way. Doing
-                # path invalidation here would decrease the network
-                # convergence time. Maybe just drop it?
-                RNS.log("Not answering path request for "+RNS.prettyhexrep(destination_hash)+interface_str+", since next hop is the requestor", RNS.LOG_DEBUG)
+
+            if attached_interface.mode == RNS.Interfaces.Interface.Interface.MODE_ROAMING and attached_interface == received_from:
+                RNS.log("Not answering path request on roaming-mode interface, since next hop is on same roaming-mode interface", RNS.LOG_DEBUG)
+            
             else:
-                RNS.log("Answering path request for "+RNS.prettyhexrep(destination_hash)+interface_str+", path is known", RNS.LOG_DEBUG)
-
-                now = time.time()
-                retries = Transport.PATHFINDER_R
-                local_rebroadcasts = 0
-                block_rebroadcasts = True
-                announce_hops      = packet.hops
-
-                if is_from_local_client:
-                    retransmit_timeout = now
+                if requestor_transport_id != None and next_hop == requestor_transport_id:
+                    # TODO: Find a bandwidth efficient way to invalidate our
+                    # known path on this signal. The obvious way of signing
+                    # path requests with transport instance keys is quite
+                    # inefficient. There is probably a better way. Doing
+                    # path invalidation here would decrease the network
+                    # convergence time. Maybe just drop it?
+                    RNS.log("Not answering path request for "+RNS.prettyhexrep(destination_hash)+interface_str+", since next hop is the requestor", RNS.LOG_DEBUG)
                 else:
-                    # TODO: Look at this timing
-                    retransmit_timeout = now + Transport.PATH_REQUEST_GRACE # + (RNS.rand() * Transport.PATHFINDER_RW)
+                    RNS.log("Answering path request for "+RNS.prettyhexrep(destination_hash)+interface_str+", path is known", RNS.LOG_DEBUG)
 
-                # This handles an edge case where a peer sends a past
-                # request for a destination just after an announce for
-                # said destination has arrived, but before it has been
-                # rebroadcast locally. In such a case the actual announce
-                # is temporarily held, and then reinserted when the path
-                # request has been served to the peer.
-                if packet.destination_hash in Transport.announce_table:
-                    held_entry = Transport.announce_table[packet.destination_hash]
-                    Transport.held_announces[packet.destination_hash] = held_entry
-                
-                Transport.announce_table[packet.destination_hash] = [now, retransmit_timeout, retries, received_from, announce_hops, packet, local_rebroadcasts, block_rebroadcasts, attached_interface]
+                    now = time.time()
+                    retries = Transport.PATHFINDER_R
+                    local_rebroadcasts = 0
+                    block_rebroadcasts = True
+                    announce_hops      = packet.hops
+
+                    if is_from_local_client:
+                        retransmit_timeout = now
+                    else:
+                        if Transport.is_local_client_interface(Transport.next_hop_interface(destination_hash)):
+                            RNS.log("Path request destination "+RNS.prettyhexrep(destination_hash)+" is on a local client interface, rebroadcasting immediately", RNS.LOG_EXTREME)
+                            retransmit_timeout = now
+
+                        else:
+                            retransmit_timeout = now + Transport.PATH_REQUEST_GRACE
+
+                            # If we are answering on a roaming-mode interface, wait a
+                            # little longer, to allow potential more well-connected
+                            # peers to answer first.
+                            if attached_interface.mode == RNS.Interfaces.Interface.Interface.MODE_ROAMING:
+                                retransmit_timeout += Transport.PATH_REQUEST_RG
+
+                    # This handles an edge case where a peer sends a past
+                    # request for a destination just after an announce for
+                    # said destination has arrived, but before it has been
+                    # rebroadcast locally. In such a case the actual announce
+                    # is temporarily held, and then reinserted when the path
+                    # request has been served to the peer.
+                    if packet.destination_hash in Transport.announce_table:
+                        held_entry = Transport.announce_table[packet.destination_hash]
+                        Transport.held_announces[packet.destination_hash] = held_entry
+                    
+                    Transport.announce_table[packet.destination_hash] = [now, retransmit_timeout, retries, received_from, announce_hops, packet, local_rebroadcasts, block_rebroadcasts, attached_interface]
 
         elif is_from_local_client:
             # Forward path request on all interfaces
@@ -2161,7 +2447,10 @@ class Transport:
                 pass
 
         for interface in detachable_interfaces:
-            interface.detach()
+            try:
+                interface.detach()
+            except Exception as e:
+                RNS.log("An error occurred while detaching "+str(interface)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
 
     @staticmethod
     def shared_connection_disappeared():
@@ -2359,7 +2648,7 @@ class Transport:
                         received_from = de[1]
                         hops = de[2]
                         expires = de[3]
-                        random_blobs = de[4]
+                        random_blobs = de[4][-Transport.PERSIST_RANDOM_BLOBS:]
                         packet_hash = de[6].get_hash()
 
                         serialised_entry = [

RNS/Utilities/rncp.py

@@ -24,6 +24,7 @@
 
 import RNS
 import argparse
+import threading
 import time
 import sys
 import os
@@ -34,9 +35,12 @@ APP_NAME = "rncp"
 allow_all = False
 allowed_identity_hashes = []
 
-def receive(configdir, verbosity = 0, quietness = 0, allowed = [], display_identity = False, limit = None, disable_auth = None, disable_announce = False):
+def listen(configdir, verbosity = 0, quietness = 0, allowed = [], display_identity = False, limit = None, disable_auth = None, announce = False):
     global allow_all, allowed_identity_hashes
+    from tempfile import TemporaryFile
     identity = None
+    if announce < 0:
+        announce = False
 
     targetloglevel = 3+verbosity-quietness
     reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
@@ -54,16 +58,48 @@ def receive(configdir, verbosity = 0, quietness = 0, allowed = [], display_ident
 
     if display_identity:
         print("Identity     : "+str(identity))
-        print("Receiving on : "+RNS.prettyhexrep(destination.hash))
+        print("Listening on : "+RNS.prettyhexrep(destination.hash))
         exit(0)
 
     if disable_auth:
         allow_all = True
     else:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        try:
+            allowed_file_name = "allowed_identities"
+            allowed_file = None
+            if os.path.isfile(os.path.expanduser("/etc/rncp/"+allowed_file_name)):
+                allowed_file = os.path.expanduser("/etc/rncp/"+allowed_file_name)
+            elif os.path.isfile(os.path.expanduser("~/.config/rncp/"+allowed_file_name)):
+                allowed_file = os.path.expanduser("~/.config/rncp/"+allowed_file_name)
+            elif os.path.isfile(os.path.expanduser("~/.rncp/"+allowed_file_name)):
+                allowed_file = os.path.expanduser("~/.rncp/"+allowed_file_name)
+            if allowed_file != None:
+                af = open(allowed_file, "r")
+                al = af.read().replace("\r", "").split("\n")
+                ali = []
+                for a in al:
+                    if len(a) == dest_len:
+                        ali.append(a)
+
+                if len(ali) > 0:
+                    if not allowed:
+                        allowed = ali
+                    else:
+                        allowed.extend(ali)
+                if len(ali) == 1:
+                    ms = "y"
+                else:
+                    ms = "ies"
+                
+                RNS.log("Loaded "+str(len(ali))+" allowed identit"+ms+" from "+str(allowed_file), RNS.LOG_VERBOSE)
+
+        except Exception as e:
+            RNS.log("Error while parsing allowed_identities file. The contained exception was: "+str(e), RNS.LOG_ERROR)
+
         if allowed != None:
             for a in allowed:
                 try:
-                    dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
                     if len(a) != dest_len:
                         raise ValueError("Allowed destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
                     try:
@@ -78,16 +114,60 @@ def receive(configdir, verbosity = 0, quietness = 0, allowed = [], display_ident
     if len(allowed_identity_hashes) < 1 and not disable_auth:
         print("Warning: No allowed identities configured, rncp will not accept any files!")
 
-    destination.set_link_established_callback(receive_link_established)
-    print("rncp ready to receive on "+RNS.prettyhexrep(destination.hash))
+    def fetch_request(path, data, request_id, link_id, remote_identity, requested_at):
+        target_link = None
+        for link in RNS.Transport.active_links:
+            if link.link_id == link_id:
+                target_link = link
 
-    if not disable_announce:
-        destination.announce()
+        file_path = os.path.expanduser(data)
+        if not os.path.isfile(file_path):
+            RNS.log("Client-requested file not found: "+str(file_path), RNS.LOG_VERBOSE)
+            return False
+        else:
+            if target_link != None:
+                RNS.log("Sending file "+str(file_path)+" to client", RNS.LOG_VERBOSE)
+
+                temp_file = TemporaryFile()
+                real_file = open(file_path, "rb")
+                filename_bytes = os.path.basename(file_path).encode("utf-8")
+                filename_len = len(filename_bytes)
+
+                if filename_len > 0xFFFF:
+                    print("Filename exceeds max size, cannot send")
+                    exit(1)
+                else:
+                    print("Preparing file...", end=" ")
+
+                temp_file.write(filename_len.to_bytes(2, "big"))
+                temp_file.write(filename_bytes)
+                temp_file.write(real_file.read())
+                temp_file.seek(0)
+
+                fetch_resource = RNS.Resource(temp_file, target_link)
+                return True
+            else:
+                return None
+
+
+    destination.set_link_established_callback(client_link_established)
+    destination.register_request_handler("fetch_file", response_generator=fetch_request, allow=RNS.Destination.ALLOW_LIST, allowed_list=allowed_identity_hashes)
+    print("rncp listening on "+RNS.prettyhexrep(destination.hash))
+
+    if announce >= 0:
+        def job():
+            destination.announce()
+            if announce > 0:
+                while True:
+                    time.sleep(announce)
+                    destination.announce()
+
+        threading.Thread(target=job, daemon=True).start()
     
     while True:
         time.sleep(1)
 
-def receive_link_established(link):
+def client_link_established(link):
     RNS.log("Incoming link established", RNS.LOG_VERBOSE)
     link.set_remote_identified_callback(receive_sender_identified)
     link.set_resource_strategy(RNS.Link.ACCEPT_APP)
@@ -181,7 +261,221 @@ def sender_progress(resource):
         resource_done = True
 
 link = None
-def send(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT):
+def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False):
+    global current_resource, resource_done, link, speed
+    targetloglevel = 3+verbosity-quietness
+
+    try:
+        dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+        if len(destination) != dest_len:
+            raise ValueError("Allowed destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+        try:
+            destination_hash = bytes.fromhex(destination)
+        except Exception as e:
+            raise ValueError("Invalid destination entered. Check your input.")
+    except Exception as e:
+        print(str(e))
+        exit(1)
+
+    reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
+
+    identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
+    if os.path.isfile(identity_path):
+        identity = RNS.Identity.from_file(identity_path)
+        if identity == None:
+            RNS.log("Could not load identity for rncp. The identity file at \""+str(identity_path)+"\" may be corrupt or unreadable.", RNS.LOG_ERROR)
+            exit(2)
+    else:
+        identity = None
+
+    if identity == None:
+        RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
+        identity = RNS.Identity()
+        identity.to_file(identity_path)
+
+    if not RNS.Transport.has_path(destination_hash):
+        RNS.Transport.request_path(destination_hash)
+        if silent:
+            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested")
+        else:
+            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
+        sys.stdout.flush()
+
+    i = 0
+    syms = "⢄⢂⢁⡁⡈⡐⡠"
+    estab_timeout = time.time()+timeout
+    while not RNS.Transport.has_path(destination_hash) and time.time() < estab_timeout:
+        if not silent:
+            time.sleep(0.1)
+            print(("\b\b"+syms[i]+" "), end="")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
+
+    if not RNS.Transport.has_path(destination_hash):
+        if silent:
+            print("Path not found")
+        else:
+            print("\r                                                            \rPath not found")
+        exit(1)
+    else:
+        if silent:
+            print("Establishing link with "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("\r                                                            \rEstablishing link with "+RNS.prettyhexrep(destination_hash)+"  ", end=" ")
+
+    listener_identity = RNS.Identity.recall(destination_hash)
+    listener_destination = RNS.Destination(
+        listener_identity,
+        RNS.Destination.OUT,
+        RNS.Destination.SINGLE,
+        APP_NAME,
+        "receive"
+    )
+
+    link = RNS.Link(listener_destination)
+    while link.status != RNS.Link.ACTIVE and time.time() < estab_timeout:
+        if not silent:
+            time.sleep(0.1)
+            print(("\b\b"+syms[i]+" "), end="")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
+
+    if not RNS.Transport.has_path(destination_hash):
+        if silent:
+            print("Could not establish link with "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("\r                                                            \rCould not establish link with "+RNS.prettyhexrep(destination_hash))
+        exit(1)
+    else:
+        if silent:
+            print("Requesting file from remote...")
+        else:
+            print("\r                                                            \rRequesting file from remote  ", end=" ")
+
+    link.identify(identity)
+
+    request_resolved = False
+    request_status = "unknown"
+    resource_resolved = False
+    resource_status = "unrequested"
+    current_resource = None
+    def request_response(request_receipt):
+        nonlocal request_resolved, request_status
+        if request_receipt.response == False:
+            request_status = "not_found"
+        elif request_receipt.response == None:
+            request_status = "remote_error"
+        else:
+            request_status = "found"
+
+        request_resolved = True
+
+    def request_failed(request_receipt):
+        nonlocal request_resolved, request_status
+        request_status = "unknown"
+        request_resolved = True
+
+    def fetch_resource_started(resource):
+        nonlocal resource_status
+        current_resource = resource
+        current_resource.progress_callback(sender_progress)
+        resource_status = "started"
+
+    def fetch_resource_concluded(resource):
+        nonlocal resource_resolved, resource_status
+        if resource.status == RNS.Resource.COMPLETE:
+            if resource.total_size > 4:
+                filename_len = int.from_bytes(resource.data.read(2), "big")
+                filename = resource.data.read(filename_len).decode("utf-8")
+
+                counter = 0
+                saved_filename = filename
+                while os.path.isfile(saved_filename):
+                    counter += 1
+                    saved_filename = filename+"."+str(counter)
+                
+                file = open(saved_filename, "wb")
+                file.write(resource.data.read())
+                file.close()
+                resource_status = "completed"
+
+            else:
+                print("Invalid data received, ignoring resource")
+                resource_status = "invalid_data"
+
+        else:
+            print("Resource failed")
+            resource_status = "failed"
+
+        resource_resolved = True
+
+    link.set_resource_strategy(RNS.Link.ACCEPT_ALL)
+    link.set_resource_started_callback(fetch_resource_started)
+    link.set_resource_concluded_callback(fetch_resource_concluded)
+    link.request("fetch_file", data=file, response_callback=request_response, failed_callback=request_failed)
+
+    syms = "⢄⢂⢁⡁⡈⡐⡠"
+    while not request_resolved:
+        if not silent:
+            time.sleep(0.1)
+            print(("\b\b"+syms[i]+" "), end="")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
+
+    if request_status == "not_found":
+        if not silent: print("\r                                                            \r", end="")
+        print("Fetch request failed, the file "+str(file)+" was not found on the remote")
+        link.teardown()
+        time.sleep(1)
+        exit(0)
+    elif request_status == "remote_error":
+        if not silent: print("\r                                                            \r", end="")
+        print("Fetch request failed due to an error on the remote system")
+        link.teardown()
+        time.sleep(1)
+        exit(0)
+    elif request_status == "unknown":
+        if not silent: print("\r                                                            \r", end="")
+        print("Fetch request failed due to an unknown error (probably not authorised)")
+        link.teardown()
+        time.sleep(1)
+        exit(0)
+    elif request_status == "found":
+        if not silent: print("\r                                                            \r", end="")
+
+    while not resource_resolved:
+        if not silent:
+            time.sleep(0.1)
+            if current_resource:
+                prg = current_resource.get_progress()
+                percent = round(prg * 100.0, 1)
+                stat_str = str(percent)+"% - " + size_str(int(prg*current_resource.total_size)) + " of " + size_str(current_resource.total_size) + " - " +size_str(speed, "b")+"ps"
+                print("\r                                                                                  \rTransferring file "+syms[i]+" "+stat_str, end=" ")
+            else:
+                print("\r                                                                                  \rWaiting for transfer to start "+syms[i]+" ", end=" ")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
+
+    if current_resource.status != RNS.Resource.COMPLETE:
+        if silent:
+            print("The transfer failed")
+        else:
+            print("\r                                                            \rThe transfer failed")
+        exit(1)
+    else:
+        if silent:
+            print(str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("\r                                                                                  \r"+str(file)+" fetched from "+RNS.prettyhexrep(destination_hash))
+        link.teardown()
+        time.sleep(0.25)
+        exit(0)
+
+    link.teardown()
+    exit(0)
+
+
+def send(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False):
     global current_resource, resource_done, link, speed
     from tempfile import TemporaryFile
     targetloglevel = 3+verbosity-quietness
@@ -226,7 +520,12 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
 
     identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
     if os.path.isfile(identity_path):
-        identity = RNS.Identity.from_file(identity_path)                
+        identity = RNS.Identity.from_file(identity_path)
+        if identity == None:
+            RNS.log("Could not load identity for rncp. The identity file at \""+str(identity_path)+"\" may be corrupt or unreadable.", RNS.LOG_ERROR)
+            exit(2)
+    else:
+        identity = None
 
     if identity == None:
         RNS.log("No valid saved identity found, creating new...", RNS.LOG_INFO)
@@ -235,23 +534,33 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
 
     if not RNS.Transport.has_path(destination_hash):
         RNS.Transport.request_path(destination_hash)
-        print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
+        if silent:
+            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested")
+        else:
+            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
         sys.stdout.flush()
 
     i = 0
     syms = "⢄⢂⢁⡁⡈⡐⡠"
     estab_timeout = time.time()+timeout
     while not RNS.Transport.has_path(destination_hash) and time.time() < estab_timeout:
-        time.sleep(0.1)
-        print(("\b\b"+syms[i]+" "), end="")
-        sys.stdout.flush()
-        i = (i+1)%len(syms)
+        if not silent:
+            time.sleep(0.1)
+            print(("\b\b"+syms[i]+" "), end="")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
 
     if not RNS.Transport.has_path(destination_hash):
-        print("\r                                                            \rPath not found")
+        if silent:
+            print("Path not found")
+        else:
+            print("\r                                                            \rPath not found")
         exit(1)
     else:
-        print("\r                                                            \rEstablishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=" ")
+        if silent:
+            print("Establishing link with "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("\r                                                            \rEstablishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=" ")
 
     receiver_identity = RNS.Identity.recall(destination_hash)
     receiver_destination = RNS.Destination(
@@ -264,48 +573,75 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
 
     link = RNS.Link(receiver_destination)
     while link.status != RNS.Link.ACTIVE and time.time() < estab_timeout:
-        time.sleep(0.1)
-        print(("\b\b"+syms[i]+" "), end="")
-        sys.stdout.flush()
-        i = (i+1)%len(syms)
+        if not silent:
+            time.sleep(0.1)
+            print(("\b\b"+syms[i]+" "), end="")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
 
-    if not RNS.Transport.has_path(destination_hash):
-        print("\r                                                            \rCould not establish link with "+RNS.prettyhexrep(destination_hash))
+    if time.time() > estab_timeout:
+        if silent:
+            print("Link establishment with "+RNS.prettyhexrep(destination_hash)+" timed out")
+        else:
+            print("\r                                                            \rLink establishment with "+RNS.prettyhexrep(destination_hash)+" timed out")
+        exit(1)
+    elif not RNS.Transport.has_path(destination_hash):
+        if silent:
+            print("No path found to "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("\r                                                            \rNo path found to "+RNS.prettyhexrep(destination_hash))
         exit(1)
     else:
-        print("\r                                                            \rAdvertising file resource  ", end=" ")
+        if silent:
+            print("Advertising file resource...")
+        else:
+            print("\r                                                            \rAdvertising file resource  ", end=" ")
 
     link.identify(identity)
     resource = RNS.Resource(temp_file, link, callback = sender_progress, progress_callback = sender_progress)
     current_resource = resource
 
     while resource.status < RNS.Resource.TRANSFERRING:
-        time.sleep(0.1)
-        print(("\b\b"+syms[i]+" "), end="")
-        sys.stdout.flush()
-        i = (i+1)%len(syms)
+        if not silent:
+            time.sleep(0.1)
+            print(("\b\b"+syms[i]+" "), end="")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
 
     
     if resource.status > RNS.Resource.COMPLETE:
-        print("\r                                                            \rFile was not accepted by "+RNS.prettyhexrep(destination_hash))
+        if silent:
+            print("File was not accepted by "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("\r                                                            \rFile was not accepted by "+RNS.prettyhexrep(destination_hash))
         exit(1)
     else:
-        print("\r                                                            \rTransferring file  ", end=" ")
+        if silent:
+            print("Transferring file...")
+        else:
+            print("\r                                                            \rTransferring file  ", end=" ")
 
     while not resource_done:
-        time.sleep(0.1)
-        prg = current_resource.get_progress()
-        percent = round(prg * 100.0, 1)
-        stat_str = str(percent)+"% - " + size_str(int(prg*current_resource.total_size)) + " of " + size_str(current_resource.total_size) + " - " +size_str(speed, "b")+"ps"
-        print("\r                                                                                  \rTransferring file "+syms[i]+" "+stat_str, end=" ")
-        sys.stdout.flush()
-        i = (i+1)%len(syms)
+        if not silent:
+            time.sleep(0.1)
+            prg = current_resource.get_progress()
+            percent = round(prg * 100.0, 1)
+            stat_str = str(percent)+"% - " + size_str(int(prg*current_resource.total_size)) + " of " + size_str(current_resource.total_size) + " - " +size_str(speed, "b")+"ps"
+            print("\r                                                                                  \rTransferring file "+syms[i]+" "+stat_str, end=" ")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
 
     if current_resource.status != RNS.Resource.COMPLETE:
-        print("\r                                                            \rThe transfer failed")
+        if silent:
+            print("The transfer failed")
+        else:
+            print("\r                                                            \rThe transfer failed")
         exit(1)
     else:
-        print("\r                                                                                  \r"+str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
+        if silent:
+            print(str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("\r                                                                                  \r"+str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
         link.teardown()
         time.sleep(0.25)
         real_file.close()
@@ -320,19 +656,21 @@ def main():
         parser.add_argument("--config", metavar="path", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
         parser.add_argument('-v', '--verbose', action='count', default=0, help="increase verbosity")
         parser.add_argument('-q', '--quiet', action='count', default=0, help="decrease verbosity")
-        parser.add_argument('-p', '--print-identity', action='store_true', default=False, help="print identity and destination info and exit")
-        parser.add_argument("-r", '--receive', action='store_true', default=False, help="wait for incoming files")
-        parser.add_argument("-b", '--no-announce', action='store_true', default=False, help="don't announce at program start")
+        parser.add_argument("-S", '--silent', action='store_true', default=False, help="disable transfer progress output")
+        parser.add_argument("-l", '--listen', action='store_true', default=False, help="listen for incoming transfer requests")
+        parser.add_argument("-f", '--fetch', action='store_true', default=False, help="fetch file from remote listener instead of sending")
+        parser.add_argument("-b", action='store', metavar="seconds", default=-1, help="announce interval, 0 to only announce at startup", type=int)
         parser.add_argument('-a', metavar="allowed_hash", dest="allowed", action='append', help="accept from this identity", type=str)
         parser.add_argument('-n', '--no-auth', action='store_true', default=False, help="accept files from anyone")
+        parser.add_argument('-p', '--print-identity', action='store_true', default=False, help="print identity and destination info and exit")
         parser.add_argument("-w", action="store", metavar="seconds", type=float, help="sender timeout before giving up", default=RNS.Transport.PATH_REQUEST_TIMEOUT)
         # parser.add_argument("--limit", action="store", metavar="files", type=float, help="maximum number of files to accept", default=None)
         parser.add_argument("--version", action="version", version="rncp {version}".format(version=__version__))
         
         args = parser.parse_args()
 
-        if args.receive or args.print_identity:
-            receive(
+        if args.listen or args.print_identity:
+            listen(
                 configdir = args.config,
                 verbosity=args.verbose,
                 quietness=args.quiet,
@@ -340,9 +678,25 @@ def main():
                 display_identity=args.print_identity,
                 # limit=args.limit,
                 disable_auth=args.no_auth,
-                disable_announce=args.no_announce,
+                announce=args.b,
             )
 
+        elif args.fetch:
+            if args.destination != None and args.file != None:
+                fetch(
+                    configdir = args.config,
+                    verbosity = args.verbose,
+                    quietness = args.quiet,
+                    destination = args.destination,
+                    file = args.file,
+                    timeout = args.w,
+                    silent = args.silent,
+                )
+            else:
+                print("")
+                parser.print_help()
+                print("")
+
         elif args.destination != None and args.file != None:
             send(
                 configdir = args.config,
@@ -351,6 +705,7 @@ def main():
                 destination = args.destination,
                 file = args.file,
                 timeout = args.w,
+                silent = args.silent,
             )
 
         else:

RNS/Utilities/rnid.py

@@ -64,19 +64,22 @@ def main():
 
         parser.add_argument("--config", metavar="path", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
         parser.add_argument("-i", "--identity", metavar="identity", action="store", default=None, help="hexadecimal Reticulum Destination hash or path to Identity file", type=str)
-        parser.add_argument("-g", "--generate", metavar="path", action="store", default=None, help="generate a new Identity")
+        parser.add_argument("-g", "--generate", metavar="file", action="store", default=None, help="generate a new Identity")
+        parser.add_argument("-m", "--import", dest="import_str", metavar="identity_data", action="store", default=None, help="import Reticulum identity in hex, base32 or base64 format", type=str)
+        parser.add_argument("-x", "--export", action="store_true", default=None, help="export identity to hex, base32 or base64 format")
+
         parser.add_argument("-v", "--verbose", action="count", default=0, help="increase verbosity")
         parser.add_argument("-q", "--quiet", action="count", default=0, help="decrease verbosity")
 
         parser.add_argument("-a", "--announce", metavar="aspects", action="store", default=None, help="announce a destination based on this Identity")
-        parser.add_argument("-H", "--hash", metavar="aspects", action="store", default=None, help="show destination hash5s for other aspects for this Identity")
-        parser.add_argument("-e", "--encrypt", metavar="path", action="store", default=None, help="encrypt file")
-        parser.add_argument("-d", "--decrypt", metavar="path", action="store", default=None, help="decrypt file")
+        parser.add_argument("-H", "--hash", metavar="aspects", action="store", default=None, help="show destination hashes for other aspects for this Identity")
+        parser.add_argument("-e", "--encrypt", metavar="file", action="store", default=None, help="encrypt file")
+        parser.add_argument("-d", "--decrypt", metavar="file", action="store", default=None, help="decrypt file")
         parser.add_argument("-s", "--sign", metavar="path", action="store", default=None, help="sign file")
         parser.add_argument("-V", "--validate", metavar="path", action="store", default=None, help="validate signature")
 
-        parser.add_argument("-r", "--read", metavar="path", action="store", default=None, help="input file path", type=str)
-        parser.add_argument("-w", "--write", metavar="path", action="store", default=None, help="output file path", type=str)
+        parser.add_argument("-r", "--read", metavar="file", action="store", default=None, help="input file path", type=str)
+        parser.add_argument("-w", "--write", metavar="file", action="store", default=None, help="output file path", type=str)
         parser.add_argument("-f", "--force", action="store_true", default=None, help="write output even if it overwrites existing files")
         parser.add_argument("-I", "--stdin", action="store_true", default=False, help=argparse.SUPPRESS) # "read input from STDIN instead of file"
         parser.add_argument("-O", "--stdout", action="store_true", default=False, help=argparse.SUPPRESS) # help="write output to STDOUT instead of file", 
@@ -86,7 +89,8 @@ def main():
         parser.add_argument("-p", "--print-identity", action="store_true", default=False, help="print identity info and exit")
         parser.add_argument("-P", "--print-private", action="store_true", default=False, help="allow displaying private keys")
 
-        parser.add_argument("-b", "--base64", action="store_true", default=False, help=argparse.SUPPRESS) # help="Use base64-encoded input and output")
+        parser.add_argument("-b", "--base64", action="store_true", default=False, help="Use base64-encoded input and output")
+        parser.add_argument("-B", "--base32", action="store_true", default=False, help="Use base32-encoded input and output")
 
         parser.add_argument("--version", action="version", version="rnid {version}".format(version=__version__))
         
@@ -110,6 +114,59 @@ def main():
                 args.read = args.sign
 
         identity_str = args.identity
+        if args.import_str:
+            identity_bytes = None
+            try:
+                if args.base64:
+                    identity_bytes = base64.urlsafe_b64decode(args.import_str)
+                elif args.base32:
+                    identity_bytes = base64.b32decode(args.import_str)
+                else:
+                    identity_bytes = bytes.fromhex(args.import_str)
+            except Exception as e:
+                print("Invalid identity data specified for import: "+str(e))
+                exit(41)
+
+            try:
+                identity = RNS.Identity.from_bytes(identity_bytes)
+            except Exception as e:
+                print("Could not create Reticulum identity from specified data: "+str(e))
+                exit(42)
+
+            RNS.log("Identity imported")
+            if args.base64:
+                RNS.log("Public Key  : "+base64.urlsafe_b64encode(identity.get_public_key()).decode("utf-8"))
+            elif args.base32:
+                RNS.log("Public Key  : "+base64.b32encode(identity.get_public_key()).decode("utf-8"))
+            else:
+                RNS.log("Public Key  : "+RNS.hexrep(identity.get_public_key(), delimit=False))
+            if identity.prv:
+                if args.print_private:
+                    if args.base64:
+                        RNS.log("Private Key : "+base64.urlsafe_b64encode(identity.get_private_key()).decode("utf-8"))
+                    elif args.base32:
+                        RNS.log("Private Key : "+base64.b32encode(identity.get_private_key()).decode("utf-8"))
+                    else:
+                        RNS.log("Private Key : "+RNS.hexrep(identity.get_private_key(), delimit=False))
+                else:
+                    RNS.log("Private Key : Hidden")
+
+            if args.write:
+                try:
+                    wp = os.path.expanduser(args.write)
+                    if not os.path.isfile(wp) or args.force:
+                        identity.to_file(wp)
+                        RNS.log("Wrote imported identity to "+str(args.write))
+                    else:
+                        print("File "+str(wp)+" already exists, not overwriting")
+                        exit(43)
+
+                except Exception as e:
+                    print("Error while writing imported identity to file: "+str(e))
+                    exit(44)
+
+            exit(0)
+
         if not args.generate and not identity_str:
             print("\nNo identity provided, cannot continue\n")
             parser.print_help()
@@ -166,8 +223,8 @@ def main():
                                 RNS.log("Identity request timed out", RNS.LOG_ERROR)
                                 exit(6)
                             else:
-                                RNS.log("Received Identity "+str(identity)+" for destination "+RNS.prettyhexrep(destination_hash)+" from the network")
                                 identity = RNS.Identity.recall(destination_hash)
+                                RNS.log("Received Identity "+str(identity)+" for destination "+RNS.prettyhexrep(destination_hash)+" from the network")
 
                     else:
                         RNS.log("Recalled Identity "+str(identity)+" for destination "+RNS.prettyhexrep(destination_hash))
@@ -196,7 +253,7 @@ def main():
                 if args.hash:
                     try:
                         aspects = args.hash.split(".")
-                        if not len(aspects) > 1:
+                        if not len(aspects) > 0:
                             RNS.log("Invalid destination aspects specified", RNS.LOG_ERROR)
                             exit(32)
                         else:
@@ -246,14 +303,38 @@ def main():
                     exit(0)
 
                 if args.print_identity:
-                    RNS.log("Public Key  : "+RNS.hexrep(identity.pub_bytes, delimit=False))
+                    if args.base64:
+                        RNS.log("Public Key  : "+base64.urlsafe_b64encode(identity.get_public_key()).decode("utf-8"))
+                    elif args.base32:
+                        RNS.log("Public Key  : "+base64.b32encode(identity.get_public_key()).decode("utf-8"))
+                    else:
+                        RNS.log("Public Key  : "+RNS.hexrep(identity.get_public_key(), delimit=False))
                     if identity.prv:
                         if args.print_private:
-                            RNS.log("Private Key : "+RNS.hexrep(identity.prv_bytes, delimit=False))
+                            if args.base64:
+                                RNS.log("Private Key : "+base64.urlsafe_b64encode(identity.get_private_key()).decode("utf-8"))
+                            elif args.base32:
+                                RNS.log("Private Key : "+base64.b32encode(identity.get_private_key()).decode("utf-8"))
+                            else:
+                                RNS.log("Private Key : "+RNS.hexrep(identity.get_private_key(), delimit=False))
                         else:
                             RNS.log("Private Key : Hidden")
                     exit(0)
 
+                if args.export:
+                    if identity.prv:
+                            if args.base64:
+                                RNS.log("Exported Identity : "+base64.urlsafe_b64encode(identity.get_private_key()).decode("utf-8"))
+                            elif args.base32:
+                                RNS.log("Exported Identity : "+base64.b32encode(identity.get_private_key()).decode("utf-8"))
+                            else:
+                                RNS.log("Exported Identity : "+RNS.hexrep(identity.get_private_key(), delimit=False))
+                    else:
+                        RNS.log("Identity doesn't hold a private key, cannot export")
+                        exit(50)
+
+                    exit(0)
+
                 if args.validate:
                     if not args.read and args.validate.lower().endswith("."+SIG_EXT):
                         args.read = str(args.validate).replace("."+SIG_EXT, "")

RNS/Utilities/rnir.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+
+# MIT License
+#
+# Copyright (c) 2023 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import RNS
+import argparse
+import time
+
+from RNS._version import __version__
+
+
+def program_setup(configdir, verbosity = 0, quietness = 0, service = False):
+    targetverbosity = verbosity-quietness
+
+    if service:
+        targetlogdest  = RNS.LOG_FILE
+        targetverbosity = None
+    else:
+        targetlogdest  = RNS.LOG_STDOUT
+
+    reticulum = RNS.Reticulum(configdir=configdir, verbosity=targetverbosity, logdest=targetlogdest)
+    exit(0)
+
+def main():
+    try:
+        parser = argparse.ArgumentParser(description="Reticulum Distributed Identity Resolver")
+        parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
+        parser.add_argument('-v', '--verbose', action='count', default=0)
+        parser.add_argument('-q', '--quiet', action='count', default=0)
+        parser.add_argument("--exampleconfig", action='store_true', default=False, help="print verbose configuration example to stdout and exit")
+        parser.add_argument("--version", action="version", version="ir {version}".format(version=__version__))
+        
+        args = parser.parse_args()
+
+        if args.exampleconfig:
+            print(__example_rns_config__)
+            exit()
+
+        if args.config:
+            configarg = args.config
+        else:
+            configarg = None
+
+        program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet)
+
+    except KeyboardInterrupt:
+        print("")
+        exit()
+
+__example_rns_config__ = '''# This is an example Identity Resolver file.
+'''
+
+if __name__ == "__main__":
+    main()

RNS/Utilities/rnpath.py

@@ -30,7 +30,7 @@ import argparse
 from RNS._version import __version__
 
 
-def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity, timeout, drop_queues):
+def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity, timeout, drop_queues, drop_via):
     if table:
         destination_hash = None
         if destination_hexhash != None:
@@ -155,6 +155,29 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
             sys.exit(1)
 
 
+    elif drop_via:
+        try:
+            dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
+            if len(destination_hexhash) != dest_len:
+                raise ValueError("Destination length is invalid, must be {hex} hexadecimal characters ({byte} bytes).".format(hex=dest_len, byte=dest_len//2))
+            try:
+                destination_hash = bytes.fromhex(destination_hexhash)
+            except Exception as e:
+                raise ValueError("Invalid destination entered. Check your input.")
+        except Exception as e:
+            print(str(e))
+            sys.exit(1)
+
+
+        reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+
+        if reticulum.drop_all_via(destination_hash):
+            print("Dropped all paths via "+RNS.prettyhexrep(destination_hash))
+        else:
+            print("Unable to drop paths via "+RNS.prettyhexrep(destination_hash)+". Does the transport instance exist?")
+            sys.exit(1)
+
+
     else:
         try:
             dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
@@ -187,15 +210,20 @@ def program_setup(configdir, table, rates, drop, destination_hexhash, verbosity,
 
         if RNS.Transport.has_path(destination_hash):
             hops = RNS.Transport.hops_to(destination_hash)
-            next_hop = RNS.prettyhexrep(reticulum.get_next_hop(destination_hash))
-            next_hop_interface = reticulum.get_next_hop_if_name(destination_hash)
-
-            if hops != 1:
-                ms = "s"
+            next_hop_bytes = reticulum.get_next_hop(destination_hash)
+            if next_hop_bytes == None:
+                print("\r                                                       \rError: Invalid path data returned")
+                sys.exit(1)
             else:
-                ms = ""
+                next_hop = RNS.prettyhexrep(next_hop_bytes)
+                next_hop_interface = reticulum.get_next_hop_if_name(destination_hash)
 
-            print("\rPath found, destination "+RNS.prettyhexrep(destination_hash)+" is "+str(hops)+" hop"+ms+" away via "+next_hop+" on "+next_hop_interface)
+                if hops != 1:
+                    ms = "s"
+                else:
+                    ms = ""
+
+                print("\rPath found, destination "+RNS.prettyhexrep(destination_hash)+" is "+str(hops)+" hop"+ms+" away via "+next_hop+" on "+next_hop_interface)
         else:
             print("\r                                                       \rPath not found")
             sys.exit(1)
@@ -251,6 +279,13 @@ def main():
             default=False
         )
 
+        parser.add_argument(
+            "-x", "--drop-via",
+            action="store_true",
+            help="drop all paths via specified transport instance",
+            default=False
+        )
+
         parser.add_argument(
             "-w",
             action="store",
@@ -277,7 +312,7 @@ def main():
         else:
             configarg = None
 
-        if not args.drop_announces and not args.table and not args.rates and not args.destination:
+        if not args.drop_announces and not args.table and not args.rates and not args.destination and not args.drop_via:
             print("")
             parser.print_help()
             print("")
@@ -291,6 +326,7 @@ def main():
                 verbosity = args.verbose,
                 timeout = args.w,
                 drop_queues = args.drop_announces,
+                drop_via = args.drop_via,
             )
             sys.exit(0)
 

RNS/Utilities/rnprobe.py

@@ -31,8 +31,10 @@ import argparse
 from RNS._version import __version__
 
 DEFAULT_PROBE_SIZE = 16
+DEFAULT_TIMEOUT = 12
 
-def program_setup(configdir, destination_hexhash, size=DEFAULT_PROBE_SIZE, full_name = None, verbosity = 0):
+def program_setup(configdir, destination_hexhash, size=None, full_name = None, verbosity = 0, timeout=None, wait=0, probes=1):
+    if size == None: size = DEFAULT_PROBE_SIZE
     if full_name == None:
         print("The full destination name including application name aspects must be specified for the destination")
         exit()
@@ -71,14 +73,19 @@ def program_setup(configdir, destination_hexhash, size=DEFAULT_PROBE_SIZE, full_
         print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
         sys.stdout.flush()
 
+    _timeout = time.time() + (timeout or DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash))
     i = 0
     syms = "⢄⢂⢁⡁⡈⡐⡠"
-    while not RNS.Transport.has_path(destination_hash):
+    while not RNS.Transport.has_path(destination_hash) and not time.time() > _timeout:
         time.sleep(0.1)
         print(("\b\b"+syms[i]+" "), end="")
         sys.stdout.flush()
         i = (i+1)%len(syms)
 
+    if time.time() > _timeout:
+        print("\r                                                          \rPath request timed out")
+        exit(1)
+
     server_identity = RNS.Identity.recall(destination_hash)
 
     request_destination = RNS.Destination(
@@ -89,101 +96,120 @@ def program_setup(configdir, destination_hexhash, size=DEFAULT_PROBE_SIZE, full_
         *aspects
     )
 
-    probe = RNS.Packet(request_destination, os.urandom(size))
-    receipt = probe.send()
+    sent = 0
+    replies = 0
+    while probes:
 
-    if more_output:
-        more = " via "+RNS.prettyhexrep(reticulum.get_next_hop(destination_hash))+" on "+str(reticulum.get_next_hop_if_name(destination_hash))
-    else:
-        more = ""
+        if sent > 0:
+            time.sleep(wait)
 
-    print("\rSent "+str(size)+" byte probe to "+RNS.prettyhexrep(destination_hash)+more+"  ", end=" ")
+        try:
+            probe = RNS.Packet(request_destination, os.urandom(size))
+            probe.pack()
+        except OSError:
+            print("Error: Probe packet size of "+str(len(probe.raw))+" bytes exceed MTU of "+str(RNS.Reticulum.MTU)+" bytes")
+            exit(3)
 
-    i = 0
-    while not receipt.status == RNS.PacketReceipt.DELIVERED:
-        time.sleep(0.1)
-        print(("\b\b"+syms[i]+" "), end="")
-        sys.stdout.flush()
-        i = (i+1)%len(syms)
+        receipt = probe.send()
+        sent += 1
 
-    print("\b\b ")
-    sys.stdout.flush()
+        if more_output:
+            nhd = reticulum.get_next_hop(destination_hash)
+            via_str = " via "+RNS.prettyhexrep(nhd) if nhd != None else ""
+            if_str  = " on "+str(reticulum.get_next_hop_if_name(destination_hash)) if reticulum.get_next_hop_if_name(destination_hash) != "None" else ""
+            more = via_str+if_str
+        else:
+            more = ""
 
-    hops = RNS.Transport.hops_to(destination_hash)
-    if hops != 1:
-        ms = "s"
-    else:
-        ms = ""
+        print("\rSent probe "+str(sent)+" ("+str(size)+" bytes) to "+RNS.prettyhexrep(destination_hash)+more+"  ", end=" ")
 
-    rtt = receipt.get_rtt()
-    if (rtt >= 1):
-        rtt = round(rtt, 3)
-        rttstring = str(rtt)+" seconds"
-    else:
-        rtt = round(rtt*1000, 3)
-        rttstring = str(rtt)+" milliseconds"
+        _timeout = time.time() + (timeout or DEFAULT_TIMEOUT+reticulum.get_first_hop_timeout(destination_hash))
+        i = 0
+        while receipt.status == RNS.PacketReceipt.SENT and not time.time() > _timeout:
+            time.sleep(0.1)
+            print(("\b\b"+syms[i]+" "), end="")
+            sys.stdout.flush()
+            i = (i+1)%len(syms)
 
-    reception_stats = ""
-    if reticulum.is_connected_to_shared_instance:
-        reception_rssi = reticulum.get_packet_rssi(receipt.proof_packet.packet_hash)
-        reception_snr  = reticulum.get_packet_snr(receipt.proof_packet.packet_hash)
-
-        if reception_rssi != None:
-            reception_stats += " [RSSI "+str(reception_rssi)+" dBm]"
+        if time.time() > _timeout:
+            print("\r                                                                \rProbe timed out")
         
-        if reception_snr != None:
-            reception_stats += " [SNR "+str(reception_snr)+" dB]"
+        else:
+            print("\b\b ")
+            sys.stdout.flush()
 
+            if receipt.status == RNS.PacketReceipt.DELIVERED:
+                replies += 1
+                hops = RNS.Transport.hops_to(destination_hash)
+                if hops != 1:
+                    ms = "s"
+                else:
+                    ms = ""
+
+                rtt = receipt.get_rtt()
+                if (rtt >= 1):
+                    rtt = round(rtt, 3)
+                    rttstring = str(rtt)+" seconds"
+                else:
+                    rtt = round(rtt*1000, 3)
+                    rttstring = str(rtt)+" milliseconds"
+
+                reception_stats = ""
+                if reticulum.is_connected_to_shared_instance:
+                    reception_rssi = reticulum.get_packet_rssi(receipt.proof_packet.packet_hash)
+                    reception_snr  = reticulum.get_packet_snr(receipt.proof_packet.packet_hash)
+                    reception_q    = reticulum.get_packet_q(receipt.proof_packet.packet_hash)
+
+                    if reception_rssi != None:
+                        reception_stats += " [RSSI "+str(reception_rssi)+" dBm]"
+                    
+                    if reception_snr != None:
+                        reception_stats += " [SNR "+str(reception_snr)+" dB]"
+                    
+                    if reception_q != None:
+                        reception_stats += " [Link Quality "+str(reception_q)+"%]"
+
+                else:
+                    if receipt.proof_packet != None:
+                        if receipt.proof_packet.rssi != None:
+                            reception_stats += " [RSSI "+str(receipt.proof_packet.rssi)+" dBm]"
+                        
+                        if receipt.proof_packet.snr != None:
+                            reception_stats += " [SNR "+str(receipt.proof_packet.snr)+" dB]"
+
+                print(
+                    "Valid reply from "+
+                    RNS.prettyhexrep(receipt.destination.hash)+
+                    "\nRound-trip time is "+rttstring+
+                    " over "+str(hops)+" hop"+ms+
+                    reception_stats+"\n"
+                )
+
+            else:
+                print("\r                                                          \rProbe timed out")
+
+        probes -= 1
+
+    loss = round((1-(replies/sent))*100, 2)
+    print(f"Sent {sent}, received {replies}, packet loss {loss}%")
+    if loss > 0:
+        exit(2)
     else:
-        if receipt.proof_packet != None:
-            if receipt.proof_packet.rssi != None:
-                reception_stats += " [RSSI "+str(receipt.proof_packet.rssi)+" dBm]"
-            
-            if receipt.proof_packet.snr != None:
-                reception_stats += " [SNR "+str(receipt.proof_packet.snr)+" dB]"
-
-    print(
-        "Valid reply received from "+
-        RNS.prettyhexrep(receipt.destination.hash)+
-        "\nRound-trip time is "+rttstring+
-        " over "+str(hops)+" hop"+ms+
-        reception_stats
-    )
-
+        exit(0)
     
 
 def main():
     try:
         parser = argparse.ArgumentParser(description="Reticulum Probe Utility")
 
-        parser.add_argument("--config",
-            action="store",
-            default=None,
-            help="path to alternative Reticulum config directory",
-            type=str
-        )
-
-        parser.add_argument(
-            "--version",
-            action="version",
-            version="rnprobe {version}".format(version=__version__)
-        )
-
-        parser.add_argument(
-            "full_name",
-            nargs="?",
-            default=None,
-            help="full destination name in dotted notation",
-            type=str
-        )
-
-        parser.add_argument(
-            "destination_hash",
-            nargs="?",
-            default=None,
-            help="hexadecimal hash of the destination",
-            type=str
-        )
+        parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
+        parser.add_argument("-s", "--size", action="store", default=None, help="size of probe packet payload in bytes", type=int)
+        parser.add_argument("-n", "--probes", action="store", default=1, help="number of probes to send", type=int)
+        parser.add_argument("-t", "--timeout", metavar="seconds", action="store", default=None, help="timeout before giving up", type=float)
+        parser.add_argument("-w", "--wait", metavar="seconds", action="store", default=0, help="time between each probe", type=float)
+        parser.add_argument("--version", action="version", version="rnprobe {version}".format(version=__version__))
+        parser.add_argument("full_name", nargs="?", default=None, help="full destination name in dotted notation", type=str)
+        parser.add_argument("destination_hash", nargs="?", default=None, help="hexadecimal hash of the destination", type=str)
 
         parser.add_argument('-v', '--verbose', action='count', default=0)
 
@@ -202,8 +228,12 @@ def main():
             program_setup(
                 configdir = configarg,
                 destination_hexhash = args.destination_hash,
+                size = args.size,
                 full_name = args.full_name,
-                verbosity = args.verbose
+                verbosity = args.verbose,
+                probes = args.probes,
+                wait = args.wait,
+                timeout = args.timeout,
             )
 
     except KeyboardInterrupt:

RNS/Utilities/rnsd.py

@@ -87,7 +87,7 @@ __example_rns_config__ = '''# This is an example Reticulum config file.
 # always-on. This directive is optional and can be removed
 # for brevity.
 
-enable_transport = False
+enable_transport = No
 
 
 # By default, the first program to launch the Reticulum
@@ -111,6 +111,18 @@ share_instance = Yes
 shared_instance_port = 37428
 instance_control_port = 37429
 
+
+# On systems where running instances may not have access
+# to the same shared Reticulum configuration directory,
+# it is still possible to allow full interactivity for
+# running instances, by manually specifying a shared RPC
+# key. In almost all cases, this option is not needed, but
+# it can be useful on operating systems such as Android.
+# The key must be specified as bytes in hexadecimal.
+
+# rpc_key = e5c032d3ec4e64a6aca9927ba8ab73336780f6d71790
+
+
 # You can configure Reticulum to panic and forcibly close
 # if an unrecoverable interface error occurs, such as the
 # hardware device for an interface disappearing. This is
@@ -120,6 +132,17 @@ instance_control_port = 37429
 panic_on_interface_error = No
 
 
+# When Transport is enabled, it is possible to allow the
+# Transport Instance to respond to probe requests from
+# the rnprobe utility. This can be a useful tool to test
+# connectivity. When this option is enabled, the probe
+# destination will be generated from the Identity of the
+# Transport Instance, and printed to the log at startup.
+# Optional, and disabled by default.
+
+respond_to_probes = No
+
+
 [logging]
 # Valid log levels are 0 through 7:
 #   0: Log only critical information

RNS/Utilities/rnstatus.py

@@ -46,7 +46,7 @@ def size_str(num, suffix='B'):
 
     return "%.2f%s%s" % (num, last_unit, suffix)
 
-def program_setup(configdir, dispall=False, verbosity=0, name_filter=None,json=False):
+def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=False, astats=False, sorting=None, sort_reverse=False):
     reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
 
     stats = None
@@ -62,16 +62,38 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None,json=F
                 if isinstance(stats[s], bytes):
                     stats[s] = RNS.hexrep(stats[s], delimit=False)
 
-                for i in stats[s]:
-                    if isinstance(i, dict):
-                        for k in i:
-                            if isinstance(i[k], bytes):
-                                i[k] = RNS.hexrep(i[k], delimit=False)
+                if isinstance(stats[s], dict):
+                    for i in stats[s]:
+                        if isinstance(i, dict):
+                            for k in i:
+                                if isinstance(i[k], bytes):
+                                    i[k] = RNS.hexrep(i[k], delimit=False)
 
             print(json.dumps(stats))
             exit()
+
+        interfaces = stats["interfaces"]
+        if sorting != None and isinstance(sorting, str):
+            sorting = sorting.lower()
+            if sorting == "rate" or sorting == "bitrate":
+                interfaces.sort(key=lambda i: i["bitrate"], reverse=not sort_reverse)
+            if sorting == "rx":
+                interfaces.sort(key=lambda i: i["rxb"], reverse=not sort_reverse)
+            if sorting == "tx":
+                interfaces.sort(key=lambda i: i["txb"], reverse=not sort_reverse)
+            if sorting == "traffic":
+                interfaces.sort(key=lambda i: i["rxb"]+i["txb"], reverse=not sort_reverse)
+            if sorting == "announces" or sorting == "announce":
+                interfaces.sort(key=lambda i: i["incoming_announce_frequency"]+i["outgoing_announce_frequency"], reverse=not sort_reverse)
+            if sorting == "arx":
+                interfaces.sort(key=lambda i: i["incoming_announce_frequency"], reverse=not sort_reverse)
+            if sorting == "atx":
+                interfaces.sort(key=lambda i: i["outgoing_announce_frequency"], reverse=not sort_reverse)
+            if sorting == "held":
+                interfaces.sort(key=lambda i: i["held_announces"], reverse=not sort_reverse)
+
             
-        for ifstat in stats["interfaces"]:
+        for ifstat in interfaces:
             name = ifstat["name"]
 
             if dispall or not (
@@ -113,7 +135,7 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None,json=F
                                 else:
                                     spec_str = " programs"
 
-                                clients_string = "Serving : "+str(cnum)+spec_str
+                                clients_string = "Serving   : "+str(cnum)+spec_str
                             elif name.startswith("I2PInterface["):
                                 if "i2p_connectable" in ifstat and ifstat["i2p_connectable"] == True:
                                     cnum = clients
@@ -122,11 +144,11 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None,json=F
                                     else:
                                         spec_str = " connected I2P endpoints"
 
-                                    clients_string = "Peers   : "+str(cnum)+spec_str
+                                    clients_string = "Peers     : "+str(cnum)+spec_str
                                 else:
                                     clients_string = ""
                             else:
-                                clients_string = "Clients : "+str(clients)
+                                clients_string = "Clients   : "+str(clients)
 
                         else:
                             clients = None
@@ -134,43 +156,63 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None,json=F
                         print(" {n}".format(n=ifstat["name"]))
 
                         if "ifac_netname" in ifstat and ifstat["ifac_netname"] != None:
-                            print("    Network : {nn}".format(nn=ifstat["ifac_netname"]))
+                            print("    Network   : {nn}".format(nn=ifstat["ifac_netname"]))
 
-                        print("    Status  : {ss}".format(ss=ss))
+                        print("    Status    : {ss}".format(ss=ss))
 
                         if clients != None and clients_string != "":
                             print("    "+clients_string)
 
                         if not (name.startswith("Shared Instance[") or name.startswith("TCPInterface[Client") or name.startswith("LocalInterface[")):
-                            print("    Mode    : {mode}".format(mode=modestr))
+                            print("    Mode      : {mode}".format(mode=modestr))
 
                         if "bitrate" in ifstat and ifstat["bitrate"] != None:
-                            print("    Rate    : {ss}".format(ss=speed_str(ifstat["bitrate"])))
+                            print("    Rate      : {ss}".format(ss=speed_str(ifstat["bitrate"])))
+
+                        if "airtime_short" in ifstat and "airtime_long" in ifstat:
+                            print("    Airtime   : {ats}% (15s), {atl}% (1h)".format(ats=str(ifstat["airtime_short"]),atl=str(ifstat["airtime_long"])))
+                        
+                        if "channel_load_short" in ifstat and "channel_load_long" in ifstat:
+                            print("    Ch.Load   : {ats}% (15s), {atl}% (1h)".format(ats=str(ifstat["channel_load_short"]),atl=str(ifstat["channel_load_long"])))
                         
                         if "peers" in ifstat and ifstat["peers"] != None:
-                            print("    Peers   : {np} reachable".format(np=ifstat["peers"]))
+                            print("    Peers     : {np} reachable".format(np=ifstat["peers"]))
 
                         if "tunnelstate" in ifstat and ifstat["tunnelstate"] != None:
-                            print("    I2P     : {ts}".format(ts=ifstat["tunnelstate"]))
+                            print("    I2P       : {ts}".format(ts=ifstat["tunnelstate"]))
 
                         if "ifac_signature" in ifstat and ifstat["ifac_signature"] != None:
                             sigstr = "<…"+RNS.hexrep(ifstat["ifac_signature"][-5:], delimit=False)+">"
-                            print("    Access  : {nb}-bit IFAC by {sig}".format(nb=ifstat["ifac_size"]*8, sig=sigstr))
+                            print("    Access    : {nb}-bit IFAC by {sig}".format(nb=ifstat["ifac_size"]*8, sig=sigstr))
                         
                         if "i2p_b32" in ifstat and ifstat["i2p_b32"] != None:
-                            print("    I2P B32 : {ep}".format(ep=str(ifstat["i2p_b32"])))
+                            print("    I2P B32   : {ep}".format(ep=str(ifstat["i2p_b32"])))
 
-                        if "announce_queue" in ifstat and ifstat["announce_queue"] != None and ifstat["announce_queue"] > 0:
+                        if astats and "announce_queue" in ifstat and ifstat["announce_queue"] != None and ifstat["announce_queue"] > 0:
                             aqn = ifstat["announce_queue"]
                             if aqn == 1:
-                                print("    Queued  : {np} announce".format(np=aqn))
+                                print("    Queued    : {np} announce".format(np=aqn))
                             else:
-                                print("    Queued  : {np} announces".format(np=aqn))
+                                print("    Queued    : {np} announces".format(np=aqn))
                         
-                        print("    Traffic : {txb}↑\n              {rxb}↓".format(rxb=size_str(ifstat["rxb"]), txb=size_str(ifstat["txb"])))
+                        if astats and "held_announces" in ifstat and ifstat["held_announces"] != None and ifstat["held_announces"] > 0:
+                            aqn = ifstat["held_announces"]
+                            if aqn == 1:
+                                print("    Held      : {np} announce".format(np=aqn))
+                            else:
+                                print("    Held      : {np} announces".format(np=aqn))
+                        
+                        if astats and "incoming_announce_frequency" in ifstat and ifstat["incoming_announce_frequency"] != None:
+                            print("    Announces : {iaf}↑".format(iaf=RNS.prettyfrequency(ifstat["outgoing_announce_frequency"])))
+                            print("                {iaf}↓".format(iaf=RNS.prettyfrequency(ifstat["incoming_announce_frequency"])))
+
+                        print("    Traffic   : {txb}↑\n                {rxb}↓".format(rxb=size_str(ifstat["rxb"]), txb=size_str(ifstat["txb"])))
 
         if "transport_id" in stats and stats["transport_id"] != None:
-            print("\n Reticulum Transport Instance "+RNS.prettyhexrep(stats["transport_id"])+" is running")
+            print("\n Transport Instance "+RNS.prettyhexrep(stats["transport_id"])+" running")
+            if "probe_responder" in stats and stats["probe_responder"] != None:
+                print(" Probe responder at "+RNS.prettyhexrep(stats["probe_responder"])+ " active")
+            print(" Uptime is "+RNS.prettytime(stats["transport_uptime"]))
 
         print("")
                 
@@ -191,6 +233,31 @@ def main():
             default=False
         )
         
+        parser.add_argument(
+            "-A",
+            "--announce-stats",
+            action="store_true",
+            help="show announce stats",
+            default=False
+        )
+        
+        parser.add_argument(
+            "-s",
+            "--sort",
+            action="store",
+            help="sort interfaces by [rate, traffic, rx, tx, announces, arx, atx, held]",
+            default=None,
+            type=str
+        )
+        
+        parser.add_argument(
+            "-r",
+            "--reverse",
+            action="store_true",
+            help="reverse sorting",
+            default=False,
+        )
+        
         parser.add_argument(
             "-j",
             "--json",
@@ -210,7 +277,16 @@ def main():
         else:
             configarg = None
 
-        program_setup(configdir = configarg, dispall = args.all, verbosity=args.verbose, name_filter=args.filter, json=args.json)
+        program_setup(
+            configdir = configarg,
+            dispall = args.all,
+            verbosity=args.verbose,
+            name_filter=args.filter,
+            json=args.json,
+            astats=args.announce_stats,
+            sorting=args.sort,
+            sort_reverse=args.reverse,
+        )
 
     except KeyboardInterrupt:
         print("")

RNS/__init__.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -38,6 +38,7 @@ from .Transport import Transport
 from .Destination import Destination
 from .Packet import Packet
 from .Packet import PacketReceipt
+from .Resolver import Resolver
 from .Resource import Resource, ResourceAdvertisement
 from .Cryptography import HKDF
 from .Cryptography import Hashes
@@ -105,7 +106,7 @@ def timestamp_str(time_s):
 
 def log(msg, level=3, _override_destination = False):
     global _always_override_destination, compact_log_fmt
-    
+    msg = str(msg)
     if loglevel >= level:
         if not compact_log_fmt:
             logstring = "["+timestamp_str(time.time())+"] ["+loglevelname(level)+"] "+msg
@@ -143,6 +144,12 @@ def rand():
     result = instance_random.random()
     return result
 
+def trace_exception(e):
+    import traceback
+    exception_info = "".join(traceback.TracebackException.from_exception(e).format())
+    log(f"An unhandled {str(type(e))} exception occurred: {str(e)}", LOG_ERROR)
+    log(exception_info, LOG_ERROR)
+
 def hexrep(data, delimit=True):
     try:
         iter(data)
@@ -160,6 +167,9 @@ def prettyhexrep(data):
     hexrep = "<"+delimiter.join("{:02x}".format(c) for c in data)+">"
     return hexrep
 
+def prettyspeed(num, suffix="b"):
+    return prettysize(num/8, suffix=suffix)+"ps"
+
 def prettysize(num, suffix='B'):
     units = ['','K','M','G','T','P','E','Z']
     last_unit = 'Y'
@@ -179,32 +189,68 @@ def prettysize(num, suffix='B'):
 
     return "%.2f%s%s" % (num, last_unit, suffix)
 
-def prettytime(time, verbose=False):
+def prettyfrequency(hz, suffix="Hz"):
+    num = hz*1e6
+    units = ["µ", "m", "", "K","M","G","T","P","E","Z"]
+    last_unit = "Y"
+
+    for unit in units:
+        if abs(num) < 1000.0:
+            return "%.2f %s%s" % (num, unit, suffix)
+        num /= 1000.0
+
+    return "%.2f%s%s" % (num, last_unit, suffix)
+
+def prettydistance(m, suffix="m"):
+    num = m*1e6
+    units = ["µ", "m", "c", ""]
+    last_unit = "K"
+
+    for unit in units:
+        divisor = 1000.0
+        if unit == "m": divisor = 10
+        if unit == "c": divisor = 100
+
+        if abs(num) < divisor:
+            return "%.2f %s%s" % (num, unit, suffix)
+        num /= divisor
+
+    return "%.2f %s%s" % (num, last_unit, suffix)
+
+def prettytime(time, verbose=False, compact=False):
     days = int(time // (24 * 3600))
     time = time % (24 * 3600)
     hours = int(time // 3600)
     time %= 3600
     minutes = int(time // 60)
     time %= 60
-    seconds = round(time, 2)
+    if compact:
+        seconds = int(time)
+    else:
+        seconds = round(time, 2)
     
     ss = "" if seconds == 1 else "s"
     sm = "" if minutes == 1 else "s"
     sh = "" if hours == 1 else "s"
     sd = "" if days == 1 else "s"
 
+    displayed = 0
     components = []
-    if days > 0:
+    if days > 0 and ((not compact) or displayed < 2):
         components.append(str(days)+" day"+sd if verbose else str(days)+"d")
+        displayed += 1
 
-    if hours > 0:
+    if hours > 0 and ((not compact) or displayed < 2):
         components.append(str(hours)+" hour"+sh if verbose else str(hours)+"h")
+        displayed += 1
 
-    if minutes > 0:
+    if minutes > 0 and ((not compact) or displayed < 2):
         components.append(str(minutes)+" minute"+sm if verbose else str(minutes)+"m")
+        displayed += 1
 
-    if seconds > 0:
+    if seconds > 0 and ((not compact) or displayed < 2):
         components.append(str(seconds)+" second"+ss if verbose else str(seconds)+"s")
+        displayed += 1
 
     i = 0
     tstr = ""
@@ -238,4 +284,4 @@ def panic():
 
 def exit():
     print("")
-    sys.exit(0)
+    sys.exit(0)

RNS/_version.py

@@ -1 +1 @@
-__version__ = "0.5.2"
+__version__ = "0.7.5"

RNS/vendor/ifaddr/niwrapper.py

@@ -11,6 +11,25 @@ def interfaces() -> List[str]:
     adapters = RNS.vendor.ifaddr.get_adapters(include_unconfigured=True)
     return [a.name for a in adapters]
 
+def interface_names_to_indexes() -> dict:
+    adapters = RNS.vendor.ifaddr.get_adapters(include_unconfigured=True)
+    results = {}
+    for adapter in adapters:
+        results[adapter.name] = adapter.index
+    return results
+
+def interface_name_to_nice_name(ifname) -> str:
+    try:
+        adapters = RNS.vendor.ifaddr.get_adapters(include_unconfigured=True)
+        for adapter in adapters:
+            if adapter.name == ifname:
+                if hasattr(adapter, "nice_name"):
+                    return adapter.nice_name
+    except:
+        return None
+
+    return None
+
 def ifaddresses(ifname) -> dict:
     adapters = RNS.vendor.ifaddr.get_adapters(include_unconfigured=True)
     ifa = {}

Roadmap.md

@@ -14,12 +14,14 @@ This document outlines the currently established development roadmap for Reticul
 ## Currently Active Work Areas
 For each release cycle of Reticulum, improvements and additions from the five [Primary Efforts](#primary-efforts) are selected as active work areas, and can be expected to be included in the upcoming releases within that cycle. While not entirely set in stone for each release cycle, they serve as a pointer of what to expect in the near future.
 
-- The current `0.5.x` release cycle aims at completing
+- The current `0.7.x` release cycle aims at completing
   - [ ] Overhauling and updating the documentation
-  - [ ] Performance and memory optimisations of the Python reference implementation
-  - [ ] Fixing potential bugs
+  - [ ] Distributed Destination Naming System
+  - [ ] Create a standalone RNS Daemon app for Android
+  - [ ] Network-wide path balancing
   - [ ] Add automatic retries to all use cases of the `Request` API
-  - [ ] Improve performance and efficiency of the `Buffer` and `Channel` API
+  - [ ] Performance and memory optimisations of the Python reference implementation
+  - [ ] Fixing bugs discovered while operating Reticulum systems and applications
 
 ## Primary Efforts
 The development path for Reticulum is currently laid out in five distinct areas: *Comprehensibility*, *Universality*, *Functionality*, *Usability & Utility* and *Interfaceability*. Conceptualising the development of Reticulum into these areas serves to advance the implementation and work towards the Foundational Goals & Values of Reticulum.
@@ -38,12 +40,12 @@ These efforts are aimed at improving the ease of which Reticulum is understood,
         - Update NomadNet screenshots
         - Update Sideband screenshots
     - Installation
-        - Add a *Reticulum On Raspberry Pi* section
-        - Update *Reticulum On Android* section if necessary
-        - Update Android install documentation.
+        - [x] Add a *Reticulum On Raspberry Pi* section
+        - [x] Update *Reticulum On Android* section if necessary
+        - [x] Update Android install documentation.
     - Communications hardware section
         - Add information about RNode external displays.
-        - Packet radio modems.
+        - [x] Packet radio modems.
         - Possibly add other relevant types here as well.
     - Setup *Best Practices For...* / *Installation Examples* section.
         - Home or office (example)
@@ -73,7 +75,6 @@ These efforts aim to expand and improve the core functionality and reliability o
 ### Usability & Utility
 These effors seek to make Reticulum easier to use and operate, and to expand the utility of the stack on deployed systems.
 
-- Add bluetooth pairing code output to rnodeconf
 - Easy way to share interface configurations, see [#19](https://github.com/markqvist/Reticulum/discussions/19)
 - Transit traffic display in rnstatus
 - rnsconfig utility

docs/manual/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: c5670e7931d15fbef274e2eee17c3561
+config: e6cf914f5d96347d4f64d2e8bcefb841
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/manual/_sources/gettingstartedfast.rst.txt

@@ -70,7 +70,8 @@ Remote Shell
 
 The `rnsh <https://github.com/acehoss/rnsh>`_ program lets you establish fully interactive
 remote shell sessions over Reticulum. It also allows you to pipe any program to or from a
-remote system, and is similar to how ``ssh`` works.
+remote system, and is similar to how ``ssh`` works. The ``rnsh`` is very efficient, and
+can facilitate fully interactive shell sessions, even over extremely low-bandwidth links.
 
 Nomad Network
 ^^^^^^^^^^^^^
@@ -179,29 +180,29 @@ and :ref:`Interfaces<interfaces-main>` chapters of this manual.
 Connecting Reticulum Instances Over the Internet
 ================================================
 Reticulum currently offers two interfaces suitable for connecting instances over the Internet: :ref:`TCP<interfaces-tcps>`
-and :ref:`I2P<interfaces-i2p>`. Each interface offers a different set of features, and Reticulum 
-users should carefully choose the interface which best suites their needs. 
+and :ref:`I2P<interfaces-i2p>`. Each interface offers a different set of features, and Reticulum
+users should carefully choose the interface which best suites their needs.
 
 The ``TCPServerInterface`` allows users to host an instance accessible over TCP/IP. This
 method is generally faster, lower latency, and more energy efficient than using ``I2PInterface``,
 however it also leaks more data about the server host.
 
 TCP connections reveal the IP address of both your instance and the server to anyone who can
-inspect the connection. Someone could use this information to determine your location or identity. Adversaries 
+inspect the connection. Someone could use this information to determine your location or identity. Adversaries
 inspecting your packets may be able to record packet metadata like time of transmission and packet size.
 Even though Reticulum encrypts traffic, TCP does not, so an adversary may be able to use
 packet inspection to learn that a system is running Reticulum, and what other IP addresses connect to it.
 Hosting a publicly reachable instance over TCP also requires a publicly reachable IP address,
 which most Internet connections don't offer anymore.
 
-The ``I2PInterface`` routes messages through the `Invisible Internet Protocol 
+The ``I2PInterface`` routes messages through the `Invisible Internet Protocol
 (I2P) <https://geti2p.net/en/>`_. To use this interface, users must also run an I2P daemon in
-parallel to ``rnsd``. For always-on I2P nodes it is recommended to use `i2pd <https://i2pd.website/>`_. 
+parallel to ``rnsd``. For always-on I2P nodes it is recommended to use `i2pd <https://i2pd.website/>`_.
 
-By default, I2P will encrypt and mix all traffic sent over the Internet, and 
-hide both the sender and receiver Reticulum instance IP addresses. Running an I2P node 
+By default, I2P will encrypt and mix all traffic sent over the Internet, and
+hide both the sender and receiver Reticulum instance IP addresses. Running an I2P node
 will also relay other I2P user's encrypted packets, which will use extra
-bandwidth and compute power, but also makes timing attacks and other forms of 
+bandwidth and compute power, but also makes timing attacks and other forms of
 deep-packet-inspection much more difficult.
 
 I2P also allows users to host globally available Reticulum instances from non-public IP's and behind firewalls and NAT.
@@ -219,11 +220,11 @@ by adding one of the following interfaces to your ``.reticulum/config`` file:
 
 .. code::
 
-  # TCP/IP interface to the Dublin hub
-  [[RNS Testnet Dublin]]
+  # TCP/IP interface to the RNS Amsterdam Hub
+  [[RNS Testnet Amsterdam]]
     type = TCPClientInterface
     enabled = yes
-    target_host = dublin.connect.reticulum.network
+    target_host = amsterdam.connect.reticulum.network
     target_port = 4965
 
   # TCP/IP interface to the BetweenTheBorders Hub (community-provided)
@@ -233,11 +234,11 @@ by adding one of the following interfaces to your ``.reticulum/config`` file:
     target_host = betweentheborders.com
     target_port = 4242
 
-  # Interface to I2P hub A
-  [[RNS Testnet I2P Hub A]]
+  # Interface to Testnet I2P Hub
+  [[RNS Testnet I2P Hub]]
     type = I2PInterface
     enabled = yes
-    peers = uxg5kubabakh3jtnvsipingbr5574dle7bubvip7llfvwx2tgrua.b32.i2p
+    peers = g3br23bvx3lq5uddcsjii74xgmn6y5q325ovrkq2zw2wbzbqgbuq.b32.i2p
 
 Many other Reticulum instances are connecting to this testnet, and you can also join it
 via other entry points if you know them. There is absolutely no control over the network
@@ -281,7 +282,7 @@ started is to install the latest release of Reticulum via pip:
 
 .. code::
 
-   pip3 install rns
+   pip install rns
 
 The above command will install Reticulum and dependencies, and you will be
 ready to import and use RNS in your own programs. The next step will most
@@ -291,7 +292,7 @@ For extended functionality, you can install optional dependencies:
 
 .. code::
 
-   pip3 install pyserial
+   pip install pyserial
 
 
 Further information can be found in the :ref:`API Reference<api-main>`.
@@ -306,7 +307,7 @@ don't use pip, but try this recipe:
 .. code::
 
     # Install dependencies
-    pip3 install cryptography pyserial
+    pip install cryptography pyserial
 
     # Clone repository
     git clone https://github.com/markqvist/Reticulum.git
@@ -316,25 +317,25 @@ don't use pip, but try this recipe:
     ln -s ../RNS ./Examples/
 
     # Run an example
-    python3 Examples/Echo.py -s
+    python Examples/Echo.py -s
 
     # Unless you've manually created a config file, Reticulum will do so now,
     # and immediately exit. Make any necessary changes to the file:
     nano ~/.reticulum/config
 
     # ... and launch the example again.
-    python3 Examples/Echo.py -s
+    python Examples/Echo.py -s
 
     # You can now repeat the process on another computer,
     # and run the same example with -h to get command line options.
-    python3 Examples/Echo.py -h
+    python Examples/Echo.py -h
 
     # Run the example in client mode to "ping" the server.
     # Replace the hash below with the actual destination hash of your server.
-    python3 Examples/Echo.py 174a64852a75682259ad8b921b8bf416
+    python Examples/Echo.py 174a64852a75682259ad8b921b8bf416
 
     # Have a look at another example
-    python3 Examples/Filetransfer.py -h
+    python Examples/Filetransfer.py -h
 
 When you have experimented with the basic examples, it's time to go read the
 :ref:`Understanding Reticulum<understanding-main>` chapter. Before submitting
@@ -414,7 +415,7 @@ locally on your device using the following command:
 
 It is also possible to include Reticulum in apps compiled and distributed as
 Android APKs. A detailed tutorial and example source code will be included
-here at a later point. Until then you can use the `Sideband source code <https://github.com/markqvist/sideband>`_ as an example and startig point.
+here at a later point. Until then you can use the `Sideband source code <https://github.com/markqvist/sideband>`_ as an example and starting point.
 
 
 ARM64
@@ -446,15 +447,15 @@ detailed in this manual.
 
 Debian Bookworm
 ^^^^^^^^^^^^^^^^^^^^^^^^
-On versions of Debian released after April 2023, it is no longer possible
+On versions of Debian released after April 2023, it is no longer possible by default
 to use ``pip`` to install packages onto your system. Unfortunately, you will need to
 use the replacement ``pipx`` command instead, which places installed packages in an
-isolated environment. This should not negatively affect Reticulum, but installation
-via this method is not fully tested yet.
+isolated environment. This should not negatively affect Reticulum, but will not work
+for including and using Reticulum in your own scripts and programs.
 
 .. code::
 
-    # Install pipx 
+    # Install pipx
     sudo apt install pipx
 
     # Make installed programs available on the command line
@@ -463,18 +464,32 @@ via this method is not fully tested yet.
     # Install Reticulum
     pipx install rns
 
+Alternatively, you can restore normal behaviour to ``pip`` by creating or editing
+the configuration file located at ``~/.config/pip/pip.conf``, and adding the
+following section:
+
+.. code:: text
+
+    [global]
+    break-system-packages = true
+
+Please note that the "break-system-packages" directive is a somewhat misleading choice
+of words. Setting it will of course not break any system packages, but will simply
+allow installing ``pip`` packages user- and system-wide. While this *could* in rare
+cases lead to version conflicts, it does not generally pose any problems.
+
 
 Ubuntu Lunar
 ^^^^^^^^^^^^^^^^^^^^^^^^
-On versions of Ubuntu released after April 2023, it is no longer possible
+On versions of Ubuntu released after April 2023, it is no longer possible by default
 to use ``pip`` to install packages onto your system. Unfortunately, you will need to
 use the replacement ``pipx`` command instead, which places installed packages in an
-isolated environment. This should not negatively affect Reticulum, but installation
-via this method is not fully tested yet.
+isolated environment. This should not negatively affect Reticulum, but will not work
+for including and using Reticulum in your own scripts and programs.
 
 .. code::
 
-    # Install pipx 
+    # Install pipx
     sudo apt install pipx
 
     # Make installed programs available on the command line
@@ -483,14 +498,24 @@ via this method is not fully tested yet.
     # Install Reticulum
     pipx install rns
 
+Alternatively, you can restore normal behaviour to ``pip`` by creating or editing
+the configuration file located at ``~/.config/pip/pip.conf``, and adding the
+following section:
+
+.. code:: text
+
+    [global]
+    break-system-packages = true
+
+Please note that the "break-system-packages" directive is a somewhat misleading choice
+of words. Setting it will of course not break any system packages, but will simply
+allow installing ``pip`` packages user- and system-wide. While this _could_ in rare
+cases lead to version conflicts, it does not generally pose any problems.
 
 Pure-Python Reticulum
 ==============================================
 In some rare cases, and on more obscure system types, it is not possible to
-install one or more dependencies
-
-On more unusual systems, and in some rare cases, it might not be possible to
-install or even compile one or more of the above modules. In such situations,
+install one or more dependencies. In such situations,
 you can use the ``rnspure`` package instead of the ``rns`` package, or use ``pip``
 with the ``--no-dependencies`` command-line option. The ``rnspure``
 package requires no external dependencies for installation. Please note that the

docs/manual/_sources/hardware.rst.txt

@@ -24,11 +24,20 @@ starting from scratch.
 This chapter will outline a few different sensible starting paths to get
 real-world functional wireless communications up and running with minimal cost
 and effort. Two fundamental devices categories will be covered, *RNodes* and
-*WiFi-based radios*.
+*WiFi-based radios*. Additionally, other common options will be briefly described.
+
+Knowing how to employ just a few different types of hardware will make it possible
+to build a wide range of useful networks with little effort.
+
+Combining Hardware Types
+========================
+
+It is useful to combine different link and hardware types when designing and
+building a network. One useful design pattern is to employ high-capacity point-to-point
+links based on WiFi or millimeter-wave radios (with high-gain directional antennas)
+for the network backbone, and using LoRa-based RNodes for covering large areas with
+connectivity for client devices.
 
-While there are many other device categories that are useful in building Reticulum
-networks, knowing how to employ just these two will make it possible to build
-a wide range of useful networks with little effort.
 
 .. _rnode-main:
 
@@ -190,13 +199,6 @@ such as serial port and on-air parameters. For v2.x firmwares, you just need to
 the Connection ID of the RNode, and Reticulum will automatically locate and connect to the
 RNode, using the parameters stored in the RNode itself.
 
-.. _rnode-suppliers:
-
-Suppliers
-^^^^^^^^^
-Get in touch if you want to have your RNode supplier listed here, or if you want help to
-get started with producing RNodes.
-
 
 WiFi-based Hardware
 ===================
@@ -231,11 +233,31 @@ that is relatively cheap while providing long range and high capacity for Reticu
 networks. As in all other cases, it is also possible for Reticulum to co-exist with IP
 networks running concurrently on such devices.
 
-Combining Hardware Types
-========================
+Ethernet-based Hardware
+=======================
 
-It is useful to combine different link and hardware types when designing and
-building a network. One useful design pattern is to employ high-capacity point-to-point
-links based on WiFi or millimeter-wave radios (with high-gain directional antennas)
-for the network backbone, and using LoRa-based RNodes for covering large areas with
-connectivity for client devices.
+Reticulum can run over any kind of hardware that can provide a switched Ethernet-based
+medium. This means that anything from a plain Ethernet switch, to fiber-optic systems,
+to data radios with Ethernet interfaces can be used by Reticulum.
+
+The Ethernet medium does not need to have any IP infrastructure such as DHCP servers
+or routing set up, but in case such infrastructure does exist, Reticulum will simply
+co-exist with.
+
+To use Reticulum over Ethernet-based mediums, it is generally enough to use the included
+:ref:`AutoInterface<interfaces-auto>`. This interface also works over any kind of
+virtual networking adapter, such as ``tun`` and ``tap`` devices in Linux.
+
+Serial Lines & Devices
+======================
+
+Using Reticulum over any kind of raw serial line is also possible with the
+:ref:`SerialInterface<interfaces-serial>`. This interface type is also useful for
+using Reticulum over communications hardware that provides a serial port interface.
+
+Packet Radio Modems
+===================
+
+Any packet radio modem that provides a standard KISS interface over USB, serial or TCP
+can be used with Reticulum. This includes virtual software modems such as
+`FreeDV TNC <https://github.com/xssfox/freedv-tnc>`_ and `Dire Wolf <https://github.com/wb2osz/direwolf>`_.

docs/manual/_sources/interfaces.rst.txt

@@ -1,9 +1,9 @@
 
 .. _interfaces-main:
 
-********************
-Supported Interfaces
-********************
+**********************
+Configuring Interfaces
+**********************
 
 Reticulum supports using many kinds of devices as networking interfaces, and
 allows you to mix and match them in any way you choose. The number of distinct
@@ -33,9 +33,20 @@ system, which should be enabled by default in almost all OSes.
 
 .. code::
 
-  # This example demonstrates a TCP server interface.
-  # It will listen for incoming connections on the
-  # specified IP address and port number.
+  # This example demonstrates a bare-minimum setup
+  # of an Auto Interface. It will allow communica-
+  # tion with all other reachable devices on all
+  # usable physical ethernet-based devices that
+  # are available on the system.
+
+  [[Default Interface]]
+    type = AutoInterface
+    interface_enabled = True
+
+  # This example demonstrates an more specifically
+  # configured Auto Interface, that only uses spe-
+  # cific physical interfaces, and has a number of
+  # other configuration options set.
   
   [[Default Interface]]
     type = AutoInterface
@@ -47,6 +58,12 @@ system, which should be enabled by default in almost all OSes.
 
     group_id = reticulum
 
+    # You can also choose the multicast address type:
+    # temporary (default, Temporary Multicast Address)
+    # or permanent (Permanent Multicast Address)
+
+    multicast_address_type = permanent
+
     # You can also select specifically which
     # kernel networking devices to use.
 
@@ -365,6 +382,7 @@ can be used, and offers full control over LoRa parameters.
     # out identification on the channel with
     # a set interval by configuring the
     # following two parameters.
+    
     # id_callsign = MYCALL-0
     # id_interval = 600
 
@@ -372,7 +390,21 @@ can be used, and offers full control over LoRa parameters.
     # with low amounts of RAM, using packet
     # flow control can be useful. By default
     # it is disabled.
-    flow_control = False
+    
+    # flow_control = False
+
+    # It is possible to limit the airtime
+    # utilisation of an RNode by using the
+    # following two configuration options.
+    # The short-term limit is applied in a
+    # window of approximately 15 seconds,
+    # and the long-term limit is enforced
+    # over a rolling 60 minute window. Both
+    # options are specified in percent.
+    
+    # airtime_limit_long = 1.5
+    # airtime_limit_short = 33
+
 
 .. _interfaces-serial:
 
@@ -746,3 +778,74 @@ conserve bandwidth, while very fast networks can support applications that
 need very frequent announces. Reticulum implements these mechanisms to ensure
 that a large span of network types can seamlessly *co-exist* and interconnect.
 
+.. _interfaces-ingress-control:
+
+New Destination Rate Limiting
+=============================
+
+On public interfaces, where anyone may connect and announce new destinations,
+it can be useful to control the rate at which announces for *new*  destinations are
+processed.
+
+If a large influx of announces for newly created or previously unknown destinations
+occur within a short amount of time, Reticulum will place these announces on hold,
+so that announce traffic for known and previously established destinations can
+continue to be processed without interruptions.
+
+After the burst subsides, and an additional waiting period has passed, the held
+announces will be released at a slow rate, until the hold queue is cleared. This
+also means, that should a node decide to connect to a public interface, announce
+a large amount of bogus destinations, and then disconnect, these destination will
+never make it into path tables and waste network bandwidth on retransmitted
+announces.
+
+**It's important to note** that the ingress control works at the level of *individual
+sub-interfaces*. As an example, this means that one client on a :ref:`TCP Server Interface<interfaces-tcps>`
+cannot disrupt processing of incoming announces for other connected clients on the same
+:ref:`TCP Server Interface<interfaces-tcps>`. All other clients on the same interface will still have new announces
+processed without interruption.
+
+By default, Reticulum will handle this automatically, and ingress announce
+control will be enabled on interface where it is sensible to do so. It should
+generally not be neccessary to modify the ingress control configuration,
+but all the parameters are exposed for configuration if needed.
+
+ * | The ``ingress_control`` option tells Reticulum whether or not
+     to enable announce ingress control on the interface. Defaults to
+     ``True``.
+
+ * | The ``ic_new_time`` option configures how long (in seconds) an
+     interface is considered newly spawned. Defaults to ``2*60*60`` seconds. This
+     option is useful on publicly accessible interfaces that spawn new
+     sub-interfaces when a new client connects. 
+
+ * | The ``ic_burst_freq_new`` option sets the maximum announce ingress
+     frequency for newly spawned interfaces. Defaults to ``3.5``
+     announces per second.
+
+ * | The ``ic_burst_freq`` option sets the maximum announce ingress
+     frequency for other interfaces. Defaults to ``12`` announces
+     per second.
+
+     *If an interface exceeds its burst frequency, incoming announces
+     for unknown destinations will be temporarily held in a queue, and
+     not processed until later.*
+
+ * | The ``ic_max_held_announces`` option sets the maximum amount of
+     unique announces that will be held in the queue. Any additional
+     unique announces will be dropped. Defaults to ``256`` announces.
+
+ * | The ``ic_burst_hold`` option sets how much time (in seconds) must
+     pass after the burst frequency drops below its threshold, for the
+     announce burst to be considered cleared. Defaults to ``60``
+     seconds.
+
+ * | The ``ic_burst_penalty`` option sets how much time (in seconds) must
+     pass after the burst is considered cleared, before held announces can
+     start being released from the queue. Defaults to ``5*60``
+     seconds.
+
+ * | The ``ic_held_release_interval`` option sets how much time (in seconds)
+     must pass between releasing each held announce from the queue. Defaults
+     to ``30`` seconds.
+

docs/manual/_sources/understanding.rst.txt

@@ -75,7 +75,7 @@ guide the design of Reticulum:
     it can be easily modified and replicated by anyone interested in doing so.
 * **Very low bandwidth requirements**
     Reticulum should be able to function reliably over links with a transmission capacity as low
-    as *500 bits per second*.
+    as *5 bits per second*.
 * **Encryption by default**
     Reticulum must use strong encryption by default for all communication.
 * **Initiator Anonymity**
@@ -595,7 +595,7 @@ or less any medium that allows you to send and receive data, which satisfies som
 minimum requirements.
 
 The communication channel must support at least half-duplex operation, and provide an average
-throughput of around 500 bits per second, and supports a physical layer MTU of 500 bytes. The
+throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The
 Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x 
 runtime environment.
 
@@ -858,7 +858,7 @@ both on general-purpose CPUs and on microcontrollers. The necessary primitives a
 
 * Ed25519 for signatures
 
-* X22519 for ECDH key exchanges
+* X25519 for ECDH key exchanges
 
 * HKDF for key derivation
 

docs/manual/_sources/using.rst.txt

@@ -99,6 +99,17 @@ configuration file is created. The default configuration looks like this:
   instance_control_port = 37429
 
 
+  # On systems where running instances may not have access
+  # to the same shared Reticulum configuration directory,
+  # it is still possible to allow full interactivity for
+  # running instances, by manually specifying a shared RPC
+  # key. In almost all cases, this option is not needed, but
+  # it can be useful on operating systems such as Android.
+  # The key must be specified as bytes in hexadecimal.
+
+  # rpc_key = e5c032d3ec4e64a6aca9927ba8ab73336780f6d71790
+
+
   # You can configure Reticulum to panic and forcibly close
   # if an unrecoverable interface error occurs, such as the
   # hardware device for an interface disappearing. This is
@@ -108,6 +119,17 @@ configuration file is created. The default configuration looks like this:
   panic_on_interface_error = No
 
 
+  # When Transport is enabled, it is possible to allow the
+  # Transport Instance to respond to probe requests from
+  # the rnprobe utility. This can be a useful tool to test
+  # connectivity. When this option is enabled, the probe
+  # destination will be generated from the Identity of the
+  # Transport Instance, and printed to the log at startup.
+  # Optional, and disabled by default.
+
+  respond_to_probes = No
+
+
   [logging]
   # Valid log levels are 0 through 7:
   #   0: Log only critical information
@@ -145,10 +167,19 @@ configuration file is created. The default configuration looks like this:
 If Reticulum infrastructure already exists locally, you probably don't need to
 change anything, and you may already be connected to a wider network. If not,
 you will probably need to add relevant *interfaces* to the configuration, in
-order to communicate with other systems. It is a good idea to read the comments
-and explanations in the above default config. It will teach you the basic
-concepts you need to understand to configure your network. Once you have done that,
-take a look at the :ref:`Interfaces<interfaces-main>` chapter of this manual.
+order to communicate with other systems.
+
+You can generate a much more verbose configuration example by running the command:
+
+``rnsd --exampleconfig``
+
+The output includes examples for most interface types supported
+by Reticulum, along with additional options and configuration parameters.
+
+It is a good idea to read the comments and explanations in the above default config.
+It will teach you the basic concepts you need to understand to configure your network.
+Once you have done that, take a look at the :ref:`Interfaces<interfaces-main>` chapter
+of this manual.
 
 Included Utility Programs
 -------------------------
@@ -170,28 +201,47 @@ When ``rnsd`` is running, it will keep all configured interfaces open, handle tr
 it is enabled, and allow any other programs to immediately utilise the
 Reticulum network it is configured for.
 
-You can even run multiple instances of rnsd with different configurations on
+You can even run multiple instances of ``rnsd`` with different configurations on
 the same system.
 
-.. code:: text
+**Usage Examples**
 
-  # Install Reticulum
-  pip3 install rns
-
-  # Run rnsd
-  rnsd
+Run ``rnsd``:
 
 .. code:: text
 
-  usage: rnsd [-h] [--config CONFIG] [-v] [-q] [--version]
+  $ rnsd
+
+  [2023-08-18 17:59:56] [Notice] Started rnsd version 0.5.8
+
+Run ``rnsd`` in service mode, ensuring all logging output is sent directly to file:
+
+.. code:: text
+
+  $ rnsd -s
+
+Generate a verbose and detailed configuration example, with explanations of all the
+various configuration options, and interface configuration examples:
+
+.. code:: text
+
+  $ rnsd --exampleconfig
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnsd.py [-h] [--config CONFIG] [-v] [-q] [-s] [--exampleconfig] [--version]
 
   Reticulum Network Stack Daemon
 
-  optional arguments:
+  options:
     -h, --help       show this help message and exit
     --config CONFIG  path to alternative Reticulum config directory
     -v, --verbose
     -q, --quiet
+    -s, --service    rnsd is running as a service and should log to file
+    --exampleconfig  print verbose configuration example to stdout and exit
     --version        show program's version number and exit
 
 You can easily add ``rnsd`` as an always-on service by :ref:`configuring a service<using-systemd>`.
@@ -202,12 +252,14 @@ The rnstatus Utility
 Using the ``rnstatus`` utility, you can view the status of configured Reticulum
 interfaces, similar to the ``ifconfig`` program.
 
+**Usage Examples**
+
+Run ``rnstatus``:
+
 .. code:: text
 
-  # Run rnstatus
-  rnstatus
+  $ rnstatus
 
-  # Example output
   Shared Instance[37428]
      Status  : Up
      Serving : 1 program
@@ -240,44 +292,175 @@ interfaces, similar to the ``ifconfig`` program.
 
   Reticulum Transport Instance <5245a8efe1788c6a1cd36144a270e13b> running
 
+Filter output to only show some interfaces:
+
 .. code:: text
 
-  usage: rnstatus [-h] [--config CONFIG] [--version] [-a] [-v]
+  $ rnstatus rnode
+
+  RNodeInterface[RNode UHF]
+     Status  : Up
+     Mode    : Access Point
+     Rate    : 1.30 kbps
+     Access  : 64-bit IFAC by <…e702c42ba8>
+     Traffic : 8.49 KB↑
+               9.23 KB↓
+
+  Reticulum Transport Instance <5245a8efe1788c6a1cd36144a270e13b> running
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnstatus.py [-h] [--config CONFIG] [--version] [-a] [-A] [-s SORT]
+                     [-r] [-j] [-v] [filter]
 
   Reticulum Network Stack Status
 
-  optional arguments:
-    -h, --help       show this help message and exit
-    --config CONFIG  path to alternative Reticulum config directory
-    --version        show program's version number and exit
-    -a, --all        show all interfaces
+  positional arguments:
+    filter                only display interfaces with names including filter
+
+  options:
+    -h, --help            show this help message and exit
+    --config CONFIG       path to alternative Reticulum config directory
+    --version             show program's version number and exit
+    -a, --all             show all interfaces
+    -A, --announce-stats  show announce stats
+    -s SORT, --sort SORT  sort interfaces by [rate, traffic, rx, tx, announces, arx, atx, held]
+    -r, --reverse         reverse sorting
+    -j, --json            output in JSON format
     -v, --verbose
 
 
+The rnid Utility
+====================
+
+With the ``rnid`` utility, you can generate, manage and view Reticulum Identities.
+The program can also calculate Destination hashes, and perform encryption and
+decryption of files.
+
+Using ``rnid``, it is possible to asymmetrically encrypt files and information for
+any Reticulum destination hash, and also to create and verify cryptographic signatures.
+
+**Usage Examples**
+
+Generate a new Identity:
+
+.. code:: text
+
+  $ rnid -g ./new_identity
+
+Display Identity key information:
+
+.. code:: text
+
+  $ rnid -i ./new_identity -p
+
+  Loaded Identity <984b74a3f768bef236af4371e6f248cd> from new_id
+  Public Key  : 0f4259fef4521ab75a3409e353fe9073eb10783b4912a6a9937c57bf44a62c1e
+  Private Key : Hidden
+
+Encrypt a file for an LXMF user:
+
+.. code:: text
+
+  $ rnid -i 8dd57a738226809646089335a6b03695 -e my_file.txt
+
+  Recalled Identity <bc7291552be7a58f361522990465165c> for destination <8dd57a738226809646089335a6b03695>
+  Encrypting my_file.txt
+  File my_file.txt encrypted for <bc7291552be7a58f361522990465165c> to my_file.txt.rfe
+
+If the Identity for the destination is not already known, you can fetch it from the network by using the ``-R`` command-line option:
+
+.. code:: text
+
+  $ rnid -R -i 30602def3b3506a28ed33db6f60cc6c9 -e my_file.txt
+
+  Requesting unknown Identity for <30602def3b3506a28ed33db6f60cc6c9>...
+  Received Identity <2b489d06eaf7c543808c76a5332a447d> for destination <30602def3b3506a28ed33db6f60cc6c9> from the network
+  Encrypting my_file.txt
+  File my_file.txt encrypted for <2b489d06eaf7c543808c76a5332a447d> to my_file.txt.rfe
+
+Decrypt a file using the Reticulum Identity it was encrypted for:
+
+.. code:: text
+
+  $ rnid -i ./my_identity -d my_file.txt.rfe
+
+  Loaded Identity <2225fdeecaf6e2db4556c3c2d7637294> from ./my_identity
+  Decrypting ./my_file.txt.rfe...
+  File ./my_file.txt.rfe decrypted with <2225fdeecaf6e2db4556c3c2d7637294> to ./my_file.txt
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnid.py [-h] [--config path] [-i identity] [-g path] [-v] [-q] [-a aspects]
+                 [-H aspects] [-e path] [-d path] [-s path] [-V path] [-r path] [-w path]
+                 [-f] [-R] [-t seconds] [-p] [-P] [--version]
+
+  Reticulum Identity & Encryption Utility
+
+  options:
+    -h, --help            show this help message and exit
+    --config path         path to alternative Reticulum config directory
+    -i identity, --identity identity
+                          hexadecimal Reticulum Destination hash or path to Identity file
+    -g path, --generate path
+                          generate a new Identity
+    -v, --verbose         increase verbosity
+    -q, --quiet           decrease verbosity
+    -a aspects, --announce aspects
+                          announce a destination based on this Identity
+    -H aspects, --hash aspects
+                          show destination hashes for other aspects for this Identity
+    -e path, --encrypt path
+                          encrypt file
+    -d path, --decrypt path
+                          decrypt file
+    -s path, --sign path  sign file
+    -V path, --validate path
+                          validate signature
+    -r path, --read path  input file path
+    -w path, --write path
+                          output file path
+    -f, --force           write output even if it overwrites existing files
+    -R, --request         request unknown Identities from the network
+    -t seconds            identity request timeout before giving up
+    -p, --print-identity  print identity info and exit
+    -P, --print-private   allow displaying private keys
+    --version             show program's version number and exit
+
+
 The rnpath Utility
 ====================
 
 With the ``rnpath`` utility, you can look up and view paths for
 destinations on the Reticulum network.
 
+**Usage Examples**
+
+Resolve path to a destination:
+
 .. code:: text
 
-  # Run rnpath
-  rnpath c89b4da064bf66d280f0e4d8abfd9806
+  $ rnpath c89b4da064bf66d280f0e4d8abfd9806
 
-  # Example output
   Path found, destination <c89b4da064bf66d280f0e4d8abfd9806> is 4 hops away via <f53a1c4278e0726bb73fcc623d6ce763> on TCPInterface[Testnet/dublin.connect.reticulum.network:4965]
 
+**All Command-Line Options**
+
 .. code:: text
 
-  usage: rnpath [-h] [--config CONFIG] [--version] [-t] [-r] [-d] [-D] [-w seconds] [-v] [destination]
-  
+  usage: rnpath.py [-h] [--config CONFIG] [--version] [-t] [-r] [-d] [-D]
+                   [-x] [-w seconds] [-v] [destination]
+
   Reticulum Path Discovery Utility
-  
+
   positional arguments:
     destination           hexadecimal hash of the destination
-  
-  optional arguments:
+
+  options:
     -h, --help            show this help message and exit
     --config CONFIG       path to alternative Reticulum config directory
     --version             show program's version number and exit
@@ -285,6 +468,7 @@ destinations on the Reticulum network.
     -r, --rates           show announce rate info
     -d, --drop            remove the path to a destination
     -D, --drop-announces  drop all queued announces
+    -x, --drop-via        drop all paths via specified transport instance
     -w seconds            timeout before giving up
     -v, --verbose
 
@@ -295,21 +479,53 @@ The rnprobe Utility
 The ``rnprobe`` utility lets you probe a destination for connectivity, similar
 to the ``ping`` program. Please note that probes will only be answered if the
 specified destination is configured to send proofs for received packets. Many
-destinations will not have this option enabled, and will not be probable.
+destinations will not have this option enabled, so most destinations will not
+be probable.
+
+You can enable a probe-reply destination on Reticulum Transport Instances by
+setting the ``respond_to_probes`` configuration directive. Reticulum will then
+print the probe destination to the log on Transport Instance startup.
+
+**Usage Examples**
+
+Probe a destination:
 
 .. code:: text
 
-  # Run rnprobe
-  rnprobe example_utilities.echo.request 2d03725b327348980d570f739a3a5708
+  $ rnprobe rnstransport.probe 2d03725b327348980d570f739a3a5708
 
-  # Example output
   Sent 16 byte probe to <2d03725b327348980d570f739a3a5708>
   Valid reply received from <2d03725b327348980d570f739a3a5708>
   Round-trip time is 38.469 milliseconds over 2 hops
 
+Send a larger probe:
+
 .. code:: text
 
-  usage: rnprobe [-h] [--config CONFIG] [--version] [-v] [full_name] [destination_hash]
+  $ rnprobe rnstransport.probe 2d03725b327348980d570f739a3a5708 -s 256
+
+  Sent 16 byte probe to <2d03725b327348980d570f739a3a5708>
+  Valid reply received from <2d03725b327348980d570f739a3a5708>
+  Round-trip time is 38.781 milliseconds over 2 hops
+
+If the interface that receives the probe replies supports reporting radio
+parameters such as **RSSI** and **SNR**, the ``rnprobe`` utility will print
+these as part of the result as well.
+
+.. code:: text
+
+  $ rnprobe rnstransport.probe e7536ee90bd4a440e130490b87a25124
+  
+  Sent 16 byte probe to <e7536ee90bd4a440e130490b87a25124>
+  Valid reply received from <e7536ee90bd4a440e130490b87a25124>
+  Round-trip time is 1.809 seconds over 1 hop [RSSI -73 dBm] [SNR 12.0 dB]
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnprobe [-h] [--config CONFIG] [--version] [-v] [-s SIZE]
+                 [full_name] [destination_hash]
 
   Reticulum Probe Utility
 
@@ -320,6 +536,7 @@ destinations will not have this option enabled, and will not be probable.
   optional arguments:
     -h, --help        show this help message and exit
     --config CONFIG   path to alternative Reticulum config directory
+    -s SIZE, --size SIZE  size of probe packet payload in bytes
     --version         show program's version number and exit
     -v, --verbose
 
@@ -330,20 +547,39 @@ The rncp Utility
 The ``rncp`` utility is a simple file transfer tool. Using it, you can transfer
 files through Reticulum.
 
+**Usage Examples**
+
+Run rncp on the receiving system, specifying which identities are allowed to send files:
+  
 .. code:: text
 
-  # Run rncp on the receiving system, specifying which identities
-  # are allowed to send files
-  rncp --receive -a 1726dbad538775b5bf9b0ea25a4079c8 -a c50cc4e4f7838b6c31f60ab9032cbc62
+  $ rncp --listen -a 1726dbad538775b5bf9b0ea25a4079c8 -a c50cc4e4f7838b6c31f60ab9032cbc62
 
-  # From another system, copy a file to the receiving system
-  rncp ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
-
-You can specify as many allowed senders as needed, or complete disable authentication.
+You can also specify allowed identity hashes (one per line) in the file ~/.rncp/allowed_identities
+and simply running the program in listener mode:
 
 .. code:: text
 
-  usage: rncp [-h] [--config path] [-v] [-q] [-p] [-r] [-b] [-a allowed_hash] [-n] [-w seconds] [--version] [file] [destination]
+  $ rncp --listen
+
+From another system, copy a file to the receiving system:
+
+.. code:: text
+
+  $ rncp ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
+
+Or fetch a file from the remote system:
+
+.. code:: text
+
+  $ rncp --fetch ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rncp.py [-h] [--config path] [-v] [-q] [-S] [-l] [-f] [-b seconds]
+                 [-a allowed_hash] [-n] [-p] [-w seconds] [--version] [file] [destination]
 
   Reticulum File Transfer Utility
 
@@ -351,19 +587,20 @@ You can specify as many allowed senders as needed, or complete disable authentic
     file                  file to be transferred
     destination           hexadecimal hash of the receiver
 
-  optional arguments:
+  options:
     -h, --help            show this help message and exit
     --config path         path to alternative Reticulum config directory
     -v, --verbose         increase verbosity
     -q, --quiet           decrease verbosity
-    -p, --print-identity  print identity and destination info and exit
-    -r, --receive         wait for incoming files
-    -b, --no-announce     don't announce at program start
+    -S, --silent          disable transfer progress output
+    -l, --listen          listen for incoming transfer requests
+    -f, --fetch           fetch file from remote listener instead of sending
+    -b seconds            announce interval, 0 to only announce at startup
     -a allowed_hash       accept from this identity
-    -n, --no-auth         accept files from anyone
+    -n, --no-auth         accept files and fetches from anyone
+    -p, --print-identity  print identity and destination info and exit
     -w seconds            sender timeout before giving up
     --version             show program's version number and exit
-    -v, --verbose
 
 
 The rnx Utility
@@ -371,32 +608,43 @@ The rnx Utility
 
 The ``rnx`` utility is a basic remote command execution program. It allows you to
 execute commands on remote systems over Reticulum, and to view returned command
-output.
+output. For a fully interactive remote shell solution, be sure to also take a look
+at the `rnsh <https://github.com/acehoss/rnsh>`_ program.
+
+**Usage Examples**
+
+Run rnx on the listening system, specifying which identities are allowed to execute commands:
 
 .. code:: text
 
-  # Run rnx on the listening system, specifying which identities
-  # are allowed to execute commands
-  rnx --listen -a 941bed5e228775e5a8079fc38b1ccf3f -a 1b03013c25f1c2ca068a4f080b844a10
+  $ rnx --listen -a 941bed5e228775e5a8079fc38b1ccf3f -a 1b03013c25f1c2ca068a4f080b844a10
 
-  # From another system, run a command 
-  rnx 7a55144adf826958a9529a3bcf08b149 "cat /proc/cpuinfo"
-
-  # Or enter the interactive mode pseudo-shell
-  rnx 7a55144adf826958a9529a3bcf08b149 -x
-
-  # The default identity file is stored in
-  # ~/.reticulum/identities/rnx, but you can use
-  # another one, which will be created if it does
-  # not already exist
-  rnx 7a55144adf826958a9529a3bcf08b149 -i /path/to/identity -x
-
-You can specify as many allowed senders as needed, or completely disable authentication.
+From another system, run a command on the remote:
 
 .. code:: text
 
-  usage: rnx [-h] [--config path] [-v] [-q] [-p] [-l] [-i identity] [-x] [-b] [-a allowed_hash] [-n] [-N] [-d] [-m] [-w seconds] [-W seconds] [--stdin STDIN] [--stdout STDOUT] [--stderr STDERR] [--version]
-             [destination] [command]
+  $ rnx 7a55144adf826958a9529a3bcf08b149 "cat /proc/cpuinfo"
+
+Or enter the interactive mode pseudo-shell:
+
+.. code:: text
+
+  $ rnx 7a55144adf826958a9529a3bcf08b149 -x
+
+The default identity file is stored in ``~/.reticulum/identities/rnx``, but you can use
+another one, which will be created if it does not already exist 
+
+.. code:: text
+
+  $ rnx 7a55144adf826958a9529a3bcf08b149 -i /path/to/identity -x
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnx [-h] [--config path] [-v] [-q] [-p] [-l] [-i identity] [-x] [-b] [-n] [-N]
+             [-d] [-m] [-a allowed_hash] [-w seconds] [-W seconds] [--stdin STDIN]
+             [--stdout STDOUT] [--stderr STDERR] [--version] [destination] [command]
 
   Reticulum Remote Execution Utility
 
@@ -433,11 +681,19 @@ The rnodeconf Utility
 The ``rnodeconf`` utility allows you to inspect and configure existing :ref:`RNodes<rnode-main>`, and
 to create and provision new :ref:`RNodes<rnode-main>` from any supported hardware devices.
 
+**All Command-Line Options**
+
 .. code:: text
 
-  usage: rnodeconf [-h] [-i] [-a] [-u] [-U] [--fw-version version] [--nocheck] [-C] [-N] [-T] [-b] [-B] [-p] [--freq Hz] [--bw Hz] [--txp dBm] [--sf factor] [--cr rate] [--eeprom-backup] [--eeprom-dump] [--eeprom-wipe] [--version] [port]
+  usage: rnodeconf.py [-h] [-i] [-a] [-u] [-U] [--fw-version version] [--nocheck] [-e]
+                      [-E] [-C] [--baud-flash baud_flash] [-N] [-T] [-b] [-B] [-p] [-D i]
+                      [--freq Hz] [--bw Hz] [--txp dBm] [--sf factor] [--cr rate]
+                      [--eeprom-backup] [--eeprom-dump] [--eeprom-wipe] [-P]
+                      [--trust-key hexbytes] [--version] [port]
 
-  RNode Configuration and firmware utility. This program allows you to change various settings and startup modes of RNode. It can also install, flash and update the firmware on supported devices.
+  RNode Configuration and firmware utility. This program allows you to change various
+  settings and startup modes of RNode. It can also install, flash and update the firmware
+  on supported devices.
 
   positional arguments:
     port                  serial port where RNode is attached
@@ -453,11 +709,14 @@ to create and provision new :ref:`RNodes<rnode-main>` from any supported hardwar
     -e, --extract         Extract firmware from connected RNode for later use
     -E, --use-extracted   Use the extracted firmware for autoinstallation or update
     -C, --clear-cache     Clear locally cached firmware files
+    --baud-flash baud_flash
+                          Set specific baud rate when flashing device. Default is 921600
     -N, --normal          Switch device to normal mode
     -T, --tnc             Switch device to TNC mode
     -b, --bluetooth-on    Turn device bluetooth on
     -B, --bluetooth-off   Turn device bluetooth off
     -p, --bluetooth-pair  Put device into bluetooth pairing mode
+    -D i, --display i     Set display intensity (0-255)
     --freq Hz             Frequency in Hz for TNC mode
     --bw Hz               Bandwidth in Hz for TNC mode
     --txp dBm             TX power in dBm for TNC mode
@@ -466,6 +725,8 @@ to create and provision new :ref:`RNodes<rnode-main>` from any supported hardwar
     --eeprom-backup       Backup EEPROM to file
     --eeprom-dump         Dump EEPROM to console
     --eeprom-wipe         Unlock and wipe EEPROM
+    -P, --public          Display public part of signing key
+    --trust-key hexbytes  Public key to trust for device verification
     --version             Print program version and exit
 
 For more information on how to create your own RNodes, please read the :ref:`Creating RNodes<rnode-creating>`
@@ -519,6 +780,9 @@ Reticulum as a System Service
 Instead of starting Reticulum manually, you can install ``rnsd`` as a system
 service and have it start automatically at boot.
 
+Systemwide Service
+^^^^^^^^^^^^^^^^^^
+
 If you installed Reticulum with ``pip``, the ``rnsd`` program will most likely
 be located in a user-local installation path only, which means ``systemd`` will not
 be able to execute it. In this case, you can simply symlink the ``rnsd`` program
@@ -566,3 +830,48 @@ If you want to automatically start ``rnsd`` at boot, run:
 .. code:: text
 
   sudo systemctl enable rnsd
+
+Userspace Service
+^^^^^^^^^^^^^^^^^
+
+Alternatively you can use a user systemd service instead of a system wide one. This way the whole setup can be done as a regular user.
+Create a user systemd service files ``~/.config/systemd/user/rnsd.service`` with the following content:
+
+.. code:: text
+
+  [Unit]
+  Description=Reticulum Network Stack Daemon
+  After=default.target
+
+  [Service]
+  # If you run Reticulum on WiFi devices,
+  # or other devices that need some extra
+  # time to initialise, you might want to
+  # add a short delay before Reticulum is
+  # started by systemd:
+  # ExecStartPre=/bin/sleep 10
+  Type=simple
+  Restart=always
+  RestartSec=3
+  ExecStart=RNS_BIN_DIR/rnsd --service
+
+  [Install]
+  WantedBy=default.target
+
+Replace ``RNS_BIN_DIR`` with the path to your Reticulum binary directory (eg. /home/USERNAMEHERE/rns/bin).
+
+Start user service:
+
+.. code:: text
+
+  systemctl --user daemon-reload
+  systemctl --user start rnsd.service
+
+If you want to automatically start ``rnsd`` without having to log in as the USERNAMEHERE, do:
+
+.. code:: text
+  
+  sudo loginctl enable-linger USERNAMEHERE
+  systemctl --user enable rnsd.service
+  
+

docs/manual/_sources/whatis.rst.txt

@@ -21,15 +21,15 @@ networks, without any need for hierarchical or beaureucratic structures to contr
 or manage them, while ensuring individuals and communities full sovereignty
 over their own network segments.
 
-Reticulum is a complete networking stack, and does not need IP or higher
+Reticulum is a **complete networking stack**, and does not need IP or higher
 layers, although it is easy to utilise IP (with TCP or UDP) as the underlying
 carrier for Reticulum. It is therefore trivial to tunnel Reticulum over the
 Internet or private IP networks. Reticulum is built directly on cryptographic
 principles, allowing resilience and stable functionality in open and trustless
 networks.
 
-No kernel modules or drivers are required. Reticulum runs completely in
-userland, and can run on practically any system that runs Python 3. Reticulum
+No kernel modules or drivers are required. Reticulum can run completely in
+userland, and will run on practically any system that runs Python 3. Reticulum
 runs well even on small single-board computers like the Pi Zero.
 
 
@@ -38,7 +38,7 @@ Current Status
 **Please know!** Reticulum should currently be considered beta software. All core protocol
 features are implemented and functioning, but additions will probably occur as
 real-world use is explored. *There will be bugs*. The API and wire-format can be
-considered stable at the moment, but could change if absolutely warranted.
+considered complete and stable at the moment, but could change if absolutely warranted.
 
 
 What does Reticulum Offer?
@@ -71,7 +71,7 @@ What does Reticulum Offer?
 
 * Efficient link establishment
 
-  * Total bandwidth cost of setting up a link is only 3 packets, totalling 297 bytes
+  * Total cost of setting up an encrypted and verified link is only 3 packets, totalling 297 bytes
 
   * Low cost of keeping links open at only 0.44 bits per second
 
@@ -91,7 +91,7 @@ What does Reticulum Offer?
 Where can Reticulum be Used?
 ============================
 Over practically any medium that can support at least a half-duplex channel
-with 500 bits per second throughput, and an MTU of 500 bytes. Data radios,
+with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios,
 modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes,
 ad-hoc WiFi, free-space optical links and similar systems are all examples
 of the types of interfaces Reticulum was designed for.

docs/manual/_static/basic.css

@@ -236,16 +236,6 @@ div.body p, div.body dd, div.body li, div.body blockquote {
 a.headerlink {
     visibility: hidden;
 }
-a.brackets:before,
-span.brackets > a:before{
-    content: "[";
-}
-
-a.brackets:after,
-span.brackets > a:after {
-    content: "]";
-}
-
 
 h1:hover > a.headerlink,
 h2:hover > a.headerlink,
@@ -334,11 +324,15 @@ aside.sidebar {
 p.sidebar-title {
     font-weight: bold;
 }
+nav.contents,
+aside.topic,
 div.admonition, div.topic, blockquote {
     clear: left;
 }
 
 /* -- topics ---------------------------------------------------------------- */
+nav.contents,
+aside.topic,
 div.topic {
     border: 1px solid #ccc;
     padding: 7px;
@@ -377,6 +371,8 @@ div.body p.centered {
 
 div.sidebar > :last-child,
 aside.sidebar > :last-child,
+nav.contents > :last-child,
+aside.topic > :last-child,
 div.topic > :last-child,
 div.admonition > :last-child {
     margin-bottom: 0;
@@ -384,6 +380,8 @@ div.admonition > :last-child {
 
 div.sidebar::after,
 aside.sidebar::after,
+nav.contents::after,
+aside.topic::after,
 div.topic::after,
 div.admonition::after,
 blockquote::after {
@@ -608,19 +606,26 @@ ol.simple p,
 ul.simple p {
     margin-bottom: 0;
 }
-dl.footnote > dt,
-dl.citation > dt {
+aside.footnote > span,
+div.citation > span {
     float: left;
-    margin-right: 0.5em;
 }
-
-dl.footnote > dd,
-dl.citation > dd {
+aside.footnote > span:last-of-type,
+div.citation > span:last-of-type {
+  padding-right: 0.5em;
+}
+aside.footnote > p {
+  margin-left: 2em;
+}
+div.citation > p {
+  margin-left: 4em;
+}
+aside.footnote > p:last-of-type,
+div.citation > p:last-of-type {
     margin-bottom: 0em;
 }
-
-dl.footnote > dd:after,
-dl.citation > dd:after {
+aside.footnote > p:last-of-type:after,
+div.citation > p:last-of-type:after {
     content: "";
     clear: both;
 }
@@ -636,10 +641,6 @@ dl.field-list > dt {
     padding-left: 0.5em;
     padding-right: 5px;
 }
-dl.field-list > dt:after {
-    content: ":";
-}
-
 
 dl.field-list > dd {
     padding-left: 0.5em;

docs/manual/_static/copybutton.css

@@ -35,7 +35,8 @@ div.highlight  {
     position: relative;
 }
 
-.highlight:hover button.copybtn {
+/* Show the copybutton */
+.highlight:hover button.copybtn, button.copybtn.success {
 	opacity: 1;
 }
 

docs/manual/_static/copybutton.js

@@ -20,7 +20,7 @@ const messages = {
   },
   'fr' : {
     'copy': 'Copier',
-    'copy_to_clipboard': 'Copié dans le presse-papier',
+    'copy_to_clipboard': 'Copier dans le presse-papier',
     'copy_success': 'Copié !',
     'copy_failure': 'Échec de la copie',
   },
@@ -102,18 +102,25 @@ const clearSelection = () => {
   }
 }
 
-// Changes tooltip text for two seconds, then changes it back
+// Changes tooltip text for a moment, then changes it back
+// We want the timeout of our `success` class to be a bit shorter than the
+// tooltip and icon change, so that we can hide the icon before changing back.
+var timeoutIcon = 2000;
+var timeoutSuccessClass = 1500;
+
 const temporarilyChangeTooltip = (el, oldText, newText) => {
   el.setAttribute('data-tooltip', newText)
   el.classList.add('success')
-  setTimeout(() => el.setAttribute('data-tooltip', oldText), 2000)
-  setTimeout(() => el.classList.remove('success'), 2000)
+  // Remove success a little bit sooner than we change the tooltip
+  // So that we can use CSS to hide the copybutton first
+  setTimeout(() => el.classList.remove('success'), timeoutSuccessClass)
+  setTimeout(() => el.setAttribute('data-tooltip', oldText), timeoutIcon)
 }
 
 // Changes the copy button icon for two seconds, then changes it back
 const temporarilyChangeIcon = (el) => {
   el.innerHTML = iconCheck;
-  setTimeout(() => {el.innerHTML = iconCopy}, 2000)
+  setTimeout(() => {el.innerHTML = iconCopy}, timeoutIcon)
 }
 
 const addCopyButtonToCodeCells = () => {
@@ -125,7 +132,8 @@ const addCopyButtonToCodeCells = () => {
   }
 
   // Add copybuttons to all of our code cells
-  const codeCells = document.querySelectorAll('div.highlight pre')
+  const COPYBUTTON_SELECTOR = 'div.highlight pre';
+  const codeCells = document.querySelectorAll(COPYBUTTON_SELECTOR)
   codeCells.forEach((codeCell, index) => {
     const id = codeCellId(index)
     codeCell.setAttribute('id', id)
@@ -141,10 +149,25 @@ function escapeRegExp(string) {
     return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
 }
 
+/**
+ * Removes excluded text from a Node.
+ *
+ * @param {Node} target Node to filter.
+ * @param {string} exclude CSS selector of nodes to exclude.
+ * @returns {DOMString} Text from `target` with text removed.
+ */
+function filterText(target, exclude) {
+    const clone = target.cloneNode(true);  // clone as to not modify the live DOM
+    if (exclude) {
+        // remove excluded nodes
+        clone.querySelectorAll(exclude).forEach(node => node.remove());
+    }
+    return clone.innerText;
+}
+
 // Callback when a copy button is clicked. Will be passed the node that was clicked
 // should then grab the text and replace pieces of text that shouldn't be used in output
 function formatCopyText(textContent, copybuttonPromptText, isRegexp = false, onlyCopyPromptLines = true, removePrompts = true, copyEmptyLines = true, lineContinuationChar = "", hereDocDelim = "") {
-
     var regexp;
     var match;
 
@@ -199,7 +222,12 @@ function formatCopyText(textContent, copybuttonPromptText, isRegexp = false, onl
 
 var copyTargetText = (trigger) => {
   var target = document.querySelector(trigger.attributes['data-clipboard-target'].value);
-  return formatCopyText(target.innerText, '', false, true, true, true, '', '')
+
+  // get filtered text
+  let exclude = '.linenos';
+
+  let text = filterText(target, exclude);
+  return formatCopyText(text, '', false, true, true, true, '', '')
 }
 
   // Initialize with a callback so we can modify the text before copy

docs/manual/_static/copybutton_funcs.js

@@ -2,10 +2,25 @@ function escapeRegExp(string) {
     return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
 }
 
+/**
+ * Removes excluded text from a Node.
+ *
+ * @param {Node} target Node to filter.
+ * @param {string} exclude CSS selector of nodes to exclude.
+ * @returns {DOMString} Text from `target` with text removed.
+ */
+export function filterText(target, exclude) {
+    const clone = target.cloneNode(true);  // clone as to not modify the live DOM
+    if (exclude) {
+        // remove excluded nodes
+        clone.querySelectorAll(exclude).forEach(node => node.remove());
+    }
+    return clone.innerText;
+}
+
 // Callback when a copy button is clicked. Will be passed the node that was clicked
 // should then grab the text and replace pieces of text that shouldn't be used in output
 export function formatCopyText(textContent, copybuttonPromptText, isRegexp = false, onlyCopyPromptLines = true, removePrompts = true, copyEmptyLines = true, lineContinuationChar = "", hereDocDelim = "") {
-
     var regexp;
     var match;
 

docs/manual/_static/documentation_options.js

@@ -1,6 +1,6 @@
 var DOCUMENTATION_OPTIONS = {
     URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'),
-    VERSION: '0.5.2 beta',
+    VERSION: '0.7.5 beta',
     LANGUAGE: 'en',
     COLLAPSE_INDEX: false,
     BUILDER: 'html',

docs/manual/_static/pygments.css

@@ -22,6 +22,7 @@
 .highlight .cs { color: #8f5902; font-style: italic } /* Comment.Special */
 .highlight .gd { color: #a40000 } /* Generic.Deleted */
 .highlight .ge { color: #000000; font-style: italic } /* Generic.Emph */
+.highlight .ges { color: #000000; font-weight: bold; font-style: italic } /* Generic.EmphStrong */
 .highlight .gr { color: #ef2929 } /* Generic.Error */
 .highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
 .highlight .gi { color: #00A000 } /* Generic.Inserted */
@@ -101,12 +102,13 @@ body[data-theme="dark"] .highlight .x { color: #d0d0d0 } /* Other */
 body[data-theme="dark"] .highlight .p { color: #d0d0d0 } /* Punctuation */
 body[data-theme="dark"] .highlight .ch { color: #ababab; font-style: italic } /* Comment.Hashbang */
 body[data-theme="dark"] .highlight .cm { color: #ababab; font-style: italic } /* Comment.Multiline */
-body[data-theme="dark"] .highlight .cp { color: #cd2828; font-weight: bold } /* Comment.Preproc */
+body[data-theme="dark"] .highlight .cp { color: #ff3a3a; font-weight: bold } /* Comment.Preproc */
 body[data-theme="dark"] .highlight .cpf { color: #ababab; font-style: italic } /* Comment.PreprocFile */
 body[data-theme="dark"] .highlight .c1 { color: #ababab; font-style: italic } /* Comment.Single */
 body[data-theme="dark"] .highlight .cs { color: #e50808; font-weight: bold; background-color: #520000 } /* Comment.Special */
 body[data-theme="dark"] .highlight .gd { color: #d22323 } /* Generic.Deleted */
 body[data-theme="dark"] .highlight .ge { color: #d0d0d0; font-style: italic } /* Generic.Emph */
+body[data-theme="dark"] .highlight .ges { color: #d0d0d0; font-weight: bold; font-style: italic } /* Generic.EmphStrong */
 body[data-theme="dark"] .highlight .gr { color: #d22323 } /* Generic.Error */
 body[data-theme="dark"] .highlight .gh { color: #ffffff; font-weight: bold } /* Generic.Heading */
 body[data-theme="dark"] .highlight .gi { color: #589819 } /* Generic.Inserted */
@@ -186,12 +188,13 @@ body:not([data-theme="light"]) .highlight .x { color: #d0d0d0 } /* Other */
 body:not([data-theme="light"]) .highlight .p { color: #d0d0d0 } /* Punctuation */
 body:not([data-theme="light"]) .highlight .ch { color: #ababab; font-style: italic } /* Comment.Hashbang */
 body:not([data-theme="light"]) .highlight .cm { color: #ababab; font-style: italic } /* Comment.Multiline */
-body:not([data-theme="light"]) .highlight .cp { color: #cd2828; font-weight: bold } /* Comment.Preproc */
+body:not([data-theme="light"]) .highlight .cp { color: #ff3a3a; font-weight: bold } /* Comment.Preproc */
 body:not([data-theme="light"]) .highlight .cpf { color: #ababab; font-style: italic } /* Comment.PreprocFile */
 body:not([data-theme="light"]) .highlight .c1 { color: #ababab; font-style: italic } /* Comment.Single */
 body:not([data-theme="light"]) .highlight .cs { color: #e50808; font-weight: bold; background-color: #520000 } /* Comment.Special */
 body:not([data-theme="light"]) .highlight .gd { color: #d22323 } /* Generic.Deleted */
 body:not([data-theme="light"]) .highlight .ge { color: #d0d0d0; font-style: italic } /* Generic.Emph */
+body:not([data-theme="light"]) .highlight .ges { color: #d0d0d0; font-weight: bold; font-style: italic } /* Generic.EmphStrong */
 body:not([data-theme="light"]) .highlight .gr { color: #d22323 } /* Generic.Error */
 body:not([data-theme="light"]) .highlight .gh { color: #ffffff; font-weight: bold } /* Generic.Heading */
 body:not([data-theme="light"]) .highlight .gi { color: #589819 } /* Generic.Inserted */

docs/manual/_static/scripts/furo.js.LICENSE.txt

@@ -1,7 +0,0 @@
-/*!
- * gumshoejs v5.1.2 (patched by @pradyunsg)
- * A simple, framework-agnostic scrollspy script.
- * (c) 2019 Chris Ferdinandi
- * MIT License
- * http://github.com/cferdinandi/gumshoe
- */

docs/manual/examples.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Support Reticulum" href="support.html" /><link rel="prev" title="Building Networks" href="networks.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Code Examples - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Code Examples - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -258,7 +258,7 @@ program.</p>
     <span class="c1"># Destinations are endpoints in Reticulum, that can be addressed</span>
     <span class="c1"># and communicated with. Destinations can also announce their</span>
     <span class="c1"># existence, which will let the network know they are reachable</span>
-    <span class="c1"># and autoomatically create paths to them, from anywhere else</span>
+    <span class="c1"># and automatically create paths to them, from anywhere else</span>
     <span class="c1"># in the network.</span>
     <span class="n">destination</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Destination</span><span class="p">(</span>
         <span class="n">identity</span><span class="p">,</span>
@@ -269,7 +269,7 @@ program.</p>
     <span class="p">)</span>
 
     <span class="c1"># We configure the destination to automatically prove all</span>
-    <span class="c1"># packets adressed to it. By doing this, RNS will automatically</span>
+    <span class="c1"># packets addressed to it. By doing this, RNS will automatically</span>
     <span class="c1"># generate a proof for each incoming packet and transmit it</span>
     <span class="c1"># back to the sender of that packet. This will let anyone that</span>
     <span class="c1"># tries to communicate with the destination know whether their</span>
@@ -375,7 +375,7 @@ notifications about announces from relevant destinations.</p>
     <span class="c1"># Destinations are endpoints in Reticulum, that can be addressed</span>
     <span class="c1"># and communicated with. Destinations can also announce their</span>
     <span class="c1"># existence, which will let the network know they are reachable</span>
-    <span class="c1"># and autoomatically create paths to them, from anywhere else</span>
+    <span class="c1"># and automatically create paths to them, from anywhere else</span>
     <span class="c1"># in the network.</span>
     <span class="n">destination_1</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Destination</span><span class="p">(</span>
         <span class="n">identity</span><span class="p">,</span>
@@ -396,7 +396,7 @@ notifications about announces from relevant destinations.</p>
     <span class="p">)</span>
 
     <span class="c1"># We configure the destinations to automatically prove all</span>
-    <span class="c1"># packets adressed to it. By doing this, RNS will automatically</span>
+    <span class="c1"># packets addressed to it. By doing this, RNS will automatically</span>
     <span class="c1"># generate a proof for each incoming packet and transmit it</span>
     <span class="c1"># back to the sender of that packet. This will let anyone that</span>
     <span class="c1"># tries to communicate with the destination know whether their</span>
@@ -656,6 +656,7 @@ the Packet interface.</p>
 <span class="c1"># of the packet.                                         #</span>
 <span class="c1">##########################################################</span>
 
+<span class="kn">import</span> <span class="nn">os</span>
 <span class="kn">import</span> <span class="nn">argparse</span>
 <span class="kn">import</span> <span class="nn">RNS</span>
 
@@ -678,8 +679,19 @@ the Packet interface.</p>
     <span class="c1"># We must first initialise Reticulum</span>
     <span class="n">reticulum</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Reticulum</span><span class="p">(</span><span class="n">configpath</span><span class="p">)</span>
     
-    <span class="c1"># Randomly create a new identity for our echo server</span>
-    <span class="n">server_identity</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Identity</span><span class="p">()</span>
+    <span class="c1"># Load identity from file if it exist or randomley create</span>
+    <span class="k">if</span> <span class="n">configpath</span><span class="p">:</span>
+        <span class="n">ifilepath</span> <span class="o">=</span> <span class="s2">&quot;</span><span class="si">%s</span><span class="s2">/storage/identitiesy/</span><span class="si">%s</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">configpath</span><span class="p">,</span><span class="n">APP_NAME</span><span class="p">)</span>
+    <span class="k">else</span><span class="p">:</span>
+        <span class="n">ifilepath</span> <span class="o">=</span> <span class="s2">&quot;</span><span class="si">%s</span><span class="s2">/storage/identities/</span><span class="si">%s</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">RNS</span><span class="o">.</span><span class="n">Reticulum</span><span class="o">.</span><span class="n">configdir</span><span class="p">,</span><span class="n">APP_NAME</span><span class="p">)</span>
+    <span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">ifilepath</span><span class="p">):</span>
+        <span class="c1"># Load identity from file</span>
+        <span class="n">server_identity</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Identity</span><span class="o">.</span><span class="n">from_file</span><span class="p">(</span><span class="n">ifilepath</span><span class="p">)</span>
+        <span class="n">RNS</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&quot;loaded identity from file: &quot;</span><span class="o">+</span><span class="n">ifilepath</span><span class="p">,</span> <span class="n">RNS</span><span class="o">.</span><span class="n">LOG_VERBOSE</span><span class="p">)</span>
+    <span class="k">else</span><span class="p">:</span>
+        <span class="c1"># Randomly create a new identity for our echo example</span>
+        <span class="n">server_identity</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Identity</span><span class="p">()</span>
+        <span class="n">RNS</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&quot;created new identity&quot;</span><span class="p">,</span> <span class="n">RNS</span><span class="o">.</span><span class="n">LOG_VERBOSE</span><span class="p">)</span>
 
     <span class="c1"># We create a destination that clients can query. We want</span>
     <span class="c1"># to be able to verify echo replies to our clients, so we</span>
@@ -697,7 +709,7 @@ the Packet interface.</p>
     <span class="p">)</span>
 
     <span class="c1"># We configure the destination to automatically prove all</span>
-    <span class="c1"># packets adressed to it. By doing this, RNS will automatically</span>
+    <span class="c1"># packets addressed to it. By doing this, RNS will automatically</span>
     <span class="c1"># generate a proof for each incoming packet and transmit it</span>
     <span class="c1"># back to the sender of that packet.</span>
     <span class="n">echo_destination</span><span class="o">.</span><span class="n">set_proof_strategy</span><span class="p">(</span><span class="n">RNS</span><span class="o">.</span><span class="n">Destination</span><span class="o">.</span><span class="n">PROVE_ALL</span><span class="p">)</span>
@@ -1018,8 +1030,20 @@ destination, and passing traffic back and forth over the link.</p>
     <span class="c1"># We must first initialise Reticulum</span>
     <span class="n">reticulum</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Reticulum</span><span class="p">(</span><span class="n">configpath</span><span class="p">)</span>
     
-    <span class="c1"># Randomly create a new identity for our link example</span>
-    <span class="n">server_identity</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Identity</span><span class="p">()</span>
+    <span class="c1"># Load identity from file if it exist or randomley create</span>
+    <span class="k">if</span> <span class="n">configpath</span><span class="p">:</span>
+        <span class="n">ifilepath</span> <span class="o">=</span> <span class="s2">&quot;</span><span class="si">%s</span><span class="s2">/storage/identitiesy/</span><span class="si">%s</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">configpath</span><span class="p">,</span><span class="n">APP_NAME</span><span class="p">)</span>
+    <span class="k">else</span><span class="p">:</span>
+        <span class="n">ifilepath</span> <span class="o">=</span> <span class="s2">&quot;</span><span class="si">%s</span><span class="s2">/storage/identities/</span><span class="si">%s</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">RNS</span><span class="o">.</span><span class="n">Reticulum</span><span class="o">.</span><span class="n">configdir</span><span class="p">,</span><span class="n">APP_NAME</span><span class="p">)</span>
+    <span class="n">RNS</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&quot;ifilepath: </span><span class="si">%s</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="n">ifilepath</span><span class="p">)</span>
+    <span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">ifilepath</span><span class="p">):</span>
+        <span class="c1"># Load identity from file</span>
+        <span class="n">server_identity</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Identity</span><span class="o">.</span><span class="n">from_file</span><span class="p">(</span><span class="n">ifilepath</span><span class="p">)</span>
+        <span class="n">RNS</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&quot;loaded identity from file: &quot;</span><span class="o">+</span><span class="n">ifilepath</span><span class="p">,</span> <span class="n">RNS</span><span class="o">.</span><span class="n">LOG_VERBOSE</span><span class="p">)</span>
+    <span class="k">else</span><span class="p">:</span>
+        <span class="c1"># Randomly create a new identity for our link example</span>
+        <span class="n">server_identity</span> <span class="o">=</span> <span class="n">RNS</span><span class="o">.</span><span class="n">Identity</span><span class="p">()</span>
+        <span class="n">RNS</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&quot;created new identity&quot;</span><span class="p">,</span> <span class="n">RNS</span><span class="o">.</span><span class="n">LOG_VERBOSE</span><span class="p">)</span>
 
     <span class="c1"># We create a destination that clients can connect to. We</span>
     <span class="c1"># want clients to create links to this destination, so we</span>
@@ -2064,7 +2088,7 @@ data between peers of a <code class="docutils literal notranslate"><span class="
     <span class="n">RNS</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&quot;Client disconnected&quot;</span><span class="p">)</span>
 
 <span class="k">def</span> <span class="nf">server_message_received</span><span class="p">(</span><span class="n">message</span><span class="p">):</span>
-    <span class="sd">&quot;&quot;&quot;</span>
+<span class="w">    </span><span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">    A message handler</span>
 <span class="sd">    @param message: An instance of a subclass of MessageBase</span>
 <span class="sd">    @return: True if message was handled</span>
@@ -2413,7 +2437,7 @@ binary data between peers of a <code class="docutils literal notranslate"><span
     <span class="n">RNS</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="s2">&quot;Client disconnected&quot;</span><span class="p">)</span>
 
 <span class="k">def</span> <span class="nf">server_buffer_ready</span><span class="p">(</span><span class="n">ready_bytes</span><span class="p">:</span> <span class="nb">int</span><span class="p">):</span>
-    <span class="sd">&quot;&quot;&quot;</span>
+<span class="w">    </span><span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">    Callback from buffer when buffer has data available</span>
 
 <span class="sd">    :param ready_bytes: The number of bytes ready to read</span>
@@ -3084,8 +3108,7 @@ interface to efficiently pass files of any size over a Reticulum <a class="refer
     <span class="c1"># And set up a small job to check for</span>
     <span class="c1"># a potential timeout in receiving the</span>
     <span class="c1"># file list</span>
-    <span class="n">thread</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">filelist_timeout_job</span><span class="p">)</span>
-    <span class="n">thread</span><span class="o">.</span><span class="n">setDaemon</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
+    <span class="n">thread</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">filelist_timeout_job</span><span class="p">,</span> <span class="n">daemon</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
     <span class="n">thread</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
 
 <span class="c1"># This job just sleeps for the specified</span>

docs/manual/forhumans.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>An Explanation of Reticulum for Human Beings - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>An Explanation of Reticulum for Human Beings - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>

docs/manual/genindex.html

@@ -4,9 +4,9 @@
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
     <meta name="color-scheme" content="light dark"><link rel="index" title="Index" href="#" /><link rel="search" title="Search" href="search.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/><title>Index - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/><title>Index - Reticulum Network Stack 0.7.5 beta documentation</title>
 <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -139,7 +139,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -165,10 +165,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -179,7 +179,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -373,6 +373,8 @@
         <li><a href="reference.html#RNS.Link.get_establishment_rate">get_establishment_rate() (RNS.Link method)</a>
 </li>
         <li><a href="reference.html#RNS.Resource.get_hash">get_hash() (RNS.Resource method)</a>
+</li>
+        <li><a href="reference.html#RNS.Reticulum.get_instance">get_instance() (RNS.Reticulum static method)</a>
 </li>
         <li><a href="reference.html#RNS.Resource.get_parts">get_parts() (RNS.Resource method)</a>
 </li>
@@ -389,6 +391,8 @@
 </li>
         </ul></li>
         <li><a href="reference.html#RNS.Identity.get_public_key">get_public_key() (RNS.Identity method)</a>
+</li>
+        <li><a href="reference.html#RNS.Link.get_q">get_q() (RNS.Link method)</a>
 </li>
     </ul></td>
     <td style="width: 33%; vertical-align: top;"><ul>
@@ -401,10 +405,14 @@
         <li><a href="reference.html#RNS.RequestReceipt.get_response">get_response() (RNS.RequestReceipt method)</a>
 </li>
         <li><a href="reference.html#RNS.RequestReceipt.get_response_time">get_response_time() (RNS.RequestReceipt method)</a>
+</li>
+        <li><a href="reference.html#RNS.Link.get_rssi">get_rssi() (RNS.Link method)</a>
 </li>
         <li><a href="reference.html#RNS.PacketReceipt.get_rtt">get_rtt() (RNS.PacketReceipt method)</a>
 </li>
         <li><a href="reference.html#RNS.Resource.get_segments">get_segments() (RNS.Resource method)</a>
+</li>
+        <li><a href="reference.html#RNS.Link.get_snr">get_snr() (RNS.Link method)</a>
 </li>
         <li><a href="reference.html#RNS.PacketReceipt.get_status">get_status() (RNS.PacketReceipt method)</a>
 
@@ -502,6 +510,8 @@
 </li>
     </ul></td>
     <td style="width: 33%; vertical-align: top;"><ul>
+        <li><a href="reference.html#RNS.Reticulum.MINIMUM_BITRATE">MINIMUM_BITRATE (RNS.Reticulum attribute)</a>
+</li>
         <li><a href="reference.html#RNS.MessageBase.MSGTYPE">MSGTYPE (RNS.MessageBase attribute)</a>
 </li>
         <li><a href="reference.html#RNS.Reticulum.MTU">MTU (RNS.Reticulum attribute)</a>
@@ -520,6 +530,8 @@
 </li>
     </ul></td>
     <td style="width: 33%; vertical-align: top;"><ul>
+        <li><a href="reference.html#RNS.Link.no_data_for">no_data_for() (RNS.Link method)</a>
+</li>
         <li><a href="reference.html#RNS.Link.no_inbound_for">no_inbound_for() (RNS.Link method)</a>
 </li>
         <li><a href="reference.html#RNS.Link.no_outbound_for">no_outbound_for() (RNS.Link method)</a>
@@ -656,10 +668,12 @@
 </li>
         <li><a href="reference.html#RNS.Identity.to_file">to_file() (RNS.Identity method)</a>
 </li>
-        <li><a href="reference.html#RNS.Transport">Transport (class in RNS)</a>
+        <li><a href="reference.html#RNS.Link.track_phy_stats">track_phy_stats() (RNS.Link method)</a>
 </li>
     </ul></td>
     <td style="width: 33%; vertical-align: top;"><ul>
+        <li><a href="reference.html#RNS.Transport">Transport (class in RNS)</a>
+</li>
         <li><a href="reference.html#RNS.Reticulum.transport_enabled">transport_enabled() (RNS.Reticulum static method)</a>
 </li>
         <li><a href="reference.html#RNS.Identity.truncated_hash">truncated_hash() (RNS.Identity static method)</a>

docs/manual/gettingstartedfast.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Using Reticulum on Your System" href="using.html" /><link rel="prev" title="What is Reticulum?" href="whatis.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Getting Started Fast - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Getting Started Fast - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -277,7 +277,8 @@ radio interfaces can then be added later.</p>
 <h3>Remote Shell<a class="headerlink" href="#remote-shell" title="Permalink to this heading">#</a></h3>
 <p>The <a class="reference external" href="https://github.com/acehoss/rnsh">rnsh</a> program lets you establish fully interactive
 remote shell sessions over Reticulum. It also allows you to pipe any program to or from a
-remote system, and is similar to how <code class="docutils literal notranslate"><span class="pre">ssh</span></code> works.</p>
+remote system, and is similar to how <code class="docutils literal notranslate"><span class="pre">ssh</span></code> works. The <code class="docutils literal notranslate"><span class="pre">rnsh</span></code> is very efficient, and
+can facilitate fully interactive shell sessions, even over extremely low-bandwidth links.</p>
 </section>
 <section id="nomad-network">
 <h3>Nomad Network<a class="headerlink" href="#nomad-network" title="Permalink to this heading">#</a></h3>
@@ -387,11 +388,11 @@ easier setup, use TCP.</p>
 <h2>Connect to the Public Testnet<a class="headerlink" href="#connect-to-the-public-testnet" title="Permalink to this heading">#</a></h2>
 <p>An experimental public testnet has been made accessible over both I2P and TCP. You can join it
 by adding one of the following interfaces to your <code class="docutils literal notranslate"><span class="pre">.reticulum/config</span></code> file:</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># TCP/IP interface to the Dublin hub</span>
-<span class="p">[[</span><span class="n">RNS</span> <span class="n">Testnet</span> <span class="n">Dublin</span><span class="p">]]</span>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># TCP/IP interface to the RNS Amsterdam Hub</span>
+<span class="p">[[</span><span class="n">RNS</span> <span class="n">Testnet</span> <span class="n">Amsterdam</span><span class="p">]]</span>
   <span class="nb">type</span> <span class="o">=</span> <span class="n">TCPClientInterface</span>
   <span class="n">enabled</span> <span class="o">=</span> <span class="n">yes</span>
-  <span class="n">target_host</span> <span class="o">=</span> <span class="n">dublin</span><span class="o">.</span><span class="n">connect</span><span class="o">.</span><span class="n">reticulum</span><span class="o">.</span><span class="n">network</span>
+  <span class="n">target_host</span> <span class="o">=</span> <span class="n">amsterdam</span><span class="o">.</span><span class="n">connect</span><span class="o">.</span><span class="n">reticulum</span><span class="o">.</span><span class="n">network</span>
   <span class="n">target_port</span> <span class="o">=</span> <span class="mi">4965</span>
 
 <span class="c1"># TCP/IP interface to the BetweenTheBorders Hub (community-provided)</span>
@@ -401,11 +402,11 @@ by adding one of the following interfaces to your <code class="docutils literal
   <span class="n">target_host</span> <span class="o">=</span> <span class="n">betweentheborders</span><span class="o">.</span><span class="n">com</span>
   <span class="n">target_port</span> <span class="o">=</span> <span class="mi">4242</span>
 
-<span class="c1"># Interface to I2P hub A</span>
-<span class="p">[[</span><span class="n">RNS</span> <span class="n">Testnet</span> <span class="n">I2P</span> <span class="n">Hub</span> <span class="n">A</span><span class="p">]]</span>
+<span class="c1"># Interface to Testnet I2P Hub</span>
+<span class="p">[[</span><span class="n">RNS</span> <span class="n">Testnet</span> <span class="n">I2P</span> <span class="n">Hub</span><span class="p">]]</span>
   <span class="nb">type</span> <span class="o">=</span> <span class="n">I2PInterface</span>
   <span class="n">enabled</span> <span class="o">=</span> <span class="n">yes</span>
-  <span class="n">peers</span> <span class="o">=</span> <span class="n">uxg5kubabakh3jtnvsipingbr5574dle7bubvip7llfvwx2tgrua</span><span class="o">.</span><span class="n">b32</span><span class="o">.</span><span class="n">i2p</span>
+  <span class="n">peers</span> <span class="o">=</span> <span class="n">g3br23bvx3lq5uddcsjii74xgmn6y5q325ovrkq2zw2wbzbqgbuq</span><span class="o">.</span><span class="n">b32</span><span class="o">.</span><span class="n">i2p</span>
 </pre></div>
 </div>
 <p>Many other Reticulum instances are connecting to this testnet, and you can also join it
@@ -426,7 +427,7 @@ digital radio transceiver, that integrates easily with Reticulum.</p>
 <p>To build one yourself requires installing a custom firmware on a supported LoRa
 development board with an auto-install script. Please see the <a class="reference internal" href="hardware.html#hardware-main"><span class="std std-ref">Communications Hardware</span></a>
 chapter for a guide. If you prefer purchasing a ready-made unit, you can refer to the
-<a class="reference internal" href="hardware.html#rnode-suppliers"><span class="std std-ref">list of suppliers</span></a>. For more information on RNode, you can also
+<span class="xref std std-ref">list of suppliers</span>. For more information on RNode, you can also
 refer to these additional external resources:</p>
 <ul class="simple">
 <li><p><a class="reference external" href="https://unsigned.io/how-to-make-your-own-rnodes/">How To Make Your Own RNodes</a></p></li>
@@ -443,14 +444,14 @@ and propose adding an interface for the hardware.</p>
 <h2>Develop a Program with Reticulum<a class="headerlink" href="#develop-a-program-with-reticulum" title="Permalink to this heading">#</a></h2>
 <p>If you want to develop programs that use Reticulum, the easiest way to get
 started is to install the latest release of Reticulum via pip:</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pip3</span> <span class="n">install</span> <span class="n">rns</span>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="n">rns</span>
 </pre></div>
 </div>
 <p>The above command will install Reticulum and dependencies, and you will be
 ready to import and use RNS in your own programs. The next step will most
 likely be to look at some <a class="reference internal" href="examples.html#examples-main"><span class="std std-ref">Example Programs</span></a>.</p>
 <p>For extended functionality, you can install optional dependencies:</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pip3</span> <span class="n">install</span> <span class="n">pyserial</span>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="n">pyserial</span>
 </pre></div>
 </div>
 <p>Further information can be found in the <a class="reference internal" href="reference.html#api-main"><span class="std std-ref">API Reference</span></a>.</p>
@@ -461,7 +462,7 @@ likely be to look at some <a class="reference internal" href="examples.html#exam
 utilities, you’ll want to get the latest source from GitHub. In that case,
 don’t use pip, but try this recipe:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Install dependencies</span>
-<span class="n">pip3</span> <span class="n">install</span> <span class="n">cryptography</span> <span class="n">pyserial</span>
+<span class="n">pip</span> <span class="n">install</span> <span class="n">cryptography</span> <span class="n">pyserial</span>
 
 <span class="c1"># Clone repository</span>
 <span class="n">git</span> <span class="n">clone</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">markqvist</span><span class="o">/</span><span class="n">Reticulum</span><span class="o">.</span><span class="n">git</span>
@@ -471,25 +472,25 @@ don’t use pip, but try this recipe:</p>
 <span class="n">ln</span> <span class="o">-</span><span class="n">s</span> <span class="o">../</span><span class="n">RNS</span> <span class="o">./</span><span class="n">Examples</span><span class="o">/</span>
 
 <span class="c1"># Run an example</span>
-<span class="n">python3</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Echo</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">s</span>
+<span class="n">python</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Echo</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">s</span>
 
 <span class="c1"># Unless you&#39;ve manually created a config file, Reticulum will do so now,</span>
 <span class="c1"># and immediately exit. Make any necessary changes to the file:</span>
 <span class="n">nano</span> <span class="o">~/.</span><span class="n">reticulum</span><span class="o">/</span><span class="n">config</span>
 
 <span class="c1"># ... and launch the example again.</span>
-<span class="n">python3</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Echo</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">s</span>
+<span class="n">python</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Echo</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">s</span>
 
 <span class="c1"># You can now repeat the process on another computer,</span>
 <span class="c1"># and run the same example with -h to get command line options.</span>
-<span class="n">python3</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Echo</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">h</span>
+<span class="n">python</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Echo</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">h</span>
 
 <span class="c1"># Run the example in client mode to &quot;ping&quot; the server.</span>
 <span class="c1"># Replace the hash below with the actual destination hash of your server.</span>
-<span class="n">python3</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Echo</span><span class="o">.</span><span class="n">py</span> <span class="mi">174</span><span class="n">a64852a75682259ad8b921b8bf416</span>
+<span class="n">python</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Echo</span><span class="o">.</span><span class="n">py</span> <span class="mi">174</span><span class="n">a64852a75682259ad8b921b8bf416</span>
 
 <span class="c1"># Have a look at another example</span>
-<span class="n">python3</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Filetransfer</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">h</span>
+<span class="n">python</span> <span class="n">Examples</span><span class="o">/</span><span class="n">Filetransfer</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">h</span>
 </pre></div>
 </div>
 <p>When you have experimented with the basic examples, it’s time to go read the
@@ -559,7 +560,7 @@ locally on your device using the following command:</p>
 </div>
 <p>It is also possible to include Reticulum in apps compiled and distributed as
 Android APKs. A detailed tutorial and example source code will be included
-here at a later point. Until then you can use the <a class="reference external" href="https://github.com/markqvist/sideband">Sideband source code</a> as an example and startig point.</p>
+here at a later point. Until then you can use the <a class="reference external" href="https://github.com/markqvist/sideband">Sideband source code</a> as an example and starting point.</p>
 </section>
 <section id="arm64">
 <h3>ARM64<a class="headerlink" href="#arm64" title="Permalink to this heading">#</a></h3>
@@ -586,11 +587,11 @@ detailed in this manual.</p>
 </section>
 <section id="debian-bookworm">
 <h3>Debian Bookworm<a class="headerlink" href="#debian-bookworm" title="Permalink to this heading">#</a></h3>
-<p>On versions of Debian released after April 2023, it is no longer possible
+<p>On versions of Debian released after April 2023, it is no longer possible by default
 to use <code class="docutils literal notranslate"><span class="pre">pip</span></code> to install packages onto your system. Unfortunately, you will need to
 use the replacement <code class="docutils literal notranslate"><span class="pre">pipx</span></code> command instead, which places installed packages in an
-isolated environment. This should not negatively affect Reticulum, but installation
-via this method is not fully tested yet.</p>
+isolated environment. This should not negatively affect Reticulum, but will not work
+for including and using Reticulum in your own scripts and programs.</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Install pipx</span>
 <span class="n">sudo</span> <span class="n">apt</span> <span class="n">install</span> <span class="n">pipx</span>
 
@@ -601,14 +602,25 @@ via this method is not fully tested yet.</p>
 <span class="n">pipx</span> <span class="n">install</span> <span class="n">rns</span>
 </pre></div>
 </div>
+<p>Alternatively, you can restore normal behaviour to <code class="docutils literal notranslate"><span class="pre">pip</span></code> by creating or editing
+the configuration file located at <code class="docutils literal notranslate"><span class="pre">~/.config/pip/pip.conf</span></code>, and adding the
+following section:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>[global]
+break-system-packages = true
+</pre></div>
+</div>
+<p>Please note that the “break-system-packages” directive is a somewhat misleading choice
+of words. Setting it will of course not break any system packages, but will simply
+allow installing <code class="docutils literal notranslate"><span class="pre">pip</span></code> packages user- and system-wide. While this <em>could</em> in rare
+cases lead to version conflicts, it does not generally pose any problems.</p>
 </section>
 <section id="ubuntu-lunar">
 <h3>Ubuntu Lunar<a class="headerlink" href="#ubuntu-lunar" title="Permalink to this heading">#</a></h3>
-<p>On versions of Ubuntu released after April 2023, it is no longer possible
+<p>On versions of Ubuntu released after April 2023, it is no longer possible by default
 to use <code class="docutils literal notranslate"><span class="pre">pip</span></code> to install packages onto your system. Unfortunately, you will need to
 use the replacement <code class="docutils literal notranslate"><span class="pre">pipx</span></code> command instead, which places installed packages in an
-isolated environment. This should not negatively affect Reticulum, but installation
-via this method is not fully tested yet.</p>
+isolated environment. This should not negatively affect Reticulum, but will not work
+for including and using Reticulum in your own scripts and programs.</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Install pipx</span>
 <span class="n">sudo</span> <span class="n">apt</span> <span class="n">install</span> <span class="n">pipx</span>
 
@@ -619,14 +631,23 @@ via this method is not fully tested yet.</p>
 <span class="n">pipx</span> <span class="n">install</span> <span class="n">rns</span>
 </pre></div>
 </div>
+<p>Alternatively, you can restore normal behaviour to <code class="docutils literal notranslate"><span class="pre">pip</span></code> by creating or editing
+the configuration file located at <code class="docutils literal notranslate"><span class="pre">~/.config/pip/pip.conf</span></code>, and adding the
+following section:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>[global]
+break-system-packages = true
+</pre></div>
+</div>
+<p>Please note that the “break-system-packages” directive is a somewhat misleading choice
+of words. Setting it will of course not break any system packages, but will simply
+allow installing <code class="docutils literal notranslate"><span class="pre">pip</span></code> packages user- and system-wide. While this _could_ in rare
+cases lead to version conflicts, it does not generally pose any problems.</p>
 </section>
 </section>
 <section id="pure-python-reticulum">
 <h2>Pure-Python Reticulum<a class="headerlink" href="#pure-python-reticulum" title="Permalink to this heading">#</a></h2>
 <p>In some rare cases, and on more obscure system types, it is not possible to
-install one or more dependencies</p>
-<p>On more unusual systems, and in some rare cases, it might not be possible to
-install or even compile one or more of the above modules. In such situations,
+install one or more dependencies. In such situations,
 you can use the <code class="docutils literal notranslate"><span class="pre">rnspure</span></code> package instead of the <code class="docutils literal notranslate"><span class="pre">rns</span></code> package, or use <code class="docutils literal notranslate"><span class="pre">pip</span></code>
 with the <code class="docutils literal notranslate"><span class="pre">--no-dependencies</span></code> command-line option. The <code class="docutils literal notranslate"><span class="pre">rnspure</span></code>
 package requires no external dependencies for installation. Please note that the

docs/manual/hardware.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
-<link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Supported Interfaces" href="interfaces.html" /><link rel="prev" title="Understanding Reticulum" href="understanding.html" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
+<link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Configuring Interfaces" href="interfaces.html" /><link rel="prev" title="Understanding Reticulum" href="understanding.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Communications Hardware - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Communications Hardware - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -240,10 +240,17 @@ starting from scratch.</p>
 <p>This chapter will outline a few different sensible starting paths to get
 real-world functional wireless communications up and running with minimal cost
 and effort. Two fundamental devices categories will be covered, <em>RNodes</em> and
-<em>WiFi-based radios</em>.</p>
-<p>While there are many other device categories that are useful in building Reticulum
-networks, knowing how to employ just these two will make it possible to build
-a wide range of useful networks with little effort.</p>
+<em>WiFi-based radios</em>. Additionally, other common options will be briefly described.</p>
+<p>Knowing how to employ just a few different types of hardware will make it possible
+to build a wide range of useful networks with little effort.</p>
+<section id="combining-hardware-types">
+<h2>Combining Hardware Types<a class="headerlink" href="#combining-hardware-types" title="Permalink to this heading">#</a></h2>
+<p>It is useful to combine different link and hardware types when designing and
+building a network. One useful design pattern is to employ high-capacity point-to-point
+links based on WiFi or millimeter-wave radios (with high-gain directional antennas)
+for the network backbone, and using LoRa-based RNodes for covering large areas with
+connectivity for client devices.</p>
+</section>
 <section id="rnode">
 <span id="rnode-main"></span><h2>RNode<a class="headerlink" href="#rnode" title="Permalink to this heading">#</a></h2>
 <p>Reliable and general-purpose long-range digital radio transceiver systems are
@@ -373,11 +380,6 @@ such as serial port and on-air parameters. For v2.x firmwares, you just need to
 the Connection ID of the RNode, and Reticulum will automatically locate and connect to the
 RNode, using the parameters stored in the RNode itself.</p>
 </section>
-<section id="suppliers">
-<span id="rnode-suppliers"></span><h3>Suppliers<a class="headerlink" href="#suppliers" title="Permalink to this heading">#</a></h3>
-<p>Get in touch if you want to have your RNode supplier listed here, or if you want help to
-get started with producing RNodes.</p>
-</section>
 </section>
 <section id="wifi-based-hardware">
 <h2>WiFi-based Hardware<a class="headerlink" href="#wifi-based-hardware" title="Permalink to this heading">#</a></h2>
@@ -403,13 +405,29 @@ that is relatively cheap while providing long range and high capacity for Reticu
 networks. As in all other cases, it is also possible for Reticulum to co-exist with IP
 networks running concurrently on such devices.</p>
 </section>
-<section id="combining-hardware-types">
-<h2>Combining Hardware Types<a class="headerlink" href="#combining-hardware-types" title="Permalink to this heading">#</a></h2>
-<p>It is useful to combine different link and hardware types when designing and
-building a network. One useful design pattern is to employ high-capacity point-to-point
-links based on WiFi or millimeter-wave radios (with high-gain directional antennas)
-for the network backbone, and using LoRa-based RNodes for covering large areas with
-connectivity for client devices.</p>
+<section id="ethernet-based-hardware">
+<h2>Ethernet-based Hardware<a class="headerlink" href="#ethernet-based-hardware" title="Permalink to this heading">#</a></h2>
+<p>Reticulum can run over any kind of hardware that can provide a switched Ethernet-based
+medium. This means that anything from a plain Ethernet switch, to fiber-optic systems,
+to data radios with Ethernet interfaces can be used by Reticulum.</p>
+<p>The Ethernet medium does not need to have any IP infrastructure such as DHCP servers
+or routing set up, but in case such infrastructure does exist, Reticulum will simply
+co-exist with.</p>
+<p>To use Reticulum over Ethernet-based mediums, it is generally enough to use the included
+<a class="reference internal" href="interfaces.html#interfaces-auto"><span class="std std-ref">AutoInterface</span></a>. This interface also works over any kind of
+virtual networking adapter, such as <code class="docutils literal notranslate"><span class="pre">tun</span></code> and <code class="docutils literal notranslate"><span class="pre">tap</span></code> devices in Linux.</p>
+</section>
+<section id="serial-lines-devices">
+<h2>Serial Lines &amp; Devices<a class="headerlink" href="#serial-lines-devices" title="Permalink to this heading">#</a></h2>
+<p>Using Reticulum over any kind of raw serial line is also possible with the
+<a class="reference internal" href="interfaces.html#interfaces-serial"><span class="std std-ref">SerialInterface</span></a>. This interface type is also useful for
+using Reticulum over communications hardware that provides a serial port interface.</p>
+</section>
+<section id="packet-radio-modems">
+<h2>Packet Radio Modems<a class="headerlink" href="#packet-radio-modems" title="Permalink to this heading">#</a></h2>
+<p>Any packet radio modem that provides a standard KISS interface over USB, serial or TCP
+can be used with Reticulum. This includes virtual software modems such as
+<a class="reference external" href="https://github.com/xssfox/freedv-tnc">FreeDV TNC</a> and <a class="reference external" href="https://github.com/wb2osz/direwolf">Dire Wolf</a>.</p>
 </section>
 </section>
 
@@ -423,7 +441,7 @@ connectivity for client devices.</p>
                 <div class="context">
                   <span>Next</span>
                 </div>
-                <div class="title">Supported Interfaces</div>
+                <div class="title">Configuring Interfaces</div>
               </div>
               <svg class="furo-related-icon"><use href="#svg-arrow-right"></use></svg>
             </a>
@@ -470,6 +488,7 @@ connectivity for client devices.</p>
           <div class="toc-tree">
             <ul>
 <li><a class="reference internal" href="#">Communications Hardware</a><ul>
+<li><a class="reference internal" href="#combining-hardware-types">Combining Hardware Types</a></li>
 <li><a class="reference internal" href="#rnode">RNode</a><ul>
 <li><a class="reference internal" href="#creating-rnodes">Creating RNodes</a></li>
 <li><a class="reference internal" href="#supported-boards">Supported Boards</a><ul>
@@ -483,11 +502,12 @@ connectivity for client devices.</p>
 </li>
 <li><a class="reference internal" href="#installation">Installation</a></li>
 <li><a class="reference internal" href="#usage-with-reticulum">Usage with Reticulum</a></li>
-<li><a class="reference internal" href="#suppliers">Suppliers</a></li>
 </ul>
 </li>
 <li><a class="reference internal" href="#wifi-based-hardware">WiFi-based Hardware</a></li>
-<li><a class="reference internal" href="#combining-hardware-types">Combining Hardware Types</a></li>
+<li><a class="reference internal" href="#ethernet-based-hardware">Ethernet-based Hardware</a></li>
+<li><a class="reference internal" href="#serial-lines-devices">Serial Lines &amp; Devices</a></li>
+<li><a class="reference internal" href="#packet-radio-modems">Packet Radio Modems</a></li>
 </ul>
 </li>
 </ul>

docs/manual/index.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="What is Reticulum?" href="whatis.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="#"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="#"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -272,6 +272,7 @@ to participate in the development of Reticulum itself.</p>
 <li class="toctree-l2"><a class="reference internal" href="using.html#included-utility-programs">Included Utility Programs</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="using.html#the-rnsd-utility">The rnsd Utility</a></li>
 <li class="toctree-l3"><a class="reference internal" href="using.html#the-rnstatus-utility">The rnstatus Utility</a></li>
+<li class="toctree-l3"><a class="reference internal" href="using.html#the-rnid-utility">The rnid Utility</a></li>
 <li class="toctree-l3"><a class="reference internal" href="using.html#the-rnpath-utility">The rnpath Utility</a></li>
 <li class="toctree-l3"><a class="reference internal" href="using.html#the-rnprobe-utility">The rnprobe Utility</a></li>
 <li class="toctree-l3"><a class="reference internal" href="using.html#the-rncp-utility">The rncp Utility</a></li>
@@ -315,19 +316,21 @@ to participate in the development of Reticulum itself.</p>
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="hardware.html#combining-hardware-types">Combining Hardware Types</a></li>
 <li class="toctree-l2"><a class="reference internal" href="hardware.html#rnode">RNode</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="hardware.html#creating-rnodes">Creating RNodes</a></li>
 <li class="toctree-l3"><a class="reference internal" href="hardware.html#supported-boards">Supported Boards</a></li>
 <li class="toctree-l3"><a class="reference internal" href="hardware.html#installation">Installation</a></li>
 <li class="toctree-l3"><a class="reference internal" href="hardware.html#usage-with-reticulum">Usage with Reticulum</a></li>
-<li class="toctree-l3"><a class="reference internal" href="hardware.html#suppliers">Suppliers</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="hardware.html#wifi-based-hardware">WiFi-based Hardware</a></li>
-<li class="toctree-l2"><a class="reference internal" href="hardware.html#combining-hardware-types">Combining Hardware Types</a></li>
+<li class="toctree-l2"><a class="reference internal" href="hardware.html#ethernet-based-hardware">Ethernet-based Hardware</a></li>
+<li class="toctree-l2"><a class="reference internal" href="hardware.html#serial-lines-devices">Serial Lines &amp; Devices</a></li>
+<li class="toctree-l2"><a class="reference internal" href="hardware.html#packet-radio-modems">Packet Radio Modems</a></li>
 </ul>
 </li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a><ul>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a><ul>
 <li class="toctree-l2"><a class="reference internal" href="interfaces.html#auto-interface">Auto Interface</a></li>
 <li class="toctree-l2"><a class="reference internal" href="interfaces.html#i2p-interface">I2P Interface</a></li>
 <li class="toctree-l2"><a class="reference internal" href="interfaces.html#tcp-server-interface">TCP Server Interface</a></li>
@@ -341,6 +344,7 @@ to participate in the development of Reticulum itself.</p>
 <li class="toctree-l2"><a class="reference internal" href="interfaces.html#common-interface-options">Common Interface Options</a></li>
 <li class="toctree-l2"><a class="reference internal" href="interfaces.html#interface-modes">Interface Modes</a></li>
 <li class="toctree-l2"><a class="reference internal" href="interfaces.html#announce-rate-control">Announce Rate Control</a></li>
+<li class="toctree-l2"><a class="reference internal" href="interfaces.html#new-destination-rate-limiting">New Destination Rate Limiting</a></li>
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a><ul>

docs/manual/interfaces.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Building Networks" href="networks.html" /><link rel="prev" title="Communications Hardware" href="hardware.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Supported Interfaces - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Configuring Interfaces - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Supported Interfaces</a></li>
+<li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -221,8 +221,8 @@
           </label>
         </div>
         <article role="main">
-          <section id="supported-interfaces">
-<span id="interfaces-main"></span><h1>Supported Interfaces<a class="headerlink" href="#supported-interfaces" title="Permalink to this heading">#</a></h1>
+          <section id="configuring-interfaces">
+<span id="interfaces-main"></span><h1>Configuring Interfaces<a class="headerlink" href="#configuring-interfaces" title="Permalink to this heading">#</a></h1>
 <p>Reticulum supports using many kinds of devices as networking interfaces, and
 allows you to mix and match them in any way you choose. The number of distinct
 network topologies you can create with Reticulum is more or less endless, but
@@ -241,9 +241,20 @@ infrastructure like routers or DHCP servers, but will require at least some
 sort of switching medium between peers (a wired switch, a hub, a WiFi access
 point or similar), and that link-local IPv6 is enabled in your operating
 system, which should be enabled by default in almost all OSes.</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># This example demonstrates a TCP server interface.</span>
-<span class="c1"># It will listen for incoming connections on the</span>
-<span class="c1"># specified IP address and port number.</span>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># This example demonstrates a bare-minimum setup</span>
+<span class="c1"># of an Auto Interface. It will allow communica-</span>
+<span class="c1"># tion with all other reachable devices on all</span>
+<span class="c1"># usable physical ethernet-based devices that</span>
+<span class="c1"># are available on the system.</span>
+
+<span class="p">[[</span><span class="n">Default</span> <span class="n">Interface</span><span class="p">]]</span>
+  <span class="nb">type</span> <span class="o">=</span> <span class="n">AutoInterface</span>
+  <span class="n">interface_enabled</span> <span class="o">=</span> <span class="kc">True</span>
+
+<span class="c1"># This example demonstrates an more specifically</span>
+<span class="c1"># configured Auto Interface, that only uses spe-</span>
+<span class="c1"># cific physical interfaces, and has a number of</span>
+<span class="c1"># other configuration options set.</span>
 
 <span class="p">[[</span><span class="n">Default</span> <span class="n">Interface</span><span class="p">]]</span>
   <span class="nb">type</span> <span class="o">=</span> <span class="n">AutoInterface</span>
@@ -255,6 +266,12 @@ system, which should be enabled by default in almost all OSes.</p>
 
   <span class="n">group_id</span> <span class="o">=</span> <span class="n">reticulum</span>
 
+  <span class="c1"># You can also choose the multicast address type:</span>
+  <span class="c1"># temporary (default, Temporary Multicast Address)</span>
+  <span class="c1"># or permanent (Permanent Multicast Address)</span>
+
+  <span class="n">multicast_address_type</span> <span class="o">=</span> <span class="n">permanent</span>
+
   <span class="c1"># You can also select specifically which</span>
   <span class="c1"># kernel networking devices to use.</span>
 
@@ -527,6 +544,7 @@ can be used, and offers full control over LoRa parameters.</p>
   <span class="c1"># out identification on the channel with</span>
   <span class="c1"># a set interval by configuring the</span>
   <span class="c1"># following two parameters.</span>
+
   <span class="c1"># id_callsign = MYCALL-0</span>
   <span class="c1"># id_interval = 600</span>
 
@@ -534,7 +552,20 @@ can be used, and offers full control over LoRa parameters.</p>
   <span class="c1"># with low amounts of RAM, using packet</span>
   <span class="c1"># flow control can be useful. By default</span>
   <span class="c1"># it is disabled.</span>
-  <span class="n">flow_control</span> <span class="o">=</span> <span class="kc">False</span>
+
+  <span class="c1"># flow_control = False</span>
+
+  <span class="c1"># It is possible to limit the airtime</span>
+  <span class="c1"># utilisation of an RNode by using the</span>
+  <span class="c1"># following two configuration options.</span>
+  <span class="c1"># The short-term limit is applied in a</span>
+  <span class="c1"># window of approximately 15 seconds,</span>
+  <span class="c1"># and the long-term limit is enforced</span>
+  <span class="c1"># over a rolling 60 minute window. Both</span>
+  <span class="c1"># options are specified in percent.</span>
+
+  <span class="c1"># airtime_limit_long = 1.5</span>
+  <span class="c1"># airtime_limit_short = 33</span>
 </pre></div>
 </div>
 </section>
@@ -923,6 +954,91 @@ conserve bandwidth, while very fast networks can support applications that
 need very frequent announces. Reticulum implements these mechanisms to ensure
 that a large span of network types can seamlessly <em>co-exist</em> and interconnect.</p>
 </section>
+<section id="new-destination-rate-limiting">
+<span id="interfaces-ingress-control"></span><h2>New Destination Rate Limiting<a class="headerlink" href="#new-destination-rate-limiting" title="Permalink to this heading">#</a></h2>
+<p>On public interfaces, where anyone may connect and announce new destinations,
+it can be useful to control the rate at which announces for <em>new</em>  destinations are
+processed.</p>
+<p>If a large influx of announces for newly created or previously unknown destinations
+occur within a short amount of time, Reticulum will place these announces on hold,
+so that announce traffic for known and previously established destinations can
+continue to be processed without interruptions.</p>
+<p>After the burst subsides, and an additional waiting period has passed, the held
+announces will be released at a slow rate, until the hold queue is cleared. This
+also means, that should a node decide to connect to a public interface, announce
+a large amount of bogus destinations, and then disconnect, these destination will
+never make it into path tables and waste network bandwidth on retransmitted
+announces.</p>
+<p><strong>It’s important to note</strong> that the ingress control works at the level of <em>individual
+sub-interfaces</em>. As an example, this means that one client on a <a class="reference internal" href="#interfaces-tcps"><span class="std std-ref">TCP Server Interface</span></a>
+cannot disrupt processing of incoming announces for other connected clients on the same
+<a class="reference internal" href="#interfaces-tcps"><span class="std std-ref">TCP Server Interface</span></a>. All other clients on the same interface will still have new announces
+processed without interruption.</p>
+<p>By default, Reticulum will handle this automatically, and ingress announce
+control will be enabled on interface where it is sensible to do so. It should
+generally not be neccessary to modify the ingress control configuration,
+but all the parameters are exposed for configuration if needed.</p>
+<blockquote>
+<div><ul>
+<li><div class="line-block">
+<div class="line">The <code class="docutils literal notranslate"><span class="pre">ingress_control</span></code> option tells Reticulum whether or not
+to enable announce ingress control on the interface. Defaults to
+<code class="docutils literal notranslate"><span class="pre">True</span></code>.</div>
+</div>
+</li>
+<li><div class="line-block">
+<div class="line">The <code class="docutils literal notranslate"><span class="pre">ic_new_time</span></code> option configures how long (in seconds) an
+interface is considered newly spawned. Defaults to <code class="docutils literal notranslate"><span class="pre">2*60*60</span></code> seconds. This
+option is useful on publicly accessible interfaces that spawn new
+sub-interfaces when a new client connects.</div>
+</div>
+</li>
+<li><div class="line-block">
+<div class="line">The <code class="docutils literal notranslate"><span class="pre">ic_burst_freq_new</span></code> option sets the maximum announce ingress
+frequency for newly spawned interfaces. Defaults to <code class="docutils literal notranslate"><span class="pre">3.5</span></code>
+announces per second.</div>
+</div>
+</li>
+<li><div class="line-block">
+<div class="line">The <code class="docutils literal notranslate"><span class="pre">ic_burst_freq</span></code> option sets the maximum announce ingress
+frequency for other interfaces. Defaults to <code class="docutils literal notranslate"><span class="pre">12</span></code> announces
+per second.</div>
+</div>
+<blockquote>
+<div><p><em>If an interface exceeds its burst frequency, incoming announces
+for unknown destinations will be temporarily held in a queue, and
+not processed until later.</em></p>
+</div></blockquote>
+</li>
+<li><div class="line-block">
+<div class="line">The <code class="docutils literal notranslate"><span class="pre">ic_max_held_announces</span></code> option sets the maximum amount of
+unique announces that will be held in the queue. Any additional
+unique announces will be dropped. Defaults to <code class="docutils literal notranslate"><span class="pre">256</span></code> announces.</div>
+</div>
+</li>
+<li><div class="line-block">
+<div class="line">The <code class="docutils literal notranslate"><span class="pre">ic_burst_hold</span></code> option sets how much time (in seconds) must
+pass after the burst frequency drops below its threshold, for the
+announce burst to be considered cleared. Defaults to <code class="docutils literal notranslate"><span class="pre">60</span></code>
+seconds.</div>
+</div>
+</li>
+<li><div class="line-block">
+<div class="line">The <code class="docutils literal notranslate"><span class="pre">ic_burst_penalty</span></code> option sets how much time (in seconds) must
+pass after the burst is considered cleared, before held announces can
+start being released from the queue. Defaults to <code class="docutils literal notranslate"><span class="pre">5*60</span></code>
+seconds.</div>
+</div>
+</li>
+<li><div class="line-block">
+<div class="line">The <code class="docutils literal notranslate"><span class="pre">ic_held_release_interval</span></code> option sets how much time (in seconds)
+must pass between releasing each held announce from the queue. Defaults
+to <code class="docutils literal notranslate"><span class="pre">30</span></code> seconds.</div>
+</div>
+</li>
+</ul>
+</div></blockquote>
+</section>
 </section>
 
         </article>
@@ -981,7 +1097,7 @@ that a large span of network types can seamlessly <em>co-exist</em> and intercon
         <div class="toc-tree-container">
           <div class="toc-tree">
             <ul>
-<li><a class="reference internal" href="#">Supported Interfaces</a><ul>
+<li><a class="reference internal" href="#">Configuring Interfaces</a><ul>
 <li><a class="reference internal" href="#auto-interface">Auto Interface</a></li>
 <li><a class="reference internal" href="#i2p-interface">I2P Interface</a></li>
 <li><a class="reference internal" href="#tcp-server-interface">TCP Server Interface</a></li>
@@ -995,6 +1111,7 @@ that a large span of network types can seamlessly <em>co-exist</em> and intercon
 <li><a class="reference internal" href="#common-interface-options">Common Interface Options</a></li>
 <li><a class="reference internal" href="#interface-modes">Interface Modes</a></li>
 <li><a class="reference internal" href="#announce-rate-control">Announce Rate Control</a></li>
+<li><a class="reference internal" href="#new-destination-rate-limiting">New Destination Rate Limiting</a></li>
 </ul>
 </li>
 </ul>

docs/manual/networks.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
-<link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Code Examples" href="examples.html" /><link rel="prev" title="Supported Interfaces" href="interfaces.html" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
+<link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Code Examples" href="examples.html" /><link rel="prev" title="Configuring Interfaces" href="interfaces.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Building Networks - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Building Networks - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -413,7 +413,7 @@ connected outliers are now an integral part of the network.</p>
                   <span>Previous</span>
                 </div>
                 
-                <div class="title">Supported Interfaces</div>
+                <div class="title">Configuring Interfaces</div>
                 
               </div>
             </a>

docs/manual/search.html

@@ -4,8 +4,8 @@
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
     <meta name="color-scheme" content="light dark"><link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="#" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/><title>Search - Reticulum Network Stack 0.5.2 beta documentation</title><link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/><title>Search - Reticulum Network Stack 0.7.5 beta documentation</title><link rel="stylesheet" type="text/css" href="_static/pygments.css" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -138,7 +138,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -164,10 +164,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="#" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -178,7 +178,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>

docs/manual/support.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="API Reference" href="reference.html" /><link rel="prev" title="Code Examples" href="examples.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Support Reticulum - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Support Reticulum - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Support Reticulum</a></li>

docs/manual/understanding.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Communications Hardware" href="hardware.html" /><link rel="prev" title="Using Reticulum on Your System" href="using.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Understanding Reticulum - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Understanding Reticulum - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -286,7 +286,7 @@ it can be easily modified and replicated by anyone interested in doing so.</p>
 </li>
 <li><dl class="simple">
 <dt><strong>Very low bandwidth requirements</strong></dt><dd><p>Reticulum should be able to function reliably over links with a transmission capacity as low
-as <em>500 bits per second</em>.</p>
+as <em>5 bits per second</em>.</p>
 </dd>
 </dl>
 </li>
@@ -831,7 +831,7 @@ note that Reticulum is designed to be usable on more or less any computing devic
 or less any medium that allows you to send and receive data, which satisfies some very low
 minimum requirements.</p>
 <p>The communication channel must support at least half-duplex operation, and provide an average
-throughput of around 500 bits per second, and supports a physical layer MTU of 500 bytes. The
+throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The
 Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x
 runtime environment.</p>
 <p>That being said, this reference setup has been outlined to provide a common platform for anyone
@@ -1061,7 +1061,7 @@ cryptographic primitives, with widely available implementations that can be used
 both on general-purpose CPUs and on microcontrollers. The necessary primitives are:</p>
 <ul class="simple">
 <li><p>Ed25519 for signatures</p></li>
-<li><p>X22519 for ECDH key exchanges</p></li>
+<li><p>X25519 for ECDH key exchanges</p></li>
 <li><p>HKDF for key derivation</p></li>
 <li><p>Fernet for encrypted tokens</p>
 <ul>

docs/manual/using.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Understanding Reticulum" href="understanding.html" /><link rel="prev" title="Getting Started Fast" href="gettingstartedfast.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>Using Reticulum on Your System - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>Using Reticulum on Your System - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -307,6 +307,17 @@ configuration file is created. The default configuration looks like this:</p>
 <span class="n">instance_control_port</span> <span class="o">=</span> <span class="mi">37429</span>
 
 
+<span class="c1"># On systems where running instances may not have access</span>
+<span class="c1"># to the same shared Reticulum configuration directory,</span>
+<span class="c1"># it is still possible to allow full interactivity for</span>
+<span class="c1"># running instances, by manually specifying a shared RPC</span>
+<span class="c1"># key. In almost all cases, this option is not needed, but</span>
+<span class="c1"># it can be useful on operating systems such as Android.</span>
+<span class="c1"># The key must be specified as bytes in hexadecimal.</span>
+
+<span class="c1"># rpc_key = e5c032d3ec4e64a6aca9927ba8ab73336780f6d71790</span>
+
+
 <span class="c1"># You can configure Reticulum to panic and forcibly close</span>
 <span class="c1"># if an unrecoverable interface error occurs, such as the</span>
 <span class="c1"># hardware device for an interface disappearing. This is</span>
@@ -316,6 +327,17 @@ configuration file is created. The default configuration looks like this:</p>
 <span class="n">panic_on_interface_error</span> <span class="o">=</span> <span class="n">No</span>
 
 
+<span class="c1"># When Transport is enabled, it is possible to allow the</span>
+<span class="c1"># Transport Instance to respond to probe requests from</span>
+<span class="c1"># the rnprobe utility. This can be a useful tool to test</span>
+<span class="c1"># connectivity. When this option is enabled, the probe</span>
+<span class="c1"># destination will be generated from the Identity of the</span>
+<span class="c1"># Transport Instance, and printed to the log at startup.</span>
+<span class="c1"># Optional, and disabled by default.</span>
+
+<span class="n">respond_to_probes</span> <span class="o">=</span> <span class="n">No</span>
+
+
 <span class="p">[</span><span class="n">logging</span><span class="p">]</span>
 <span class="c1"># Valid log levels are 0 through 7:</span>
 <span class="c1">#   0: Log only critical information</span>
@@ -354,10 +376,15 @@ configuration file is created. The default configuration looks like this:</p>
 <p>If Reticulum infrastructure already exists locally, you probably don’t need to
 change anything, and you may already be connected to a wider network. If not,
 you will probably need to add relevant <em>interfaces</em> to the configuration, in
-order to communicate with other systems. It is a good idea to read the comments
-and explanations in the above default config. It will teach you the basic
-concepts you need to understand to configure your network. Once you have done that,
-take a look at the <a class="reference internal" href="interfaces.html#interfaces-main"><span class="std std-ref">Interfaces</span></a> chapter of this manual.</p>
+order to communicate with other systems.</p>
+<p>You can generate a much more verbose configuration example by running the command:</p>
+<p><code class="docutils literal notranslate"><span class="pre">rnsd</span> <span class="pre">--exampleconfig</span></code></p>
+<p>The output includes examples for most interface types supported
+by Reticulum, along with additional options and configuration parameters.</p>
+<p>It is a good idea to read the comments and explanations in the above default config.
+It will teach you the basic concepts you need to understand to configure your network.
+Once you have done that, take a look at the <a class="reference internal" href="interfaces.html#interfaces-main"><span class="std std-ref">Interfaces</span></a> chapter
+of this manual.</p>
 </section>
 <section id="included-utility-programs">
 <h2>Included Utility Programs<a class="headerlink" href="#included-utility-programs" title="Permalink to this heading">#</a></h2>
@@ -374,24 +401,36 @@ other programs, applications and services can utilise.</p>
 When <code class="docutils literal notranslate"><span class="pre">rnsd</span></code> is running, it will keep all configured interfaces open, handle transport if
 it is enabled, and allow any other programs to immediately utilise the
 Reticulum network it is configured for.</p>
-<p>You can even run multiple instances of rnsd with different configurations on
+<p>You can even run multiple instances of <code class="docutils literal notranslate"><span class="pre">rnsd</span></code> with different configurations on
 the same system.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span># Install Reticulum
-pip3 install rns
+<p><strong>Usage Examples</strong></p>
+<p>Run <code class="docutils literal notranslate"><span class="pre">rnsd</span></code>:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnsd
 
-# Run rnsd
-rnsd
+[2023-08-18 17:59:56] [Notice] Started rnsd version 0.5.8
 </pre></div>
 </div>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnsd [-h] [--config CONFIG] [-v] [-q] [--version]
+<p>Run <code class="docutils literal notranslate"><span class="pre">rnsd</span></code> in service mode, ensuring all logging output is sent directly to file:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnsd -s
+</pre></div>
+</div>
+<p>Generate a verbose and detailed configuration example, with explanations of all the
+various configuration options, and interface configuration examples:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnsd --exampleconfig
+</pre></div>
+</div>
+<p><strong>All Command-Line Options</strong></p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnsd.py [-h] [--config CONFIG] [-v] [-q] [-s] [--exampleconfig] [--version]
 
 Reticulum Network Stack Daemon
 
-optional arguments:
+options:
   -h, --help       show this help message and exit
   --config CONFIG  path to alternative Reticulum config directory
   -v, --verbose
   -q, --quiet
+  -s, --service    rnsd is running as a service and should log to file
+  --exampleconfig  print verbose configuration example to stdout and exit
   --version        show program&#39;s version number and exit
 </pre></div>
 </div>
@@ -401,10 +440,10 @@ optional arguments:
 <h3>The rnstatus Utility<a class="headerlink" href="#the-rnstatus-utility" title="Permalink to this heading">#</a></h3>
 <p>Using the <code class="docutils literal notranslate"><span class="pre">rnstatus</span></code> utility, you can view the status of configured Reticulum
 interfaces, similar to the <code class="docutils literal notranslate"><span class="pre">ifconfig</span></code> program.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span># Run rnstatus
-rnstatus
+<p><strong>Usage Examples</strong></p>
+<p>Run <code class="docutils literal notranslate"><span class="pre">rnstatus</span></code>:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnstatus
 
-# Example output
 Shared Instance[37428]
    Status  : Up
    Serving : 1 program
@@ -438,38 +477,147 @@ RNodeInterface[RNode UHF]
 Reticulum Transport Instance &lt;5245a8efe1788c6a1cd36144a270e13b&gt; running
 </pre></div>
 </div>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnstatus [-h] [--config CONFIG] [--version] [-a] [-v]
+<p>Filter output to only show some interfaces:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnstatus rnode
+
+RNodeInterface[RNode UHF]
+   Status  : Up
+   Mode    : Access Point
+   Rate    : 1.30 kbps
+   Access  : 64-bit IFAC by &lt;…e702c42ba8&gt;
+   Traffic : 8.49 KB↑
+             9.23 KB↓
+
+Reticulum Transport Instance &lt;5245a8efe1788c6a1cd36144a270e13b&gt; running
+</pre></div>
+</div>
+<p><strong>All Command-Line Options</strong></p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnstatus.py [-h] [--config CONFIG] [--version] [-a] [-A] [-s SORT]
+                   [-r] [-j] [-v] [filter]
 
 Reticulum Network Stack Status
 
-optional arguments:
-  -h, --help       show this help message and exit
-  --config CONFIG  path to alternative Reticulum config directory
-  --version        show program&#39;s version number and exit
-  -a, --all        show all interfaces
+positional arguments:
+  filter                only display interfaces with names including filter
+
+options:
+  -h, --help            show this help message and exit
+  --config CONFIG       path to alternative Reticulum config directory
+  --version             show program&#39;s version number and exit
+  -a, --all             show all interfaces
+  -A, --announce-stats  show announce stats
+  -s SORT, --sort SORT  sort interfaces by [rate, traffic, rx, tx, announces, arx, atx, held]
+  -r, --reverse         reverse sorting
+  -j, --json            output in JSON format
   -v, --verbose
 </pre></div>
 </div>
 </section>
+<section id="the-rnid-utility">
+<h3>The rnid Utility<a class="headerlink" href="#the-rnid-utility" title="Permalink to this heading">#</a></h3>
+<p>With the <code class="docutils literal notranslate"><span class="pre">rnid</span></code> utility, you can generate, manage and view Reticulum Identities.
+The program can also calculate Destination hashes, and perform encryption and
+decryption of files.</p>
+<p>Using <code class="docutils literal notranslate"><span class="pre">rnid</span></code>, it is possible to asymmetrically encrypt files and information for
+any Reticulum destination hash, and also to create and verify cryptographic signatures.</p>
+<p><strong>Usage Examples</strong></p>
+<p>Generate a new Identity:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnid -g ./new_identity
+</pre></div>
+</div>
+<p>Display Identity key information:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnid -i ./new_identity -p
+
+Loaded Identity &lt;984b74a3f768bef236af4371e6f248cd&gt; from new_id
+Public Key  : 0f4259fef4521ab75a3409e353fe9073eb10783b4912a6a9937c57bf44a62c1e
+Private Key : Hidden
+</pre></div>
+</div>
+<p>Encrypt a file for an LXMF user:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnid -i 8dd57a738226809646089335a6b03695 -e my_file.txt
+
+Recalled Identity &lt;bc7291552be7a58f361522990465165c&gt; for destination &lt;8dd57a738226809646089335a6b03695&gt;
+Encrypting my_file.txt
+File my_file.txt encrypted for &lt;bc7291552be7a58f361522990465165c&gt; to my_file.txt.rfe
+</pre></div>
+</div>
+<p>If the Identity for the destination is not already known, you can fetch it from the network by using the <code class="docutils literal notranslate"><span class="pre">-R</span></code> command-line option:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnid -R -i 30602def3b3506a28ed33db6f60cc6c9 -e my_file.txt
+
+Requesting unknown Identity for &lt;30602def3b3506a28ed33db6f60cc6c9&gt;...
+Received Identity &lt;2b489d06eaf7c543808c76a5332a447d&gt; for destination &lt;30602def3b3506a28ed33db6f60cc6c9&gt; from the network
+Encrypting my_file.txt
+File my_file.txt encrypted for &lt;2b489d06eaf7c543808c76a5332a447d&gt; to my_file.txt.rfe
+</pre></div>
+</div>
+<p>Decrypt a file using the Reticulum Identity it was encrypted for:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnid -i ./my_identity -d my_file.txt.rfe
+
+Loaded Identity &lt;2225fdeecaf6e2db4556c3c2d7637294&gt; from ./my_identity
+Decrypting ./my_file.txt.rfe...
+File ./my_file.txt.rfe decrypted with &lt;2225fdeecaf6e2db4556c3c2d7637294&gt; to ./my_file.txt
+</pre></div>
+</div>
+<p><strong>All Command-Line Options</strong></p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnid.py [-h] [--config path] [-i identity] [-g path] [-v] [-q] [-a aspects]
+               [-H aspects] [-e path] [-d path] [-s path] [-V path] [-r path] [-w path]
+               [-f] [-R] [-t seconds] [-p] [-P] [--version]
+
+Reticulum Identity &amp; Encryption Utility
+
+options:
+  -h, --help            show this help message and exit
+  --config path         path to alternative Reticulum config directory
+  -i identity, --identity identity
+                        hexadecimal Reticulum Destination hash or path to Identity file
+  -g path, --generate path
+                        generate a new Identity
+  -v, --verbose         increase verbosity
+  -q, --quiet           decrease verbosity
+  -a aspects, --announce aspects
+                        announce a destination based on this Identity
+  -H aspects, --hash aspects
+                        show destination hashes for other aspects for this Identity
+  -e path, --encrypt path
+                        encrypt file
+  -d path, --decrypt path
+                        decrypt file
+  -s path, --sign path  sign file
+  -V path, --validate path
+                        validate signature
+  -r path, --read path  input file path
+  -w path, --write path
+                        output file path
+  -f, --force           write output even if it overwrites existing files
+  -R, --request         request unknown Identities from the network
+  -t seconds            identity request timeout before giving up
+  -p, --print-identity  print identity info and exit
+  -P, --print-private   allow displaying private keys
+  --version             show program&#39;s version number and exit
+</pre></div>
+</div>
+</section>
 <section id="the-rnpath-utility">
 <h3>The rnpath Utility<a class="headerlink" href="#the-rnpath-utility" title="Permalink to this heading">#</a></h3>
 <p>With the <code class="docutils literal notranslate"><span class="pre">rnpath</span></code> utility, you can look up and view paths for
 destinations on the Reticulum network.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span># Run rnpath
-rnpath c89b4da064bf66d280f0e4d8abfd9806
+<p><strong>Usage Examples</strong></p>
+<p>Resolve path to a destination:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnpath c89b4da064bf66d280f0e4d8abfd9806
 
-# Example output
 Path found, destination &lt;c89b4da064bf66d280f0e4d8abfd9806&gt; is 4 hops away via &lt;f53a1c4278e0726bb73fcc623d6ce763&gt; on TCPInterface[Testnet/dublin.connect.reticulum.network:4965]
 </pre></div>
 </div>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnpath [-h] [--config CONFIG] [--version] [-t] [-r] [-d] [-D] [-w seconds] [-v] [destination]
+<p><strong>All Command-Line Options</strong></p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnpath.py [-h] [--config CONFIG] [--version] [-t] [-r] [-d] [-D]
+                 [-x] [-w seconds] [-v] [destination]
 
 Reticulum Path Discovery Utility
 
 positional arguments:
   destination           hexadecimal hash of the destination
 
-optional arguments:
+options:
   -h, --help            show this help message and exit
   --config CONFIG       path to alternative Reticulum config directory
   --version             show program&#39;s version number and exit
@@ -477,6 +625,7 @@ optional arguments:
   -r, --rates           show announce rate info
   -d, --drop            remove the path to a destination
   -D, --drop-announces  drop all queued announces
+  -x, --drop-via        drop all paths via specified transport instance
   -w seconds            timeout before giving up
   -v, --verbose
 </pre></div>
@@ -487,17 +636,41 @@ optional arguments:
 <p>The <code class="docutils literal notranslate"><span class="pre">rnprobe</span></code> utility lets you probe a destination for connectivity, similar
 to the <code class="docutils literal notranslate"><span class="pre">ping</span></code> program. Please note that probes will only be answered if the
 specified destination is configured to send proofs for received packets. Many
-destinations will not have this option enabled, and will not be probable.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span># Run rnprobe
-rnprobe example_utilities.echo.request 2d03725b327348980d570f739a3a5708
+destinations will not have this option enabled, so most destinations will not
+be probable.</p>
+<p>You can enable a probe-reply destination on Reticulum Transport Instances by
+setting the <code class="docutils literal notranslate"><span class="pre">respond_to_probes</span></code> configuration directive. Reticulum will then
+print the probe destination to the log on Transport Instance startup.</p>
+<p><strong>Usage Examples</strong></p>
+<p>Probe a destination:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnprobe rnstransport.probe 2d03725b327348980d570f739a3a5708
 
-# Example output
 Sent 16 byte probe to &lt;2d03725b327348980d570f739a3a5708&gt;
 Valid reply received from &lt;2d03725b327348980d570f739a3a5708&gt;
 Round-trip time is 38.469 milliseconds over 2 hops
 </pre></div>
 </div>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnprobe [-h] [--config CONFIG] [--version] [-v] [full_name] [destination_hash]
+<p>Send a larger probe:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnprobe rnstransport.probe 2d03725b327348980d570f739a3a5708 -s 256
+
+Sent 16 byte probe to &lt;2d03725b327348980d570f739a3a5708&gt;
+Valid reply received from &lt;2d03725b327348980d570f739a3a5708&gt;
+Round-trip time is 38.781 milliseconds over 2 hops
+</pre></div>
+</div>
+<p>If the interface that receives the probe replies supports reporting radio
+parameters such as <strong>RSSI</strong> and <strong>SNR</strong>, the <code class="docutils literal notranslate"><span class="pre">rnprobe</span></code> utility will print
+these as part of the result as well.</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnprobe rnstransport.probe e7536ee90bd4a440e130490b87a25124
+
+Sent 16 byte probe to &lt;e7536ee90bd4a440e130490b87a25124&gt;
+Valid reply received from &lt;e7536ee90bd4a440e130490b87a25124&gt;
+Round-trip time is 1.809 seconds over 1 hop [RSSI -73 dBm] [SNR 12.0 dB]
+</pre></div>
+</div>
+<p><strong>All Command-Line Options</strong></p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnprobe [-h] [--config CONFIG] [--version] [-v] [-s SIZE]
+               [full_name] [destination_hash]
 
 Reticulum Probe Utility
 
@@ -508,6 +681,7 @@ positional arguments:
 optional arguments:
   -h, --help        show this help message and exit
   --config CONFIG   path to alternative Reticulum config directory
+  -s SIZE, --size SIZE  size of probe packet payload in bytes
   --version         show program&#39;s version number and exit
   -v, --verbose
 </pre></div>
@@ -517,16 +691,27 @@ optional arguments:
 <h3>The rncp Utility<a class="headerlink" href="#the-rncp-utility" title="Permalink to this heading">#</a></h3>
 <p>The <code class="docutils literal notranslate"><span class="pre">rncp</span></code> utility is a simple file transfer tool. Using it, you can transfer
 files through Reticulum.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span># Run rncp on the receiving system, specifying which identities
-# are allowed to send files
-rncp --receive -a 1726dbad538775b5bf9b0ea25a4079c8 -a c50cc4e4f7838b6c31f60ab9032cbc62
-
-# From another system, copy a file to the receiving system
-rncp ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
+<p><strong>Usage Examples</strong></p>
+<p>Run rncp on the receiving system, specifying which identities are allowed to send files:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rncp --listen -a 1726dbad538775b5bf9b0ea25a4079c8 -a c50cc4e4f7838b6c31f60ab9032cbc62
 </pre></div>
 </div>
-<p>You can specify as many allowed senders as needed, or complete disable authentication.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rncp [-h] [--config path] [-v] [-q] [-p] [-r] [-b] [-a allowed_hash] [-n] [-w seconds] [--version] [file] [destination]
+<p>You can also specify allowed identity hashes (one per line) in the file ~/.rncp/allowed_identities
+and simply running the program in listener mode:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rncp --listen
+</pre></div>
+</div>
+<p>From another system, copy a file to the receiving system:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rncp ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
+</pre></div>
+</div>
+<p>Or fetch a file from the remote system:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rncp --fetch ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
+</pre></div>
+</div>
+<p><strong>All Command-Line Options</strong></p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rncp.py [-h] [--config path] [-v] [-q] [-S] [-l] [-f] [-b seconds]
+               [-a allowed_hash] [-n] [-p] [-w seconds] [--version] [file] [destination]
 
 Reticulum File Transfer Utility
 
@@ -534,19 +719,20 @@ positional arguments:
   file                  file to be transferred
   destination           hexadecimal hash of the receiver
 
-optional arguments:
+options:
   -h, --help            show this help message and exit
   --config path         path to alternative Reticulum config directory
   -v, --verbose         increase verbosity
   -q, --quiet           decrease verbosity
-  -p, --print-identity  print identity and destination info and exit
-  -r, --receive         wait for incoming files
-  -b, --no-announce     don&#39;t announce at program start
+  -S, --silent          disable transfer progress output
+  -l, --listen          listen for incoming transfer requests
+  -f, --fetch           fetch file from remote listener instead of sending
+  -b seconds            announce interval, 0 to only announce at startup
   -a allowed_hash       accept from this identity
-  -n, --no-auth         accept files from anyone
+  -n, --no-auth         accept files and fetches from anyone
+  -p, --print-identity  print identity and destination info and exit
   -w seconds            sender timeout before giving up
   --version             show program&#39;s version number and exit
-  -v, --verbose
 </pre></div>
 </div>
 </section>
@@ -554,27 +740,30 @@ optional arguments:
 <h3>The rnx Utility<a class="headerlink" href="#the-rnx-utility" title="Permalink to this heading">#</a></h3>
 <p>The <code class="docutils literal notranslate"><span class="pre">rnx</span></code> utility is a basic remote command execution program. It allows you to
 execute commands on remote systems over Reticulum, and to view returned command
-output.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span># Run rnx on the listening system, specifying which identities
-# are allowed to execute commands
-rnx --listen -a 941bed5e228775e5a8079fc38b1ccf3f -a 1b03013c25f1c2ca068a4f080b844a10
-
-# From another system, run a command
-rnx 7a55144adf826958a9529a3bcf08b149 &quot;cat /proc/cpuinfo&quot;
-
-# Or enter the interactive mode pseudo-shell
-rnx 7a55144adf826958a9529a3bcf08b149 -x
-
-# The default identity file is stored in
-# ~/.reticulum/identities/rnx, but you can use
-# another one, which will be created if it does
-# not already exist
-rnx 7a55144adf826958a9529a3bcf08b149 -i /path/to/identity -x
+output. For a fully interactive remote shell solution, be sure to also take a look
+at the <a class="reference external" href="https://github.com/acehoss/rnsh">rnsh</a> program.</p>
+<p><strong>Usage Examples</strong></p>
+<p>Run rnx on the listening system, specifying which identities are allowed to execute commands:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnx --listen -a 941bed5e228775e5a8079fc38b1ccf3f -a 1b03013c25f1c2ca068a4f080b844a10
 </pre></div>
 </div>
-<p>You can specify as many allowed senders as needed, or completely disable authentication.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnx [-h] [--config path] [-v] [-q] [-p] [-l] [-i identity] [-x] [-b] [-a allowed_hash] [-n] [-N] [-d] [-m] [-w seconds] [-W seconds] [--stdin STDIN] [--stdout STDOUT] [--stderr STDERR] [--version]
-           [destination] [command]
+<p>From another system, run a command on the remote:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnx 7a55144adf826958a9529a3bcf08b149 &quot;cat /proc/cpuinfo&quot;
+</pre></div>
+</div>
+<p>Or enter the interactive mode pseudo-shell:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnx 7a55144adf826958a9529a3bcf08b149 -x
+</pre></div>
+</div>
+<p>The default identity file is stored in <code class="docutils literal notranslate"><span class="pre">~/.reticulum/identities/rnx</span></code>, but you can use
+another one, which will be created if it does not already exist</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>$ rnx 7a55144adf826958a9529a3bcf08b149 -i /path/to/identity -x
+</pre></div>
+</div>
+<p><strong>All Command-Line Options</strong></p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnx [-h] [--config path] [-v] [-q] [-p] [-l] [-i identity] [-x] [-b] [-n] [-N]
+           [-d] [-m] [-a allowed_hash] [-w seconds] [-W seconds] [--stdin STDIN]
+           [--stdout STDOUT] [--stderr STDERR] [--version] [destination] [command]
 
 Reticulum Remote Execution Utility
 
@@ -610,9 +799,16 @@ optional arguments:
 <h3>The rnodeconf Utility<a class="headerlink" href="#the-rnodeconf-utility" title="Permalink to this heading">#</a></h3>
 <p>The <code class="docutils literal notranslate"><span class="pre">rnodeconf</span></code> utility allows you to inspect and configure existing <a class="reference internal" href="hardware.html#rnode-main"><span class="std std-ref">RNodes</span></a>, and
 to create and provision new <a class="reference internal" href="hardware.html#rnode-main"><span class="std std-ref">RNodes</span></a> from any supported hardware devices.</p>
-<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnodeconf [-h] [-i] [-a] [-u] [-U] [--fw-version version] [--nocheck] [-C] [-N] [-T] [-b] [-B] [-p] [--freq Hz] [--bw Hz] [--txp dBm] [--sf factor] [--cr rate] [--eeprom-backup] [--eeprom-dump] [--eeprom-wipe] [--version] [port]
+<p><strong>All Command-Line Options</strong></p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>usage: rnodeconf.py [-h] [-i] [-a] [-u] [-U] [--fw-version version] [--nocheck] [-e]
+                    [-E] [-C] [--baud-flash baud_flash] [-N] [-T] [-b] [-B] [-p] [-D i]
+                    [--freq Hz] [--bw Hz] [--txp dBm] [--sf factor] [--cr rate]
+                    [--eeprom-backup] [--eeprom-dump] [--eeprom-wipe] [-P]
+                    [--trust-key hexbytes] [--version] [port]
 
-RNode Configuration and firmware utility. This program allows you to change various settings and startup modes of RNode. It can also install, flash and update the firmware on supported devices.
+RNode Configuration and firmware utility. This program allows you to change various
+settings and startup modes of RNode. It can also install, flash and update the firmware
+on supported devices.
 
 positional arguments:
   port                  serial port where RNode is attached
@@ -628,11 +824,14 @@ options:
   -e, --extract         Extract firmware from connected RNode for later use
   -E, --use-extracted   Use the extracted firmware for autoinstallation or update
   -C, --clear-cache     Clear locally cached firmware files
+  --baud-flash baud_flash
+                        Set specific baud rate when flashing device. Default is 921600
   -N, --normal          Switch device to normal mode
   -T, --tnc             Switch device to TNC mode
   -b, --bluetooth-on    Turn device bluetooth on
   -B, --bluetooth-off   Turn device bluetooth off
   -p, --bluetooth-pair  Put device into bluetooth pairing mode
+  -D i, --display i     Set display intensity (0-255)
   --freq Hz             Frequency in Hz for TNC mode
   --bw Hz               Bandwidth in Hz for TNC mode
   --txp dBm             TX power in dBm for TNC mode
@@ -641,6 +840,8 @@ options:
   --eeprom-backup       Backup EEPROM to file
   --eeprom-dump         Dump EEPROM to console
   --eeprom-wipe         Unlock and wipe EEPROM
+  -P, --public          Display public part of signing key
+  --trust-key hexbytes  Public key to trust for device verification
   --version             Print program version and exit
 </pre></div>
 </div>
@@ -685,6 +886,8 @@ assignment varies from one boot to another.</p>
 <span id="using-systemd"></span><h3>Reticulum as a System Service<a class="headerlink" href="#reticulum-as-a-system-service" title="Permalink to this heading">#</a></h3>
 <p>Instead of starting Reticulum manually, you can install <code class="docutils literal notranslate"><span class="pre">rnsd</span></code> as a system
 service and have it start automatically at boot.</p>
+<section id="systemwide-service">
+<h4>Systemwide Service<a class="headerlink" href="#systemwide-service" title="Permalink to this heading">#</a></h4>
 <p>If you installed Reticulum with <code class="docutils literal notranslate"><span class="pre">pip</span></code>, the <code class="docutils literal notranslate"><span class="pre">rnsd</span></code> program will most likely
 be located in a user-local installation path only, which means <code class="docutils literal notranslate"><span class="pre">systemd</span></code> will not
 be able to execute it. In this case, you can simply symlink the <code class="docutils literal notranslate"><span class="pre">rnsd</span></code> program
@@ -725,6 +928,43 @@ WantedBy=multi-user.target
 </pre></div>
 </div>
 </section>
+<section id="userspace-service">
+<h4>Userspace Service<a class="headerlink" href="#userspace-service" title="Permalink to this heading">#</a></h4>
+<p>Alternatively you can use a user systemd service instead of a system wide one. This way the whole setup can be done as a regular user.
+Create a user systemd service files <code class="docutils literal notranslate"><span class="pre">~/.config/systemd/user/rnsd.service</span></code> with the following content:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>[Unit]
+Description=Reticulum Network Stack Daemon
+After=default.target
+
+[Service]
+# If you run Reticulum on WiFi devices,
+# or other devices that need some extra
+# time to initialise, you might want to
+# add a short delay before Reticulum is
+# started by systemd:
+# ExecStartPre=/bin/sleep 10
+Type=simple
+Restart=always
+RestartSec=3
+ExecStart=RNS_BIN_DIR/rnsd --service
+
+[Install]
+WantedBy=default.target
+</pre></div>
+</div>
+<p>Replace <code class="docutils literal notranslate"><span class="pre">RNS_BIN_DIR</span></code> with the path to your Reticulum binary directory (eg. /home/USERNAMEHERE/rns/bin).</p>
+<p>Start user service:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>systemctl --user daemon-reload
+systemctl --user start rnsd.service
+</pre></div>
+</div>
+<p>If you want to automatically start <code class="docutils literal notranslate"><span class="pre">rnsd</span></code> without having to log in as the USERNAMEHERE, do:</p>
+<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>sudo loginctl enable-linger USERNAMEHERE
+systemctl --user enable rnsd.service
+</pre></div>
+</div>
+</section>
+</section>
 </section>
 </section>
 
@@ -789,6 +1029,7 @@ WantedBy=multi-user.target
 <li><a class="reference internal" href="#included-utility-programs">Included Utility Programs</a><ul>
 <li><a class="reference internal" href="#the-rnsd-utility">The rnsd Utility</a></li>
 <li><a class="reference internal" href="#the-rnstatus-utility">The rnstatus Utility</a></li>
+<li><a class="reference internal" href="#the-rnid-utility">The rnid Utility</a></li>
 <li><a class="reference internal" href="#the-rnpath-utility">The rnpath Utility</a></li>
 <li><a class="reference internal" href="#the-rnprobe-utility">The rnprobe Utility</a></li>
 <li><a class="reference internal" href="#the-rncp-utility">The rncp Utility</a></li>
@@ -798,7 +1039,11 @@ WantedBy=multi-user.target
 </li>
 <li><a class="reference internal" href="#improving-system-configuration">Improving System Configuration</a><ul>
 <li><a class="reference internal" href="#fixed-serial-port-names">Fixed Serial Port Names</a></li>
-<li><a class="reference internal" href="#reticulum-as-a-system-service">Reticulum as a System Service</a></li>
+<li><a class="reference internal" href="#reticulum-as-a-system-service">Reticulum as a System Service</a><ul>
+<li><a class="reference internal" href="#systemwide-service">Systemwide Service</a></li>
+<li><a class="reference internal" href="#userspace-service">Userspace Service</a></li>
+</ul>
+</li>
 </ul>
 </li>
 </ul>

docs/manual/whatis.html

@@ -2,13 +2,13 @@
 <html class="no-js" lang="en">
   <head><meta charset="utf-8"/>
     <meta name="viewport" content="width=device-width,initial-scale=1"/>
-    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
+    <meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
 <link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="Getting Started Fast" href="gettingstartedfast.html" /><link rel="prev" title="Reticulum Network Stack Manual" href="index.html" />
 
-    <meta name="generator" content="sphinx-5.2.2, furo 2022.09.29"/>
-        <title>What is Reticulum? - Reticulum Network Stack 0.5.2 beta documentation</title>
+    <meta name="generator" content="sphinx-5.3.0, furo 2022.09.29.dev1"/>
+        <title>What is Reticulum? - Reticulum Network Stack 0.7.5 beta documentation</title>
       <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
-    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=d81277517bee4d6b0349d71bb2661d4890b5617c" />
+    <link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=189ec851f9bb375a2509b67be1f64f0cf212b702" />
     <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
     <link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
     <link rel="stylesheet" type="text/css" href="_static/custom.css" />
@@ -141,7 +141,7 @@
       </label>
     </div>
     <div class="header-center">
-      <a href="index.html"><div class="brand">Reticulum Network Stack 0.5.2 beta documentation</div></a>
+      <a href="index.html"><div class="brand">Reticulum Network Stack 0.7.5 beta documentation</div></a>
     </div>
     <div class="header-right">
       <div class="theme-toggle-container theme-toggle-header">
@@ -167,10 +167,10 @@
     <img class="sidebar-logo" src="_static/rns_logo_512.png" alt="Logo"/>
   </div>
   
-  <span class="sidebar-brand-text">Reticulum Network Stack 0.5.2 beta documentation</span>
+  <span class="sidebar-brand-text">Reticulum Network Stack 0.7.5 beta documentation</span>
   
 </a><form class="sidebar-search-container" method="get" action="search.html" role="search">
-  <input class="sidebar-search" placeholder=Search name="q" aria-label="Search">
+  <input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
   <input type="hidden" name="check_keywords" value="yes">
   <input type="hidden" name="area" value="default">
 </form>
@@ -181,7 +181,7 @@
 <li class="toctree-l1"><a class="reference internal" href="using.html">Using Reticulum on Your System</a></li>
 <li class="toctree-l1"><a class="reference internal" href="understanding.html">Understanding Reticulum</a></li>
 <li class="toctree-l1"><a class="reference internal" href="hardware.html">Communications Hardware</a></li>
-<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Supported Interfaces</a></li>
+<li class="toctree-l1"><a class="reference internal" href="interfaces.html">Configuring Interfaces</a></li>
 <li class="toctree-l1"><a class="reference internal" href="networks.html">Building Networks</a></li>
 <li class="toctree-l1"><a class="reference internal" href="examples.html">Code Examples</a></li>
 <li class="toctree-l1"><a class="reference internal" href="support.html">Support Reticulum</a></li>
@@ -238,21 +238,21 @@ outside control, manipulation or censorship.</p>
 networks, without any need for hierarchical or beaureucratic structures to control
 or manage them, while ensuring individuals and communities full sovereignty
 over their own network segments.</p>
-<p>Reticulum is a complete networking stack, and does not need IP or higher
+<p>Reticulum is a <strong>complete networking stack</strong>, and does not need IP or higher
 layers, although it is easy to utilise IP (with TCP or UDP) as the underlying
 carrier for Reticulum. It is therefore trivial to tunnel Reticulum over the
 Internet or private IP networks. Reticulum is built directly on cryptographic
 principles, allowing resilience and stable functionality in open and trustless
 networks.</p>
-<p>No kernel modules or drivers are required. Reticulum runs completely in
-userland, and can run on practically any system that runs Python 3. Reticulum
+<p>No kernel modules or drivers are required. Reticulum can run completely in
+userland, and will run on practically any system that runs Python 3. Reticulum
 runs well even on small single-board computers like the Pi Zero.</p>
 <section id="current-status">
 <h2>Current Status<a class="headerlink" href="#current-status" title="Permalink to this heading">#</a></h2>
 <p><strong>Please know!</strong> Reticulum should currently be considered beta software. All core protocol
 features are implemented and functioning, but additions will probably occur as
 real-world use is explored. <em>There will be bugs</em>. The API and wire-format can be
-considered stable at the moment, but could change if absolutely warranted.</p>
+considered complete and stable at the moment, but could change if absolutely warranted.</p>
 </section>
 <section id="what-does-reticulum-offer">
 <h2>What does Reticulum Offer?<a class="headerlink" href="#what-does-reticulum-offer" title="Permalink to this heading">#</a></h2>
@@ -275,7 +275,7 @@ considered stable at the moment, but could change if absolutely warranted.</p>
 <li><p>An intuitive and developer-friendly API</p></li>
 <li><p>Efficient link establishment</p>
 <ul>
-<li><p>Total bandwidth cost of setting up a link is only 3 packets, totalling 297 bytes</p></li>
+<li><p>Total cost of setting up an encrypted and verified link is only 3 packets, totalling 297 bytes</p></li>
 <li><p>Low cost of keeping links open at only 0.44 bits per second</p></li>
 </ul>
 </li>
@@ -293,7 +293,7 @@ considered stable at the moment, but could change if absolutely warranted.</p>
 <section id="where-can-reticulum-be-used">
 <h2>Where can Reticulum be Used?<a class="headerlink" href="#where-can-reticulum-be-used" title="Permalink to this heading">#</a></h2>
 <p>Over practically any medium that can support at least a half-duplex channel
-with 500 bits per second throughput, and an MTU of 500 bytes. Data radios,
+with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios,
 modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes,
 ad-hoc WiFi, free-space optical links and similar systems are all examples
 of the types of interfaces Reticulum was designed for.</p>

docs/source/gettingstartedfast.rst

@@ -70,7 +70,8 @@ Remote Shell
 
 The `rnsh <https://github.com/acehoss/rnsh>`_ program lets you establish fully interactive
 remote shell sessions over Reticulum. It also allows you to pipe any program to or from a
-remote system, and is similar to how ``ssh`` works.
+remote system, and is similar to how ``ssh`` works. The ``rnsh`` is very efficient, and
+can facilitate fully interactive shell sessions, even over extremely low-bandwidth links.
 
 Nomad Network
 ^^^^^^^^^^^^^
@@ -179,29 +180,29 @@ and :ref:`Interfaces<interfaces-main>` chapters of this manual.
 Connecting Reticulum Instances Over the Internet
 ================================================
 Reticulum currently offers two interfaces suitable for connecting instances over the Internet: :ref:`TCP<interfaces-tcps>`
-and :ref:`I2P<interfaces-i2p>`. Each interface offers a different set of features, and Reticulum 
-users should carefully choose the interface which best suites their needs. 
+and :ref:`I2P<interfaces-i2p>`. Each interface offers a different set of features, and Reticulum
+users should carefully choose the interface which best suites their needs.
 
 The ``TCPServerInterface`` allows users to host an instance accessible over TCP/IP. This
 method is generally faster, lower latency, and more energy efficient than using ``I2PInterface``,
 however it also leaks more data about the server host.
 
 TCP connections reveal the IP address of both your instance and the server to anyone who can
-inspect the connection. Someone could use this information to determine your location or identity. Adversaries 
+inspect the connection. Someone could use this information to determine your location or identity. Adversaries
 inspecting your packets may be able to record packet metadata like time of transmission and packet size.
 Even though Reticulum encrypts traffic, TCP does not, so an adversary may be able to use
 packet inspection to learn that a system is running Reticulum, and what other IP addresses connect to it.
 Hosting a publicly reachable instance over TCP also requires a publicly reachable IP address,
 which most Internet connections don't offer anymore.
 
-The ``I2PInterface`` routes messages through the `Invisible Internet Protocol 
+The ``I2PInterface`` routes messages through the `Invisible Internet Protocol
 (I2P) <https://geti2p.net/en/>`_. To use this interface, users must also run an I2P daemon in
-parallel to ``rnsd``. For always-on I2P nodes it is recommended to use `i2pd <https://i2pd.website/>`_. 
+parallel to ``rnsd``. For always-on I2P nodes it is recommended to use `i2pd <https://i2pd.website/>`_.
 
-By default, I2P will encrypt and mix all traffic sent over the Internet, and 
-hide both the sender and receiver Reticulum instance IP addresses. Running an I2P node 
+By default, I2P will encrypt and mix all traffic sent over the Internet, and
+hide both the sender and receiver Reticulum instance IP addresses. Running an I2P node
 will also relay other I2P user's encrypted packets, which will use extra
-bandwidth and compute power, but also makes timing attacks and other forms of 
+bandwidth and compute power, but also makes timing attacks and other forms of
 deep-packet-inspection much more difficult.
 
 I2P also allows users to host globally available Reticulum instances from non-public IP's and behind firewalls and NAT.
@@ -219,11 +220,11 @@ by adding one of the following interfaces to your ``.reticulum/config`` file:
 
 .. code::
 
-  # TCP/IP interface to the Dublin hub
-  [[RNS Testnet Dublin]]
+  # TCP/IP interface to the RNS Amsterdam Hub
+  [[RNS Testnet Amsterdam]]
     type = TCPClientInterface
     enabled = yes
-    target_host = dublin.connect.reticulum.network
+    target_host = amsterdam.connect.reticulum.network
     target_port = 4965
 
   # TCP/IP interface to the BetweenTheBorders Hub (community-provided)
@@ -233,11 +234,11 @@ by adding one of the following interfaces to your ``.reticulum/config`` file:
     target_host = betweentheborders.com
     target_port = 4242
 
-  # Interface to I2P hub A
-  [[RNS Testnet I2P Hub A]]
+  # Interface to Testnet I2P Hub
+  [[RNS Testnet I2P Hub]]
     type = I2PInterface
     enabled = yes
-    peers = uxg5kubabakh3jtnvsipingbr5574dle7bubvip7llfvwx2tgrua.b32.i2p
+    peers = g3br23bvx3lq5uddcsjii74xgmn6y5q325ovrkq2zw2wbzbqgbuq.b32.i2p
 
 Many other Reticulum instances are connecting to this testnet, and you can also join it
 via other entry points if you know them. There is absolutely no control over the network
@@ -281,7 +282,7 @@ started is to install the latest release of Reticulum via pip:
 
 .. code::
 
-   pip3 install rns
+   pip install rns
 
 The above command will install Reticulum and dependencies, and you will be
 ready to import and use RNS in your own programs. The next step will most
@@ -291,7 +292,7 @@ For extended functionality, you can install optional dependencies:
 
 .. code::
 
-   pip3 install pyserial
+   pip install pyserial
 
 
 Further information can be found in the :ref:`API Reference<api-main>`.
@@ -306,7 +307,7 @@ don't use pip, but try this recipe:
 .. code::
 
     # Install dependencies
-    pip3 install cryptography pyserial
+    pip install cryptography pyserial
 
     # Clone repository
     git clone https://github.com/markqvist/Reticulum.git
@@ -316,25 +317,25 @@ don't use pip, but try this recipe:
     ln -s ../RNS ./Examples/
 
     # Run an example
-    python3 Examples/Echo.py -s
+    python Examples/Echo.py -s
 
     # Unless you've manually created a config file, Reticulum will do so now,
     # and immediately exit. Make any necessary changes to the file:
     nano ~/.reticulum/config
 
     # ... and launch the example again.
-    python3 Examples/Echo.py -s
+    python Examples/Echo.py -s
 
     # You can now repeat the process on another computer,
     # and run the same example with -h to get command line options.
-    python3 Examples/Echo.py -h
+    python Examples/Echo.py -h
 
     # Run the example in client mode to "ping" the server.
     # Replace the hash below with the actual destination hash of your server.
-    python3 Examples/Echo.py 174a64852a75682259ad8b921b8bf416
+    python Examples/Echo.py 174a64852a75682259ad8b921b8bf416
 
     # Have a look at another example
-    python3 Examples/Filetransfer.py -h
+    python Examples/Filetransfer.py -h
 
 When you have experimented with the basic examples, it's time to go read the
 :ref:`Understanding Reticulum<understanding-main>` chapter. Before submitting
@@ -414,7 +415,7 @@ locally on your device using the following command:
 
 It is also possible to include Reticulum in apps compiled and distributed as
 Android APKs. A detailed tutorial and example source code will be included
-here at a later point. Until then you can use the `Sideband source code <https://github.com/markqvist/sideband>`_ as an example and startig point.
+here at a later point. Until then you can use the `Sideband source code <https://github.com/markqvist/sideband>`_ as an example and starting point.
 
 
 ARM64
@@ -446,15 +447,15 @@ detailed in this manual.
 
 Debian Bookworm
 ^^^^^^^^^^^^^^^^^^^^^^^^
-On versions of Debian released after April 2023, it is no longer possible
+On versions of Debian released after April 2023, it is no longer possible by default
 to use ``pip`` to install packages onto your system. Unfortunately, you will need to
 use the replacement ``pipx`` command instead, which places installed packages in an
-isolated environment. This should not negatively affect Reticulum, but installation
-via this method is not fully tested yet.
+isolated environment. This should not negatively affect Reticulum, but will not work
+for including and using Reticulum in your own scripts and programs.
 
 .. code::
 
-    # Install pipx 
+    # Install pipx
     sudo apt install pipx
 
     # Make installed programs available on the command line
@@ -463,18 +464,32 @@ via this method is not fully tested yet.
     # Install Reticulum
     pipx install rns
 
+Alternatively, you can restore normal behaviour to ``pip`` by creating or editing
+the configuration file located at ``~/.config/pip/pip.conf``, and adding the
+following section:
+
+.. code:: text
+
+    [global]
+    break-system-packages = true
+
+Please note that the "break-system-packages" directive is a somewhat misleading choice
+of words. Setting it will of course not break any system packages, but will simply
+allow installing ``pip`` packages user- and system-wide. While this *could* in rare
+cases lead to version conflicts, it does not generally pose any problems.
+
 
 Ubuntu Lunar
 ^^^^^^^^^^^^^^^^^^^^^^^^
-On versions of Ubuntu released after April 2023, it is no longer possible
+On versions of Ubuntu released after April 2023, it is no longer possible by default
 to use ``pip`` to install packages onto your system. Unfortunately, you will need to
 use the replacement ``pipx`` command instead, which places installed packages in an
-isolated environment. This should not negatively affect Reticulum, but installation
-via this method is not fully tested yet.
+isolated environment. This should not negatively affect Reticulum, but will not work
+for including and using Reticulum in your own scripts and programs.
 
 .. code::
 
-    # Install pipx 
+    # Install pipx
     sudo apt install pipx
 
     # Make installed programs available on the command line
@@ -483,14 +498,24 @@ via this method is not fully tested yet.
     # Install Reticulum
     pipx install rns
 
+Alternatively, you can restore normal behaviour to ``pip`` by creating or editing
+the configuration file located at ``~/.config/pip/pip.conf``, and adding the
+following section:
+
+.. code:: text
+
+    [global]
+    break-system-packages = true
+
+Please note that the "break-system-packages" directive is a somewhat misleading choice
+of words. Setting it will of course not break any system packages, but will simply
+allow installing ``pip`` packages user- and system-wide. While this _could_ in rare
+cases lead to version conflicts, it does not generally pose any problems.
 
 Pure-Python Reticulum
 ==============================================
 In some rare cases, and on more obscure system types, it is not possible to
-install one or more dependencies
-
-On more unusual systems, and in some rare cases, it might not be possible to
-install or even compile one or more of the above modules. In such situations,
+install one or more dependencies. In such situations,
 you can use the ``rnspure`` package instead of the ``rns`` package, or use ``pip``
 with the ``--no-dependencies`` command-line option. The ``rnspure``
 package requires no external dependencies for installation. Please note that the

docs/source/hardware.rst

@@ -24,11 +24,20 @@ starting from scratch.
 This chapter will outline a few different sensible starting paths to get
 real-world functional wireless communications up and running with minimal cost
 and effort. Two fundamental devices categories will be covered, *RNodes* and
-*WiFi-based radios*.
+*WiFi-based radios*. Additionally, other common options will be briefly described.
+
+Knowing how to employ just a few different types of hardware will make it possible
+to build a wide range of useful networks with little effort.
+
+Combining Hardware Types
+========================
+
+It is useful to combine different link and hardware types when designing and
+building a network. One useful design pattern is to employ high-capacity point-to-point
+links based on WiFi or millimeter-wave radios (with high-gain directional antennas)
+for the network backbone, and using LoRa-based RNodes for covering large areas with
+connectivity for client devices.
 
-While there are many other device categories that are useful in building Reticulum
-networks, knowing how to employ just these two will make it possible to build
-a wide range of useful networks with little effort.
 
 .. _rnode-main:
 
@@ -190,13 +199,6 @@ such as serial port and on-air parameters. For v2.x firmwares, you just need to
 the Connection ID of the RNode, and Reticulum will automatically locate and connect to the
 RNode, using the parameters stored in the RNode itself.
 
-.. _rnode-suppliers:
-
-Suppliers
-^^^^^^^^^
-Get in touch if you want to have your RNode supplier listed here, or if you want help to
-get started with producing RNodes.
-
 
 WiFi-based Hardware
 ===================
@@ -231,11 +233,31 @@ that is relatively cheap while providing long range and high capacity for Reticu
 networks. As in all other cases, it is also possible for Reticulum to co-exist with IP
 networks running concurrently on such devices.
 
-Combining Hardware Types
-========================
+Ethernet-based Hardware
+=======================
 
-It is useful to combine different link and hardware types when designing and
-building a network. One useful design pattern is to employ high-capacity point-to-point
-links based on WiFi or millimeter-wave radios (with high-gain directional antennas)
-for the network backbone, and using LoRa-based RNodes for covering large areas with
-connectivity for client devices.
+Reticulum can run over any kind of hardware that can provide a switched Ethernet-based
+medium. This means that anything from a plain Ethernet switch, to fiber-optic systems,
+to data radios with Ethernet interfaces can be used by Reticulum.
+
+The Ethernet medium does not need to have any IP infrastructure such as DHCP servers
+or routing set up, but in case such infrastructure does exist, Reticulum will simply
+co-exist with.
+
+To use Reticulum over Ethernet-based mediums, it is generally enough to use the included
+:ref:`AutoInterface<interfaces-auto>`. This interface also works over any kind of
+virtual networking adapter, such as ``tun`` and ``tap`` devices in Linux.
+
+Serial Lines & Devices
+======================
+
+Using Reticulum over any kind of raw serial line is also possible with the
+:ref:`SerialInterface<interfaces-serial>`. This interface type is also useful for
+using Reticulum over communications hardware that provides a serial port interface.
+
+Packet Radio Modems
+===================
+
+Any packet radio modem that provides a standard KISS interface over USB, serial or TCP
+can be used with Reticulum. This includes virtual software modems such as
+`FreeDV TNC <https://github.com/xssfox/freedv-tnc>`_ and `Dire Wolf <https://github.com/wb2osz/direwolf>`_.

docs/source/interfaces.rst

@@ -1,9 +1,9 @@
 
 .. _interfaces-main:
 
-********************
-Supported Interfaces
-********************
+**********************
+Configuring Interfaces
+**********************
 
 Reticulum supports using many kinds of devices as networking interfaces, and
 allows you to mix and match them in any way you choose. The number of distinct
@@ -33,9 +33,20 @@ system, which should be enabled by default in almost all OSes.
 
 .. code::
 
-  # This example demonstrates a TCP server interface.
-  # It will listen for incoming connections on the
-  # specified IP address and port number.
+  # This example demonstrates a bare-minimum setup
+  # of an Auto Interface. It will allow communica-
+  # tion with all other reachable devices on all
+  # usable physical ethernet-based devices that
+  # are available on the system.
+
+  [[Default Interface]]
+    type = AutoInterface
+    interface_enabled = True
+
+  # This example demonstrates an more specifically
+  # configured Auto Interface, that only uses spe-
+  # cific physical interfaces, and has a number of
+  # other configuration options set.
   
   [[Default Interface]]
     type = AutoInterface
@@ -47,6 +58,12 @@ system, which should be enabled by default in almost all OSes.
 
     group_id = reticulum
 
+    # You can also choose the multicast address type:
+    # temporary (default, Temporary Multicast Address)
+    # or permanent (Permanent Multicast Address)
+
+    multicast_address_type = permanent
+
     # You can also select specifically which
     # kernel networking devices to use.
 
@@ -365,6 +382,7 @@ can be used, and offers full control over LoRa parameters.
     # out identification on the channel with
     # a set interval by configuring the
     # following two parameters.
+    
     # id_callsign = MYCALL-0
     # id_interval = 600
 
@@ -372,7 +390,21 @@ can be used, and offers full control over LoRa parameters.
     # with low amounts of RAM, using packet
     # flow control can be useful. By default
     # it is disabled.
-    flow_control = False
+    
+    # flow_control = False
+
+    # It is possible to limit the airtime
+    # utilisation of an RNode by using the
+    # following two configuration options.
+    # The short-term limit is applied in a
+    # window of approximately 15 seconds,
+    # and the long-term limit is enforced
+    # over a rolling 60 minute window. Both
+    # options are specified in percent.
+    
+    # airtime_limit_long = 1.5
+    # airtime_limit_short = 33
+
 
 .. _interfaces-serial:
 
@@ -746,3 +778,74 @@ conserve bandwidth, while very fast networks can support applications that
 need very frequent announces. Reticulum implements these mechanisms to ensure
 that a large span of network types can seamlessly *co-exist* and interconnect.
 
+.. _interfaces-ingress-control:
+
+New Destination Rate Limiting
+=============================
+
+On public interfaces, where anyone may connect and announce new destinations,
+it can be useful to control the rate at which announces for *new*  destinations are
+processed.
+
+If a large influx of announces for newly created or previously unknown destinations
+occur within a short amount of time, Reticulum will place these announces on hold,
+so that announce traffic for known and previously established destinations can
+continue to be processed without interruptions.
+
+After the burst subsides, and an additional waiting period has passed, the held
+announces will be released at a slow rate, until the hold queue is cleared. This
+also means, that should a node decide to connect to a public interface, announce
+a large amount of bogus destinations, and then disconnect, these destination will
+never make it into path tables and waste network bandwidth on retransmitted
+announces.
+
+**It's important to note** that the ingress control works at the level of *individual
+sub-interfaces*. As an example, this means that one client on a :ref:`TCP Server Interface<interfaces-tcps>`
+cannot disrupt processing of incoming announces for other connected clients on the same
+:ref:`TCP Server Interface<interfaces-tcps>`. All other clients on the same interface will still have new announces
+processed without interruption.
+
+By default, Reticulum will handle this automatically, and ingress announce
+control will be enabled on interface where it is sensible to do so. It should
+generally not be neccessary to modify the ingress control configuration,
+but all the parameters are exposed for configuration if needed.
+
+ * | The ``ingress_control`` option tells Reticulum whether or not
+     to enable announce ingress control on the interface. Defaults to
+     ``True``.
+
+ * | The ``ic_new_time`` option configures how long (in seconds) an
+     interface is considered newly spawned. Defaults to ``2*60*60`` seconds. This
+     option is useful on publicly accessible interfaces that spawn new
+     sub-interfaces when a new client connects. 
+
+ * | The ``ic_burst_freq_new`` option sets the maximum announce ingress
+     frequency for newly spawned interfaces. Defaults to ``3.5``
+     announces per second.
+
+ * | The ``ic_burst_freq`` option sets the maximum announce ingress
+     frequency for other interfaces. Defaults to ``12`` announces
+     per second.
+
+     *If an interface exceeds its burst frequency, incoming announces
+     for unknown destinations will be temporarily held in a queue, and
+     not processed until later.*
+
+ * | The ``ic_max_held_announces`` option sets the maximum amount of
+     unique announces that will be held in the queue. Any additional
+     unique announces will be dropped. Defaults to ``256`` announces.
+
+ * | The ``ic_burst_hold`` option sets how much time (in seconds) must
+     pass after the burst frequency drops below its threshold, for the
+     announce burst to be considered cleared. Defaults to ``60``
+     seconds.
+
+ * | The ``ic_burst_penalty`` option sets how much time (in seconds) must
+     pass after the burst is considered cleared, before held announces can
+     start being released from the queue. Defaults to ``5*60``
+     seconds.
+
+ * | The ``ic_held_release_interval`` option sets how much time (in seconds)
+     must pass between releasing each held announce from the queue. Defaults
+     to ``30`` seconds.
+

docs/source/understanding.rst

@@ -75,7 +75,7 @@ guide the design of Reticulum:
     it can be easily modified and replicated by anyone interested in doing so.
 * **Very low bandwidth requirements**
     Reticulum should be able to function reliably over links with a transmission capacity as low
-    as *500 bits per second*.
+    as *5 bits per second*.
 * **Encryption by default**
     Reticulum must use strong encryption by default for all communication.
 * **Initiator Anonymity**
@@ -595,7 +595,7 @@ or less any medium that allows you to send and receive data, which satisfies som
 minimum requirements.
 
 The communication channel must support at least half-duplex operation, and provide an average
-throughput of around 500 bits per second, and supports a physical layer MTU of 500 bytes. The
+throughput of 5 bits per second or greater, and supports a physical layer MTU of 500 bytes. The
 Reticulum stack should be able to run on more or less any hardware that can provide a Python 3.x 
 runtime environment.
 
@@ -858,7 +858,7 @@ both on general-purpose CPUs and on microcontrollers. The necessary primitives a
 
 * Ed25519 for signatures
 
-* X22519 for ECDH key exchanges
+* X25519 for ECDH key exchanges
 
 * HKDF for key derivation
 

docs/source/using.rst

@@ -99,6 +99,17 @@ configuration file is created. The default configuration looks like this:
   instance_control_port = 37429
 
 
+  # On systems where running instances may not have access
+  # to the same shared Reticulum configuration directory,
+  # it is still possible to allow full interactivity for
+  # running instances, by manually specifying a shared RPC
+  # key. In almost all cases, this option is not needed, but
+  # it can be useful on operating systems such as Android.
+  # The key must be specified as bytes in hexadecimal.
+
+  # rpc_key = e5c032d3ec4e64a6aca9927ba8ab73336780f6d71790
+
+
   # You can configure Reticulum to panic and forcibly close
   # if an unrecoverable interface error occurs, such as the
   # hardware device for an interface disappearing. This is
@@ -108,6 +119,17 @@ configuration file is created. The default configuration looks like this:
   panic_on_interface_error = No
 
 
+  # When Transport is enabled, it is possible to allow the
+  # Transport Instance to respond to probe requests from
+  # the rnprobe utility. This can be a useful tool to test
+  # connectivity. When this option is enabled, the probe
+  # destination will be generated from the Identity of the
+  # Transport Instance, and printed to the log at startup.
+  # Optional, and disabled by default.
+
+  respond_to_probes = No
+
+
   [logging]
   # Valid log levels are 0 through 7:
   #   0: Log only critical information
@@ -145,10 +167,19 @@ configuration file is created. The default configuration looks like this:
 If Reticulum infrastructure already exists locally, you probably don't need to
 change anything, and you may already be connected to a wider network. If not,
 you will probably need to add relevant *interfaces* to the configuration, in
-order to communicate with other systems. It is a good idea to read the comments
-and explanations in the above default config. It will teach you the basic
-concepts you need to understand to configure your network. Once you have done that,
-take a look at the :ref:`Interfaces<interfaces-main>` chapter of this manual.
+order to communicate with other systems.
+
+You can generate a much more verbose configuration example by running the command:
+
+``rnsd --exampleconfig``
+
+The output includes examples for most interface types supported
+by Reticulum, along with additional options and configuration parameters.
+
+It is a good idea to read the comments and explanations in the above default config.
+It will teach you the basic concepts you need to understand to configure your network.
+Once you have done that, take a look at the :ref:`Interfaces<interfaces-main>` chapter
+of this manual.
 
 Included Utility Programs
 -------------------------
@@ -170,28 +201,47 @@ When ``rnsd`` is running, it will keep all configured interfaces open, handle tr
 it is enabled, and allow any other programs to immediately utilise the
 Reticulum network it is configured for.
 
-You can even run multiple instances of rnsd with different configurations on
+You can even run multiple instances of ``rnsd`` with different configurations on
 the same system.
 
-.. code:: text
+**Usage Examples**
 
-  # Install Reticulum
-  pip3 install rns
-
-  # Run rnsd
-  rnsd
+Run ``rnsd``:
 
 .. code:: text
 
-  usage: rnsd [-h] [--config CONFIG] [-v] [-q] [--version]
+  $ rnsd
+
+  [2023-08-18 17:59:56] [Notice] Started rnsd version 0.5.8
+
+Run ``rnsd`` in service mode, ensuring all logging output is sent directly to file:
+
+.. code:: text
+
+  $ rnsd -s
+
+Generate a verbose and detailed configuration example, with explanations of all the
+various configuration options, and interface configuration examples:
+
+.. code:: text
+
+  $ rnsd --exampleconfig
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnsd.py [-h] [--config CONFIG] [-v] [-q] [-s] [--exampleconfig] [--version]
 
   Reticulum Network Stack Daemon
 
-  optional arguments:
+  options:
     -h, --help       show this help message and exit
     --config CONFIG  path to alternative Reticulum config directory
     -v, --verbose
     -q, --quiet
+    -s, --service    rnsd is running as a service and should log to file
+    --exampleconfig  print verbose configuration example to stdout and exit
     --version        show program's version number and exit
 
 You can easily add ``rnsd`` as an always-on service by :ref:`configuring a service<using-systemd>`.
@@ -202,12 +252,14 @@ The rnstatus Utility
 Using the ``rnstatus`` utility, you can view the status of configured Reticulum
 interfaces, similar to the ``ifconfig`` program.
 
+**Usage Examples**
+
+Run ``rnstatus``:
+
 .. code:: text
 
-  # Run rnstatus
-  rnstatus
+  $ rnstatus
 
-  # Example output
   Shared Instance[37428]
      Status  : Up
      Serving : 1 program
@@ -240,44 +292,175 @@ interfaces, similar to the ``ifconfig`` program.
 
   Reticulum Transport Instance <5245a8efe1788c6a1cd36144a270e13b> running
 
+Filter output to only show some interfaces:
+
 .. code:: text
 
-  usage: rnstatus [-h] [--config CONFIG] [--version] [-a] [-v]
+  $ rnstatus rnode
+
+  RNodeInterface[RNode UHF]
+     Status  : Up
+     Mode    : Access Point
+     Rate    : 1.30 kbps
+     Access  : 64-bit IFAC by <…e702c42ba8>
+     Traffic : 8.49 KB↑
+               9.23 KB↓
+
+  Reticulum Transport Instance <5245a8efe1788c6a1cd36144a270e13b> running
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnstatus.py [-h] [--config CONFIG] [--version] [-a] [-A] [-s SORT]
+                     [-r] [-j] [-v] [filter]
 
   Reticulum Network Stack Status
 
-  optional arguments:
-    -h, --help       show this help message and exit
-    --config CONFIG  path to alternative Reticulum config directory
-    --version        show program's version number and exit
-    -a, --all        show all interfaces
+  positional arguments:
+    filter                only display interfaces with names including filter
+
+  options:
+    -h, --help            show this help message and exit
+    --config CONFIG       path to alternative Reticulum config directory
+    --version             show program's version number and exit
+    -a, --all             show all interfaces
+    -A, --announce-stats  show announce stats
+    -s SORT, --sort SORT  sort interfaces by [rate, traffic, rx, tx, announces, arx, atx, held]
+    -r, --reverse         reverse sorting
+    -j, --json            output in JSON format
     -v, --verbose
 
 
+The rnid Utility
+====================
+
+With the ``rnid`` utility, you can generate, manage and view Reticulum Identities.
+The program can also calculate Destination hashes, and perform encryption and
+decryption of files.
+
+Using ``rnid``, it is possible to asymmetrically encrypt files and information for
+any Reticulum destination hash, and also to create and verify cryptographic signatures.
+
+**Usage Examples**
+
+Generate a new Identity:
+
+.. code:: text
+
+  $ rnid -g ./new_identity
+
+Display Identity key information:
+
+.. code:: text
+
+  $ rnid -i ./new_identity -p
+
+  Loaded Identity <984b74a3f768bef236af4371e6f248cd> from new_id
+  Public Key  : 0f4259fef4521ab75a3409e353fe9073eb10783b4912a6a9937c57bf44a62c1e
+  Private Key : Hidden
+
+Encrypt a file for an LXMF user:
+
+.. code:: text
+
+  $ rnid -i 8dd57a738226809646089335a6b03695 -e my_file.txt
+
+  Recalled Identity <bc7291552be7a58f361522990465165c> for destination <8dd57a738226809646089335a6b03695>
+  Encrypting my_file.txt
+  File my_file.txt encrypted for <bc7291552be7a58f361522990465165c> to my_file.txt.rfe
+
+If the Identity for the destination is not already known, you can fetch it from the network by using the ``-R`` command-line option:
+
+.. code:: text
+
+  $ rnid -R -i 30602def3b3506a28ed33db6f60cc6c9 -e my_file.txt
+
+  Requesting unknown Identity for <30602def3b3506a28ed33db6f60cc6c9>...
+  Received Identity <2b489d06eaf7c543808c76a5332a447d> for destination <30602def3b3506a28ed33db6f60cc6c9> from the network
+  Encrypting my_file.txt
+  File my_file.txt encrypted for <2b489d06eaf7c543808c76a5332a447d> to my_file.txt.rfe
+
+Decrypt a file using the Reticulum Identity it was encrypted for:
+
+.. code:: text
+
+  $ rnid -i ./my_identity -d my_file.txt.rfe
+
+  Loaded Identity <2225fdeecaf6e2db4556c3c2d7637294> from ./my_identity
+  Decrypting ./my_file.txt.rfe...
+  File ./my_file.txt.rfe decrypted with <2225fdeecaf6e2db4556c3c2d7637294> to ./my_file.txt
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnid.py [-h] [--config path] [-i identity] [-g path] [-v] [-q] [-a aspects]
+                 [-H aspects] [-e path] [-d path] [-s path] [-V path] [-r path] [-w path]
+                 [-f] [-R] [-t seconds] [-p] [-P] [--version]
+
+  Reticulum Identity & Encryption Utility
+
+  options:
+    -h, --help            show this help message and exit
+    --config path         path to alternative Reticulum config directory
+    -i identity, --identity identity
+                          hexadecimal Reticulum Destination hash or path to Identity file
+    -g path, --generate path
+                          generate a new Identity
+    -v, --verbose         increase verbosity
+    -q, --quiet           decrease verbosity
+    -a aspects, --announce aspects
+                          announce a destination based on this Identity
+    -H aspects, --hash aspects
+                          show destination hashes for other aspects for this Identity
+    -e path, --encrypt path
+                          encrypt file
+    -d path, --decrypt path
+                          decrypt file
+    -s path, --sign path  sign file
+    -V path, --validate path
+                          validate signature
+    -r path, --read path  input file path
+    -w path, --write path
+                          output file path
+    -f, --force           write output even if it overwrites existing files
+    -R, --request         request unknown Identities from the network
+    -t seconds            identity request timeout before giving up
+    -p, --print-identity  print identity info and exit
+    -P, --print-private   allow displaying private keys
+    --version             show program's version number and exit
+
+
 The rnpath Utility
 ====================
 
 With the ``rnpath`` utility, you can look up and view paths for
 destinations on the Reticulum network.
 
+**Usage Examples**
+
+Resolve path to a destination:
+
 .. code:: text
 
-  # Run rnpath
-  rnpath c89b4da064bf66d280f0e4d8abfd9806
+  $ rnpath c89b4da064bf66d280f0e4d8abfd9806
 
-  # Example output
   Path found, destination <c89b4da064bf66d280f0e4d8abfd9806> is 4 hops away via <f53a1c4278e0726bb73fcc623d6ce763> on TCPInterface[Testnet/dublin.connect.reticulum.network:4965]
 
+**All Command-Line Options**
+
 .. code:: text
 
-  usage: rnpath [-h] [--config CONFIG] [--version] [-t] [-r] [-d] [-D] [-w seconds] [-v] [destination]
-  
+  usage: rnpath.py [-h] [--config CONFIG] [--version] [-t] [-r] [-d] [-D]
+                   [-x] [-w seconds] [-v] [destination]
+
   Reticulum Path Discovery Utility
-  
+
   positional arguments:
     destination           hexadecimal hash of the destination
-  
-  optional arguments:
+
+  options:
     -h, --help            show this help message and exit
     --config CONFIG       path to alternative Reticulum config directory
     --version             show program's version number and exit
@@ -285,6 +468,7 @@ destinations on the Reticulum network.
     -r, --rates           show announce rate info
     -d, --drop            remove the path to a destination
     -D, --drop-announces  drop all queued announces
+    -x, --drop-via        drop all paths via specified transport instance
     -w seconds            timeout before giving up
     -v, --verbose
 
@@ -295,21 +479,53 @@ The rnprobe Utility
 The ``rnprobe`` utility lets you probe a destination for connectivity, similar
 to the ``ping`` program. Please note that probes will only be answered if the
 specified destination is configured to send proofs for received packets. Many
-destinations will not have this option enabled, and will not be probable.
+destinations will not have this option enabled, so most destinations will not
+be probable.
+
+You can enable a probe-reply destination on Reticulum Transport Instances by
+setting the ``respond_to_probes`` configuration directive. Reticulum will then
+print the probe destination to the log on Transport Instance startup.
+
+**Usage Examples**
+
+Probe a destination:
 
 .. code:: text
 
-  # Run rnprobe
-  rnprobe example_utilities.echo.request 2d03725b327348980d570f739a3a5708
+  $ rnprobe rnstransport.probe 2d03725b327348980d570f739a3a5708
 
-  # Example output
   Sent 16 byte probe to <2d03725b327348980d570f739a3a5708>
   Valid reply received from <2d03725b327348980d570f739a3a5708>
   Round-trip time is 38.469 milliseconds over 2 hops
 
+Send a larger probe:
+
 .. code:: text
 
-  usage: rnprobe [-h] [--config CONFIG] [--version] [-v] [full_name] [destination_hash]
+  $ rnprobe rnstransport.probe 2d03725b327348980d570f739a3a5708 -s 256
+
+  Sent 16 byte probe to <2d03725b327348980d570f739a3a5708>
+  Valid reply received from <2d03725b327348980d570f739a3a5708>
+  Round-trip time is 38.781 milliseconds over 2 hops
+
+If the interface that receives the probe replies supports reporting radio
+parameters such as **RSSI** and **SNR**, the ``rnprobe`` utility will print
+these as part of the result as well.
+
+.. code:: text
+
+  $ rnprobe rnstransport.probe e7536ee90bd4a440e130490b87a25124
+  
+  Sent 16 byte probe to <e7536ee90bd4a440e130490b87a25124>
+  Valid reply received from <e7536ee90bd4a440e130490b87a25124>
+  Round-trip time is 1.809 seconds over 1 hop [RSSI -73 dBm] [SNR 12.0 dB]
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnprobe [-h] [--config CONFIG] [--version] [-v] [-s SIZE]
+                 [full_name] [destination_hash]
 
   Reticulum Probe Utility
 
@@ -320,6 +536,7 @@ destinations will not have this option enabled, and will not be probable.
   optional arguments:
     -h, --help        show this help message and exit
     --config CONFIG   path to alternative Reticulum config directory
+    -s SIZE, --size SIZE  size of probe packet payload in bytes
     --version         show program's version number and exit
     -v, --verbose
 
@@ -330,20 +547,39 @@ The rncp Utility
 The ``rncp`` utility is a simple file transfer tool. Using it, you can transfer
 files through Reticulum.
 
+**Usage Examples**
+
+Run rncp on the receiving system, specifying which identities are allowed to send files:
+  
 .. code:: text
 
-  # Run rncp on the receiving system, specifying which identities
-  # are allowed to send files
-  rncp --receive -a 1726dbad538775b5bf9b0ea25a4079c8 -a c50cc4e4f7838b6c31f60ab9032cbc62
+  $ rncp --listen -a 1726dbad538775b5bf9b0ea25a4079c8 -a c50cc4e4f7838b6c31f60ab9032cbc62
 
-  # From another system, copy a file to the receiving system
-  rncp ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
-
-You can specify as many allowed senders as needed, or complete disable authentication.
+You can also specify allowed identity hashes (one per line) in the file ~/.rncp/allowed_identities
+and simply running the program in listener mode:
 
 .. code:: text
 
-  usage: rncp [-h] [--config path] [-v] [-q] [-p] [-r] [-b] [-a allowed_hash] [-n] [-w seconds] [--version] [file] [destination]
+  $ rncp --listen
+
+From another system, copy a file to the receiving system:
+
+.. code:: text
+
+  $ rncp ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
+
+Or fetch a file from the remote system:
+
+.. code:: text
+
+  $ rncp --fetch ~/path/to/file.tgz 73cbd378bb0286ed11a707c13447bb1e
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rncp.py [-h] [--config path] [-v] [-q] [-S] [-l] [-f] [-b seconds]
+                 [-a allowed_hash] [-n] [-p] [-w seconds] [--version] [file] [destination]
 
   Reticulum File Transfer Utility
 
@@ -351,19 +587,20 @@ You can specify as many allowed senders as needed, or complete disable authentic
     file                  file to be transferred
     destination           hexadecimal hash of the receiver
 
-  optional arguments:
+  options:
     -h, --help            show this help message and exit
     --config path         path to alternative Reticulum config directory
     -v, --verbose         increase verbosity
     -q, --quiet           decrease verbosity
-    -p, --print-identity  print identity and destination info and exit
-    -r, --receive         wait for incoming files
-    -b, --no-announce     don't announce at program start
+    -S, --silent          disable transfer progress output
+    -l, --listen          listen for incoming transfer requests
+    -f, --fetch           fetch file from remote listener instead of sending
+    -b seconds            announce interval, 0 to only announce at startup
     -a allowed_hash       accept from this identity
-    -n, --no-auth         accept files from anyone
+    -n, --no-auth         accept files and fetches from anyone
+    -p, --print-identity  print identity and destination info and exit
     -w seconds            sender timeout before giving up
     --version             show program's version number and exit
-    -v, --verbose
 
 
 The rnx Utility
@@ -371,32 +608,43 @@ The rnx Utility
 
 The ``rnx`` utility is a basic remote command execution program. It allows you to
 execute commands on remote systems over Reticulum, and to view returned command
-output.
+output. For a fully interactive remote shell solution, be sure to also take a look
+at the `rnsh <https://github.com/acehoss/rnsh>`_ program.
+
+**Usage Examples**
+
+Run rnx on the listening system, specifying which identities are allowed to execute commands:
 
 .. code:: text
 
-  # Run rnx on the listening system, specifying which identities
-  # are allowed to execute commands
-  rnx --listen -a 941bed5e228775e5a8079fc38b1ccf3f -a 1b03013c25f1c2ca068a4f080b844a10
+  $ rnx --listen -a 941bed5e228775e5a8079fc38b1ccf3f -a 1b03013c25f1c2ca068a4f080b844a10
 
-  # From another system, run a command 
-  rnx 7a55144adf826958a9529a3bcf08b149 "cat /proc/cpuinfo"
-
-  # Or enter the interactive mode pseudo-shell
-  rnx 7a55144adf826958a9529a3bcf08b149 -x
-
-  # The default identity file is stored in
-  # ~/.reticulum/identities/rnx, but you can use
-  # another one, which will be created if it does
-  # not already exist
-  rnx 7a55144adf826958a9529a3bcf08b149 -i /path/to/identity -x
-
-You can specify as many allowed senders as needed, or completely disable authentication.
+From another system, run a command on the remote:
 
 .. code:: text
 
-  usage: rnx [-h] [--config path] [-v] [-q] [-p] [-l] [-i identity] [-x] [-b] [-a allowed_hash] [-n] [-N] [-d] [-m] [-w seconds] [-W seconds] [--stdin STDIN] [--stdout STDOUT] [--stderr STDERR] [--version]
-             [destination] [command]
+  $ rnx 7a55144adf826958a9529a3bcf08b149 "cat /proc/cpuinfo"
+
+Or enter the interactive mode pseudo-shell:
+
+.. code:: text
+
+  $ rnx 7a55144adf826958a9529a3bcf08b149 -x
+
+The default identity file is stored in ``~/.reticulum/identities/rnx``, but you can use
+another one, which will be created if it does not already exist 
+
+.. code:: text
+
+  $ rnx 7a55144adf826958a9529a3bcf08b149 -i /path/to/identity -x
+
+**All Command-Line Options**
+
+.. code:: text
+
+  usage: rnx [-h] [--config path] [-v] [-q] [-p] [-l] [-i identity] [-x] [-b] [-n] [-N]
+             [-d] [-m] [-a allowed_hash] [-w seconds] [-W seconds] [--stdin STDIN]
+             [--stdout STDOUT] [--stderr STDERR] [--version] [destination] [command]
 
   Reticulum Remote Execution Utility
 
@@ -433,11 +681,19 @@ The rnodeconf Utility
 The ``rnodeconf`` utility allows you to inspect and configure existing :ref:`RNodes<rnode-main>`, and
 to create and provision new :ref:`RNodes<rnode-main>` from any supported hardware devices.
 
+**All Command-Line Options**
+
 .. code:: text
 
-  usage: rnodeconf [-h] [-i] [-a] [-u] [-U] [--fw-version version] [--nocheck] [-C] [-N] [-T] [-b] [-B] [-p] [--freq Hz] [--bw Hz] [--txp dBm] [--sf factor] [--cr rate] [--eeprom-backup] [--eeprom-dump] [--eeprom-wipe] [--version] [port]
+  usage: rnodeconf.py [-h] [-i] [-a] [-u] [-U] [--fw-version version] [--nocheck] [-e]
+                      [-E] [-C] [--baud-flash baud_flash] [-N] [-T] [-b] [-B] [-p] [-D i]
+                      [--freq Hz] [--bw Hz] [--txp dBm] [--sf factor] [--cr rate]
+                      [--eeprom-backup] [--eeprom-dump] [--eeprom-wipe] [-P]
+                      [--trust-key hexbytes] [--version] [port]
 
-  RNode Configuration and firmware utility. This program allows you to change various settings and startup modes of RNode. It can also install, flash and update the firmware on supported devices.
+  RNode Configuration and firmware utility. This program allows you to change various
+  settings and startup modes of RNode. It can also install, flash and update the firmware
+  on supported devices.
 
   positional arguments:
     port                  serial port where RNode is attached
@@ -453,11 +709,14 @@ to create and provision new :ref:`RNodes<rnode-main>` from any supported hardwar
     -e, --extract         Extract firmware from connected RNode for later use
     -E, --use-extracted   Use the extracted firmware for autoinstallation or update
     -C, --clear-cache     Clear locally cached firmware files
+    --baud-flash baud_flash
+                          Set specific baud rate when flashing device. Default is 921600
     -N, --normal          Switch device to normal mode
     -T, --tnc             Switch device to TNC mode
     -b, --bluetooth-on    Turn device bluetooth on
     -B, --bluetooth-off   Turn device bluetooth off
     -p, --bluetooth-pair  Put device into bluetooth pairing mode
+    -D i, --display i     Set display intensity (0-255)
     --freq Hz             Frequency in Hz for TNC mode
     --bw Hz               Bandwidth in Hz for TNC mode
     --txp dBm             TX power in dBm for TNC mode
@@ -466,6 +725,8 @@ to create and provision new :ref:`RNodes<rnode-main>` from any supported hardwar
     --eeprom-backup       Backup EEPROM to file
     --eeprom-dump         Dump EEPROM to console
     --eeprom-wipe         Unlock and wipe EEPROM
+    -P, --public          Display public part of signing key
+    --trust-key hexbytes  Public key to trust for device verification
     --version             Print program version and exit
 
 For more information on how to create your own RNodes, please read the :ref:`Creating RNodes<rnode-creating>`
@@ -519,6 +780,9 @@ Reticulum as a System Service
 Instead of starting Reticulum manually, you can install ``rnsd`` as a system
 service and have it start automatically at boot.
 
+Systemwide Service
+^^^^^^^^^^^^^^^^^^
+
 If you installed Reticulum with ``pip``, the ``rnsd`` program will most likely
 be located in a user-local installation path only, which means ``systemd`` will not
 be able to execute it. In this case, you can simply symlink the ``rnsd`` program
@@ -566,3 +830,48 @@ If you want to automatically start ``rnsd`` at boot, run:
 .. code:: text
 
   sudo systemctl enable rnsd
+
+Userspace Service
+^^^^^^^^^^^^^^^^^
+
+Alternatively you can use a user systemd service instead of a system wide one. This way the whole setup can be done as a regular user.
+Create a user systemd service files ``~/.config/systemd/user/rnsd.service`` with the following content:
+
+.. code:: text
+
+  [Unit]
+  Description=Reticulum Network Stack Daemon
+  After=default.target
+
+  [Service]
+  # If you run Reticulum on WiFi devices,
+  # or other devices that need some extra
+  # time to initialise, you might want to
+  # add a short delay before Reticulum is
+  # started by systemd:
+  # ExecStartPre=/bin/sleep 10
+  Type=simple
+  Restart=always
+  RestartSec=3
+  ExecStart=RNS_BIN_DIR/rnsd --service
+
+  [Install]
+  WantedBy=default.target
+
+Replace ``RNS_BIN_DIR`` with the path to your Reticulum binary directory (eg. /home/USERNAMEHERE/rns/bin).
+
+Start user service:
+
+.. code:: text
+
+  systemctl --user daemon-reload
+  systemctl --user start rnsd.service
+
+If you want to automatically start ``rnsd`` without having to log in as the USERNAMEHERE, do:
+
+.. code:: text
+  
+  sudo loginctl enable-linger USERNAMEHERE
+  systemctl --user enable rnsd.service
+  
+

docs/source/whatis.rst

@@ -21,15 +21,15 @@ networks, without any need for hierarchical or beaureucratic structures to contr
 or manage them, while ensuring individuals and communities full sovereignty
 over their own network segments.
 
-Reticulum is a complete networking stack, and does not need IP or higher
+Reticulum is a **complete networking stack**, and does not need IP or higher
 layers, although it is easy to utilise IP (with TCP or UDP) as the underlying
 carrier for Reticulum. It is therefore trivial to tunnel Reticulum over the
 Internet or private IP networks. Reticulum is built directly on cryptographic
 principles, allowing resilience and stable functionality in open and trustless
 networks.
 
-No kernel modules or drivers are required. Reticulum runs completely in
-userland, and can run on practically any system that runs Python 3. Reticulum
+No kernel modules or drivers are required. Reticulum can run completely in
+userland, and will run on practically any system that runs Python 3. Reticulum
 runs well even on small single-board computers like the Pi Zero.
 
 
@@ -38,7 +38,7 @@ Current Status
 **Please know!** Reticulum should currently be considered beta software. All core protocol
 features are implemented and functioning, but additions will probably occur as
 real-world use is explored. *There will be bugs*. The API and wire-format can be
-considered stable at the moment, but could change if absolutely warranted.
+considered complete and stable at the moment, but could change if absolutely warranted.
 
 
 What does Reticulum Offer?
@@ -71,7 +71,7 @@ What does Reticulum Offer?
 
 * Efficient link establishment
 
-  * Total bandwidth cost of setting up a link is only 3 packets, totalling 297 bytes
+  * Total cost of setting up an encrypted and verified link is only 3 packets, totalling 297 bytes
 
   * Low cost of keeping links open at only 0.44 bits per second
 
@@ -91,7 +91,7 @@ What does Reticulum Offer?
 Where can Reticulum be Used?
 ============================
 Over practically any medium that can support at least a half-duplex channel
-with 500 bits per second throughput, and an MTU of 500 bytes. Data radios,
+with greater throughput than 5 bits per second, and an MTU of 500 bytes. Data radios,
 modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes,
 ad-hoc WiFi, free-space optical links and similar systems are all examples
 of the types of interfaces Reticulum was designed for.

setup.py

@@ -49,6 +49,7 @@ setuptools.setup(
             'rnid=RNS.Utilities.rnid:main',
             'rncp=RNS.Utilities.rncp:main',
             'rnx=RNS.Utilities.rnx:main',
+            'rnir=RNS.Utilities.rnir:main',
             'rnodeconf=RNS.Utilities.rnodeconf:main',
         ]
     },

tests/channel.py

@@ -404,104 +404,104 @@ class TestChannel(unittest.TestCase):
 
             self.assertEqual(data, decoded)
 
-    # def test_buffer_big(self):
-    #     writer = RNS.Buffer.create_writer(15, self.h.channel)
-    #     reader = RNS.Buffer.create_reader(15, self.h.channel)
-    #     data = "01234556789"*1024  # 10 KB
-    #     count = 0
-    #     write_finished = False
+    def test_buffer_big(self):
+        writer = RNS.Buffer.create_writer(15, self.h.channel)
+        reader = RNS.Buffer.create_reader(15, self.h.channel)
+        data = "01234556789"*1024*5  # 50 KB
+        count = 0
+        write_finished = False
 
-    #     def write_thread():
-    #         nonlocal count, write_finished
-    #         count = writer.write(data.encode("utf-8"))
-    #         writer.flush()
-    #         writer.close()
-    #         write_finished = True
-    #     threading.Thread(target=write_thread, name="Write Thread", daemon=True).start()
+        def write_thread():
+            nonlocal count, write_finished
+            count = writer.write(data.encode("utf-8"))
+            writer.flush()
+            writer.close()
+            write_finished = True
+        threading.Thread(target=write_thread, name="Write Thread", daemon=True).start()
 
-    #     while not write_finished or next(filter(lambda x: x.state != MessageState.MSGSTATE_DELIVERED,
-    #                                             self.h.outlet.packets), None) is not None:
-    #         with self.h.outlet.lock:
-    #             for packet in self.h.outlet.packets:
-    #                 if packet.state != MessageState.MSGSTATE_DELIVERED:
-    #                     self.h.channel._receive(packet.raw)
-    #                     packet.delivered()
-    #         time.sleep(0.0001)
+        while not write_finished or next(filter(lambda x: x.state != MessageState.MSGSTATE_DELIVERED,
+                                                self.h.outlet.packets), None) is not None:
+            with self.h.outlet.lock:
+                for packet in self.h.outlet.packets:
+                    if packet.state != MessageState.MSGSTATE_DELIVERED:
+                        self.h.channel._receive(packet.raw)
+                        packet.delivered()
+            time.sleep(0.0001)
 
-    #     self.assertEqual(len(data), count)
+        self.assertEqual(len(data), count)
 
-    #     read_finished = False
-    #     result = bytes()
+        read_finished = False
+        result = bytes()
 
-    #     def read_thread():
-    #         nonlocal read_finished, result
-    #         result = reader.read()
-    #         read_finished = True
-    #     threading.Thread(target=read_thread, name="Read Thread", daemon=True).start()
+        def read_thread():
+            nonlocal read_finished, result
+            result = reader.read()
+            read_finished = True
+        threading.Thread(target=read_thread, name="Read Thread", daemon=True).start()
 
-    #     timeout_at = time.time() + 7
-    #     while not read_finished and time.time() < timeout_at:
-    #         time.sleep(0.001)
+        timeout_at = time.time() + 7
+        while not read_finished and time.time() < timeout_at:
+            time.sleep(0.001)
 
-    #     self.assertTrue(read_finished)
-    #     self.assertEqual(len(data), len(result))
+        self.assertTrue(read_finished)
+        self.assertEqual(len(data), len(result))
 
-    #     decoded = result.decode("utf-8")
+        decoded = result.decode("utf-8")
 
-    #     self.assertSequenceEqual(data, decoded)
+        self.assertSequenceEqual(data, decoded)
 
-    # def test_buffer_small_with_callback(self):
-    #     callbacks = 0
-    #     last_cb_value = None
+    def test_buffer_small_with_callback(self):
+        callbacks = 0
+        last_cb_value = None
 
-    #     def callback(ready: int):
-    #         nonlocal callbacks, last_cb_value
-    #         callbacks += 1
-    #         last_cb_value = ready
+        def callback(ready: int):
+            nonlocal callbacks, last_cb_value
+            callbacks += 1
+            last_cb_value = ready
 
-    #     data = "Hello\n"
-    #     with RNS.RawChannelWriter(0, self.h.channel) as writer, RNS.RawChannelReader(0, self.h.channel) as reader:
-    #         reader.add_ready_callback(callback)
-    #         count = writer.write(data.encode("utf-8"))
-    #         writer.flush()
+        data = "Hello\n"
+        with RNS.RawChannelWriter(0, self.h.channel) as writer, RNS.RawChannelReader(0, self.h.channel) as reader:
+            reader.add_ready_callback(callback)
+            count = writer.write(data.encode("utf-8"))
+            writer.flush()
 
-    #         self.assertEqual(len(data), count)
-    #         self.assertEqual(1, len(self.h.outlet.packets))
+            self.assertEqual(len(data), count)
+            self.assertEqual(1, len(self.h.outlet.packets))
 
-    #         packet = self.h.outlet.packets[0]
-    #         self.h.channel._receive(packet.raw)
-    #         packet.delivered()
+            packet = self.h.outlet.packets[0]
+            self.h.channel._receive(packet.raw)
+            packet.delivered()
 
-    #         self.assertEqual(1, callbacks)
-    #         self.assertEqual(len(data), last_cb_value)
+            self.assertEqual(1, callbacks)
+            self.assertEqual(len(data), last_cb_value)
 
-    #         result = reader.readline()
+            result = reader.readline()
 
-    #         self.assertIsNotNone(result)
-    #         self.assertEqual(len(result), len(data))
+            self.assertIsNotNone(result)
+            self.assertEqual(len(result), len(data))
 
-    #         decoded = result.decode("utf-8")
+            decoded = result.decode("utf-8")
 
-    #         self.assertEqual(data, decoded)
-    #         self.assertEqual(1, len(self.h.outlet.packets))
+            self.assertEqual(data, decoded)
+            self.assertEqual(1, len(self.h.outlet.packets))
 
-    #         result = reader.read(1)
+            result = reader.read(1)
 
-    #         self.assertIsNone(result)
-    #         self.assertTrue(self.h.channel.is_ready_to_send())
+            self.assertIsNone(result)
+            self.assertTrue(self.h.channel.is_ready_to_send())
 
-    #         writer.close()
+            writer.close()
 
-    #         self.assertEqual(2, len(self.h.outlet.packets))
+            self.assertEqual(2, len(self.h.outlet.packets))
 
-    #         packet = self.h.outlet.packets[1]
-    #         self.h.channel._receive(packet.raw)
-    #         packet.delivered()
+            packet = self.h.outlet.packets[1]
+            self.h.channel._receive(packet.raw)
+            packet.delivered()
 
-    #         result = reader.read(1)
+            result = reader.read(1)
 
-    #         self.assertIsNotNone(result)
-    #         self.assertTrue(len(result) == 0)
+            self.assertIsNotNone(result)
+            self.assertTrue(len(result) == 0)
 
 
 if __name__ == '__main__':

tests/link.py

@@ -4,6 +4,7 @@ import subprocess
 import shlex
 import threading
 import time
+import random
 from unittest import skipIf
 import RNS
 import os
@@ -23,6 +24,8 @@ fixed_keys = [
     ("08bb35f92b06a0832991165a0d9b4fd91af7b7765ce4572aa6222070b11b767092b61b0fd18b3a59cae6deb9db6d4bfb1c7fcfe076cfd66eea7ddd5f877543b9", "d13712efc45ef87674fb5ac26c37c912"),
 ]
 
+BUFFER_TEST_TARGET = 32000
+
 def targets_job(caller):
     cmd = "python -c \"from tests.link import targets; targets()\""
     print("Opening subprocess for "+str(cmd)+"...", RNS.LOG_VERBOSE)
@@ -96,6 +99,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
 
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
@@ -117,6 +123,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
 
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
@@ -137,6 +146,7 @@ class TestLink(unittest.TestCase):
         pstart = time.time()
         print("Sending "+str(num_packets)+" link packets of "+str(packet_size)+" bytes...")
         for i in range(0, num_packets):
+            time.sleep(0.003)
             b += packet_size
             data = os.urandom(packet_size)
             start = time.time()
@@ -185,6 +195,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
 
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
@@ -200,6 +213,7 @@ class TestLink(unittest.TestCase):
         resource = RNS.Resource(data, l1, timeout=resource_timeout)
         start = time.time()
 
+        # This is a hack, don't do it. Use the callbacks instead.
         while resource.status < RNS.Resource.COMPLETE:
             time.sleep(0.01)
 
@@ -221,6 +235,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
 
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
@@ -236,6 +253,7 @@ class TestLink(unittest.TestCase):
         resource = RNS.Resource(data, l1, timeout=resource_timeout)
         start = time.time()
 
+        # This is a hack, don't do it. Use the callbacks instead.
         while resource.status < RNS.Resource.COMPLETE:
             time.sleep(0.01)
 
@@ -257,6 +275,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
         
@@ -271,6 +292,7 @@ class TestLink(unittest.TestCase):
         resource = RNS.Resource(data, l1, timeout=resource_timeout)
         start = time.time()
 
+        # This is a hack, don't do it. Use the callbacks instead.
         while resource.status < RNS.Resource.COMPLETE:
             time.sleep(0.01)
 
@@ -297,6 +319,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
         
@@ -311,6 +336,7 @@ class TestLink(unittest.TestCase):
         resource = RNS.Resource(data, l1, timeout=resource_timeout)
         start = time.time()
 
+        # This is a hack, don't do it. Use the callbacks instead.
         while resource.status < RNS.Resource.COMPLETE:
             time.sleep(0.01)
 
@@ -322,6 +348,10 @@ class TestLink(unittest.TestCase):
         time.sleep(0.5)
         self.assertEqual(l1.status, RNS.Link.CLOSED)
 
+    large_resource_status = None
+    def lr_callback(self, resource):
+        TestLink.large_resource_status = resource.status
+
     @skipIf(os.getenv('SKIP_NORMAL_TESTS') != None, "Skipping")
     def test_9_large_resource(self):
         if RNS.Cryptography.backend() == "internal":
@@ -336,6 +366,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
         
@@ -344,17 +377,18 @@ class TestLink(unittest.TestCase):
         self.assertEqual(l1.status, RNS.Link.ACTIVE)
 
         resource_timeout = 120
-        resource_size = 35*1000*1000
+        resource_size = 50*1000*1000
         data = os.urandom(resource_size)
         print("Sending "+self.size_str(resource_size)+" resource...")
-        resource = RNS.Resource(data, l1, timeout=resource_timeout)
+        resource = RNS.Resource(data, l1, timeout=resource_timeout, callback=self.lr_callback)
         start = time.time()
 
-        while resource.status < RNS.Resource.COMPLETE:
+        TestLink.large_resource_status = resource.status
+        while TestLink.large_resource_status < RNS.Resource.COMPLETE:
             time.sleep(0.01)
 
         t = time.time() - start
-        self.assertEqual(resource.status, RNS.Resource.COMPLETE)
+        self.assertEqual(TestLink.large_resource_status, RNS.Resource.COMPLETE)
         print("Resource completed at "+self.size_str(resource_size/t, "b")+"ps")
 
         l1.teardown()
@@ -372,6 +406,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
 
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
@@ -421,6 +458,9 @@ class TestLink(unittest.TestCase):
         id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
         self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+        RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+        time.sleep(0.2)
+
         dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
 
         self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
@@ -455,7 +495,7 @@ class TestLink(unittest.TestCase):
 
     # @skipIf(os.getenv('SKIP_NORMAL_TESTS') != None and os.getenv('RUN_SLOW_TESTS') == None, "Skipping")
     def test_12_buffer_round_trip_big(self, local_bitrate = None):
-        global c_rns
+        global c_rns, buffer_read_target
         init_rns(self)
         print("")
         print("Buffer round trip test")
@@ -474,6 +514,9 @@ class TestLink(unittest.TestCase):
             id1 = RNS.Identity.from_bytes(bytes.fromhex(fixed_keys[0][0]))
             self.assertEqual(id1.hash, bytes.fromhex(fixed_keys[0][1]))
 
+            RNS.Transport.request_path(bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
+            time.sleep(0.2)
+
             dest = RNS.Destination(id1, RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, "link", "establish")
 
             self.assertEqual(dest.hash, bytes.fromhex("fb48da0e82e6e01ba0c014513f74540d"))
@@ -490,9 +533,9 @@ class TestLink(unittest.TestCase):
 
             buffer = None
             received = []
+
             def handle_data(ready_bytes: int):
-                # TODO: Remove
-                RNS.log("Handling data")
+                global received_bytes
                 data = buffer.read(ready_bytes)
                 received.append(data)
 
@@ -509,10 +552,11 @@ class TestLink(unittest.TestCase):
             if local_interface.bitrate < 1000:
                 target_bytes = 3000
             else:
-                target_bytes = 16000
+                target_bytes = BUFFER_TEST_TARGET
             
-
-            message = os.urandom(target_bytes)
+            random.seed(154889)
+            message = random.randbytes(target_bytes)
+            buffer_read_target = len(message)
             
             # the return message will have an appendage string " back at you"
             # for every StreamDataMessage that arrives. To verify, we need
@@ -527,35 +571,24 @@ class TestLink(unittest.TestCase):
             # since the segments will be received at max length for a
             # StreamDataMessage, the appended text will end up in a
             # separate packet.
-            expected_chunk_count = ceil(len(message)/StreamDataMessage.MAX_DATA_LEN * 2)-1
-            print("Sending " + str(len(message)) + " bytes, receiving " + str(len(expected_rx_message)) + " bytes, " +
-                  "expecting " + str(expected_chunk_count) + " chunks of " + str(StreamDataMessage.MAX_DATA_LEN) + " bytes")
+            print("Sending " + str(len(message)) + " bytes, receiving " + str(len(expected_rx_message)) + " bytes, ")
             
             buffer.write(message)
             buffer.flush()
 
-            # delay a reasonable time for the send and receive
-            # a chunk each way plus a little more for a proof each way
-            # while time.time() < expected_ready_time and len(received) < expected_chunk_count:
-            #     time.sleep(0.1)
-            # # sleep for at least one more chunk round trip in case there
-            # # are more chunks than expected
-            # if time.time() < expected_ready_time:
-            #     time.sleep(max(c_rns.MTU * 2 / local_interface.bitrate * 8, 1))
-
-            timeout = time.time() + 10
-            while len(received) < expected_chunk_count and not time.time() > timeout:
-                time.sleep(2)
-                print(f"Received {len(received)} out of {expected_chunk_count} chunks so far")
-            time.sleep(2)
-            print(f"Received {len(received)} out of {expected_chunk_count} chunks")
+            timeout = time.time() + 4
+            while not time.time() > timeout:
+                time.sleep(1)
+                print(f"Received {len(received)} chunks so far")
+            time.sleep(1)
 
             data = bytearray()
             for rx in received:
                 data.extend(rx)
-
             rx_message = data
 
+            print(f"Received {len(received)} chunks, totalling {len(rx_message)} bytes")
+
             self.assertEqual(len(expected_rx_message), len(rx_message))
             for i in range(0, len(expected_rx_message)):
                 self.assertEqual(expected_rx_message[i], rx_message[i])
@@ -598,7 +631,7 @@ class TestLink(unittest.TestCase):
 if __name__ == '__main__':
     unittest.main(verbosity=1)
 
-
+buffer_read_len = 0
 def targets(yp=False):
     if yp:
         import yappi
@@ -645,10 +678,26 @@ def targets(yp=False):
 
         buffer = None
 
+        response_data = []
         def handle_buffer(ready_bytes: int):
+            global buffer_read_len, BUFFER_TEST_TARGET
             data = buffer.read(ready_bytes)
-            buffer.write(data + " back at you".encode("utf-8"))
-            buffer.flush()
+            buffer_read_len += len(data)
+            response_data.append(data)
+
+            if data == "Hi there".encode("utf-8"):
+                RNS.log("Sending response")
+                for data in response_data:
+                    buffer.write(data + " back at you".encode("utf-8"))
+                    buffer.flush()
+                    buffer_read_len = 0
+
+            if buffer_read_len == BUFFER_TEST_TARGET:
+                RNS.log("Sending response")
+                for data in response_data:
+                    buffer.write(data + " back at you".encode("utf-8"))
+                    buffer.flush()
+                    buffer_read_len = 0
 
         buffer = RNS.Buffer.create_bidirectional_buffer(0, 0, channel, handle_buffer)
 

tests/rnsconfig/config

@@ -9,4 +9,4 @@
   loglevel = 1
 
 [interfaces]
-  # No interfaces, only local traffic
+  # No interfaces, only local traffic