Fixed occasional io thread hang on shutdown

add rnodeconf support for SeeedStudio XIAO esp32s3 wio

.github/ISSUE_TEMPLATE/🐛-bug-report.md

@@ -12,10 +12,14 @@ Before creating a bug report on this issue tracker, you **must** read the [Contr
 
 - The issue tracker is used by developers of this project. **Do not use it to ask general questions, or for support requests**.
 - Ideas and feature requests can be made on the [Discussions](https://github.com/markqvist/Reticulum/discussions). **Only** feature requests accepted by maintainers and developers are tracked and included on the issue tracker. **Do not post feature requests here**.
-- After reading the [Contribution Guidelines](https://github.com/markqvist/Reticulum/blob/master/Contributing.md), delete this section from your bug report.
+- After reading the [Contribution Guidelines](https://github.com/markqvist/Reticulum/blob/master/Contributing.md), **delete this section only** (*"Read the Contribution Guidelines"*) from your bug report, **and fill in all the other sections**.
 
 **Describe the Bug**
-A clear and concise description of what the bug is.
+First of all: Is this really a bug? Is it reproducible?
+
+If this is a request for help because something is not working as you expected, stop right here, and go to the [discussions](https://github.com/markqvist/Reticulum/discussions) instead, where you can post your questions and get help from other users.
+
+If this really is a bug or issue with the software, remove this section of the template, and provide **a clear and concise description of what the bug is**.
 
 **To Reproduce**
 Describe in detail how to reproduce the bug.
@@ -24,7 +28,7 @@ Describe in detail how to reproduce the bug.
 A clear and concise description of what you expected to happen.
 
 **Logs & Screenshots**
-Please include any relevant log output. If applicable, also add screenshots to help explain your problem.
+Please include any relevant log output. If applicable, also add screenshots to help explain your problem. In most cases, without any relevant log output, we will not be able to determine the cause of the bug, or reproduce it.
 
 **System Information**
 - OS and version

.github/workflows/build.yml

@@ -0,0 +1,96 @@
+name: Build Reticulum
+
+on:
+  push:
+    branches: 
+      - '*'
+    tags:
+      - "[0-9]+.[0-9]+.[0-9]+*"
+  pull_request:
+    branches: 
+      - master
+    paths-ignore:
+      - .gitignore
+      - LICENSE
+
+permissions:
+  contents: write
+
+concurrency: 
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.x
+      - run: make test
+
+  package:
+    needs: test
+    if: startsWith(github.ref, 'refs/tags/')
+    runs-on: ubuntu-latest
+    environment: ${{ contains(github.ref, '-') && 'development' || 'production' }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.x
+      - run: |
+          python -m pip install -q build wheel setuptools
+          make remove_symlinks
+          make build_wheel
+          make build_pure_wheel
+          make create_symlinks
+      - uses: actions/upload-artifact@v4
+        with:
+          name: package
+          path: dist/*.whl
+
+  # documentation:
+  #   needs: test
+  #   if: startsWith(github.ref, 'refs/tags/')
+  #   runs-on: ubuntu-latest
+  #   environment: ${{ contains(github.ref, '-') && 'development' || 'production' }}
+  #   steps:
+  #     - uses: actions/checkout@v4
+  #     - uses: actions/setup-python@v5
+  #       with:
+  #         python-version: 3.x
+  #     - run: |
+  #         sudo apt-get -qq update && sudo apt-get -qq install latexmk texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended
+  #         python -m pip -q install sphinx sphinx-copybutton
+  #         cd docs && make latexpdf && make epub
+  #     - uses: actions/upload-artifact@v4
+  #       with:
+  #         name: documentation
+  #         path: |
+  #           docs/build/latex/*.pdf
+  #           docs/build/epub/*.epub
+
+  release:
+    needs: [package]
+    if: startsWith(github.ref, 'refs/tags/')
+    runs-on: ubuntu-latest
+    environment: ${{ contains(github.ref, '-') && 'development' || 'production' }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/download-artifact@v4
+        with:
+          path: .artifacts
+      - uses: softprops/action-gh-release@v2
+        with:
+          files: |
+#            .artifacts/package/**.whl
+#            .artifacts/documentation/latex/reticulumnetworkstack.pdf
+#            .artifacts/documentation/epub/ReticulumNetworkStack.epub
+          draft: true
+          generate_release_notes: true
+          prerelease: ${{ contains(github.ref, '-') }}
+          fail_on_unmatched_files: true

.github/workflows/python-app.yml

@@ -1,28 +0,0 @@
-# This workflow will install Python dependencies, run tests and lint with a single version of Python
-# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
-
-name: Test suite
-
-on:
-  push:
-    branches: [ "master" ]
-  pull_request:
-    branches: [ "master" ]
-
-permissions:
-  contents: read
-
-jobs:
-  build:
-
-    runs-on: ubuntu-latest
-
-    steps:
-    - uses: actions/checkout@v4
-    - name: Set up Python 3.10
-      uses: actions/setup-python@v3
-      with:
-        python-version: "3.10"
-    - name: Test
-      run: |
-        make test

CRNS/__init__.py

@@ -0,0 +1,26 @@
+compiling = False
+noticed = False
+notice_delay = 0.3
+import time
+import sys
+import threading
+from importlib.util import find_spec
+if find_spec("pyximport") and find_spec("cython"):
+    import pyximport; pyxloader = pyximport.install(pyimport=True, language_level=3)[1]
+
+def notice_job():
+    global noticed
+    started = time.time()
+    while compiling:
+        if time.time() > started+notice_delay and compiling:
+            noticed = True
+            print("Compiling RNS object code... ", end="")
+            sys.stdout.flush()
+            break
+        time.sleep(0.1)
+
+
+compiling = True
+threading.Thread(target=notice_job, daemon=True).start()
+import RNS; compiling = False
+if noticed: print("Done."); sys.stdout.flush()

Changelog.md

@@ -1,3 +1,287 @@
+### 2025-03-13: RNS β 0.9.3
+
+This maintenance release improves performance and fixes a number of bugs.
+
+**Changes**
+- Enabled link MTU discovery by default
+- Added on-demand object code compilation and loader shim
+- Added link API methods
+- Added child interface spawning for AutoInterface
+- Fixed corrupt ratchet files not being removed on maintenance cleaning
+- Fixed `rnid` not waiting for announce timebase tick before announcing
+
+**Release Hashes**
+```
+0270c988a2b898b28348cd78138667115d4ef3f7e09c86531baaefbee35ef851 rns-0.9.3-py3-none-any.whl
+eee1a6c4c9c0f04bb17b12b8fb37b9c4cec12a99c87a046730eb7c9a6ffd999f rnspure-0.9.3-py3-none-any.whl
+```
+
+### 2025-01-19: RNS β 0.9.2
+
+This maintenance release fixes a number of bugs.
+
+**Changes**
+- Fixed missing RX/TX bytes statistics assignment
+- Fixed potential daemon thread IO buffer deadlock on externally mediated shutdown signal
+- Fixed missing check for path announce emission timestamp in lower hop-count announce processing
+
+**Release Hashes**
+```
+068eb4408b332ea6eec1a58fb4644fba3531c9ca10dcd79ecf893aaaf40e720d rns-0.9.2-py3-none-any.whl
+1e7c123d244cc14c287568f3a99953cc11ffc1e79a72a029aa1be72fa8eff24e rnspure-0.9.2-py3-none-any.whl
+```
+
+### 2025-01-19: RNS β 0.9.1
+
+This maintenance release adds reject signalling mechanism to resource transfers, fixes inconsistencies in the code examples, and improves thread configuration in the transport core.
+
+**Changes**
+- Added resource reject signalling
+- Added error reporting on configured radio parameter mismatch on Android
+- Improved thread configuration for transport core threads
+- Updated examples
+
+**Release Hashes**
+```
+49288a562ad6d4b5647c3afec051a6bb6497b75e3f165a972436134d4a93ad76 rns-0.9.1-py3-none-any.whl
+abd6c4bdead2fc25d0b9b2cda5708586e8cb776b088f2a901a5f262e2ed901ae rnspure-0.9.1-py3-none-any.whl
+```
+
+### 2025-01-17: RNS β 0.9.0
+
+This release lays the groundwork for future performance and resource utilisation optimisations. Most importantly, this release adds **link MTU autodiscovery**, which allow established links to use much higher MTUs than the base MTU of 500 bytes.
+
+**Please note!** To actually use link MTU discovery, all transport nodes along the path must be upgraded to at least version `0.9.0`. Since this is the first release to add support for this feature, *it is currently **not** activated by default*, and no clients or applications will use it yet. Using link MTU autodiscovery by default will be enabled by default in RNS version `0.9.1`. Please upgrade your nodes!
+
+Additionally, this release adds several new features, performance improvements and bug fixes, as well as support for RNodes running firmware version `1.81`.
+
+**Changes**
+- Added MTU autoconfiguration on interfaces that support higher MTUs
+- Added link MTU autodiscovery and path clamping
+- Added dynamic SDU calculations based on link MTU to `Resource`, `Channel` and `Buffer`
+- Added resource EIFR continuity to split resource handling
+- Added interference status to `RNodeInterface`
+- Fixed a display bug in `rnstatus`
+- Added live traffic stats to `rnstatus`
+- Added T3S3 support to `rnodeconf`
+- Added Heltec T114 support to `rnodeconf`
+- Added LilyGO T-Echo support to `rnodeconf`
+- Added option to print device configuration to `rnodeconf`
+- Improved CPU utilisation and memory consumption
+- Improved `rnsd` restart time on systems with many interfaces
+- Improved `rncp` status output
+- Improved packet filter performance
+- Improved interface detachment handling
+- Improved resource transfer timing and performance
+- Improved Transport core efficiency
+- Improved reliability of ratchet reloads if I/O conflicts occur
+- Improved logging
+- Improved built-in profiler
+- Fixed a potential deadlock in logging
+- Fixed time formatters not handling negative times
+- Updated example code
+
+**Release Hashes**
+```
+1ee60634cf0627c45b93f4e6c9adaf1fcdf9c1a8dfd4dd3dcd499e029554ab4f rns-0.9.0-py3-none-any.whl
+b67eec583fdb224ba8174b317e66b8f7344e338e93760ed1a90f0bafea8cf09e rnspure-0.9.0-py3-none-any.whl
+```
+
+### 2025-01-09: RNS β 0.8.9
+
+This maintenance release adds a number of configuration options to `rnodeconf`.
+
+**Changes**
+- Added noise floor output to `rnstatus` for supported interfaces
+- Added channel noise floor and CSMA parameter reporting to `RNodeInterface`
+- Added ability to set display rotation in `rnodeconf`
+- Added ability to configure interference avoidance to `rnodeconf`
+- Fixed missing console image install on Heltec V3 in `rnodeconf`
+
+**Release Hashes**
+```
+b54fe8bc296f83a3a70569c9d1e9db3096249789c18f8d0217671479fa6881a1 rns-0.8.9-py3-none-any.whl
+52fd992e5f9478d5a1f61f8f37dc0ee2d268fdd0b8a4e6656d33d632490afc5a rnspure-0.8.9-py3-none-any.whl
+```
+
+### 2024-12-11: RNS β 0.8.8
+
+This maintenance release adds a single API function and fixes a bug.
+
+**Changes**
+- Allow announce handlers to receive announce packet hash
+- Fix packet RSSI/SNR/Q cache not being available on standalone instances
+
+**Release Hashes**
+```
+9c1755a81049c67b051ecb9fe4b2c5f7d98bf09d20ed52d6ce6a410298b0527b rns-0.8.8-py3-none-any.whl
+d8871d69cde4b0a0b99b383f324d651dc77a2f44ec9641be828902c778a8d128 rnspure-0.8.8-py3-none-any.whl
+```
+
+### 2024-12-09: RNS β 0.8.7
+
+This maintenance release adds support for OpenWRT packaging, and brings several minor improvements and bugfixes.
+
+Thanks to @gretel and @jacobeva, who contributed to this release!
+
+**Changes**
+- Added support for packaging RNS to OpenWRT
+- Added ability to run `rnstatus` as application-local imported module
+- Added ability to reflect RNS log output to app-internal log handler callback
+- Added display read functionality to `RNodeInterface`
+- Fixed a regression in `RNodeMultiInterface` caused by earlier refactoring
+- Imrpoved documentation
+
+**Release Hashes**
+```
+e76ba8feeeae2c8df27e9906deebd7c721f0f0e887ad3fbd26df0212d6ce907a rns-0.8.7-py3-none-any.whl
+046608539bc235d52c970c7f3c54e7aa01a86016ae00263f8a55fc796b6939f5 rnspure-0.8.7-py3-none-any.whl
+```
+
+### 2024-11-24: RNS β 0.8.6
+
+This release adds full interface modularity and custom interface loading to RNS. Users can now easily create and use their own custom interfaces for communicating over practically anything. Support for IPv6 has also been added to the TCP-based interfaces.
+
+In addition, several bugs have been fixed, and various internal improvements to code consistency and naming conventions have been carried out.
+
+Thanks to @gretel and @deavmi, who contributed to this release!
+
+**Changes**
+- Added ability to load and configure custom, user-supplied interfaces
+- Added IPv6 support to `TCPClientInterface` and `TCPServerInterface`
+- Added an init option to the API for requiring an existing shared instance
+- Changed `rnstatus` behaviour to only show status if Reticulum is already running
+- Fixed `KISSInterface` beacon length for compatibility with software modems
+- Fixed interface client count sometimes reporting incorrect values on TCP and I2P interfaces
+- Refactored and improved interface initialisation and configuration handling
+- Refactored interface code to be more consistent
+- Refactored various deprecated references and names
+- Updated documentation and manual
+
+**Release Hashes**
+```
+60be127f003cd7838149bf8f01020206f829a7bd192706a608e39d8d7193d07b rns-0.8.6-py3-none-any.whl
+d8701e19279d292b5b8af9da7c67b6ac88a992ca65109f8182c3e5c761a9ebeb rnspure-0.8.6-py3-none-any.whl
+```
+
+### 2024-10-20: RNS β 0.8.5
+
+This maintenance release fixes a number of bugs. Thanks to @faragher for contributing to this release!
+
+**Changes**
+- Fixed missing close of file handles
+- Fixed invalid values returned from `get_snr()` and `get_q()` physical layer stats API functions
+
+**Release Hashes**
+```
+1757e809e083585bf4c23b6fe0f29954e5a1586ce14081099e38e606a75831df rns-0.8.5-py3-none-any.whl
+44254630634f4dbb1ce3242247fe8180379d27bff15d183263b1856fd662f88d rnspure-0.8.5-py3-none-any.whl
+```
+
+### 2024-10-11: RNS β 0.8.4
+
+This release fixes a number of bugs and improves reliability of automatic reconnection when BLE-connected RNodes unexpectedly disappear or lose connection.
+
+**Changes**
+- Improved RNode BLE reconnection realiability
+- Added RNode battery state to `rnstatus` output
+- Fixed resource transfer hanging for a long time over slow links if proof packet is lost
+- Fixed missing import on Android
+
+**Release Hashes**
+```
+d3f7a9fddc6c1e59b1e4895756fe602408ac6ef09de377ee65ec62d09fff97a3 rns-0.8.4-py3-none-any.whl
+eb3843bcab1428be0adb097988991229a4c03156ab40cc9c6e2d9c590d8b850b rnspure-0.8.4-py3-none-any.whl
+```
+
+### 2024-10-10: RNS β 0.8.3
+
+This release fixes a bug in resource transfer progress calculation, improves RNode error handling, and brings minor improvements to the `rncp` utility.
+
+**Changes**
+- Fixed a bug in resource transfer progress calculations
+- Added physical layer transfer rate output option to `rncp`
+- Added save directory option to `rncp`
+- Improved path handling for the fetch-jail option of of `rncp`
+- Added error detection for modem communication timeouts on connected RNode devices
+
+**Release Hashes**
+```
+54ddab32769081045db5fe45b27492cc012bf2fad64bc65ed37011f3651469fb rns-0.8.3-py3-none-any.whl
+a04915111d65b05a5f2ef2687ed208813034196c0c5e711cb01e6db72faa23ef rnspure-0.8.3-py3-none-any.whl
+```
+
+### 2024-10-06: RNS β 0.8.2
+
+This release adds several new boards to `rnodeconf`, fixes a range of bugs and improves transport reliability.
+
+Thanks to @jacobeva, @prusnak and @deavmi who contributed to this release!
+
+**Changes**
+- Added support for T-Beam Supreme devices to `rnodeconf`
+- Added support for T3S3 devices to `rnodeconf`
+- Added support for T-Deck devices to `rnodeconf`
+- Added support for new hardware error codes from connected RNodes
+- Added the ability to control the display on nRF52-based RNodes
+- Improved resource transfers over very slow links, by adding more suitable `MAX_WINDOW` cap if link speed is continously below threshold.
+- Improved `rnodeconf` flashing so manual resets for some devices are no longer required
+- Added edge case handling for receiving a link proof after the link had timed out and been closed, but before it having been purged from active links table
+- Updated supported hardware section of the manual with new boards
+- Tuned path request timing for roaming instances
+- Fixed a bug that caused RNS to fail to initialise in Termux on Android
+- Fixed a bug in RNodeInterface firmware version comparison
+- Fixed a bug in the serial framing of RNodeMultiInterface
+- Fixed a bug in sub-interface spawning of RNodeMultiInterface
+
+**Release Hashes**
+```
+db720a727a09c0c9d76288dec5a995a30146e65d6a4c5c034f47fb60a78f4962 rns-0.8.2-py3-none-any.whl
+ee412535edba48817551658247fb0c843d17e1c97cad9d2a819a7fc627c5ba28 rnspure-0.8.2-py3-none-any.whl
+```
+
+### 2024-10-02: RNS β 0.8.1
+
+This release adds BLE support to RNodeInterface, and support for configuring additional options to `rnodeconf`.
+
+**Changes**
+- Added Bluetooth Low Energy support to RNodeInterface
+- Added RNode battery information to `rnstatus` output
+- Added display blanking configuration to `rnodeconf`
+- Added NeoPixel intensity configuration to `rnodeconf`
+
+**Release Hashes**
+```
+f4b6b99b67d6b33b8a4562e5d5d5ac54c76814fff26e6c7a79950b82bd80123f rns-0.8.1-py3-none-any.whl
+c2e540b4bf0f272bb51ae3e33a02f9c07f2619746d069d7ed83d88017bf7ea30 rnspure-0.8.1-py3-none-any.whl
+```
+
+### 2024-09-25: RNS β 0.8.0
+
+This maintenance release improves the interface statistics API, and updates documentation.
+
+**Changes**
+- Added additional information to interface statistics
+- Updated documentation
+
+**Release Hashes**
+```
+fa5ff6d98230693be6805bb9a94585a6f54ec0af9cba15b771d4e676f140dc43 rns-0.8.0-py3-none-any.whl
+ba20f688b69ae861c8aced251e10242a358fea15da6c22df10d4fc8846c9bf48 rnspure-0.8.0-py3-none-any.whl
+```
+
+### 2024-09-24: RNS β 0.7.9
+
+This maintenance release improves transport reliability in certain (rare) cases.
+
+**Changes**
+- Added handling of a transport edge-case
+
+**Release Hashes**
+```
+4c20c46df021d366386d497145024396f904666b0de22a92f9e5c937886ea39d rns-0.7.9-py3-none-any.whl
+97d26282df929eca732a15523bc9d7f66387a93ffd911e8063c94c3f8f6ad73c rnspure-0.7.9-py3-none-any.whl
+```
+
 ### 2024-09-18: RNS β 0.7.8
 
 This maintenance release adds support for the openCom XL to `rnodeconf`, fixes a number of bugs, and also includes a few fine-tunings of timing parameters.
@@ -5,7 +289,6 @@ This maintenance release adds support for the openCom XL to `rnodeconf`, fixes a
 Thanks to @liamcottle and @jacobeva for contributing to this release!
 
 **Changes**
-- 
 - Added interface prioritisation according to reported bitrate
 - Added support for openCom XL to `rnodeconf`
 - Added performance profiler to built-in debugging tools

Contributing.md

@@ -6,7 +6,7 @@ Apart from writing code, there are many ways in which you can contribute. Before
 
 ## Expected Conduct
 
-First and foremost, there is one simple requirement for taking part in this community: While we primarily interact virtually, your actions matter and have real consequences. Therefore: **Act like a responsible, civilized person** - also in the face of disputes and heated disagreements. Speak your mind here, discussions are welcome. Just do so in the spirit of being face-to-face with everyone else. Thank you.
+First and foremost, there is one simple requirement for taking part in this community: While we primarily interact virtually, your actions matter and have real consequences. Therefore: **Act like a responsible, civilized person** - especially in the face of disputes and heated disagreements. Speak your mind here; discussions are welcome. Just do so in the spirit of being face-to-face with everyone else. Thank you.
 
 ## Asking Questions
 

Examples/Announce.py

@@ -6,6 +6,7 @@
 
 import argparse
 import random
+import sys
 import RNS
 
 # Let's define an app name. We'll use this for all
@@ -168,4 +169,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Broadcast.py

@@ -118,4 +118,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Buffer.py

@@ -157,7 +157,7 @@ def client(destination_hexhash, configpath):
         destination_hash = bytes.fromhex(destination_hexhash)
     except:
         RNS.log("Invalid destination entered. Check your input!\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -254,9 +254,8 @@ def link_closed(link):
     else:
         RNS.log("Link closed, exiting now")
     
-    RNS.Reticulum.exit_handler()
     time.sleep(1.5)
-    os._exit(0)
+    sys.exit(0)
 
 # When the buffer has new data, read it and write it to the terminal.
 def client_buffer_ready(ready_bytes: int):
@@ -320,4 +319,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Channel.py

@@ -124,7 +124,7 @@ def server(configpath):
 def server_loop(destination):
     # Let the user know that everything is ready
     RNS.log(
-        "Link example "+
+        "Channel example "+
         RNS.prettyhexrep(destination.hash)+
         " running, waiting for a connection."
     )
@@ -212,7 +212,7 @@ def client(destination_hexhash, configpath):
         destination_hash = bytes.fromhex(destination_hexhash)
     except:
         RNS.log("Invalid destination entered. Check your input!\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -276,7 +276,7 @@ def client_loop():
                 packed_size = len(message.pack())
                 channel = server_link.get_channel()
                 if channel.is_ready_to_send():
-                    if packed_size <= channel.MDU:
+                    if packed_size <= channel.mdu:
                         channel.send(message)
                     else:
                         RNS.log(
@@ -321,9 +321,8 @@ def link_closed(link):
     else:
         RNS.log("Link closed, exiting now")
     
-    RNS.Reticulum.exit_handler()
     time.sleep(1.5)
-    os._exit(0)
+    sys.exit(0)
 
 # When a packet is received over the channel, we
 # simply print out the data.
@@ -387,4 +386,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Echo.py

@@ -6,6 +6,7 @@
 ##########################################################
 
 import argparse
+import sys
 import RNS
 
 # Let's define an app name. We'll use this for all
@@ -130,7 +131,7 @@ def client(destination_hexhash, configpath, timeout=None):
     except Exception as e:
         RNS.log("Invalid destination entered. Check your input!")
         RNS.log(str(e)+"\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -328,4 +329,4 @@ if __name__ == "__main__":
                 client(args.destination, configarg, timeout=timeoutarg)
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/ExampleInterface.py

@@ -0,0 +1,299 @@
+# MIT License - Copyright (c) 2024 Mark Qvist / unsigned.io
+
+# This example illustrates creating a custom interface
+# definition, that can be loaded and used by Reticulum at
+# runtime. Any number of custom interfaces can be created
+# and loaded. To use the interface place it in the folder
+# ~/.reticulum/interfaces, and add an interface entry to
+# your Reticulum configuration file similar to this:
+
+#  [[Example Custom Interface]]
+#    type = ExampleInterface
+#    enabled = no
+#    mode = gateway
+#    port = /dev/ttyUSB0
+#    speed = 115200
+#    databits = 8
+#    parity = none
+#    stopbits = 1
+
+from time import sleep
+import sys
+import threading
+import time
+
+# This HDLC helper class is used by the interface
+# to delimit and packetize data over the physical
+# medium - in this case a serial connection.
+class HDLC():
+    # This example interface packetizes data using
+    # simplified HDLC framing, similar to PPP
+    FLAG     = 0x7E
+    ESC      = 0x7D
+    ESC_MASK = 0x20
+
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
+        data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
+        return data
+
+# Let's define our custom interface class. It must
+# be a sub-class of the RNS "Interface" class.
+class ExampleInterface(Interface):
+    # All interface classes must define a default
+    # IFAC size, used in IFAC setup when the user
+    # has not specified a custom IFAC size. This
+    # option is specified in bytes.
+    DEFAULT_IFAC_SIZE = 8
+
+    # The following properties are local to this
+    # particular interface implementation.
+    owner    = None
+    port     = None
+    speed    = None
+    databits = None
+    parity   = None
+    stopbits = None
+    serial   = None
+
+    # All Reticulum interfaces must have an __init__
+    # method that takes 2 positional arguments:
+    # The owner RNS Transport instance, and a dict
+    # of configuration values.
+    def __init__(self, owner, configuration):
+
+        # The following lines demonstrate handling
+        # potential dependencies required for the
+        # interface to function correctly.
+        import importlib
+        if importlib.util.find_spec('serial') != None:
+            import serial
+        else:
+            RNS.log("Using this interface requires a serial communication module to be installed.", RNS.LOG_CRITICAL)
+            RNS.log("You can install one with the command: python3 -m pip install pyserial", RNS.LOG_CRITICAL)
+            RNS.panic()
+
+        # We start out by initialising the super-class
+        super().__init__()
+
+        # To make sure the configuration data is in the
+        # correct format, we parse it through the following
+        # method on the generic Interface class. This step
+        # is required to ensure compatibility on all the
+        # platforms that Reticulum supports.
+        ifconf    = Interface.get_config_obj(configuration)
+
+        # Read the interface name from the configuration
+        # and set it on our interface instance.
+        name      = ifconf["name"]
+        self.name = name
+
+        # We read configuration parameters from the supplied
+        # configuration data, and provide default values in
+        # case any are missing.
+        port      = ifconf["port"] if "port" in ifconf else None
+        speed     = int(ifconf["speed"]) if "speed" in ifconf else 9600
+        databits  = int(ifconf["databits"]) if "databits" in ifconf else 8
+        parity    = ifconf["parity"] if "parity" in ifconf else "N"
+        stopbits  = int(ifconf["stopbits"]) if "stopbits" in ifconf else 1
+
+        # In case no port is specified, we abort setup by
+        # raising an exception.
+        if port == None:
+            raise ValueError(f"No port specified for {self}")
+
+        # All interfaces must supply a hardware MTU value
+        # to the RNS Transport instance. This value should
+        # be the maximum data packet payload size that the
+        # underlying medium is capable of handling in all
+        # cases without any segmentation.
+        self.HW_MTU = 564
+
+        # We initially set the "online" property to false,
+        # since the interface has not actually been fully
+        # initialised and connected yet.
+        self.online   = False
+
+        # In this case, we can also set the indicated bit-
+        # rate of the interface to the serial port speed.
+        self.bitrate  = speed
+        
+        # Configure internal properties on the interface
+        # according to the supplied configuration.
+        self.pyserial = serial
+        self.serial   = None
+        self.owner    = owner
+        self.port     = port
+        self.speed    = speed
+        self.databits = databits
+        self.parity   = serial.PARITY_NONE
+        self.stopbits = stopbits
+        self.timeout  = 100
+
+        if parity.lower() == "e" or parity.lower() == "even":
+            self.parity = serial.PARITY_EVEN
+
+        if parity.lower() == "o" or parity.lower() == "odd":
+            self.parity = serial.PARITY_ODD
+
+        # Since all required parameters are now configured,
+        # we will try opening the serial port.
+        try:
+            self.open_port()
+        except Exception as e:
+            RNS.log("Could not open serial port for interface "+str(self), RNS.LOG_ERROR)
+            raise e
+
+        # If opening the port succeeded, run any post-open
+        # configuration required.
+        if self.serial.is_open:
+            self.configure_device()
+        else:
+            raise IOError("Could not open serial port")
+
+    # Open the serial port with supplied configuration
+    # parameters and store a reference to the open port.
+    def open_port(self):
+        RNS.log("Opening serial port "+self.port+"...", RNS.LOG_VERBOSE)
+        self.serial = self.pyserial.Serial(
+            port = self.port,
+            baudrate = self.speed,
+            bytesize = self.databits,
+            parity = self.parity,
+            stopbits = self.stopbits,
+            xonxoff = False,
+            rtscts = False,
+            timeout = 0,
+            inter_byte_timeout = None,
+            write_timeout = None,
+            dsrdtr = False,
+        )
+
+    # The only thing required after opening the port
+    # is to wait a small amount of time for the
+    # hardware to initialise and then start a thread
+    # that reads any incoming data from the device.
+    def configure_device(self):
+        sleep(0.5)
+        thread = threading.Thread(target=self.read_loop)
+        thread.daemon = True
+        thread.start()
+        self.online = True
+        RNS.log("Serial port "+self.port+" is now open", RNS.LOG_VERBOSE)
+
+
+    # This method will be called from our read-loop
+    # whenever a full packet has been received over
+    # the underlying medium.
+    def process_incoming(self, data):
+        # Update our received bytes counter
+        self.rxb += len(data)            
+
+        # And send the data packet to the Transport
+        # instance for processing.
+        self.owner.inbound(data, self)
+
+    # The running Reticulum Transport instance will
+    # call this method on the interface whenever the
+    # interface must transmit a packet.
+    def process_outgoing(self,data):
+        if self.online:
+            # First, escape and packetize the data
+            # according to HDLC framing.
+            data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+
+            # Then write the framed data to the port
+            written = self.serial.write(data)
+
+            # Update the transmitted bytes counter
+            # and ensure that all data was written
+            self.txb += len(data)            
+            if written != len(data):
+                raise IOError("Serial interface only wrote "+str(written)+" bytes of "+str(len(data)))
+
+    # This read loop runs in a thread and continously
+    # receives bytes from the underlying serial port.
+    # When a full packet has been received, it will
+    # be sent to the process_incoming methed, which
+    # will in turn pass it to the Transport instance.
+    def read_loop(self):
+        try:
+            in_frame = False
+            escape = False
+            data_buffer = b""
+            last_read_ms = int(time.time()*1000)
+
+            while self.serial.is_open:
+                if self.serial.in_waiting:
+                    byte = ord(self.serial.read(1))
+                    last_read_ms = int(time.time()*1000)
+
+                    if (in_frame and byte == HDLC.FLAG):
+                        in_frame = False
+                        self.process_incoming(data_buffer)
+                    elif (byte == HDLC.FLAG):
+                        in_frame = True
+                        data_buffer = b""
+                    elif (in_frame and len(data_buffer) < self.HW_MTU):
+                        if (byte == HDLC.ESC):
+                            escape = True
+                        else:
+                            if (escape):
+                                if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
+                                    byte = HDLC.FLAG
+                                if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
+                                    byte = HDLC.ESC
+                                escape = False
+                            data_buffer = data_buffer+bytes([byte])
+                        
+                else:
+                    time_since_last = int(time.time()*1000) - last_read_ms
+                    if len(data_buffer) > 0 and time_since_last > self.timeout:
+                        data_buffer = b""
+                        in_frame = False
+                        escape = False
+                    sleep(0.08)
+                    
+        except Exception as e:
+            self.online = False
+            RNS.log("A serial port error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is now offline.", RNS.LOG_ERROR)
+            
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+            RNS.log("Reticulum will attempt to reconnect the interface periodically.", RNS.LOG_ERROR)
+
+        self.online = False
+        self.serial.close()
+        self.reconnect_port()
+
+    # This method handles serial port disconnects.
+    def reconnect_port(self):
+        while not self.online:
+            try:
+                time.sleep(5)
+                RNS.log("Attempting to reconnect serial port "+str(self.port)+" for "+str(self)+"...", RNS.LOG_VERBOSE)
+                self.open_port()
+                if self.serial.is_open:
+                    self.configure_device()
+            except Exception as e:
+                RNS.log("Error while reconnecting port, the contained exception was: "+str(e), RNS.LOG_ERROR)
+
+        RNS.log("Reconnected serial port for "+str(self))
+
+    # Signal to Reticulum that this interface should
+    # not perform any ingress limiting.
+    def should_ingress_limit(self):
+        return False
+
+    # We must provide a string representation of this
+    # interface, that is used whenever the interface
+    # is printed in logs or external programs.
+    def __str__(self):
+        return "ExampleInterface["+self.name+"]"
+
+# Finally, register the defined interface class as the
+# target class for Reticulum to use as an interface
+interface_class = ExampleInterface

Examples/Filetransfer.py

@@ -224,7 +224,7 @@ def client(destination_hexhash, configpath):
         destination_hash = bytes.fromhex(destination_hexhash)
     except:
         RNS.log("Invalid destination entered. Check your input!\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -462,7 +462,7 @@ def filelist_timeout_job():
     global server_files
     if len(server_files) == 0:
         RNS.log("Timed out waiting for filelist, exiting")
-        os._exit(0)
+        sys.exit(0)
 
 
 # When a link is closed, we'll inform the
@@ -475,9 +475,8 @@ def link_closed(link):
     else:
         RNS.log("Link closed, exiting now")
     
-    RNS.Reticulum.exit_handler()
     time.sleep(1.5)
-    os._exit(0)
+    sys.exit(0)
 
 # When RNS detects that the download has
 # started, we'll update our menu state
@@ -601,4 +600,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Identify.py

@@ -133,7 +133,7 @@ def client(destination_hexhash, configpath):
         destination_hash = bytes.fromhex(destination_hexhash)
     except:
         RNS.log("Invalid destination entered. Check your input!\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -245,9 +245,8 @@ def link_closed(link):
     else:
         RNS.log("Link closed, exiting now")
     
-    RNS.Reticulum.exit_handler()
     time.sleep(1.5)
-    os._exit(0)
+    sys.exit(0)
 
 # When a packet is received over the link, we
 # simply print out the data.
@@ -311,4 +310,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Link.py

@@ -119,7 +119,7 @@ def client(destination_hexhash, configpath):
         destination_hash = bytes.fromhex(destination_hexhash)
     except:
         RNS.log("Invalid destination entered. Check your input!\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -222,9 +222,8 @@ def link_closed(link):
     else:
         RNS.log("Link closed, exiting now")
     
-    RNS.Reticulum.exit_handler()
     time.sleep(1.5)
-    os._exit(0)
+    sys.exit(0)
 
 # When a packet is received over the link, we
 # simply print out the data.
@@ -288,4 +287,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Minimal.py

@@ -5,6 +5,7 @@
 ##########################################################
 
 import argparse
+import sys
 import RNS
 
 # Let's define an app name. We'll use this for all
@@ -98,4 +99,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Ratchets.py

@@ -5,6 +5,7 @@
 ##########################################################
 
 import argparse
+import sys
 import RNS
 
 # Let's define an app name. We'll use this for all
@@ -25,9 +26,6 @@ def server(configpath):
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
-
-    # TODO: Remove
-    RNS.loglevel = RNS.LOG_DEBUG
     
     # Randomly create a new identity for our echo server
     server_identity = RNS.Identity()
@@ -141,7 +139,7 @@ def client(destination_hexhash, configpath, timeout=None):
     except Exception as e:
         RNS.log("Invalid destination entered. Check your input!")
         RNS.log(str(e)+"\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -340,4 +338,4 @@ if __name__ == "__main__":
                 client(args.destination, configarg, timeout=timeoutarg)
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Request.py

@@ -119,7 +119,7 @@ def client(destination_hexhash, configpath):
         destination_hash = bytes.fromhex(destination_hexhash)
     except:
         RNS.log("Invalid destination entered. Check your input!\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -226,9 +226,8 @@ def link_closed(link):
     else:
         RNS.log("Link closed, exiting now")
     
-    RNS.Reticulum.exit_handler()
     time.sleep(1.5)
-    os._exit(0)
+    sys.exit(0)
 
 
 ##########################################################
@@ -284,4 +283,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Examples/Speedtest.py

@@ -149,8 +149,6 @@ def server_packet_received(message, packet):
         time.sleep(0.2)
         rc = 0
         received_data = 0
-        # latest_client_link.teardown()
-        # os._exit(0)
 
 
 ##########################################################
@@ -159,6 +157,7 @@ def server_packet_received(message, packet):
 
 # A reference to the server link
 server_link = None
+should_quit = False
 
 # This initialisation is executed when the users chooses
 # to run as a client
@@ -175,7 +174,7 @@ def client(destination_hexhash, configpath):
         destination_hash = bytes.fromhex(destination_hexhash)
     except:
         RNS.log("Invalid destination entered. Check your input!\n")
-        exit()
+        sys.exit(0)
 
     # We must first initialise Reticulum
     reticulum = RNS.Reticulum(configpath)
@@ -216,7 +215,7 @@ def client(destination_hexhash, configpath):
     client_loop()
 
 def client_loop():
-    global server_link
+    global server_link, should_quit
 
     # Wait for the link to become active
     while not server_link:
@@ -224,16 +223,7 @@ def client_loop():
 
     should_quit = False
     while not should_quit:
-        try:
-            text = input()
-
-            # Check if we should quit the example
-            if text == "quit" or text == "q" or text == "exit":
-                should_quit = True
-                server_link.teardown()
-
-        except Exception as e:
-            raise e
+        time.sleep(0.2)
 
 # This function is called when a link
 # has been established with the server
@@ -246,8 +236,8 @@ def link_established(link):
 
     # Inform the user that the server is
     # connected
-    RNS.log("Link established with server,sending...")
-    rd = os.urandom(RNS.Link.MDU)
+    RNS.log("Link established with server, sending...")
+    rd = os.urandom(link.mdu)
     started = time.time()
     while link.status == RNS.Link.ACTIVE and data_sent < data_cap*1.25:
         RNS.Packet(server_link, rd, create_receipt=False).send()
@@ -276,17 +266,17 @@ def link_established(link):
 # When a link is closed, we'll inform the
 # user, and exit the program
 def link_closed(link):
+    global should_quit
     if link.teardown_reason == RNS.Link.TIMEOUT:
         RNS.log("The link timed out, exiting now")
     elif link.teardown_reason == RNS.Link.DESTINATION_CLOSED:
         RNS.log("The link was closed by the server, exiting now")
     else:
         RNS.log("Link closed, exiting now")
-    
-    RNS.Reticulum.exit_handler()
 
+    should_quit = True
     time.sleep(1.5)
-    os._exit(0)
+    sys.exit(0)
 
 def client_packet_received(message, packet):
     pass
@@ -344,4 +334,4 @@ if __name__ == "__main__":
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        sys.exit(0)

Makefile

@@ -24,6 +24,12 @@ clean:
 	@make -C docs clean
 	@echo Done
 
+purge_docs:
+	@echo Purging documentation build...
+	@-rm -rf ./docs/manual
+	@-rm -rf ./docs/*.pdf
+	@-rm -rf ./docs/*.epub
+
 remove_symlinks:
 	@echo Removing symlinks for build...
 	-rm Examples/RNS
@@ -34,14 +40,14 @@ create_symlinks:
 	-ln -s ../RNS ./Examples/
 	-ln -s ../../RNS ./RNS/Utilities/
 
-build_sdist_only:
+build_sdist: purge_docs
 	python3 setup.py sdist
 
 build_wheel:
-	python3 setup.py sdist bdist_wheel
+	python3 setup.py bdist_wheel
 
 build_pure_wheel:
-	python3 setup.py sdist bdist_wheel --pure
+	python3 setup.py bdist_wheel --pure
 
 documentation:
 	make -C docs html
@@ -49,7 +55,7 @@ documentation:
 manual:
 	make -C docs latexpdf epub
 
-release: test remove_symlinks build_wheel build_pure_wheel documentation manual create_symlinks
+release: test remove_symlinks build_sdist build_wheel build_pure_wheel documentation manual create_symlinks
 
 debug: remove_symlinks build_wheel build_pure_wheel create_symlinks
 

README.md

@@ -1,4 +1,4 @@
-Reticulum Network Stack β <img align="right" src="https://static.pepy.tech/personalized-badge/rns?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Installs" style="padding-left:10px"/><a href="https://github.com/markqvist/reticulum/actions/workflows/python-app.yml"><img align="right" src="https://github.com/markqvist/reticulum/actions/workflows/python-app.yml/badge.svg"/></a>
+Reticulum Network Stack β <img align="right" src="https://static.pepy.tech/personalized-badge/rns?period=month&units=international_system&left_color=grey&right_color=blue&left_text=Installs/month" style="padding-left:10px"/><a href="https://github.com/markqvist/Reticulum/actions/workflows/build.yml"><img align="right" src="https://github.com/markqvist/Reticulum/actions/workflows/build.yml/badge.svg"/></a>
 ==========
 
 <p align="center"><img width="200" src="https://raw.githubusercontent.com/markqvist/Reticulum/master/docs/source/graphics/rns_logo_512.png"></p>
@@ -41,21 +41,32 @@ For more info, see [reticulum.network](https://reticulum.network/) and [the FAQ
 
 ## Notable Features
 - Coordination-less globally unique addressing and identification
-- Fully self-configuring multi-hop routing
+- Fully self-configuring multi-hop routing over heterogeneous carriers
+- Flexible scalability over heterogeneous topologies
+  - Reticulum can carry data over any mixture of physical mediums and topologies
+  - Low-bandwidth networks can co-exist and interoperate with large, high-bandwidth networks
 - Initiator anonymity, communicate without revealing your identity
+  - Reticulum does not include source addresses on any packets
 - Asymmetric X25519 encryption and Ed25519 signatures as a basis for all communication
-- Forward Secrecy with ephemeral Elliptic Curve Diffie-Hellman keys on Curve25519
+  - The foundational Reticulum Identity Keys are 512-bit Elliptic Curve keysets
+- Forward Secrecy is available for all communication types, both for single packets and over links
 - Reticulum uses the following format for encrypted tokens:
-  - Keys are ephemeral and derived from an ECDH key exchange on Curve25519
+  - Ephemeral per-packet and link keys and derived from an ECDH key exchange on Curve25519
   - AES-128 in CBC mode with PKCS7 padding
   - HMAC using SHA256 for authentication
   - IVs are generated through os.urandom()
 - Unforgeable packet delivery confirmations
-- A variety of supported interface types
+- Flexible and extensible interface system
+  - Reticulum includes a large variety of built-in interface types
+  - Ability to load and utilise custom user- or community-supplied interface types
+  - Easily create your own custom interfaces for communicating over anything
+- Authentication and virtual network segmentation on all supported interface types
 - An intuitive and easy-to-use API
+  - Simpler and easier to use than sockets APIs and simpler, but more powerful
+  - Makes building distributed and decentralised applications much simpler
 - Reliable and efficient transfer of arbitrary amounts of data
   - Reticulum can handle a few bytes of data or files of many gigabytes
-  - Sequencing, transfer coordination and checksumming are automatic
+  - Sequencing, compression, transfer coordination and checksumming are automatic
   - The API is very easy to use, and provides transfer progress
 - Lightweight, flexible and expandable Request/Response mechanism
 - Efficient link establishment
@@ -170,11 +181,12 @@ program.
 
 Reticulum implements a range of generalised interface types that covers most of
 the communications hardware that Reticulum can run over. If your hardware is
-not supported, it's relatively simple to implement an interface class. I will
-gratefully accept pull requests for custom interfaces if they are generally
-useful.
+not supported, it's [simple to implement a custom interface module](https://markqvist.github.io/Reticulum/manual/interfaces.html#custom-interfaces).
 
-Currently, the following interfaces are supported:
+Pull requests for custom interfaces are gratefully accepted, provided they are
+generally useful and well-tested in real-world usage.
+
+Currently, the following built-in interfaces are supported:
 
 - Any Ethernet device
 - LoRa using [RNode](https://unsigned.io/rnode/)
@@ -298,14 +310,15 @@ Are certain features in the development roadmap are important to you or your
 organisation? Make them a reality quickly by sponsoring their implementation.
 
 ## Cryptographic Primitives
-Reticulum uses a simple suite of efficient, strong and modern cryptographic
+Reticulum uses a simple suite of efficient, strong and well-tested cryptographic
 primitives, with widely available implementations that can be used both on
-general-purpose CPUs and on microcontrollers. The necessary primitives are:
+general-purpose CPUs and on microcontrollers. The utilised primitives are:
 
-- Ed25519 for signatures
-- X22519 for ECDH key exchanges
+- Reticulum Identity Keys are 512-bit Curve25519 keysets
+  - A 256-bit Ed25519 key for signatures
+  - A 256-bit X22519 key for ECDH key exchanges
 - HKDF for key derivation
-- Modified Fernet for encrypted tokens
+- Encrypted tokens are based on the [Fernet spec](https://github.com/fernet/spec/)
   - Ephemeral keys derived from an ECDH key exchange on Curve25519
   - AES-128 in CBC mode with PKCS7 padding
   - HMAC using SHA256 for message authentication
@@ -319,12 +332,12 @@ In the default installation configuration, the `X25519`, `Ed25519` and
 (via the [PyCA/cryptography](https://github.com/pyca/cryptography) package).
 The hashing functions `SHA-256` and `SHA-512` are provided by the standard
 Python [hashlib](https://docs.python.org/3/library/hashlib.html). The `HKDF`,
-`HMAC`, `Fernet` primitives, and the `PKCS7` padding function are always
+`HMAC`, `Token` primitives, and the `PKCS7` padding function are always
 provided by the following internal implementations:
 
 - [HKDF.py](RNS/Cryptography/HKDF.py)
 - [HMAC.py](RNS/Cryptography/HMAC.py)
-- [Fernet.py](RNS/Cryptography/Fernet.py)
+- [Token.py](RNS/Cryptography/Token.py)
 - [PKCS7.py](RNS/Cryptography/PKCS7.py)
 
 
@@ -364,7 +377,6 @@ projects:
 - [I2Plib](https://github.com/l-n-s/i2plib) by [Viktor Villainov](https://github.com/l-n-s)
 - [PySerial](https://github.com/pyserial/pyserial) by Chris Liechti, *BSD License*
 - [Configobj](https://github.com/DiffSK/configobj) by Michael Foord, Nicola Larosa, Rob Dennis & Eli Courtwright, *BSD License*
-- [Six](https://github.com/benjaminp/six) by [Benjamin Peterson](https://github.com/benjaminp), *MIT License*
-- [ifaddr](https://github.com/pydron/ifaddr) by [Pydron](https://github.com/pydron), *MIT License*
+- [ifaddr](https://github.com/pydron/ifaddr) by Stefan C. Mueller, *MIT License*
 - [Umsgpack.py](https://github.com/vsergeev/u-msgpack-python) by [Ivan A. Sergeev](https://github.com/vsergeev)
 - [Python](https://www.python.org)

RNS/Buffer.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -45,7 +45,8 @@ class StreamDataMessage(MessageBase):
     The stream id is limited to 2 bytes - 2 bit
     """
 
-    MAX_DATA_LEN = RNS.Link.MDU - 2 - 6  # 2 for stream data message header, 6 for channel envelope
+    OVERHEAD     = 2 + 6  # 2 for stream data message header, 6 for channel envelope
+    MAX_DATA_LEN = RNS.Link.MDU - OVERHEAD
     """
     When the Buffer package is imported, this value is
     calculcated based on the value of OVERHEAD
@@ -215,6 +216,7 @@ class RawChannelWriter(RawIOBase, AbstractContextManager):
         self._stream_id = stream_id
         self._channel = channel
         self._eof = False
+        self._mdu = channel.mdu - StreamDataMessage.OVERHEAD
 
     def write(self, __b: bytes) -> int | None:
         try:

RNS/Channel.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -602,7 +602,7 @@ class Channel(contextlib.AbstractContextManager):
         return envelope
 
     @property
-    def MDU(self):
+    def mdu(self):
         """
         Maximum Data Unit: the number of bytes available
         for a message to consume in a single send. This
@@ -611,7 +611,10 @@ class Channel(contextlib.AbstractContextManager):
 
         :return: number of bytes available
         """
-        return self._outlet.mdu - 6  # sizeof(msgtype) + sizeof(length) + sizeof(sequence)
+        mdu = self._outlet.mdu - 6  # sizeof(msgtype) + sizeof(length) + sizeof(sequence)
+        if mdu > 0xFFFF:
+            mdu = 0xFFFF
+        return mdu
 
 
 class LinkChannelOutlet(ChannelOutletBase):
@@ -639,7 +642,7 @@ class LinkChannelOutlet(ChannelOutletBase):
 
     @property
     def mdu(self):
-        return self.link.MDU
+        return self.link.mdu
 
     @property
     def rtt(self):

RNS/Cryptography/AES.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2022 Mark Qvist / unsigned.io
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal

RNS/Cryptography/Ed25519.py

@@ -1,3 +1,25 @@
+# MIT License
+#
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 import os
 from .pure25519 import ed25519_oop as ed25519
 

RNS/Cryptography/HKDF.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2022 Mark Qvist / unsigned.io
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -48,7 +48,7 @@ def hkdf(length=None, derive_from=None, salt=None, context=None):
     derived = b""
 
     for i in range(ceil(length / hash_len)):
-        block = hmac_sha256(pseudorandom_key, block + context + bytes([i + 1]))
+        block = hmac_sha256(pseudorandom_key, block + context + bytes([(i + 1)%(0xFF+1)]))
         derived += block
 
     return derived[:length]

RNS/Cryptography/Hashes.py

@@ -1,4 +1,26 @@
-import importlib
+# MIT License
+#
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import importlib.util
 if importlib.util.find_spec('hashlib') != None:
     import hashlib
 else:

RNS/Cryptography/PKCS7.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2022 Mark Qvist / unsigned.io
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal

RNS/Cryptography/Provider.py

@@ -1,16 +1,39 @@
-import importlib
+# MIT License
+#
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import importlib.util
 
 PROVIDER_NONE     = 0x00
 PROVIDER_INTERNAL = 0x01
 PROVIDER_PYCA     = 0x02
 
+FORCE_INTERNAL = False
 PROVIDER = PROVIDER_NONE
 
 pyca_v = None
 use_pyca = False
 
 try:
-    if importlib.util.find_spec('cryptography') != None:
+    if not FORCE_INTERNAL and importlib.util.find_spec('cryptography') != None:
         import cryptography
         pyca_v = cryptography.__version__
         v = pyca_v.split(".")

RNS/Cryptography/Proxies.py

@@ -1,3 +1,25 @@
+# MIT License
+#
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 from cryptography.hazmat.primitives import serialization
 from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey, Ed25519PublicKey
 from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey, X25519PublicKey

RNS/Cryptography/Token.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2022 Mark Qvist / unsigned.io
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -27,7 +27,7 @@ from RNS.Cryptography import HMAC
 from RNS.Cryptography import PKCS7
 from RNS.Cryptography.AES import AES_128_CBC
 
-class Fernet():
+class Token():
     """
     This class provides a slightly modified implementation of the Fernet spec
     found at: https://github.com/fernet/spec/blob/master/Spec.md
@@ -37,7 +37,7 @@ class Fernet():
     not relevant to Reticulum. They are therefore stripped from this
     implementation, since they incur overhead and leak initiator metadata.
     """
-    FERNET_OVERHEAD  = 48 # Bytes
+    TOKEN_OVERHEAD  = 48 # Bytes
 
     @staticmethod
     def generate_key():

RNS/Cryptography/X25519.py

@@ -82,10 +82,13 @@ def _fix_secret(n):
     n |= 64 << 8 * 31
     return n
 
+def _fix_base_point(n):
+    n &= ~(2**255)
+    return n
 
 def curve25519(base_point_raw, secret_raw):
     """Raise the base point to a given power"""
-    base_point = _unpack_number(base_point_raw)
+    base_point = _fix_base_point(_unpack_number(base_point_raw))
     secret = _fix_secret(_unpack_number(secret_raw))
     return _pack_number(_raw_curve25519(base_point, secret))
 

RNS/Cryptography/__init__.py

@@ -1,3 +1,25 @@
+# MIT License
+#
+# Copyright (c) 2022-2025 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
 import os
 import glob
 
@@ -5,7 +27,7 @@ from .Hashes import sha256
 from .Hashes import sha512
 from .HKDF import hkdf
 from .PKCS7 import PKCS7
-from .Fernet import Fernet
+from .Token import Token
 from .Provider import backend
 
 import RNS.Cryptography.Provider as cp
@@ -20,5 +42,7 @@ elif cp.PROVIDER == cp.PROVIDER_PYCA:
     from RNS.Cryptography.Proxies import Ed25519PrivateKeyProxy as Ed25519PrivateKey
     from RNS.Cryptography.Proxies import Ed25519PublicKeyProxy as Ed25519PublicKey
 
-modules = glob.glob(os.path.dirname(__file__)+"/*.py")
-__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]
+py_modules  = glob.glob(os.path.dirname(__file__)+"/*.py")
+pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
+modules     = py_modules+pyc_modules
+__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))

RNS/Destination.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2024 Mark Qvist / unsigned.io and contributors
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -26,7 +26,7 @@ import time
 import threading
 import RNS
 
-from RNS.Cryptography import Fernet
+from RNS.Cryptography import Token
 from .vendor import umsgpack as umsgpack
 
 class Callbacks:
@@ -192,7 +192,7 @@ class Destination:
         """
         :returns: A human-readable representation of the destination including addressable hash and full name.
         """
-        return "<"+self.name+"/"+self.hexhash+">"
+        return "<"+self.name+":"+self.hexhash+">"
 
     def _clean_ratchets(self):
         if self.ratchets != None:
@@ -424,7 +424,7 @@ class Destination:
     def _reload_ratchets(self, ratchets_path):
         if os.path.isfile(ratchets_path):
             with self.ratchet_file_lock:
-                try:
+                def load_attempt():
                     ratchets_file = open(ratchets_path, "rb")
                     persisted_data = umsgpack.unpackb(ratchets_file.read())
                     if "signature" in persisted_data and "ratchets" in persisted_data:
@@ -433,10 +433,22 @@ class Destination:
                             self.ratchets_path = ratchets_path
                         else:
                             raise KeyError("Invalid ratchet file signature")
+                
+                try:
+                    try:
+                        load_attempt()
+
+                    except Exception as e:
+                        RNS.trace_exception(e)
+                        RNS.log(f"First ratchet reload attempt for {self} failed. Possible I/O conflict. Retrying in 500ms.", RNS.LOG_ERROR)
+                        time.sleep(0.5)
+                        load_attempt()
+                        RNS.log(f"Ratchet reload retry succeeded", RNS.LOG_DEBUG)
 
                 except Exception as e:
                     self.ratchets = None
                     self.ratchets_path = None
+                    RNS.trace_exception(e)
                     raise OSError("Could not read ratchet file contents for "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
         else:
             RNS.log("No existing ratchet data found, initialising new ratchet file for "+str(self), RNS.LOG_DEBUG)
@@ -525,8 +537,8 @@ class Destination:
             raise TypeError("A single destination holds keys through an Identity instance")
 
         if self.type == Destination.GROUP:
-            self.prv_bytes = Fernet.generate_key()
-            self.prv = Fernet(self.prv_bytes)
+            self.prv_bytes = Token.generate_key()
+            self.prv = Token(self.prv_bytes)
 
     def get_private_key(self):
         """
@@ -556,7 +568,7 @@ class Destination:
 
         if self.type == Destination.GROUP:
             self.prv_bytes = key
-            self.prv = Fernet(self.prv_bytes)
+            self.prv = Token(self.prv_bytes)
 
     def load_public_key(self, key):
         if self.type != Destination.SINGLE:

RNS/Identity.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2024 Mark Qvist / unsigned.io and contributors.
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -31,7 +31,7 @@ import threading
 from .vendor import umsgpack as umsgpack
 
 from RNS.Cryptography import X25519PrivateKey, X25519PublicKey, Ed25519PrivateKey, Ed25519PublicKey
-from RNS.Cryptography import Fernet
+from RNS.Cryptography import Token
 
 
 class Identity:
@@ -66,7 +66,7 @@ class Identity:
     """
 
     # Non-configurable constants
-    FERNET_OVERHEAD           = RNS.Cryptography.Fernet.FERNET_OVERHEAD
+    TOKEN_OVERHEAD            = RNS.Cryptography.Token.TOKEN_OVERHEAD
     AES128_BLOCKSIZE          = 16          # In bytes
     HASHLENGTH                = 256         # In bits
     SIGLENGTH                 = KEYSIZE     # In bits
@@ -93,29 +93,47 @@ class Identity:
 
 
     @staticmethod
-    def recall(destination_hash):
+    def recall(target_hash, from_identity_hash=False):
         """
-        Recall identity for a destination hash.
+        Recall identity for a destination or identity hash. By default, this function
+        will return the identity associated with a given *destination* hash. As an
+        example, if you know the ``lxmf.delivery`` destination hash of an endpoint,
+        this function will return the associated underlying identity. You can also
+        search for an identity from a known *identity hash*, by setting the
+        ``from_identity_hash`` argument.
 
-        :param destination_hash: Destination hash as *bytes*.
+        :param target_hash: Destination or identity hash as *bytes*.
+        :param from_identity_hash: Whether to search based on identity hash instead of destination hash as *bool*.
         :returns: An :ref:`RNS.Identity<api-identity>` instance that can be used to create an outgoing :ref:`RNS.Destination<api-destination>`, or *None* if the destination is unknown.
         """
-        if destination_hash in Identity.known_destinations:
-            identity_data = Identity.known_destinations[destination_hash]
-            identity = Identity(create_keys=False)
-            identity.load_public_key(identity_data[2])
-            identity.app_data = identity_data[3]
-            return identity
-        else:
-            for registered_destination in RNS.Transport.destinations:
-                if destination_hash == registered_destination.hash:
+        if from_identity_hash:
+            for destination_hash in Identity.known_destinations:
+                if target_hash == Identity.truncated_hash(Identity.known_destinations[destination_hash][2]):
+                    identity_data = Identity.known_destinations[destination_hash]
                     identity = Identity(create_keys=False)
-                    identity.load_public_key(registered_destination.identity.get_public_key())
-                    identity.app_data = None
+                    identity.load_public_key(identity_data[2])
+                    identity.app_data = identity_data[3]
                     return identity
 
             return None
 
+        else:
+            if target_hash in Identity.known_destinations:
+                identity_data = Identity.known_destinations[target_hash]
+                identity = Identity(create_keys=False)
+                identity.load_public_key(identity_data[2])
+                identity.app_data = identity_data[3]
+                return identity
+            else:
+                for registered_destination in RNS.Transport.destinations:
+                    if target_hash == registered_destination.hash:
+                        identity = Identity(create_keys=False)
+                        identity.load_public_key(registered_destination.identity.get_public_key())
+                        identity.app_data = None
+                        return identity
+
+                return None
+
     @staticmethod
     def recall_app_data(destination_hash):
         """
@@ -155,9 +173,9 @@ class Identity:
             storage_known_destinations = {}
             if os.path.isfile(RNS.Reticulum.storagepath+"/known_destinations"):
                 try:
-                    file = open(RNS.Reticulum.storagepath+"/known_destinations","rb")
-                    storage_known_destinations = umsgpack.load(file)
-                    file.close()
+                    with open(RNS.Reticulum.storagepath+"/known_destinations","rb") as file:
+                        storage_known_destinations = umsgpack.load(file)
+ 
                 except:
                     pass
 
@@ -169,9 +187,9 @@ class Identity:
                 RNS.log("Skipped recombining known destinations from disk, since an error occurred: "+str(e), RNS.LOG_WARNING)
 
             RNS.log("Saving "+str(len(Identity.known_destinations))+" known destinations to storage...", RNS.LOG_DEBUG)
-            file = open(RNS.Reticulum.storagepath+"/known_destinations","wb")
-            umsgpack.dump(Identity.known_destinations, file)
-            file.close()
+            with open(RNS.Reticulum.storagepath+"/known_destinations","wb") as file:
+                umsgpack.dump(Identity.known_destinations, file)
+            
 
             save_time = time.time() - save_start
             if save_time < 1:
@@ -191,9 +209,8 @@ class Identity:
     def load_known_destinations():
         if os.path.isfile(RNS.Reticulum.storagepath+"/known_destinations"):
             try:
-                file = open(RNS.Reticulum.storagepath+"/known_destinations","rb")
-                loaded_known_destinations = umsgpack.load(file)
-                file.close()
+                with open(RNS.Reticulum.storagepath+"/known_destinations","rb") as file:
+                    loaded_known_destinations = umsgpack.load(file)
 
                 Identity.known_destinations = {}
                 for known_destination in loaded_known_destinations:
@@ -267,31 +284,34 @@ class Identity:
 
     @staticmethod
     def _remember_ratchet(destination_hash, ratchet):
-        # TODO: Remove at some point, and only log new ratchets
-        RNS.log(f"Remembering ratchet {RNS.prettyhexrep(Identity._get_ratchet_id(ratchet))} for {RNS.prettyhexrep(destination_hash)}", RNS.LOG_EXTREME)
         try:
-            Identity.known_ratchets[destination_hash] = ratchet
+            if destination_hash in Identity.known_ratchets and Identity.known_ratchets[destination_hash] == ratchet:
+                ratchet_exists = True
+            else:
+                ratchet_exists = False
 
-            if not RNS.Transport.owner.is_connected_to_shared_instance:
-                def persist_job():
-                    with Identity.ratchet_persist_lock:
-                        hexhash = RNS.hexrep(destination_hash, delimit=False)
-                        ratchet_data = {"ratchet": ratchet, "received": time.time()}
+            if not ratchet_exists:
+                RNS.log(f"Remembering ratchet {RNS.prettyhexrep(Identity._get_ratchet_id(ratchet))} for {RNS.prettyhexrep(destination_hash)}", RNS.LOG_EXTREME)
+                Identity.known_ratchets[destination_hash] = ratchet
+                if not RNS.Transport.owner.is_connected_to_shared_instance:
+                    def persist_job():
+                        with Identity.ratchet_persist_lock:
+                            hexhash = RNS.hexrep(destination_hash, delimit=False)
+                            ratchet_data = {"ratchet": ratchet, "received": time.time()}
 
-                        ratchetdir = RNS.Reticulum.storagepath+"/ratchets"
-                        
-                        if not os.path.isdir(ratchetdir):
-                            os.makedirs(ratchetdir)
+                            ratchetdir = RNS.Reticulum.storagepath+"/ratchets"
+                            
+                            if not os.path.isdir(ratchetdir):
+                                os.makedirs(ratchetdir)
 
-                        outpath   = f"{ratchetdir}/{hexhash}.out"
-                        finalpath = f"{ratchetdir}/{hexhash}"
-                        ratchet_file = open(outpath, "wb")
-                        ratchet_file.write(umsgpack.packb(ratchet_data))
-                        ratchet_file.close()
-                        os.replace(outpath, finalpath)
+                            outpath   = f"{ratchetdir}/{hexhash}.out"
+                            finalpath = f"{ratchetdir}/{hexhash}"
+                            with open(outpath, "wb") as ratchet_file:
+                                ratchet_file.write(umsgpack.packb(ratchet_data))
+                            os.replace(outpath, finalpath)
 
-                
-                threading.Thread(target=persist_job, daemon=True).start()
+                    
+                    threading.Thread(target=persist_job, daemon=True).start()
 
         except Exception as e:
             RNS.log(f"Could not persist ratchet for {RNS.prettyhexrep(destination_hash)} to storage.", RNS.LOG_ERROR)
@@ -308,12 +328,19 @@ class Identity:
                 for filename in os.listdir(ratchetdir):
                     try:
                         expired = False
+                        corrupted = False
                         with open(f"{ratchetdir}/{filename}", "rb") as rf:
-                            ratchet_data = umsgpack.unpackb(rf.read())
-                            if now > ratchet_data["received"]+Identity.RATCHET_EXPIRY:
-                                expired = True
+                            # TODO: Remove individual ratchet file if corrupt
+                            try:
+                                ratchet_data = umsgpack.unpackb(rf.read())
+                                if now > ratchet_data["received"]+Identity.RATCHET_EXPIRY:
+                                    expired = True
 
-                        if expired:
+                            except Exception as e:
+                                RNS.log(f"Corrupted ratchet data while reading {ratchetdir}/{filename}, removing file", RNS.LOG_ERROR)
+                                corrupted = True
+
+                        if expired or corrupted:
                             os.unlink(f"{ratchetdir}/{filename}")
 
                     except Exception as e:
@@ -331,12 +358,12 @@ class Identity:
             ratchet_path = f"{ratchetdir}/{hexhash}"
             if os.path.isfile(ratchet_path):
                 try:
-                    ratchet_file = open(ratchet_path, "rb")
-                    ratchet_data = umsgpack.unpackb(ratchet_file.read())
-                    if time.time() < ratchet_data["received"]+Identity.RATCHET_EXPIRY and len(ratchet_data["ratchet"]) == Identity.RATCHETSIZE//8:
-                        Identity.known_ratchets[destination_hash] = ratchet_data["ratchet"]
-                    else:
-                        return None
+                    with open(ratchet_path, "rb") as ratchet_file:
+                        ratchet_data = umsgpack.unpackb(ratchet_file.read())
+                        if time.time() < ratchet_data["received"]+Identity.RATCHET_EXPIRY and len(ratchet_data["ratchet"]) == Identity.RATCHETSIZE//8:
+                            Identity.known_ratchets[destination_hash] = ratchet_data["ratchet"]
+                        else:
+                            return None
                 
                 except Exception as e:
                     RNS.log(f"An error occurred while loading ratchet data for {RNS.prettyhexrep(destination_hash)} from storage.", RNS.LOG_ERROR)
@@ -648,8 +675,8 @@ class Identity:
                 context=self.get_context(),
             )
 
-            fernet = Fernet(derived_key)
-            ciphertext = fernet.encrypt(plaintext)
+            token = Token(derived_key)
+            ciphertext = token.encrypt(plaintext)
             token = ephemeral_pub_bytes+ciphertext
 
             return token
@@ -686,8 +713,8 @@ class Identity:
                                     context=self.get_context(),
                                 )
 
-                                fernet = Fernet(derived_key)
-                                plaintext = fernet.decrypt(ciphertext)
+                                token = Token(derived_key)
+                                plaintext = token.decrypt(ciphertext)
                                 if ratchet_id_receiver:
                                     ratchet_id_receiver.latest_ratchet_id = ratchet_id
                                 
@@ -711,8 +738,8 @@ class Identity:
                             context=self.get_context(),
                         )
 
-                        fernet = Fernet(derived_key)
-                        plaintext = fernet.decrypt(ciphertext)
+                        token = Token(derived_key)
+                        plaintext = token.decrypt(ciphertext)
                         if ratchet_id_receiver:
                             ratchet_id_receiver.latest_ratchet_id = None
 

RNS/Interfaces/AX25KISSInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 from time import sleep
 import sys
 import threading
@@ -59,6 +59,7 @@ class AX25():
 class AX25KISSInterface(Interface):
     MAX_CHUNK = 32768
     BITRATE_GUESS = 1200
+    DEFAULT_IFAC_SIZE = 8
 
     owner    = None
     port     = None
@@ -68,8 +69,8 @@ class AX25KISSInterface(Interface):
     stopbits = None
     serial   = None
 
-    def __init__(self, owner, name, callsign, ssid, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control):
-        import importlib
+    def __init__(self, owner, configuration):
+        import importlib.util
         if importlib.util.find_spec('serial') != None:
             import serial
         else:
@@ -79,6 +80,25 @@ class AX25KISSInterface(Interface):
 
         super().__init__()
 
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        preamble = int(c["preamble"]) if "preamble" in c else None
+        txtail = int(c["txtail"]) if "txtail" in c else None
+        persistence = int(c["persistence"]) if "persistence" in c else None
+        slottime = int(c["slottime"]) if "slottime" in c else None
+        flow_control = c.as_bool("flow_control") if "flow_control" in c else False
+        port = c["port"] if "port" in c else None
+        speed = int(c["speed"]) if "speed" in c else 9600
+        databits = int(c["databits"]) if "databits" in c else 8
+        parity = c["parity"] if "parity" in c else "N"
+        stopbits = int(c["stopbits"]) if "stopbits" in c else 1
+
+        callsign = c["callsign"] if "callsign" in c else ""
+        ssid = int(c["ssid"]) if "ssid" in c else -1
+
+        if port == None:
+            raise ValueError("No port specified for serial interface")
+
         self.HW_MTU = 564
         
         self.pyserial = serial
@@ -225,13 +245,13 @@ class AX25KISSInterface(Interface):
                 raise IOError("Could not enable AX.25 KISS interface flow control")
 
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         if (len(data) > AX25.HEADER_SIZE):
             self.rxb += len(data)
             self.owner.inbound(data[AX25.HEADER_SIZE:], self)
 
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         datalen = len(data)
         if self.online:
             if self.interface_ready:
@@ -281,7 +301,7 @@ class AX25KISSInterface(Interface):
         if len(self.packet_queue) > 0:
             data = self.packet_queue.pop(0)
             self.interface_ready = True
-            self.processOutgoing(data)
+            self.process_outgoing(data)
         elif len(self.packet_queue) == 0:
             self.interface_ready = True
 
@@ -300,7 +320,7 @@ class AX25KISSInterface(Interface):
 
                     if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
                         in_frame = False
-                        self.processIncoming(data_buffer)
+                        self.process_incoming(data_buffer)
                     elif (byte == KISS.FEND):
                         in_frame = True
                         command = KISS.CMD_UNKNOWN

RNS/Interfaces/Android/KISSInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -52,6 +52,7 @@ class KISS():
 class KISSInterface(Interface):
     MAX_CHUNK = 32768
     BITRATE_GUESS = 1200
+    DEFAULT_IFAC_SIZE = 8
 
     owner    = None
     port     = None
@@ -61,8 +62,8 @@ class KISSInterface(Interface):
     stopbits = None
     serial   = None
 
-    def __init__(self, owner, name, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control, beacon_interval, beacon_data):
-        import importlib
+    def __init__(self, owner, configuration):
+        import importlib.util
         if RNS.vendor.platformutils.is_android():
             self.on_android  = True
             if importlib.util.find_spec('usbserial4a') != None:
@@ -83,6 +84,21 @@ class KISSInterface(Interface):
             raise SystemError("Android-specific interface was used on non-Android OS")
 
         super().__init__()
+
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        preamble = int(c["preamble"]) if "preamble" in c else None
+        txtail = int(c["txtail"]) if "txtail" in c else None
+        persistence = int(c["persistence"]) if "persistence" in c else None
+        slottime = int(c["slottime"]) if "slottime" in c else None
+        flow_control = c.as_bool("flow_control") if "flow_control" in c else False
+        port = c["port"] if "port" in c else None
+        speed = int(c["speed"]) if "speed" in c else 9600
+        databits = int(c["databits"]) if "databits" in c else 8
+        parity = c["parity"] if "parity" in c else "N"
+        stopbits = int(c["stopbits"]) if "stopbits" in c else 1
+        beacon_interval = int(c["beacon_interval"]) if "beacon_interval" in c and c["beacon_interval"] != None else None
+        beacon_data = c["beacon_data"] if "beacon_data" in c else None
         
         self.HW_MTU = 564
         
@@ -267,13 +283,13 @@ class KISSInterface(Interface):
                 raise IOError("Could not enable KISS interface flow control")
 
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)
         def af():
             self.owner.inbound(data, self)
         threading.Thread(target=af, daemon=True).start()
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         datalen = len(data)
         if self.online:
             if self.interface_ready:
@@ -307,7 +323,7 @@ class KISSInterface(Interface):
         if len(self.packet_queue) > 0:
             data = self.packet_queue.pop(0)
             self.interface_ready = True
-            self.processOutgoing(data)
+            self.process_outgoing(data)
         elif len(self.packet_queue) == 0:
             self.interface_ready = True
 
@@ -328,7 +344,7 @@ class KISSInterface(Interface):
 
                     if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
                         in_frame = False
-                        self.processIncoming(data_buffer)
+                        self.process_incoming(data_buffer)
                     elif (byte == KISS.FEND):
                         in_frame = True
                         command = KISS.CMD_UNKNOWN
@@ -373,7 +389,13 @@ class KISSInterface(Interface):
                             if time.time() > self.first_tx + self.beacon_i:
                                 RNS.log("Interface "+str(self)+" is transmitting beacon data: "+str(self.beacon_d.decode("utf-8")), RNS.LOG_DEBUG)
                                 self.first_tx = None
-                                self.processOutgoing(self.beacon_d)
+
+                                # Pad to minimum length
+                                frame = bytearray(self.beacon_d)
+                                while len(frame) < 15:
+                                    frame.append(0x00)
+
+                                self.process_outgoing(bytes(frame))
 
         except Exception as e:
             self.online = False

RNS/Interfaces/Android/SerialInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -42,6 +42,7 @@ class HDLC():
 
 class SerialInterface(Interface):
     MAX_CHUNK = 32768
+    DEFAULT_IFAC_SIZE = 8
 
     owner    = None
     port     = None
@@ -51,8 +52,8 @@ class SerialInterface(Interface):
     stopbits = None
     serial   = None
 
-    def __init__(self, owner, name, port, speed, databits, parity, stopbits):
-        import importlib
+    def __init__(self, owner, configuration):
+        import importlib.util
         if RNS.vendor.platformutils.is_android():
             self.on_android  = True
             if importlib.util.find_spec('usbserial4a') != None:
@@ -74,6 +75,17 @@ class SerialInterface(Interface):
 
         super().__init__()
 
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        port = c["port"] if "port" in c else None
+        speed = int(c["speed"]) if "speed" in c else 9600
+        databits = int(c["databits"]) if "databits" in c else 8
+        parity = c["parity"] if "parity" in c else "N"
+        stopbits = int(c["stopbits"]) if "stopbits" in c else 1
+
+        if port == None:
+            raise ValueError("No port specified for serial interface")
+
         self.HW_MTU = 564
         
         self.pyserial = serial
@@ -172,13 +184,13 @@ class SerialInterface(Interface):
         RNS.log("Serial port "+self.port+" is now open", RNS.LOG_VERBOSE)
 
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)
         def af():
             self.owner.inbound(data, self)
         threading.Thread(target=af, daemon=True).start()
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         if self.online:
             data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
             written = self.serial.write(data)
@@ -202,7 +214,7 @@ class SerialInterface(Interface):
 
                     if (in_frame and byte == HDLC.FLAG):
                         in_frame = False
-                        self.processIncoming(data_buffer)
+                        self.process_incoming(data_buffer)
                     elif (byte == HDLC.FLAG):
                         in_frame = True
                         data_buffer = b""

RNS/Interfaces/Android/__init__.py

@@ -23,5 +23,7 @@
 import os
 import glob
 
-modules = glob.glob(os.path.dirname(__file__)+"/*.py")
-__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]
+py_modules  = glob.glob(os.path.dirname(__file__)+"/*.py")
+pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
+modules     = py_modules+pyc_modules
+__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))

RNS/Interfaces/AutoInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 from collections import deque
 import socketserver
 import threading
@@ -33,9 +33,13 @@ import RNS
 
 
 class AutoInterface(Interface):
+    HW_MTU = 1196
+    FIXED_MTU = True
+
     DEFAULT_DISCOVERY_PORT = 29716
     DEFAULT_DATA_PORT      = 42671
     DEFAULT_GROUP_ID       = "reticulum".encode("utf-8")
+    DEFAULT_IFAC_SIZE      = 16
 
     SCOPE_LINK         = "2"
     SCOPE_ADMIN        = "4"
@@ -46,7 +50,7 @@ class AutoInterface(Interface):
     MULTICAST_PERMANENT_ADDRESS_TYPE = "0"
     MULTICAST_TEMPORARY_ADDRESS_TYPE = "1"
 
-    PEERING_TIMEOUT    = 7.5
+    PEERING_TIMEOUT    = 10.0
 
     ALL_IGNORE_IFS     = ["lo0"]
     DARWIN_IGNORE_IFS  = ["awdl0", "llw0", "lo0", "en5"]
@@ -78,7 +82,6 @@ class AutoInterface(Interface):
         return ifas
 
     def interface_name_to_index(self, ifname):
-
         # socket.if_nametoindex doesn't work with uuid interface names on windows, it wants the ethernet_0 style
         # we will just get the index from netinfo instead as it seems to work
         if RNS.vendor.platformutils.is_windows():
@@ -86,16 +89,27 @@ class AutoInterface(Interface):
 
         return socket.if_nametoindex(ifname)
 
-    def __init__(self, owner, name, group_id=None, discovery_scope=None, discovery_port=None, multicast_address_type=None, data_port=None, allowed_interfaces=None, ignored_interfaces=None, configured_bitrate=None):
-        from RNS.vendor.ifaddr import niwrapper
+    def __init__(self, owner, configuration):
+        c                      = Interface.get_config_obj(configuration)
+        name                   = c["name"]
+        group_id               = c["group_id"] if "group_id" in c else None
+        discovery_scope        = c["discovery_scope"] if "discovery_scope" in c else None
+        discovery_port         = int(c["discovery_port"]) if "discovery_port" in c else None
+        multicast_address_type = c["multicast_address_type"] if "multicast_address_type" in c else None
+        data_port              = int(c["data_port"]) if "data_port" in c else None
+        allowed_interfaces     = c.as_list("devices") if "devices" in c else None
+        ignored_interfaces     = c.as_list("ignored_devices") if "ignored_devices" in c else None
+        configured_bitrate     = c["configured_bitrate"] if "configured_bitrate" in c else None
+
+        from RNS.Interfaces import netinfo
         super().__init__()
-        self.netinfo = niwrapper
-
-        self.HW_MTU = 1064
+        self.netinfo = netinfo
 
+        self.HW_MTU = AutoInterface.HW_MTU
         self.IN  = True
         self.OUT = False
         self.name = name
+        self.owner = owner
         self.online = False
         self.peers = {}
         self.link_local_addresses = []
@@ -103,6 +117,8 @@ class AutoInterface(Interface):
         self.interface_servers = {}
         self.multicast_echoes = {}
         self.timed_out_interfaces = {}
+        self.spawned_interfaces = {}
+        self.write_lock = threading.Lock()
         self.mif_deque = deque(maxlen=AutoInterface.MULTI_IF_DEQUE_LEN)
         self.mif_deque_times = deque(maxlen=AutoInterface.MULTI_IF_DEQUE_LEN)
         self.carrier_changed = False
@@ -118,7 +134,7 @@ class AutoInterface(Interface):
         # Increase peering timeout on Android, due to potential
         # low-power modes implemented on many chipsets.
         if RNS.vendor.platformutils.is_android():
-            self.peering_timeout *= 3
+            self.peering_timeout *= 2.5
 
         if allowed_interfaces == None:
             self.allowed_interfaces = []
@@ -276,32 +292,31 @@ class AutoInterface(Interface):
             else:
                 self.bitrate = AutoInterface.BITRATE_GUESS
 
-            peering_wait = self.announce_interval*1.2
-            RNS.log(str(self)+" discovering peers for "+str(round(peering_wait, 2))+" seconds...", RNS.LOG_VERBOSE)
+    def final_init(self):
+        peering_wait = self.announce_interval*1.2
+        RNS.log(str(self)+" discovering peers for "+str(round(peering_wait, 2))+" seconds...", RNS.LOG_VERBOSE)
 
-            self.owner = owner
-            socketserver.UDPServer.address_family = socket.AF_INET6
+        socketserver.UDPServer.address_family = socket.AF_INET6
 
-            for ifname in self.adopted_interfaces:
-                local_addr = self.adopted_interfaces[ifname]+"%"+str(self.interface_name_to_index(ifname))
-                addr_info = socket.getaddrinfo(local_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
-                address = addr_info[0][4]
+        for ifname in self.adopted_interfaces:
+            local_addr = self.adopted_interfaces[ifname]+"%"+str(self.interface_name_to_index(ifname))
+            addr_info = socket.getaddrinfo(local_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
+            address = addr_info[0][4]
 
-                udp_server = socketserver.UDPServer(address, self.handler_factory(self.processIncoming))
-                self.interface_servers[ifname] = udp_server
-                
-                thread = threading.Thread(target=udp_server.serve_forever)
-                thread.daemon = True
-                thread.start()
+            udp_server = socketserver.UDPServer(address, self.handler_factory(self.process_incoming))
+            self.interface_servers[ifname] = udp_server
+            
+            thread = threading.Thread(target=udp_server.serve_forever)
+            thread.daemon = True
+            thread.start()
 
-            job_thread = threading.Thread(target=self.peer_jobs)
-            job_thread.daemon = True
-            job_thread.start()
+        job_thread = threading.Thread(target=self.peer_jobs)
+        job_thread.daemon = True
+        job_thread.start()
 
-            time.sleep(peering_wait)
-
-            self.online = True
+        time.sleep(peering_wait)
 
+        self.online = True
 
     def discovery_handler(self, socket, ifname):
         def announce_loop():
@@ -313,8 +328,9 @@ class AutoInterface(Interface):
         
         while True:
             data, ipv6_src = socket.recvfrom(1024)
+            peering_hash = data[:RNS.Identity.HASHLENGTH//8]
             expected_hash = RNS.Identity.full_hash(self.group_id+ipv6_src[0].encode("utf-8"))
-            if data == expected_hash:
+            if peering_hash == expected_hash:
                 self.add_peer(ipv6_src[0], ifname)
             else:
                 RNS.log(str(self)+" received peering packet on "+str(ifname)+" from "+str(ipv6_src[0])+", but authentication hash was incorrect.", RNS.LOG_DEBUG)
@@ -335,6 +351,10 @@ class AutoInterface(Interface):
             # Remove any timed out peers
             for peer_addr in timed_out_peers:
                 removed_peer = self.peers.pop(peer_addr)
+                if peer_addr in self.spawned_interfaces:
+                    spawned_interface = self.spawned_interfaces[peer_addr]
+                    spawned_interface.detach()
+                    spawned_interface.teardown()
                 RNS.log(str(self)+" removed peer "+str(peer_addr)+" on "+str(removed_peer[0]), RNS.LOG_DEBUG)
 
             for ifname in self.adopted_interfaces:
@@ -369,7 +389,7 @@ class AutoInterface(Interface):
 
                                         RNS.log("Starting new UDP listener for "+str(self)+" "+str(ifname), RNS.LOG_DEBUG)
 
-                                        udp_server = socketserver.UDPServer(listen_address, self.handler_factory(self.processIncoming))
+                                        udp_server = socketserver.UDPServer(listen_address, self.handler_factory(self.process_incoming))
                                         self.interface_servers[ifname] = udp_server
 
                                         thread = threading.Thread(target=udp_server.serve_forever)
@@ -421,6 +441,10 @@ class AutoInterface(Interface):
             else:
                 pass
 
+    @property
+    def peer_count(self):
+        return len(self.spawned_interfaces)
+
     def add_peer(self, addr, ifname):
         if addr in self.link_local_addresses:
             ifname = None
@@ -436,53 +460,151 @@ class AutoInterface(Interface):
         else:
             if not addr in self.peers:
                 self.peers[addr] = [ifname, time.time()]
+
+                spawned_interface = AutoInterfacePeer(self, addr, ifname)
+                spawned_interface.OUT = self.OUT
+                spawned_interface.IN  = self.IN
+                spawned_interface.parent_interface = self
+                spawned_interface.bitrate = self.bitrate
+                
+                spawned_interface.ifac_size = self.ifac_size
+                spawned_interface.ifac_netname = self.ifac_netname
+                spawned_interface.ifac_netkey = self.ifac_netkey
+                if spawned_interface.ifac_netname != None or spawned_interface.ifac_netkey != None:
+                    ifac_origin = b""
+                    if spawned_interface.ifac_netname != None:
+                        ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netname.encode("utf-8"))
+                    if spawned_interface.ifac_netkey != None:
+                        ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netkey.encode("utf-8"))
+
+                    ifac_origin_hash = RNS.Identity.full_hash(ifac_origin)
+                    spawned_interface.ifac_key = RNS.Cryptography.hkdf(
+                        length=64,
+                        derive_from=ifac_origin_hash,
+                        salt=RNS.Reticulum.IFAC_SALT,
+                        context=None
+                    )
+                    spawned_interface.ifac_identity = RNS.Identity.from_bytes(spawned_interface.ifac_key)
+                    spawned_interface.ifac_signature = spawned_interface.ifac_identity.sign(RNS.Identity.full_hash(spawned_interface.ifac_key))
+
+                spawned_interface.announce_rate_target = self.announce_rate_target
+                spawned_interface.announce_rate_grace = self.announce_rate_grace
+                spawned_interface.announce_rate_penalty = self.announce_rate_penalty
+                spawned_interface.mode = self.mode
+                spawned_interface.HW_MTU = self.HW_MTU
+                spawned_interface.online = True
+                RNS.Transport.interfaces.append(spawned_interface)
+                if addr in self.spawned_interfaces:
+                    self.spawned_interfaces[addr].detach()
+                    self.spawned_interfaces[addr].teardown()
+                    self.spawned_interfaces.pop(spawned_interface)
+                self.spawned_interfaces[addr] = spawned_interface
+
                 RNS.log(str(self)+" added peer "+str(addr)+" on "+str(ifname), RNS.LOG_DEBUG)
             else:
                 self.refresh_peer(addr)
 
     def refresh_peer(self, addr):
-        self.peers[addr][1] = time.time()
+        try:
+            self.peers[addr][1] = time.time()
+        except Exception as e:
+            RNS.log(f"An error occurred while refreshing peer {addr} on {self}: {e}", RNS.LOG_ERROR)
 
-    def processIncoming(self, data):
-        data_hash = RNS.Identity.full_hash(data)
-        deque_hit = False
-        if data_hash in self.mif_deque:
-            for te in self.mif_deque_times:
-                if te[0] == data_hash and time.time() < te[1]+AutoInterface.MULTI_IF_DEQUE_TTL:
-                    deque_hit = True
-                    break
+    def process_incoming(self, data, addr=None):
+        if self.online and addr in self.spawned_interfaces:
+            self.spawned_interfaces[addr].process_incoming(data, addr)
 
-        if not deque_hit:
-            self.mif_deque.append(data_hash)
-            self.mif_deque_times.append([data_hash, time.time()])
-            self.rxb += len(data)
-            self.owner.inbound(data, self)
-
-    def processOutgoing(self,data):
-            for peer in self.peers:
-                try:
-                    if self.outbound_udp_socket == None:
-                        self.outbound_udp_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
-
-                    peer_addr = str(peer)+"%"+str(self.interface_name_to_index(self.peers[peer][0]))
-                    addr_info = socket.getaddrinfo(peer_addr, self.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
-                    self.outbound_udp_socket.sendto(data, addr_info[0][4])
-
-                except Exception as e:
-                    RNS.log("Could not transmit on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
-
-            
-            self.txb += len(data)
-            
+    def process_outgoing(self,data):
+        pass
 
     # Until per-device sub-interfacing is implemented,
     # ingress limiting should be disabled on AutoInterface
     def should_ingress_limit(self):
         return False
 
+    def detach(self):
+        self.online = False
+
     def __str__(self):
         return "AutoInterface["+self.name+"]"
 
+class AutoInterfacePeer(Interface):
+
+    def __init__(self, owner, addr, ifname):
+        super().__init__()
+        self.owner = owner
+        self.parent_interface = owner
+        self.addr = addr
+        self.ifname = ifname
+        self.peer_addr = None
+        self.addr_info = None
+        self.HW_MTU = self.owner.HW_MTU
+        self.FIXED_MTU = self.owner.FIXED_MTU
+
+    def __str__(self):
+        return f"AutoInterfacePeer[{self.ifname}/{self.addr}]"
+
+    def process_incoming(self, data, addr=None):
+        if self.online and self.owner.online:
+            data_hash = RNS.Identity.full_hash(data)
+            deque_hit = False
+            if data_hash in self.owner.mif_deque:
+                for te in self.owner.mif_deque_times:
+                    if te[0] == data_hash and time.time() < te[1]+AutoInterface.MULTI_IF_DEQUE_TTL:
+                        deque_hit = True
+                        break
+
+            if not deque_hit:
+                self.owner.refresh_peer(self.addr)
+                self.owner.mif_deque.append(data_hash)
+                self.owner.mif_deque_times.append([data_hash, time.time()])
+                self.rxb += len(data)
+                self.owner.rxb += len(data)
+                self.owner.owner.inbound(data, self)
+
+    def process_outgoing(self, data):
+        if self.online:
+            with self.owner.write_lock:
+                try:
+                    if self.owner.outbound_udp_socket == None: self.owner.outbound_udp_socket = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
+                    if self.peer_addr == None: self.peer_addr = str(self.addr)+"%"+str(self.owner.interface_name_to_index(self.ifname))
+                    if self.addr_info == None: self.addr_info = socket.getaddrinfo(self.peer_addr, self.owner.data_port, socket.AF_INET6, socket.SOCK_DGRAM)
+                    self.owner.outbound_udp_socket.sendto(data, self.addr_info[0][4])
+                    self.txb += len(data)
+                    self.owner.txb += len(data)
+                except Exception as e:
+                    RNS.log("Could not transmit on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+
+    def detach(self):
+        self.online = False
+        self.detached = True
+        
+    def teardown(self):
+        if not self.detached:
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down.", RNS.LOG_ERROR)
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+        else:
+            RNS.log("The interface "+str(self)+" is being torn down.", RNS.LOG_VERBOSE)
+
+        self.online = False
+        self.OUT = False
+        self.IN = False
+
+        if self.addr in self.owner.spawned_interfaces:
+            try: self.owner.spawned_interfaces.pop(self.addr)
+            except Exception as e:
+                RNS.log(f"Could not remove {self} from parent interface on detach. The contained exception was: {e}", RNS.LOG_ERROR)
+
+        if self in RNS.Transport.interfaces:
+            RNS.Transport.interfaces.remove(self)
+
+    # Until per-device sub-interfacing is implemented,
+    # ingress limiting should be disabled on AutoInterface
+    def should_ingress_limit(self):
+        return False
+
 class AutoInterfaceHandler(socketserver.BaseRequestHandler):
     def __init__(self, callback, *args, **keys):
         self.callback = callback
@@ -490,4 +612,5 @@ class AutoInterfaceHandler(socketserver.BaseRequestHandler):
 
     def handle(self):
         data = self.request[0]
-        self.callback(data)
+        addr = self.client_address[0]
+        self.callback(data, addr)

RNS/Interfaces/BackboneInterface.py

@@ -0,0 +1,677 @@
+# MIT License
+#
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+from RNS.Interfaces.Interface import Interface
+import threading
+import socket
+import select
+import time
+import sys
+import os
+import RNS
+
+class HDLC():
+    FLAG              = 0x7E
+    ESC               = 0x7D
+    ESC_MASK          = 0x20
+
+    @staticmethod
+    def escape(data):
+        data = data.replace(bytes([HDLC.ESC]), bytes([HDLC.ESC, HDLC.ESC^HDLC.ESC_MASK]))
+        data = data.replace(bytes([HDLC.FLAG]), bytes([HDLC.ESC, HDLC.FLAG^HDLC.ESC_MASK]))
+        return data
+
+class BackboneInterface(Interface):
+    HW_MTU            = 1048576
+    BITRATE_GUESS     = 1_000_000_000
+    DEFAULT_IFAC_SIZE = 16
+    AUTOCONFIGURE_MTU = True
+
+    epoll = None
+    listener_filenos = {}
+    spawned_interface_filenos = {}
+    epoll = None
+    _job_active = False
+    _job_lock = threading.Lock()
+
+    @staticmethod
+    def get_address_for_if(name, bind_port, prefer_ipv6=False):
+        from RNS.Interfaces import netinfo
+        ifaddr = netinfo.ifaddresses(name)
+        if len(ifaddr) < 1:
+            raise SystemError(f"No addresses available on specified kernel interface \"{name}\" for BackboneInterface to bind to")
+
+        if (prefer_ipv6 or not netinfo.AF_INET in ifaddr) and netinfo.AF_INET6 in ifaddr:
+            bind_ip = ifaddr[netinfo.AF_INET6][0]["addr"]
+            if bind_ip.lower().startswith("fe80::"):
+                # We'll need to add the interface as scope for link-local addresses
+                return BackboneInterface.get_address_for_host(f"{bind_ip}%{name}", bind_port, prefer_ipv6)
+            else:
+                return BackboneInterface.get_address_for_host(bind_ip, bind_port, prefer_ipv6)
+        elif netinfo.AF_INET in ifaddr:
+            bind_ip = ifaddr[netinfo.AF_INET][0]["addr"]
+            return (bind_ip, bind_port)
+        else:
+            raise SystemError(f"No addresses available on specified kernel interface \"{name}\" for BackboneInterface to bind to")
+
+    @staticmethod
+    def get_address_for_host(name, bind_port, prefer_ipv6=False):
+        address_infos = socket.getaddrinfo(name, bind_port, proto=socket.IPPROTO_TCP)
+        address_info  = address_infos[0]
+        for entry in address_infos:
+            if prefer_ipv6 and entry[0] == socket.AF_INET6:
+                address_info = entry; break
+            elif not prefer_ipv6 and entry[0] == socket.AF_INET:
+                address_info = entry; break
+
+        if address_info[0] == socket.AF_INET6:
+            return (name, bind_port, address_info[4][2], address_info[4][3])
+        elif address_info[0] == socket.AF_INET:
+            return (name, bind_port)
+        else:
+            raise SystemError(f"No suitable kernel interface available for address \"{name}\" for BackboneInterface to bind to")
+
+
+    @property
+    def clients(self):
+        return len(self.spawned_interfaces)
+
+    def __init__(self, owner, configuration):
+        if not RNS.vendor.platformutils.is_linux() and not RNS.vendor.platformutils.is_android():
+            raise OSError("BackboneInterface is only supported on Linux-based operating systems")
+
+        super().__init__()
+
+        c            = Interface.get_config_obj(configuration)
+        name         = c["name"]
+        device       = c["device"] if "device" in c else None
+        port         = int(c["port"]) if "port" in c else None
+        bindip       = c["listen_ip"] if "listen_ip" in c else None
+        bindport     = int(c["listen_port"]) if "listen_port" in c else None
+        prefer_ipv6  = c.as_bool("prefer_ipv6") if "prefer_ipv6" in c else False
+
+        if port != None: bindport = port
+
+        self.HW_MTU = BackboneInterface.HW_MTU
+        self.online = False
+        self.IN  = True
+        self.OUT = False
+        self.name = name
+        self.detached = False
+        self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
+        self.spawned_interfaces = []
+
+        if bindport == None:
+            raise SystemError(f"No TCP port configured for interface \"{name}\"")
+        else:
+            self.bind_port = bindport
+
+        bind_address = None
+        if device != None:
+            bind_address = self.get_address_for_if(device, self.bind_port, prefer_ipv6)
+        else:
+            if bindip == None:
+                raise SystemError(f"No TCP bind IP configured for interface \"{name}\"")
+            bind_address = self.get_address_for_host(bindip, self.bind_port, prefer_ipv6)
+
+        if bind_address != None:
+            self.receives = True
+            self.bind_ip = bind_address[0]
+            self.owner = owner
+
+            if len(bind_address) == 2  : BackboneInterface.add_listener(self, bind_address, socket_type=socket.AF_INET)
+            elif len(bind_address) == 4: BackboneInterface.add_listener(self, bind_address, socket_type=socket.AF_INET6)
+
+            self.bitrate = self.BITRATE_GUESS
+            self.online = True
+
+        else:
+            raise SystemError("Insufficient parameters to create listener")
+
+    @staticmethod
+    def start():
+        if not BackboneInterface._job_active: threading.Thread(target=BackboneInterface.__job, daemon=True).start()
+
+    @staticmethod
+    def ensure_epoll():
+        if not BackboneInterface.epoll: BackboneInterface.epoll = select.epoll()
+
+    @staticmethod
+    def add_listener(interface, bind_address, socket_type=socket.AF_INET):
+        BackboneInterface.ensure_epoll()
+        if socket_type == socket.AF_INET:
+            server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            server_socket.bind(bind_address)
+        elif socket_type == socket.AF_INET6:
+            server_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
+            server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            server_socket.bind(bind_address)
+        elif socket_type == socket.AF_UNIX:
+            server_socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+            server_socket.bind(bind_address)
+        else: raise TypeError(f"Invalid socket type {socket_type} for {interface}")
+
+        server_socket.listen(1)
+        server_socket.setblocking(0)
+        BackboneInterface.listener_filenos[server_socket.fileno()] = (interface, server_socket)
+        BackboneInterface.epoll.register(server_socket.fileno(), select.EPOLLIN)
+        BackboneInterface.start()
+
+    @staticmethod
+    def add_client_socket(client_socket, interface):
+        BackboneInterface.ensure_epoll()
+        BackboneInterface.spawned_interface_filenos[client_socket.fileno()] = interface
+        BackboneInterface.register_in(client_socket.fileno())
+        BackboneInterface.start()
+
+    @staticmethod
+    def register_in(fileno):
+        if fileno < 0:
+            RNS.log(f"Attempt to register invalid file descriptor {fileno}", RNS.LOG_ERROR)
+            return
+
+        try: BackboneInterface.epoll.register(fileno, select.EPOLLIN)
+        except Exception as e:
+            RNS.log(f"An error occurred while registering EPOLL_IN for file descriptor {fileno}: {e}", RNS.LOG_ERROR)
+
+    @staticmethod
+    def deregister_fileno(fileno):
+        if fileno < 0:
+            RNS.log(f"Attempt to deregister invalid file descriptor {fileno}", RNS.LOG_ERROR)
+            return
+
+        try: BackboneInterface.epoll.unregister(fileno)
+        except Exception as e:
+            RNS.log(f"An error occurred while deregistering file descriptor {fileno}: {e}", RNS.LOG_DEBUG)
+
+    @staticmethod
+    def deregister_listeners():
+        for fileno in BackboneInterface.listener_filenos:
+            owner_interface, server_socket = BackboneInterface.listener_filenos[fileno]
+            fileno = server_socket.fileno()
+            BackboneInterface.deregister_fileno(fileno)
+            server_socket.close()
+
+        BackboneInterface.listener_filenos.clear()
+
+    @staticmethod
+    def tx_ready(interface):
+        if interface.socket:
+            fileno = interface.socket.fileno()
+            if fileno in BackboneInterface.spawned_interface_filenos:
+                try:
+                    BackboneInterface.epoll.modify(interface.socket.fileno(), select.EPOLLOUT)
+                except Exception as e:
+                    RNS.trace_exception(e)
+
+    @staticmethod
+    def __job():
+        with BackboneInterface._job_lock:
+            if BackboneInterface._job_active: return
+            else:
+                BackboneInterface._job_active = True
+                BackboneInterface.ensure_epoll()
+                try:
+                    while True:
+                        events = BackboneInterface.epoll.poll(1)
+                        for fileno, event in BackboneInterface.epoll.poll(1):
+                            if fileno in BackboneInterface.spawned_interface_filenos:
+                                spawned_interface = BackboneInterface.spawned_interface_filenos[fileno]
+                                client_socket = spawned_interface.socket
+                                if client_socket and fileno == client_socket.fileno() and (event & select.EPOLLIN):
+                                    try: received_bytes = client_socket.recv(spawned_interface.HW_MTU)
+                                    except Exception as e:
+                                        RNS.log(f"Error while reading from {spawned_interface}: {e}", RNS.LOG_DEBUG)
+                                        received_bytes = b""
+
+                                    if len(received_bytes): spawned_interface.receive(received_bytes)
+                                    else:
+                                        BackboneInterface.deregister_fileno(fileno); client_socket.close()
+                                        try:
+                                            if fileno in BackboneInterface.spawned_interface_filenos: BackboneInterface.spawned_interface_filenos.pop(fileno)
+                                        except Exception as e: RNS.log(f"Error while removing spawned interface file descriptor from BackboneInterface I/O handler: {e}", RNS.LOG_ERROR)
+
+                                        try:
+                                            if spawned_interface.parent_interface:
+                                                pif = spawned_interface.parent_interface
+                                                if pif.spawned_interfaces != None:
+                                                    while spawned_interface in pif.spawned_interfaces: pif.spawned_interfaces.remove(spawned_interface)
+                                        except Exception as e: RNS.log(f"Error while removing spawned interface from {pif}: {e}", RNS.LOG_ERROR)
+
+                                        spawned_interface.receive(received_bytes)
+                                
+                                elif client_socket and fileno == client_socket.fileno() and (event & select.EPOLLOUT):
+                                    try:
+                                        written = client_socket.send(spawned_interface.transmit_buffer)
+                                    except Exception as e:
+                                        written = 0
+                                        if not spawned_interface.detached: RNS.log(f"Error while writing to {spawned_interface}: {e}", RNS.LOG_DEBUG)
+                                        BackboneInterface.deregister_fileno(fileno)
+
+                                        try:
+                                            if fileno in BackboneInterface.spawned_interface_filenos: BackboneInterface.spawned_interface_filenos.pop(fileno)
+                                        except Exception as e: RNS.log(f"Error while removing spawned interface file descriptor from BackboneInterface I/O handler: {e}", RNS.LOG_ERROR)
+                                        
+                                        try:
+                                            if spawned_interface.parent_interface:
+                                                pif = spawned_interface.parent_interface
+                                                if pif.spawned_interfaces != None:
+                                                    while spawned_interface in pif.spawned_interfaces: pif.spawned_interfaces.remove(spawned_interface)
+                                        except Exception as e: RNS.log(f"Error while removing spawned interface from {pif}: {e}", RNS.LOG_ERROR)
+
+                                        try: client_socket.close()
+                                        except Exception as e: RNS.log(f"Error while closing socket for {spawned_interface}: {e}", RNS.LOG_ERROR)
+                                        spawned_interface.receive(b"")
+
+                                    spawned_interface.transmit_buffer = spawned_interface.transmit_buffer[written:]
+                                    if len(spawned_interface.transmit_buffer) == 0: BackboneInterface.epoll.modify(fileno, select.EPOLLIN)
+                                    spawned_interface.txb += written
+                                    if spawned_interface.parent_interface: spawned_interface.parent_interface.txb += written
+                                
+                                elif client_socket and fileno == client_socket.fileno() and event & (select.EPOLLHUP):
+                                    BackboneInterface.deregister_fileno(fileno)
+                                    try:
+                                        if fileno in BackboneInterface.spawned_interface_filenos: BackboneInterface.spawned_interface_filenos.pop(fileno)
+                                    except Exception as e: RNS.log(f"Error while removing spawned interface file descriptor from BackboneInterface I/O handler: {e}", RNS.LOG_ERROR)
+
+                                    try:
+                                        if spawned_interface.parent_interface:
+                                            pif = spawned_interface.parent_interface
+                                            if pif.spawned_interfaces != None:
+                                                while spawned_interface in pif.spawned_interfaces: pif.spawned_interfaces.remove(spawned_interface)
+                                    except Exception as e: RNS.log(f"Error while removing spawned interface from {pif}: {e}", RNS.LOG_ERROR)
+
+                                    try: client_socket.close()
+                                    except Exception as e: RNS.log(f"Error while closing socket for {spawned_interface}: {e}", RNS.LOG_ERROR)
+                                    spawned_interface.receive(b"")
+
+                            elif fileno in BackboneInterface.listener_filenos:
+                                owner_interface, server_socket = BackboneInterface.listener_filenos[fileno]
+                                if fileno == server_socket.fileno() and (event & select.EPOLLIN):
+                                    client_socket, address = server_socket.accept()
+                                    client_socket.setblocking(0)
+                                    if not owner_interface.incoming_connection(client_socket):
+                                        client_socket.close()
+                                
+                                elif fileno == server_socket.fileno() and (event & select.EPOLLHUP):
+                                    try: BackboneInterface.deregister_fileno(fileno)
+                                    except Exception as e: RNS.log(f"Error while deregistering listener file descriptor {fileno}: {e}", RNS.LOG_ERROR)
+
+                                    try: server_socket.close()
+                                    except Exception as e: RNS.log(f"Error while closing listener socket for {server_socket}: {e}", RNS.LOG_ERROR)
+
+                except Exception as e:
+                    RNS.log(f"BackboneInterface error: {e}", RNS.LOG_ERROR)
+                    RNS.trace_exception(e)
+
+                finally:
+                    BackboneInterface.deregister_listeners()
+    
+    def incoming_connection(self, socket):
+        RNS.log("Accepting incoming connection", RNS.LOG_VERBOSE)
+        spawned_configuration = {"name": "Client on "+self.name, "target_host": None, "target_port": None}
+        spawned_interface = BackboneClientInterface(self.owner, spawned_configuration, connected_socket=socket)
+        spawned_interface.OUT = self.OUT
+        spawned_interface.IN  = self.IN
+        spawned_interface.socket = socket
+        spawned_interface.target_ip = socket.getpeername()[0]
+        spawned_interface.target_port = str(socket.getpeername()[1])
+        spawned_interface.parent_interface = self
+        spawned_interface.bitrate = self.bitrate
+        spawned_interface.optimise_mtu()
+        
+        spawned_interface.ifac_size = self.ifac_size
+        spawned_interface.ifac_netname = self.ifac_netname
+        spawned_interface.ifac_netkey = self.ifac_netkey
+        if spawned_interface.ifac_netname != None or spawned_interface.ifac_netkey != None:
+            ifac_origin = b""
+            if spawned_interface.ifac_netname != None:
+                ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netname.encode("utf-8"))
+            if spawned_interface.ifac_netkey != None:
+                ifac_origin += RNS.Identity.full_hash(spawned_interface.ifac_netkey.encode("utf-8"))
+
+            ifac_origin_hash = RNS.Identity.full_hash(ifac_origin)
+            spawned_interface.ifac_key = RNS.Cryptography.hkdf(
+                length=64,
+                derive_from=ifac_origin_hash,
+                salt=RNS.Reticulum.IFAC_SALT,
+                context=None
+            )
+            spawned_interface.ifac_identity = RNS.Identity.from_bytes(spawned_interface.ifac_key)
+            spawned_interface.ifac_signature = spawned_interface.ifac_identity.sign(RNS.Identity.full_hash(spawned_interface.ifac_key))
+
+        spawned_interface.announce_rate_target = self.announce_rate_target
+        spawned_interface.announce_rate_grace = self.announce_rate_grace
+        spawned_interface.announce_rate_penalty = self.announce_rate_penalty
+        spawned_interface.mode = self.mode
+        spawned_interface.HW_MTU = self.HW_MTU
+        spawned_interface.online = True
+        RNS.log("Spawned new BackboneClient Interface: "+str(spawned_interface), RNS.LOG_VERBOSE)
+        RNS.Transport.interfaces.append(spawned_interface)
+        while spawned_interface in self.spawned_interfaces: self.spawned_interfaces.remove(spawned_interface)
+        self.spawned_interfaces.append(spawned_interface)
+        BackboneInterface.add_client_socket(socket, spawned_interface)
+
+        return True
+
+    def received_announce(self, from_spawned=False):
+        if from_spawned: self.ia_freq_deque.append(time.time())
+
+    def sent_announce(self, from_spawned=False):
+        if from_spawned: self.oa_freq_deque.append(time.time())
+
+    def process_outgoing(self, data):
+        pass
+
+    def detach(self):
+        self.detached = True
+        self.online = False
+        detached = []
+        for fileno in BackboneInterface.listener_filenos:
+            owner_interface, listener_socket = BackboneInterface.listener_filenos[fileno]
+            if owner_interface == self:
+                if hasattr(listener_socket, "shutdown"):
+                    if callable(listener_socket.shutdown):
+                        try: listener_socket.shutdown(socket.SHUT_RDWR)
+                        except Exception as e: RNS.log("Error while shutting down socket for "+str(self)+": "+str(e), RNS.LOG_ERROR)
+
+    def __str__(self):
+        if ":" in self.bind_ip:
+            ip_str = f"[{self.bind_ip}]"
+        else:
+            ip_str = f"{self.bind_ip}"
+
+        return "BackboneInterface["+self.name+"/"+ip_str+":"+str(self.bind_port)+"]"
+
+
+class BackboneClientInterface(Interface):
+    BITRATE_GUESS = 100_000_000
+    DEFAULT_IFAC_SIZE = 16
+    AUTOCONFIGURE_MTU = True
+
+    RECONNECT_WAIT = 5
+    RECONNECT_MAX_TRIES = None
+
+    # TCP socket options
+    TCP_USER_TIMEOUT = 24
+    TCP_PROBE_AFTER = 5
+    TCP_PROBE_INTERVAL = 2
+    TCP_PROBES = 12
+
+    INITIAL_CONNECT_TIMEOUT = 5
+    SYNCHRONOUS_START = True
+
+    def __init__(self, owner, configuration, connected_socket=None):
+        super().__init__()
+
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        target_ip = c["target_host"] if "target_host" in c and c["target_host"] != None else None
+        target_port = int(c["target_port"]) if "target_port" in c and c["target_host"] != None else None
+        i2p_tunneled = c.as_bool("i2p_tunneled") if "i2p_tunneled" in c else False
+        connect_timeout = c.as_int("connect_timeout") if "connect_timeout" in c else None
+        max_reconnect_tries = c.as_int("max_reconnect_tries") if "max_reconnect_tries" in c else None
+        prefer_ipv6  = c.as_bool("prefer_ipv6") if "prefer_ipv6" in c else False
+        
+        self.HW_MTU           = BackboneInterface.HW_MTU
+        self.IN               = True
+        self.OUT              = False
+        self.socket           = None
+        self.parent_interface = None
+        self.name             = name
+        self.initiator        = False
+        self.reconnecting     = False
+        self.never_connected  = True
+        self.owner            = owner
+        self.online           = False
+        self.detached         = False
+        self.prefer_ipv6      = prefer_ipv6
+        self.i2p_tunneled     = i2p_tunneled
+        self.mode             = RNS.Interfaces.Interface.Interface.MODE_FULL
+        self.bitrate          = BackboneClientInterface.BITRATE_GUESS
+        self.frame_buffer     = b""
+        self.transmit_buffer  = b""
+        
+        if max_reconnect_tries == None:
+            self.max_reconnect_tries = BackboneClientInterface.RECONNECT_MAX_TRIES
+        else:
+            self.max_reconnect_tries = max_reconnect_tries
+
+        if connected_socket != None:
+            self.receives    = True
+            self.target_ip   = None
+            self.target_port = None
+            self.socket      = connected_socket
+
+            self.set_timeouts_linux()
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+
+        elif target_ip != None and target_port != None:
+            self.receives    = True
+            self.target_ip   = target_ip
+            self.target_port = target_port
+            self.initiator   = True
+
+            if connect_timeout != None:
+                self.connect_timeout = connect_timeout
+            else:
+                self.connect_timeout = BackboneClientInterface.INITIAL_CONNECT_TIMEOUT
+            
+            if BackboneClientInterface.SYNCHRONOUS_START:
+                self.initial_connect()
+            else:
+                thread = threading.Thread(target=self.initial_connect)
+                thread.daemon = True
+                thread.start()
+            
+    def initial_connect(self):
+        if not self.connect(initial=True):
+            thread = threading.Thread(target=self.reconnect)
+            thread.daemon = True
+            thread.start()
+        else:
+            self.wants_tunnel = True
+
+    def set_timeouts_linux(self):
+        self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_USER_TIMEOUT, int(BackboneClientInterface.TCP_USER_TIMEOUT * 1000))
+        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
+        self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, int(BackboneClientInterface.TCP_PROBE_AFTER))
+        self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, int(BackboneClientInterface.TCP_PROBE_INTERVAL))
+        self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, int(BackboneClientInterface.TCP_PROBES))
+
+    def detach(self):
+        self.online = False
+        if self.socket != None:
+            if hasattr(self.socket, "close"):
+                if callable(self.socket.close):
+                    self.detached = True
+                    
+                    try:
+                        if self.socket != None: self.socket.shutdown(socket.SHUT_RDWR)
+                    except Exception as e: RNS.log("Error while shutting down socket for "+str(self)+": "+str(e), RNS.LOG_ERROR)
+
+                    try:
+                        if self.socket != None: self.socket.close()
+                    except Exception as e: RNS.log("Error while closing socket for "+str(self)+": "+str(e), RNS.LOG_ERROR)
+
+                    self.socket = None
+
+    def connect(self, initial=False):
+        try:
+            if initial:
+                RNS.log("Establishing TCP connection for "+str(self)+"...", RNS.LOG_DEBUG)
+
+            address_infos = socket.getaddrinfo(self.target_ip, self.target_port, proto=socket.IPPROTO_TCP)
+            address_info  = address_infos[0]
+            for entry in address_infos:
+                if self.prefer_ipv6 and entry[0] == socket.AF_INET6:
+                    address_info = entry; break
+                elif not self.prefer_ipv6 and entry[0] == socket.AF_INET:
+                    address_info = entry; break
+
+            address_family = address_info[0]
+            target_address = address_info[4]
+
+            self.socket = socket.socket(address_family, socket.SOCK_STREAM)
+            self.socket.settimeout(BackboneClientInterface.INITIAL_CONNECT_TIMEOUT)
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+            self.socket.connect(target_address)
+            self.socket.settimeout(None)
+
+            BackboneInterface.add_client_socket(self.socket, self)
+            self.online  = True
+
+            if initial:
+                RNS.log("TCP connection for "+str(self)+" established", RNS.LOG_DEBUG)
+        
+        except Exception as e:
+            if initial:
+                RNS.log("Initial connection for "+str(self)+" could not be established: "+str(e), RNS.LOG_ERROR)
+                RNS.log("Leaving unconnected and retrying connection in "+str(BackboneClientInterface.RECONNECT_WAIT)+" seconds.", RNS.LOG_ERROR)
+                return False
+            
+            else:
+                raise e
+
+        self.set_timeouts_linux()
+        
+        self.online  = True
+        self.never_connected = False
+
+        return True
+
+    def reconnect(self):
+        if self.initiator:
+            if not self.reconnecting:
+                self.reconnecting = True
+                attempts = 0
+                while not self.online:
+                    time.sleep(BackboneClientInterface.RECONNECT_WAIT)
+                    attempts += 1
+
+                    if self.max_reconnect_tries != None and attempts > self.max_reconnect_tries:
+                        RNS.log("Max reconnection attempts reached for "+str(self), RNS.LOG_ERROR)
+                        self.teardown()
+                        break
+
+                    try:
+                        self.connect()
+
+                    except Exception as e:
+                        RNS.log("Connection attempt for "+str(self)+" failed: "+str(e), RNS.LOG_DEBUG)
+
+                if not self.never_connected:
+                    RNS.log("Reconnected socket for "+str(self)+".", RNS.LOG_INFO)
+
+                self.reconnecting = False
+                RNS.Transport.synthesize_tunnel(self)
+
+        else:
+            RNS.log("Attempt to reconnect on a non-initiator TCP interface. This should not happen.", RNS.LOG_ERROR)
+            raise IOError("Attempt to reconnect on a non-initiator TCP interface")
+
+    def process_incoming(self, data):
+        if self.online and not self.detached:
+            self.rxb += len(data)
+            if hasattr(self, "parent_interface") and self.parent_interface != None:
+                self.parent_interface.rxb += len(data)
+                        
+            self.owner.inbound(data, self)
+
+    def process_outgoing(self, data):
+        if self.online and not self.detached:
+            try:
+                self.transmit_buffer += bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+                BackboneInterface.tx_ready(self)
+
+            except Exception as e:
+                RNS.log("Exception occurred while transmitting via "+str(self)+", tearing down interface", RNS.LOG_ERROR)
+                RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+                self.teardown()
+
+    def receive(self, data_in):
+        try:
+            if len(data_in) > 0:
+                self.frame_buffer += data_in
+                flags_remaining = True
+                while flags_remaining:
+                    frame_start = self.frame_buffer.find(HDLC.FLAG)
+                    if frame_start != -1:
+                        frame_end = self.frame_buffer.find(HDLC.FLAG, frame_start+1)
+                        if frame_end != -1:
+                            frame = self.frame_buffer[frame_start+1:frame_end]
+                            frame = frame.replace(bytes([HDLC.ESC, HDLC.FLAG ^ HDLC.ESC_MASK]), bytes([HDLC.FLAG]))
+                            frame = frame.replace(bytes([HDLC.ESC, HDLC.ESC  ^ HDLC.ESC_MASK]), bytes([HDLC.ESC]))
+                            if len(frame) > RNS.Reticulum.HEADER_MINSIZE:
+                                self.process_incoming(frame)
+                            self.frame_buffer = self.frame_buffer[frame_end:]
+                        else:
+                            flags_remaining = False
+                    else:
+                        flags_remaining = False
+
+            else:
+                self.online = False
+                if self.initiator and not self.detached:
+                    RNS.log("The socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
+                    self.reconnect()
+                else:
+                    RNS.log("The socket for remote client "+str(self)+" was closed.", RNS.LOG_VERBOSE)
+                    self.teardown()
+                
+        except Exception as e:
+            self.online = False
+            RNS.log("An interface error occurred for "+str(self)+", the contained exception was: "+str(e), RNS.LOG_WARNING)
+
+            if self.initiator:
+                RNS.log("Attempting to reconnect...", RNS.LOG_WARNING)
+                self.reconnect()
+            else:
+                self.teardown()
+
+    def teardown(self):
+        if self.initiator and not self.detached:
+            RNS.log("The interface "+str(self)+" experienced an unrecoverable error and is being torn down. Restart Reticulum to attempt to open this interface again.", RNS.LOG_ERROR)
+            if RNS.Reticulum.panic_on_interface_error:
+                RNS.panic()
+
+        else:
+            RNS.log("The interface "+str(self)+" is being torn down.", RNS.LOG_VERBOSE)
+
+        self.online = False
+        self.OUT = False
+        self.IN = False
+
+        if hasattr(self, "parent_interface") and self.parent_interface != None:
+            while self in self.parent_interface.spawned_interfaces:
+                self.parent_interface.spawned_interfaces.remove(self)
+
+        if self in RNS.Transport.interfaces:
+            if not self.initiator:
+                RNS.Transport.interfaces.remove(self)
+
+
+    def __str__(self):
+        if ":" in self.target_ip: ip_str = f"[{self.target_ip}]"
+        else: ip_str = f"{self.target_ip}"
+        return "BackboneInterface["+str(self.name)+"/"+ip_str+":"+str(self.target_port)+"]"

RNS/Interfaces/I2PInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 import socketserver
 import threading
 import platform
@@ -627,14 +627,14 @@ class I2PInterfacePeer(Interface):
             RNS.log("Attempt to reconnect on a non-initiator I2P interface. This should not happen.", RNS.LOG_ERROR)
             raise IOError("Attempt to reconnect on a non-initiator I2P interface")
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)
         if hasattr(self, "parent_interface") and self.parent_interface != None and self.parent_count:
             self.parent_interface.rxb += len(data)
                     
         self.owner.inbound(data, self)
 
-    def processOutgoing(self, data):
+    def process_outgoing(self, data):
         if self.online:
             while self.writing:
                 time.sleep(0.001)
@@ -732,7 +732,7 @@ class I2PInterfacePeer(Interface):
                             # Read loop for KISS framing
                             if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
                                 in_frame = False
-                                self.processIncoming(data_buffer)
+                                self.process_incoming(data_buffer)
                             elif (byte == KISS.FEND):
                                 in_frame = True
                                 command = KISS.CMD_UNKNOWN
@@ -759,7 +759,7 @@ class I2PInterfacePeer(Interface):
                             # Read loop for HDLC framing
                             if (in_frame and byte == HDLC.FLAG):
                                 in_frame = False
-                                self.processIncoming(data_buffer)
+                                self.process_incoming(data_buffer)
                             elif (byte == HDLC.FLAG):
                                 in_frame = True
                                 data_buffer = b""
@@ -815,8 +815,8 @@ class I2PInterfacePeer(Interface):
         self.IN = False
 
         if hasattr(self, "parent_interface") and self.parent_interface != None:
-            if self.parent_interface.clients > 0:
-                self.parent_interface.clients -= 1
+            while self in self.parent_interface.spawned_interfaces:
+                self.parent_interface.spawned_interfaces.remove(self)
 
         if self in RNS.Transport.interfaces:
             if not self.initiator:
@@ -829,14 +829,28 @@ class I2PInterfacePeer(Interface):
 
 class I2PInterface(Interface):
     BITRATE_GUESS      = 256*1000
+    DEFAULT_IFAC_SIZE  = 16
 
-    def __init__(self, owner, name, rns_storagepath, peers, connectable = False, ifac_size = 16, ifac_netname = None, ifac_netkey = None):
+    @property
+    def clients(self):
+        return len(self.spawned_interfaces)
+
+    def __init__(self, owner, configuration):
         super().__init__()
+
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        rns_storagepath = c["storagepath"]
+        peers = c.as_list("peers") if "peers" in c else None
+        connectable = c.as_bool("connectable") if "connectable" in c else False
+        ifac_size = c["ifac_size"] if "ifac_size" in c else None
+        ifac_netname = c["ifac_netname"] if "ifac_netname" in c else None
+        ifac_netkey = c["ifac_netkey"] if "ifac_netkey" in c else None
         
         self.HW_MTU = 1064
 
         self.online = False
-        self.clients = 0
+        self.spawned_interfaces = []
         self.owner = owner
         self.connectable = connectable
         self.i2p_tunneled = True
@@ -956,10 +970,12 @@ class I2PInterface(Interface):
         spawned_interface.HW_MTU = self.HW_MTU
         RNS.log("Spawned new I2PInterface Peer: "+str(spawned_interface), RNS.LOG_VERBOSE)
         RNS.Transport.interfaces.append(spawned_interface)
-        self.clients += 1
+        while spawned_interface in self.spawned_interfaces:
+            self.spawned_interfaces.remove(spawned_interface)
+        self.spawned_interfaces.append(spawned_interface)
         spawned_interface.read_loop()
 
-    def processOutgoing(self, data):
+    def process_outgoing(self, data):
         pass
 
     def received_announce(self, from_spawned=False):

RNS/Interfaces/Interface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2023 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -24,6 +24,7 @@ import RNS
 import time
 import threading
 from collections import deque
+from RNS.vendor.configobj import ConfigObj
 
 class Interface:
     IN  = False
@@ -63,13 +64,21 @@ class Interface:
     IC_BURST_PENALTY         = 5*60
     IC_HELD_RELEASE_INTERVAL = 30
 
-    def __init__(self):
-        self.rxb = 0
-        self.txb = 0
-        self.created = time.time()
-        self.online = False
-        self.bitrate = 1e6
+    AUTOCONFIGURE_MTU = False
+    FIXED_MTU         = False
 
+    def __init__(self):
+        self.rxb      = 0
+        self.txb      = 0
+        self.created  = time.time()
+        self.detached = False
+        self.online   = False
+        self.bitrate  = 62500
+        self.HW_MTU   = None
+
+        self.parent_interface = None
+        self.spawned_interfaces = None
+        self.tunnel_id = None
         self.ingress_control = True
         self.ic_max_held_announces = Interface.MAX_HELD_ANNOUNCES
         self.ic_burst_hold = Interface.IC_BURST_HOLD
@@ -116,6 +125,33 @@ class Interface:
         else:
             return False
 
+    def optimise_mtu(self):
+        if self.AUTOCONFIGURE_MTU:
+            if self.bitrate   > 500_000_000:
+                self.HW_MTU = 524288
+            elif self.bitrate > 16_000_000:
+                self.HW_MTU = 262144
+            elif self.bitrate > 8_000_000:
+                self.HW_MTU = 131072
+            elif self.bitrate > 4_000_000:
+                self.HW_MTU = 65536
+            elif self.bitrate > 2_000_000:
+                self.HW_MTU = 32768
+            elif self.bitrate > 1_000_000:
+                self.HW_MTU = 16384
+            elif self.bitrate > 500_000:
+                self.HW_MTU = 8192
+            elif self.bitrate > 250_000:
+                self.HW_MTU = 4096
+            elif self.bitrate > 125_000:
+                self.HW_MTU = 2048
+            elif self.bitrate > 62_500:
+                self.HW_MTU = 1024
+            else:
+                self.HW_MTU = None
+
+        RNS.log(f"{self} hardware MTU set to {self.HW_MTU}", RNS.LOG_DEBUG) # TODO: Remove debug
+
     def age(self):
         return time.time()-self.created
 
@@ -151,12 +187,12 @@ class Interface:
             RNS.log("An error occurred while processing held announces for "+str(self), RNS.LOG_ERROR)
             RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
 
-    def received_announce(self):
+    def received_announce(self, from_spawned=False):
         self.ia_freq_deque.append(time.time())
         if hasattr(self, "parent_interface") and self.parent_interface != None:
             self.parent_interface.received_announce(from_spawned=True)
 
-    def sent_announce(self):
+    def sent_announce(self, from_spawned=False):
         self.oa_freq_deque.append(time.time())
         if hasattr(self, "parent_interface") and self.parent_interface != None:
             self.parent_interface.sent_announce(from_spawned=True)
@@ -222,7 +258,7 @@ class Interface:
                     wait_time = (tx_time / self.announce_cap)
                     self.announce_allowed_at = now + wait_time
 
-                    self.processOutgoing(selected["raw"])
+                    self.process_outgoing(selected["raw"])
                     self.sent_announce()
 
                     if selected in self.announce_queue:
@@ -237,5 +273,19 @@ class Interface:
                 RNS.log("Error while processing announce queue on "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
                 RNS.log("The announce queue for this interface has been cleared.", RNS.LOG_ERROR)
 
+    def final_init(self):
+        pass
+
     def detach(self):
-        pass
+        pass
+
+    @staticmethod
+    def get_config_obj(config_in):
+        if type(config_in) == ConfigObj:
+            return config_in
+        else:
+            try:
+                return ConfigObj(config_in)
+            except Exception as e:
+                RNS.log(f"Could not parse supplied configuration data. The contained exception was: {e}", RNS.LOG_ERROR)
+                raise SystemError("Invalid configuration data supplied")

RNS/Interfaces/KISSInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 from time import sleep
 import sys
 import threading
@@ -52,6 +52,7 @@ class KISS():
 class KISSInterface(Interface):
     MAX_CHUNK = 32768
     BITRATE_GUESS = 1200
+    DEFAULT_IFAC_SIZE = 8
 
     owner    = None
     port     = None
@@ -61,8 +62,8 @@ class KISSInterface(Interface):
     stopbits = None
     serial   = None
 
-    def __init__(self, owner, name, port, speed, databits, parity, stopbits, preamble, txtail, persistence, slottime, flow_control, beacon_interval, beacon_data):
-        import importlib
+    def __init__(self, owner, configuration):
+        import importlib.util
         if importlib.util.find_spec('serial') != None:
             import serial
         else:
@@ -71,6 +72,24 @@ class KISSInterface(Interface):
             RNS.panic()
 
         super().__init__()
+
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        preamble = int(c["preamble"]) if "preamble" in c else None
+        txtail = int(c["txtail"]) if "txtail" in c else None
+        persistence = int(c["persistence"]) if "persistence" in c else None
+        slottime = int(c["slottime"]) if "slottime" in c else None
+        flow_control = c.as_bool("flow_control") if "flow_control" in c else False
+        port = c["port"] if "port" in c else None
+        speed = int(c["speed"]) if "speed" in c else 9600
+        databits = int(c["databits"]) if "databits" in c else 8
+        parity = c["parity"] if "parity" in c else "N"
+        stopbits = int(c["stopbits"]) if "stopbits" in c else 1
+        beacon_interval = int(c["id_interval"]) if "id_interval" in c else None
+        beacon_data = c["id_callsign"] if "id_callsign" in c else None
+
+        if port == None:
+            raise ValueError("No port specified for serial interface")
         
         self.HW_MTU = 564
         
@@ -217,12 +236,12 @@ class KISSInterface(Interface):
                 raise IOError("Could not enable KISS interface flow control")
 
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)  
         self.owner.inbound(data, self)
 
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         datalen = len(data)
         if self.online:
             if self.interface_ready:
@@ -256,7 +275,7 @@ class KISSInterface(Interface):
         if len(self.packet_queue) > 0:
             data = self.packet_queue.pop(0)
             self.interface_ready = True
-            self.processOutgoing(data)
+            self.process_outgoing(data)
         elif len(self.packet_queue) == 0:
             self.interface_ready = True
 
@@ -275,7 +294,7 @@ class KISSInterface(Interface):
 
                     if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
                         in_frame = False
-                        self.processIncoming(data_buffer)
+                        self.process_incoming(data_buffer)
                     elif (byte == KISS.FEND):
                         in_frame = True
                         command = KISS.CMD_UNKNOWN
@@ -319,7 +338,13 @@ class KISSInterface(Interface):
                             if time.time() > self.first_tx + self.beacon_i:
                                 RNS.log("Interface "+str(self)+" is transmitting beacon data: "+str(self.beacon_d.decode("utf-8")), RNS.LOG_DEBUG)
                                 self.first_tx = None
-                                self.processOutgoing(self.beacon_d)
+                                
+                                # Pad to minimum length
+                                frame = bytearray(self.beacon_d)
+                                while len(frame) < 15:
+                                    frame.append(0x00)
+                                    
+                                self.process_outgoing(bytes(frame))
 
         except Exception as e:
             self.online = False

RNS/Interfaces/LocalInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2023 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,8 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
+from RNS.Interfaces.BackboneInterface import BackboneInterface
 import socketserver
 import threading
 import socket
@@ -52,16 +53,17 @@ class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
 
 class LocalClientInterface(Interface):
     RECONNECT_WAIT = 8
+    AUTOCONFIGURE_MTU = True
 
-    def __init__(self, owner, name, target_port = None, connected_socket=None):
+    def __init__(self, owner, name, target_port = None, connected_socket=None, socket_path=None):
         super().__init__()
 
-        # TODO: Remove at some point
-        # self.rxptime = 0
+        self.epoll_backend    = False
+        self.HW_MTU           = 262144
+        self.online           = False
         
-        self.HW_MTU = 1064
-
-        self.online  = False
+        if socket_path != None and RNS.vendor.platformutils.use_af_unix(): self.socket_path = f"\0rns/{socket_path}"
+        else: self.socket_path = None
         
         self.IN               = True
         self.OUT              = False
@@ -72,16 +74,29 @@ class LocalClientInterface(Interface):
         self.detached         = False
         self.name             = name
         self.mode             = RNS.Interfaces.Interface.Interface.MODE_FULL
+        self.frame_buffer     = b""
+        self.transmit_buffer  = b""
+
+        if RNS.vendor.platformutils.use_epoll():
+            self.epoll_backend = True
 
         if connected_socket != None:
             self.receives    = True
             self.target_ip   = None
             self.target_port = None
             self.socket      = connected_socket
-            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+
+            if self.socket.family == socket.AF_INET:
+                self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
 
             self.is_connected_to_shared_instance = False
 
+        elif self.socket_path != None:
+            self.receives    = True
+            self.target_ip   = None
+            self.target_port = None
+            self.connect()
+
         elif target_port != None:
             self.receives    = True
             self.target_ip   = "127.0.0.1"
@@ -89,7 +104,7 @@ class LocalClientInterface(Interface):
             self.connect()
 
         self.owner   = owner
-        self.bitrate = 1000*1000*1000
+        self.bitrate = 1_000_000_000
         self.online  = True
         self.writing = False
 
@@ -100,22 +115,30 @@ class LocalClientInterface(Interface):
         self.announce_rate_penalty = None
 
         if connected_socket == None:
-            thread = threading.Thread(target=self.read_loop)
-            thread.daemon = True
-            thread.start()
+            if not self.epoll_backend:
+                thread = threading.Thread(target=self.read_loop)
+                thread.daemon = True
+                thread.start()
 
     def should_ingress_limit(self):
         return False
 
     def connect(self):
-        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
-        self.socket.connect((self.target_ip, self.target_port))
+        if self.socket_path != None:
+            self.socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+            self.socket.connect(self.socket_path)
+        
+        else:
+            self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+            self.socket.connect((self.target_ip, self.target_port))
 
         self.online = True
         self.is_connected_to_shared_instance = True
         self.never_connected = False
 
+        if self.epoll_backend: BackboneInterface.add_client_socket(self.socket, self)
+
         return True
 
 
@@ -139,9 +162,11 @@ class LocalClientInterface(Interface):
                     RNS.log("Reconnected socket for "+str(self)+".", RNS.LOG_INFO)
 
                 self.reconnecting = False
-                thread = threading.Thread(target=self.read_loop)
-                thread.daemon = True
-                thread.start()
+                if not self.epoll_backend:
+                    thread = threading.Thread(target=self.read_loop)
+                    thread.daemon = True
+                    thread.start()
+
                 def job():
                     time.sleep(LocalClientInterface.RECONNECT_WAIT+2)
                     RNS.Transport.shared_connection_reappeared()
@@ -152,77 +177,92 @@ class LocalClientInterface(Interface):
             raise IOError("Attempt to reconnect on a non-initiator local interface")
 
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)
-        if hasattr(self, "parent_interface") and self.parent_interface != None:
-            self.parent_interface.rxb += len(data)
+        if self.parent_interface != None: self.parent_interface.rxb += len(data)
         
-        # TODO: Remove at some point
-        # processing_start = time.time()
-        
-        self.owner.inbound(data, self)
+        try:
+            self.owner.inbound(data, self)
+        except Exception as e:
+            RNS.log(f"An error in the processing of an incoming frame for {self}: {e}", RNS.LOG_ERROR)
+            RNS.trace_exception(e)
 
-        # TODO: Remove at some point
-        # duration = time.time() - processing_start
-        # self.rxptime += duration
-
-    def processOutgoing(self, data):
+    def process_outgoing(self, data):
         if self.online:
             try:
-                self.writing = True
+                if self.epoll_backend:
+                    self.transmit_buffer += bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+                    BackboneInterface.tx_ready(self)
 
-                if self._force_bitrate:
-                    if not hasattr(self, "send_lock"):
-                        self.send_lock = Lock()
+                else:
+                    self.writing = True
 
-                    with self.send_lock:
-                        s = len(data) / self.bitrate * 8
-                        RNS.log(f"Simulating latency of {RNS.prettytime(s)} for {len(data)} bytes")
-                        time.sleep(s)
+                    if self._force_bitrate:
+                        if not hasattr(self, "send_lock"):
+                            self.send_lock = Lock()
 
-                data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
-                self.socket.sendall(data)
-                self.writing = False
-                self.txb += len(data)
-                if hasattr(self, "parent_interface") and self.parent_interface != None:
-                    self.parent_interface.txb += len(data)
+                        with self.send_lock:
+                            # RNS.log(f"Simulating latency of {RNS.prettytime(s)} for {len(data)} bytes", RNS.LOG_EXTREME)
+                            s = len(data) / self.bitrate * 8
+                            time.sleep(s)
+
+                    data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
+                    self.socket.sendall(data)
+                    self.writing = False
+                    self.txb += len(data)
+                    if hasattr(self, "parent_interface") and self.parent_interface != None:
+                        self.parent_interface.txb += len(data)
 
             except Exception as e:
                 RNS.log("Exception occurred while transmitting via "+str(self)+", tearing down interface", RNS.LOG_ERROR)
                 RNS.log("The contained exception was: "+str(e), RNS.LOG_ERROR)
+                RNS.trace_exception(e)
                 self.teardown()
 
+    def handle_hdlc(self, data_in):
+        self.frame_buffer += data_in
+        flags_remaining = True
+        while flags_remaining:
+            frame_start = self.frame_buffer.find(HDLC.FLAG)
+            if frame_start != -1:
+                frame_end = self.frame_buffer.find(HDLC.FLAG, frame_start+1)
+                if frame_end != -1:
+                    frame = self.frame_buffer[frame_start+1:frame_end]
+                    frame = frame.replace(bytes([HDLC.ESC, HDLC.FLAG ^ HDLC.ESC_MASK]), bytes([HDLC.FLAG]))
+                    frame = frame.replace(bytes([HDLC.ESC, HDLC.ESC  ^ HDLC.ESC_MASK]), bytes([HDLC.ESC]))
+                    if len(frame) > RNS.Reticulum.HEADER_MINSIZE:
+                        self.process_incoming(frame)
+                    self.frame_buffer = self.frame_buffer[frame_end:]
+                else:
+                    flags_remaining = False
+            else:
+                flags_remaining = False
+
+    def receive(self, data_in):
+        try:
+            if len(data_in) > 0: self.handle_hdlc(data_in)
+            else:
+                self.online = False
+                if self.is_connected_to_shared_instance and not self.detached:
+                    RNS.log("Socket for "+str(self)+" was closed, attempting to reconnect...", RNS.LOG_WARNING)
+                    RNS.Transport.shared_connection_disappeared()
+                    self.reconnect()
+                else:
+                    self.teardown(nowarning=True)
+                
+        except Exception as e:
+            self.online = False
+            RNS.log("An interface error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
+            RNS.log("Tearing down "+str(self), RNS.LOG_ERROR)
+            self.teardown()
 
     def read_loop(self):
         try:
-            in_frame = False
-            escape = False
-            data_buffer = b""
-
+            self.frame_buffer = b""
+            data_in = b""
             while True:
                 data_in = self.socket.recv(4096)
-                if len(data_in) > 0:
-                    pointer = 0
-                    while pointer < len(data_in):
-                        byte = data_in[pointer]
-                        pointer += 1
-                        if (in_frame and byte == HDLC.FLAG):
-                            in_frame = False
-                            self.processIncoming(data_buffer)
-                        elif (byte == HDLC.FLAG):
-                            in_frame = True
-                            data_buffer = b""
-                        elif (in_frame and len(data_buffer) < self.HW_MTU):
-                            if (byte == HDLC.ESC):
-                                escape = True
-                            else:
-                                if (escape):
-                                    if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
-                                        byte = HDLC.FLAG
-                                    if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
-                                        byte = HDLC.ESC
-                                    escape = False
-                                data_buffer = data_buffer+bytes([byte])
+                if len(data_in) > 0: self.handle_hdlc(data_in)
                 else:
                     self.online = False
                     if self.is_connected_to_shared_instance and not self.detached:
@@ -234,7 +274,6 @@ class LocalClientInterface(Interface):
 
                     break
 
-                
         except Exception as e:
             self.online = False
             RNS.log("An interface error occurred, the contained exception was: "+str(e), RNS.LOG_ERROR)
@@ -249,12 +288,14 @@ class LocalClientInterface(Interface):
                     self.detached = True
                     
                     try:
-                        self.socket.shutdown(socket.SHUT_RDWR)
+                        if self.socket != None:
+                            self.socket.shutdown(socket.SHUT_RDWR)
                     except Exception as e:
                         RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
 
                     try:
-                        self.socket.close()
+                        if self.socket != None:
+                            self.socket.close()
                     except Exception as e:
                         RNS.log("Error while closing socket for "+str(self)+": "+str(e))
 
@@ -288,69 +329,115 @@ class LocalClientInterface(Interface):
 
 
     def __str__(self):
-        return "LocalInterface["+str(self.target_port)+"]"
+        if self.socket_path: return "Shared Instance["+str(self.socket_path.replace("\0", ""))+"]"
+        else: return "Shared Instance["+str(self.target_port)+"]"
 
 
 class LocalServerInterface(Interface):
+    AUTOCONFIGURE_MTU = True
 
-    def __init__(self, owner, bindport=None):
+    def __init__(self, owner, bindport=None, socket_path=None):
         super().__init__()
+        self.epoll_backend = False
         self.online = False
         self.clients = 0
         
+        if socket_path != None and RNS.vendor.platformutils.use_af_unix(): self.socket_path = f"\0rns/{socket_path}"
+        else: self.socket_path = None
+        
         self.IN  = True
         self.OUT = False
         self.name = "Reticulum"
         self.mode = RNS.Interfaces.Interface.Interface.MODE_FULL
 
-        if (bindport != None):
+        if RNS.vendor.platformutils.use_epoll():
+            self.epoll_backend = True
+
+        if socket_path != None and self.epoll_backend:
+            self.receives = True
+            self.bind_ip = None
+            self.bind_port = None
+
+            self.owner = owner
+            self.is_local_shared_instance = True
+            BackboneInterface.add_listener(self, self.socket_path, socket_type=socket.AF_UNIX)
+
+        elif bindport != None:
             self.receives = True
             self.bind_ip = "127.0.0.1"
             self.bind_port = bindport
 
-            def handlerFactory(callback):
-                def createHandler(*args, **keys):
-                    return LocalInterfaceHandler(callback, *args, **keys)
-                return createHandler
-
             self.owner = owner
             self.is_local_shared_instance = True
 
             address = (self.bind_ip, self.bind_port)
+            if self.epoll_backend: BackboneInterface.add_listener(self, address)
+            else:
+                def handlerFactory(callback):
+                    def createHandler(*args, **keys):
+                        return LocalInterfaceHandler(callback, *args, **keys)
+                    return createHandler
 
-            self.server = ThreadingTCPServer(address, handlerFactory(self.incoming_connection))
-
-            thread = threading.Thread(target=self.server.serve_forever)
-            thread.daemon = True
-            thread.start()
-
-            self.announce_rate_target  = None
-            self.announce_rate_grace   = None
-            self.announce_rate_penalty = None
-
-            self.bitrate = 1000*1000*1000
-            self.online = True
+                self.server = ThreadingTCPServer(address, handlerFactory(self.incoming_connection))
+                self.server.daemon_threads = True
+                thread = threading.Thread(target=self.server.serve_forever)
+                thread.daemon = True
+                thread.start()
 
+        self.announce_rate_target  = None
+        self.announce_rate_grace   = None
+        self.announce_rate_penalty = None
 
+        self.bitrate = 1000*1000*1000
+        self.online = True
 
     def incoming_connection(self, handler):
-        interface_name = str(str(handler.client_address[1]))
-        spawned_interface = LocalClientInterface(self.owner, name=interface_name, connected_socket=handler.request)
-        spawned_interface.OUT = self.OUT
-        spawned_interface.IN  = self.IN
-        spawned_interface.target_ip = handler.client_address[0]
-        spawned_interface.target_port = str(handler.client_address[1])
-        spawned_interface.parent_interface = self
-        spawned_interface.bitrate = self.bitrate
-        if hasattr(self, "_force_bitrate"):
-            spawned_interface._force_bitrate = self._force_bitrate
-        # RNS.log("Accepting new connection to shared instance: "+str(spawned_interface), RNS.LOG_EXTREME)
-        RNS.Transport.interfaces.append(spawned_interface)
-        RNS.Transport.local_client_interfaces.append(spawned_interface)
-        self.clients += 1
-        spawned_interface.read_loop()
+        if self.epoll_backend:
+            client_socket = handler
+            if client_socket.family == socket.AF_INET:
+                interface_name = str(str(client_socket.getpeername()[1]))
+            elif client_socket.family == socket.AF_UNIX:
+                interface_name = f"{self.clients}@{self.socket_path}"
 
-    def processOutgoing(self, data):
+            spawned_interface = LocalClientInterface(self.owner, name=interface_name, connected_socket=client_socket)
+            spawned_interface.OUT = self.OUT
+            spawned_interface.IN  = self.IN
+            spawned_interface.socket = client_socket
+            spawned_interface.parent_interface = self
+            spawned_interface.bitrate = self.bitrate
+
+            if client_socket.family == socket.AF_INET:
+                spawned_interface.target_ip = client_socket.getpeername()[0]
+                spawned_interface.target_port = str(client_socket.getpeername()[1])
+
+            elif client_socket.family == socket.AF_UNIX:
+                spawned_interface.target_ip = None
+                spawned_interface.target_port = interface_name
+                spawned_interface.socket_path = self.socket_path
+
+            if hasattr(self, "_force_bitrate"): spawned_interface._force_bitrate = self._force_bitrate
+            RNS.Transport.interfaces.append(spawned_interface)
+            RNS.Transport.local_client_interfaces.append(spawned_interface)
+            BackboneInterface.add_client_socket(client_socket, spawned_interface)
+            self.clients += 1
+            return True
+
+        else:
+            interface_name = str(str(handler.client_address[1]))
+            spawned_interface = LocalClientInterface(self.owner, name=interface_name, connected_socket=handler.request)
+            spawned_interface.OUT = self.OUT
+            spawned_interface.IN  = self.IN
+            spawned_interface.target_ip = handler.client_address[0]
+            spawned_interface.target_port = str(handler.client_address[1])
+            spawned_interface.parent_interface = self
+            spawned_interface.bitrate = self.bitrate
+            if hasattr(self, "_force_bitrate"): spawned_interface._force_bitrate = self._force_bitrate
+            RNS.Transport.interfaces.append(spawned_interface)
+            RNS.Transport.local_client_interfaces.append(spawned_interface)
+            self.clients += 1
+            spawned_interface.read_loop()
+
+    def process_outgoing(self, data):
         pass
 
     def received_announce(self, from_spawned=False):
@@ -360,7 +447,8 @@ class LocalServerInterface(Interface):
         if from_spawned: self.oa_freq_deque.append(time.time())
 
     def __str__(self):
-        return "Shared Instance["+str(self.bind_port)+"]"
+        if self.socket_path: return "Shared Instance["+str(self.socket_path.replace("\0", ""))+"]"
+        else: return "Shared Instance["+str(self.bind_port)+"]"
 
 class LocalInterfaceHandler(socketserver.BaseRequestHandler):
     def __init__(self, callback, *args, **keys):

RNS/Interfaces/PipeInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 from time import sleep
 import sys
 import threading
@@ -46,16 +46,25 @@ class HDLC():
 class PipeInterface(Interface):
     MAX_CHUNK = 32768
     BITRATE_GUESS = 1*1000*1000
+    DEFAULT_IFAC_SIZE = 8
 
     owner    = None
     command  = None
     
-    def __init__(self, owner, name, command, respawn_delay):
+    def __init__(self, owner, configuration):
+        super().__init__()
+
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        command = c["command"] if "command" in c else None
+        respawn_delay = c.as_float("respawn_delay") if "respawn_delay" in c else None
+
+        if command == None:
+            raise ValueError("No command specified for PipeInterface")
+
         if respawn_delay == None:
             respawn_delay = 5
 
-        super().__init__()
-
         self.HW_MTU = 1064
         
         self.owner    = owner
@@ -101,12 +110,12 @@ class PipeInterface(Interface):
         RNS.log("Subprocess pipe for "+str(self)+" is now connected", RNS.LOG_VERBOSE)
 
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)            
         self.owner.inbound(data, self)
 
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         if self.online:
             data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
             written = self.process.stdin.write(data)
@@ -134,7 +143,7 @@ class PipeInterface(Interface):
 
                     if (in_frame and byte == HDLC.FLAG):
                         in_frame = False
-                        self.processIncoming(data_buffer)
+                        self.process_incoming(data_buffer)
                     elif (byte == HDLC.FLAG):
                         in_frame = True
                         data_buffer = b""

RNS/Interfaces/RNodeInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2023 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 from time import sleep
 import sys
 import threading
@@ -54,10 +54,13 @@ class KISS():
     CMD_STAT_SNR    = 0x24
     CMD_STAT_CHTM   = 0x25
     CMD_STAT_PHYPRM = 0x26
+    CMD_STAT_BAT    = 0x27
+    CMD_STAT_CSMA   = 0x28
     CMD_BLINK       = 0x30
     CMD_RANDOM      = 0x40
     CMD_FB_EXT      = 0x41
     CMD_FB_READ     = 0x42
+    CMD_DISP_READ   = 0x66
     CMD_FB_WRITE    = 0x43
     CMD_BT_CTRL     = 0x46
     CMD_PLATFORM    = 0x48
@@ -77,9 +80,13 @@ class KISS():
     ERROR_INITRADIO     = 0x01
     ERROR_TXFAILED      = 0x02
     ERROR_EEPROM_LOCKED = 0x03
+    ERROR_QUEUE_FULL    = 0x04
+    ERROR_MEMORY_LOW    = 0x05
+    ERROR_MODEM_TIMEOUT = 0x06
 
     PLATFORM_AVR   = 0x90
     PLATFORM_ESP32 = 0x80
+    PLATFORM_NRF52 = 0x70
 
     @staticmethod
     def escape(data):
@@ -90,6 +97,7 @@ class KISS():
 
 class RNodeInterface(Interface):
     MAX_CHUNK = 32768
+    DEFAULT_IFAC_SIZE = 8
 
     FREQ_MIN = 137000000
     FREQ_MAX = 3000000000
@@ -107,11 +115,18 @@ class RNodeInterface(Interface):
     Q_SNR_MAX      = 6
     Q_SNR_STEP     = 2
 
-    def __init__(self, owner, name, port, frequency = None, bandwidth = None, txpower = None, sf = None, cr = None, flow_control = False, id_interval = None, id_callsign = None, st_alock = None, lt_alock = None):
+    BATTERY_STATE_UNKNOWN     = 0x00
+    BATTERY_STATE_DISCHARGING = 0x01
+    BATTERY_STATE_CHARGING    = 0x02
+    BATTERY_STATE_CHARGED     = 0x03
+
+    DISPLAY_READ_INTERVAL     = 1.0
+
+    def __init__(self, owner, configuration):
         if RNS.vendor.platformutils.is_android():
             raise SystemError("Invalid interface type. The Android-specific RNode interface must be used on Android")
 
-        import importlib
+        import importlib.util
         if importlib.util.find_spec('serial') != None:
             import serial
         else:
@@ -121,6 +136,41 @@ class RNodeInterface(Interface):
 
         super().__init__()
 
+        c = Interface.get_config_obj(configuration)
+        name         = c["name"]
+        frequency    = int(c["frequency"]) if "frequency" in c else 0
+        bandwidth    = int(c["bandwidth"]) if "bandwidth" in c else 0
+        txpower      = int(c["txpower"]) if "txpower" in c else 0
+        sf           = int(c["spreadingfactor"]) if "spreadingfactor" in c else 0
+        cr           = int(c["codingrate"]) if "codingrate" in c else 0
+        flow_control = c.as_bool("flow_control") if "flow_control" in c else False
+        id_interval  = int(c["id_interval"]) if "id_interval" in c else None
+        id_callsign  = c["id_callsign"] if "id_callsign" in c else None
+        st_alock     = float(c["airtime_limit_short"]) if "airtime_limit_short" in c else None
+        lt_alock     = float(c["airtime_limit_long"]) if "airtime_limit_long" in c else None
+
+        force_ble = False
+        ble_name = None
+        ble_addr = None
+
+        port = c["port"] if "port" in c else None
+
+        if port == None:
+            raise ValueError("No port specified for RNode interface")
+
+        if port != None:
+            ble_uri_scheme = "ble://"
+            if port.lower().startswith(ble_uri_scheme):
+                force_ble = True
+                ble_string = port[len(ble_uri_scheme):]
+                port = None
+                if len(ble_string) == 0:
+                    pass
+                elif len(ble_string.split(":")) == 6 and len(ble_string) == 17:
+                    ble_addr = ble_string
+                else:
+                    ble_name = ble_string
+
         self.HW_MTU = 508
         
         self.pyserial    = serial
@@ -135,6 +185,16 @@ class RNodeInterface(Interface):
         self.online      = False
         self.detached    = False
         self.reconnecting= False
+        self.hw_errors   = []
+
+        self.use_ble     = False
+        self.ble_name    = ble_name
+        self.ble_addr    = ble_addr
+        self.ble         = None
+        self.ble_rx_lock = threading.Lock()
+        self.ble_tx_lock = threading.Lock()
+        self.ble_rx_queue= b""
+        self.ble_tx_queue= b""
 
         self.frequency   = frequency
         self.bandwidth   = bandwidth
@@ -175,16 +235,38 @@ class RNodeInterface(Interface):
         self.r_airtime_long       = 0.0
         self.r_channel_load_short = 0.0
         self.r_channel_load_long  = 0.0
-        self.r_symbol_time_ms = None
-        self.r_symbol_rate = None
-        self.r_preamble_symbols = None
-        self.r_premable_time_ms = None
+        self.r_symbol_time_ms     = None
+        self.r_symbol_rate        = None
+        self.r_preamble_symbols   = None
+        self.r_premable_time_ms   = None
+        self.r_csma_slot_time_ms  = None
+        self.r_csma_difs_ms       = None
+        self.r_csma_cw_band       = None
+        self.r_csma_cw_min        = None
+        self.r_csma_cw_max        = None
+        self.r_current_rssi       = None
+        self.r_noise_floor        = None
+
+        self.r_battery_state = RNodeInterface.BATTERY_STATE_UNKNOWN
+        self.r_battery_percent = 0
+        self.r_framebuffer = b""
+        self.r_framebuffer_readtime = 0
+        self.r_framebuffer_latency = 0
+        self.r_disp = b""
+        self.r_disp_readtime = 0
+        self.r_disp_latency = 0
+
+        self.should_read_display   = False
+        self.read_display_interval = RNodeInterface.DISPLAY_READ_INTERVAL
 
         self.packet_queue    = []
         self.flow_control    = flow_control
         self.interface_ready = False
         self.announce_rate_target = None
 
+        if force_ble or self.ble_addr != None or self.ble_name != None:
+            self.use_ble = True
+
         self.validcfg  = True
         if (self.frequency < RNodeInterface.FREQ_MIN or self.frequency > RNodeInterface.FREQ_MAX):
             RNS.log("Invalid frequency configured for "+str(self), RNS.LOG_ERROR)
@@ -248,23 +330,38 @@ class RNodeInterface(Interface):
 
 
     def open_port(self):
-        RNS.log("Opening serial port "+self.port+"...")
-        self.serial = self.pyserial.Serial(
-            port = self.port,
-            baudrate = self.speed,
-            bytesize = self.databits,
-            parity = self.pyserial.PARITY_NONE,
-            stopbits = self.stopbits,
-            xonxoff = False,
-            rtscts = False,
-            timeout = 0,
-            inter_byte_timeout = None,
-            write_timeout = None,
-            dsrdtr = False,
-        )
+        if not self.use_ble:
+            RNS.log("Opening serial port "+self.port+"...")
+            self.serial = self.pyserial.Serial(
+                port = self.port,
+                baudrate = self.speed,
+                bytesize = self.databits,
+                parity = self.pyserial.PARITY_NONE,
+                stopbits = self.stopbits,
+                xonxoff = False,
+                rtscts = False,
+                timeout = 0,
+                inter_byte_timeout = None,
+                write_timeout = None,
+                dsrdtr = False,
+            )
+        
+        else:
+            RNS.log(f"Opening BLE connection for {self}...")
+            if self.ble != None and self.ble.running == False:
+                self.ble.close()
+                self.ble.cleanup()
+                self.ble = None
 
+            if self.ble == None:
+                self.ble = BLEConnection(owner=self, target_name=self.ble_name, target_bt_addr=self.ble_addr)
+                self.serial = self.ble
 
-    def configure_device(self):
+            open_time = time.time()
+            while not self.ble.connected and time.time() < open_time + self.ble.CONNECT_TIMEOUT:
+                time.sleep(1)
+
+    def reset_radio_state(self):
         self.r_frequency = None
         self.r_bandwidth = None
         self.r_txpower   = None
@@ -272,6 +369,10 @@ class RNodeInterface(Interface):
         self.r_cr        = None
         self.r_state     = None
         self.r_lock      = None
+        self.detected    = False
+
+    def configure_device(self):
+        self.reset_radio_state()
         sleep(2.0)
 
         thread = threading.Thread(target=self.readLoop)
@@ -279,13 +380,23 @@ class RNodeInterface(Interface):
         thread.start()
 
         self.detect()
-        sleep(0.2)
+        if not self.use_ble:
+            sleep(0.2)
+        else:
+            ble_detect_timeout = 5
+            detect_time = time.time()
+            while not self.detected and time.time() < detect_time + ble_detect_timeout:
+                time.sleep(0.1)
+            if self.detected:
+                detect_time = RNS.prettytime(time.time()-detect_time)
+            else:
+                RNS.log(f"RNode detect timed out over {self.port}", RNS.LOG_ERROR)
         
         if not self.detected:
             RNS.log("Could not detect device for "+str(self), RNS.LOG_ERROR)
             self.serial.close()
         else:
-            if self.platform == KISS.PLATFORM_ESP32:
+            if self.platform == KISS.PLATFORM_ESP32 or self.platform == KISS.PLATFORM_NRF52:
                 self.display = True
 
         RNS.log("Serial port "+self.port+" is now open")
@@ -313,6 +424,9 @@ class RNodeInterface(Interface):
         self.setLTALock()
         self.setRadioState(KISS.RADIO_STATE_ON)
 
+        if self.use_ble:
+            time.sleep(2)
+
     def detect(self):
         kiss_command = bytes([KISS.FEND, KISS.CMD_DETECT, KISS.DETECT_REQ, KISS.FEND, KISS.CMD_FW_VERSION, 0x00, KISS.FEND, KISS.CMD_PLATFORM, 0x00, KISS.FEND, KISS.CMD_MCU, 0x00, KISS.FEND])
         written = self.serial.write(kiss_command)
@@ -361,7 +475,34 @@ class RNodeInterface(Interface):
             
             written = self.serial.write(kiss_command)
             if written != len(kiss_command):
-                raise IOError("An IO error occurred while writing framebuffer data device")
+                raise IOError("An IO error occurred while writing framebuffer data to device")
+
+    def read_framebuffer(self):
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_FB_READ])+bytes([0x01])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        self.r_framebuffer_readtime = time.time()
+        if written != len(kiss_command):
+            raise IOError("An IO error occurred while sending framebuffer read command")
+
+    def read_display(self):
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_DISP_READ])+bytes([0x01])+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        self.r_disp_readtime = time.time()
+        if written != len(kiss_command):
+            raise IOError("An IO error occurred while sending display read command")
+
+    def _read_display_job(self):
+        while self.should_read_display:
+            self.read_display()
+            time.sleep(self.read_display_interval)
+
+    def start_display_updates(self):
+        if not self.should_read_display:
+            self.should_read_display = True
+            threading.Thread(target=self._read_display_job, daemon=True).start()
+
+    def stop_display_updates(self):
+        self.should_read_display = False
 
     def hard_reset(self):
         kiss_command = bytes([KISS.FEND, KISS.CMD_RESET, 0xf8, KISS.FEND])
@@ -447,9 +588,12 @@ class RNodeInterface(Interface):
             raise IOError("An IO error occurred while configuring radio state for "+str(self))
 
     def validate_firmware(self):
-        if (self.maj_version >= RNodeInterface.REQUIRED_FW_VER_MAJ):
-            if (self.min_version >= RNodeInterface.REQUIRED_FW_VER_MIN):
-                self.firmware_ok = True
+        if (self.maj_version > RNodeInterface.REQUIRED_FW_VER_MAJ):
+            self.firmware_ok = True
+        else:
+            if (self.maj_version >= RNodeInterface.REQUIRED_FW_VER_MAJ):
+                if (self.min_version >= RNodeInterface.REQUIRED_FW_VER_MIN):
+                    self.firmware_ok = True
         
         if self.firmware_ok:
             return
@@ -462,7 +606,14 @@ class RNodeInterface(Interface):
 
     def validateRadioState(self):
         RNS.log("Waiting for radio configuration validation for "+str(self)+"...", RNS.LOG_VERBOSE)
-        sleep(0.25);
+        if self.use_ble:
+            sleep(1.00)
+        else:
+            sleep(0.25)
+
+        if self.use_ble and self.ble != None and self.ble.device_disappeared:
+            RNS.log(f"Device disappeared during radio state validation for {self}", RNS.LOG_ERROR)
+            return False
 
         self.validcfg = True
         if (self.r_frequency != None and abs(self.frequency - int(self.r_frequency)) > 100):
@@ -495,14 +646,14 @@ class RNodeInterface(Interface):
         except:
             self.bitrate = 0
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)
         self.owner.inbound(data, self)
         self.r_stat_rssi = None
         self.r_stat_snr = None
 
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         datalen = len(data)
         if self.online:
             if self.interface_ready:
@@ -533,7 +684,7 @@ class RNodeInterface(Interface):
         if len(self.packet_queue) > 0:
             data = self.packet_queue.pop(0)
             self.interface_ready = True
-            self.processOutgoing(data)
+            self.process_outgoing(data)
         elif len(self.packet_queue) == 0:
             self.interface_ready = True
 
@@ -553,7 +704,7 @@ class RNodeInterface(Interface):
 
                     if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
                         in_frame = False
-                        self.processIncoming(data_buffer)
+                        self.process_incoming(data_buffer)
                         data_buffer = b""
                         command_buffer = b""
                     elif (byte == KISS.FEND):
@@ -728,16 +879,31 @@ class RNodeInterface(Interface):
                                         byte = KISS.FESC
                                     escape = False
                                 command_buffer = command_buffer+bytes([byte])
-                                if (len(command_buffer) == 8):
+                                if (len(command_buffer) == 11):
                                     ats = command_buffer[0] << 8 | command_buffer[1]
                                     atl = command_buffer[2] << 8 | command_buffer[3]
                                     cus = command_buffer[4] << 8 | command_buffer[5]
                                     cul = command_buffer[6] << 8 | command_buffer[7]
+                                    crs = command_buffer[8]
+                                    nfl = command_buffer[9]
+                                    ntf = command_buffer[10]
                                     
                                     self.r_airtime_short      = ats/100.0
                                     self.r_airtime_long       = atl/100.0
                                     self.r_channel_load_short = cus/100.0
                                     self.r_channel_load_long  = cul/100.0
+                                    self.r_current_rssi       = crs-RNodeInterface.RSSI_OFFSET
+                                    self.r_noise_floor        = nfl-RNodeInterface.RSSI_OFFSET
+                                    if ntf == 0xFF:
+                                        self.r_interference   = None
+                                    else:
+                                        self.r_interference   = ntf-RNodeInterface.RSSI_OFFSET
+                                    
+                                    if self.r_interference != None:
+                                        RNS.log(f"{self} Radio detected interference at {self.r_interference} dBm", RNS.LOG_DEBUG)
+
+                                    # TODO: Remove debug
+                                    # RNS.log(f"RSSI: {self.r_current_rssi}, Noise floor: {self.r_noise_floor}, Interference: {self.r_interference}", RNS.LOG_EXTREME)
                         elif (command == KISS.CMD_STAT_PHYPRM):
                             if (byte == KISS.FESC):
                                 escape = True
@@ -749,22 +915,68 @@ class RNodeInterface(Interface):
                                         byte = KISS.FESC
                                     escape = False
                                 command_buffer = command_buffer+bytes([byte])
-                                if (len(command_buffer) == 10):
+                                if (len(command_buffer) == 12):
                                     lst = (command_buffer[0] << 8 | command_buffer[1])/1000.0
                                     lsr = command_buffer[2] << 8 | command_buffer[3]
                                     prs = command_buffer[4] << 8 | command_buffer[5]
                                     prt = command_buffer[6] << 8 | command_buffer[7]
                                     cst = command_buffer[8] << 8 | command_buffer[9]
+                                    dft = command_buffer[10] << 8 | command_buffer[11]
 
-                                    if lst != self.r_symbol_time_ms or lsr != self.r_symbol_rate or prs != self.r_preamble_symbols or prt != self.r_premable_time_ms or cst != self.r_csma_slot_time_ms:
+                                    if lst != self.r_symbol_time_ms or lsr != self.r_symbol_rate or prs != self.r_preamble_symbols or prt != self.r_premable_time_ms or cst != self.r_csma_slot_time_ms or dft != self.r_csma_difs_ms:
                                         self.r_symbol_time_ms    = lst
                                         self.r_symbol_rate       = lsr
                                         self.r_preamble_symbols  = prs
                                         self.r_premable_time_ms  = prt
                                         self.r_csma_slot_time_ms = cst
-                                        RNS.log(str(self)+" Radio reporting symbol time is "+str(round(self.r_symbol_time_ms,2))+"ms (at "+str(self.r_symbol_rate)+" baud)", RNS.LOG_DEBUG)
-                                        RNS.log(str(self)+" Radio reporting preamble is "+str(self.r_preamble_symbols)+" symbols ("+str(self.r_premable_time_ms)+"ms)", RNS.LOG_DEBUG)
-                                        RNS.log(str(self)+" Radio reporting CSMA slot time is "+str(self.r_csma_slot_time_ms)+"ms", RNS.LOG_DEBUG)
+                                        self.r_csma_difs_ms      = dft
+                                        RNS.log(f"{self} Radio reporting symbol time is "+str(round(self.r_symbol_time_ms,2))+"ms ("+str(self.r_symbol_rate)+" baud)", RNS.LOG_DEBUG)
+                                        RNS.log(f"{self} Radio reporting preamble is "+str(self.r_preamble_symbols)+" symbols ("+str(self.r_premable_time_ms)+"ms)", RNS.LOG_DEBUG)
+                                        RNS.log(f"{self} Radio reporting CSMA slot time is "+str(self.r_csma_slot_time_ms)+"ms", RNS.LOG_DEBUG)
+                                        RNS.log(f"{self} Radio reporting DIFS time is "+str(self.r_csma_difs_ms)+"ms", RNS.LOG_DEBUG)
+                        elif (command == KISS.CMD_STAT_CSMA):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 3):
+                                    cbw = command_buffer[0]
+                                    cbl = command_buffer[1]
+                                    cbh = command_buffer[2]
+
+                                    if cbw != self.r_csma_cw_band or cbl != self.r_csma_cw_min or cbh != self.r_csma_cw_max:
+                                        self.r_csma_cw_band = cbw
+                                        self.r_csma_cw_min  = cbl
+                                        self.r_csma_cw_max  = cbh
+                                        # TODO: Remove debug
+                                        # RNS.log(f"{self} Radio reporting contention window band is {self.r_csma_cw_band}", RNS.LOG_EXTREME)
+                                        # RNS.log(f"{self} Radio reporting minimum contention window is {self.r_csma_cw_min}", RNS.LOG_EXTREME)
+                                        # RNS.log(f"{self} Radio reporting maximum contention window is {self.r_csma_cw_max}", RNS.LOG_EXTREME)
+                        elif (command == KISS.CMD_STAT_BAT):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 2):
+                                    bat_percent = command_buffer[1]
+                                    if bat_percent > 100:
+                                        bat_percent = 100
+                                    if bat_percent < 0:
+                                        bat_percent = 0
+                                    self.r_battery_state   = command_buffer[0]
+                                    self.r_battery_percent = bat_percent
                         elif (command == KISS.CMD_RANDOM):
                             self.r_random = byte
                         elif (command == KISS.CMD_PLATFORM):
@@ -778,6 +990,12 @@ class RNodeInterface(Interface):
                             elif (byte == KISS.ERROR_TXFAILED):
                                 RNS.log(str(self)+" hardware TX error (code "+RNS.hexrep(byte)+")", RNS.LOG_ERROR)
                                 raise IOError("Hardware transmit failure")
+                            elif (byte == KISS.ERROR_MEMORY_LOW):
+                                RNS.log(str(self)+" hardware error (code "+RNS.hexrep(byte)+"): Memory exhausted", RNS.LOG_ERROR)
+                                self.hw_errors.append({"error": KISS.ERROR_MEMORY_LOW, "description": "Memory exhausted on connected device"})
+                            elif (byte == KISS.ERROR_MODEM_TIMEOUT):
+                                RNS.log(str(self)+" hardware error (code "+RNS.hexrep(byte)+"): Modem communication timed out", RNS.LOG_ERROR)
+                                self.hw_errors.append({"error": KISS.ERROR_MODEM_TIMEOUT, "description": "Modem communication timed out on connected device"})
                             else:
                                 RNS.log(str(self)+" hardware error (code "+RNS.hexrep(byte)+")", RNS.LOG_ERROR)
                                 raise IOError("Unknown hardware failure")
@@ -789,6 +1007,36 @@ class RNodeInterface(Interface):
                                         raise IOError("ESP32 reset")
                         elif (command == KISS.CMD_READY):
                             self.process_queue()
+                        elif (command == KISS.CMD_FB_READ):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 512):
+                                    self.r_framebuffer_latency = time.time() - self.r_framebuffer_readtime
+                                    self.r_framebuffer = command_buffer
+
+                        elif (command == KISS.CMD_DISP_READ):
+                            if (byte == KISS.FESC):
+                                escape = True
+                            else:
+                                if (escape):
+                                    if (byte == KISS.TFEND):
+                                        byte = KISS.FEND
+                                    if (byte == KISS.TFESC):
+                                        byte = KISS.FESC
+                                    escape = False
+                                command_buffer = command_buffer+bytes([byte])
+                                if (len(command_buffer) == 1024):
+                                    self.r_disp_latency = time.time() - self.r_disp_readtime
+                                    self.r_disp = command_buffer
+
                         elif (command == KISS.CMD_DETECT):
                             if byte == KISS.DETECT_RESP:
                                 self.detected = True
@@ -808,7 +1056,7 @@ class RNodeInterface(Interface):
                         if self.first_tx != None:
                             if time.time() > self.first_tx + self.id_interval:
                                 RNS.log("Interface "+str(self)+" is transmitting beacon data: "+str(self.id_callsign.decode("utf-8")), RNS.LOG_DEBUG)
-                                self.processOutgoing(self.id_callsign)
+                                self.process_outgoing(self.id_callsign)
 
                     sleep(0.08)
 
@@ -852,9 +1100,219 @@ class RNodeInterface(Interface):
         self.disable_external_framebuffer()
         self.setRadioState(KISS.RADIO_STATE_OFF)
         self.leave()
+        
+        if self.use_ble:
+            self.ble.close()
 
     def should_ingress_limit(self):
         return False
 
+    def get_battery_state(self):
+        return self.r_battery_state
+
+    def get_battery_state_string(self):
+        if self.r_battery_state == RNodeInterface.BATTERY_STATE_CHARGED:
+            return "charged"
+        elif  self.r_battery_state == RNodeInterface.BATTERY_STATE_CHARGING:
+            return "charging"
+        elif self.r_battery_state == RNodeInterface.BATTERY_STATE_DISCHARGING:
+            return "discharging"
+        else:
+            return "unknown"
+
+    def get_battery_percent(self):
+        return self.r_battery_percent
+
+    def ble_receive(self, data):
+        with self.ble_rx_lock:
+            self.ble_rx_queue += data
+
+    def ble_waiting(self):
+        return len(self.ble_tx_queue) > 0
+
+    def get_ble_waiting(self, n):
+        with self.ble_tx_lock:
+            data = self.ble_tx_queue[:n]
+            self.ble_tx_queue = self.ble_tx_queue[n:]
+            return data
+
     def __str__(self):
         return "RNodeInterface["+str(self.name)+"]"
+
+class BLEConnection():
+    UART_SERVICE_UUID = "6E400001-B5A3-F393-E0A9-E50E24DCCA9E"
+    UART_RX_CHAR_UUID = "6E400002-B5A3-F393-E0A9-E50E24DCCA9E"
+    UART_TX_CHAR_UUID = "6E400003-B5A3-F393-E0A9-E50E24DCCA9E"
+    bleak = None
+
+    SCAN_TIMEOUT = 2.0
+    CONNECT_TIMEOUT = 5.0
+
+    @property
+    def is_open(self):
+        return self.connected
+
+    @property
+    def in_waiting(self):
+        buflen = len(self.owner.ble_rx_queue)
+        return buflen > 0
+
+    def write(self, data_bytes):
+        with self.owner.ble_tx_lock:
+            self.owner.ble_tx_queue += data_bytes
+            return len(data_bytes)
+
+    def read(self, n):
+        with self.owner.ble_rx_lock:
+            data = self.owner.ble_rx_queue[:n]
+            self.owner.ble_rx_queue = self.owner.ble_rx_queue[n:]
+            return data
+
+    def close(self):
+        if self.connected and self.ble_device:
+            RNS.log(f"Disconnecting BLE device from {self.owner}", RNS.LOG_DEBUG)
+            self.must_disconnect = True
+
+            while self.connect_job_running:
+                time.sleep(0.1)
+
+    def __init__(self, owner=None, target_name=None, target_bt_addr=None):
+        self.owner = owner
+        self.target_name = target_name
+        self.target_bt_addr = target_bt_addr
+        self.scan_timeout = BLEConnection.SCAN_TIMEOUT
+        self.ble_device = None
+        self.last_client = None
+        self.connected = False
+        self.running = False
+        self.should_run = False
+        self.must_disconnect = False
+        self.connect_job_running = False
+        self.device_disappeared = False
+
+        import importlib.util
+        if BLEConnection.bleak == None:
+            if importlib.util.find_spec("bleak") != None:
+                import bleak
+                BLEConnection.bleak = bleak
+                
+                import asyncio
+                BLEConnection.asyncio = asyncio
+            else:
+                RNS.log("Using the RNode interface over BLE requires a the \"bleak\" module to be installed.", RNS.LOG_CRITICAL)
+                RNS.log("You can install one with the command: python3 -m pip install bleak", RNS.LOG_CRITICAL)
+                RNS.panic()
+
+        self.should_run = True
+        self.connection_thread = threading.Thread(target=self.connection_job, daemon=True).start()
+
+    def cleanup(self):
+        try:
+            if self.last_client != None:
+                self.asyncio.run(self.last_client.disconnect())
+        except Exception as e:
+            RNS.log(f"Error while disconnecting BLE device on cleanup for {self.owner}", RNS.LOG_ERROR)
+
+        self.should_run = False
+
+    def connection_job(self):
+        while self.should_run:
+            if self.ble_device == None:
+                self.ble_device = self.find_target_device()
+
+            if type(self.ble_device) == self.bleak.backends.device.BLEDevice:
+                if not self.connected:
+                    self.connect_device()
+
+            time.sleep(1)
+
+        self.cleanup()
+        self.running = False
+        RNS.log(f"BLE connection job for {self.owner} ended", RNS.LOG_DEBUG)
+
+    def connect_device(self):
+        if self.ble_device != None and type(self.ble_device) == self.bleak.backends.device.BLEDevice:
+            RNS.log(f"Connecting BLE device {self.ble_device} for {self.owner}...", RNS.LOG_DEBUG)
+
+            async def connect_job():
+                self.connect_job_running = True
+                async with self.bleak.BleakClient(self.ble_device, disconnected_callback=self.device_disconnected) as ble_client:
+                    def handle_rx(device, data):
+                        if self.owner != None:
+                            self.owner.ble_receive(data)
+
+                    self.connected = True
+                    self.ble_device = ble_client
+                    self.last_client = ble_client
+                    self.owner.port = str(f"ble://{ble_client.address}")
+
+                    loop = self.asyncio.get_running_loop()
+                    uart_service = ble_client.services.get_service(BLEConnection.UART_SERVICE_UUID)
+                    rx_characteristic = uart_service.get_characteristic(BLEConnection.UART_RX_CHAR_UUID)
+                    await ble_client.start_notify(BLEConnection.UART_TX_CHAR_UUID, handle_rx)
+
+                    while self.connected:
+                        if self.owner != None and self.owner.ble_waiting():
+                            outbound_data = self.owner.get_ble_waiting(rx_characteristic.max_write_without_response_size)
+                            await ble_client.write_gatt_char(rx_characteristic, outbound_data, response=False)
+                        elif self.must_disconnect:
+                            await ble_client.disconnect()
+                        else:
+                            await self.asyncio.sleep(0.1)
+
+
+            try:
+                self.asyncio.run(connect_job())
+            except Exception as e:
+                RNS.log(f"Could not connect BLE device {self.ble_device} for {self.owner}. Possibly missing authentication.", RNS.LOG_ERROR)
+
+            self.connect_job_running = False
+
+    def device_disconnected(self, device):
+        RNS.log(f"BLE device for {self.owner} disconnected", RNS.LOG_NOTICE)
+        self.connected = False
+        self.ble_device = None
+        self.device_disappeared = True
+
+    def find_target_device(self):
+        RNS.log(f"Searching for attachable BLE device for {self.owner}...", RNS.LOG_EXTREME)
+        def device_filter(device: self.bleak.backends.device.BLEDevice, adv: self.bleak.backends.scanner.AdvertisementData):
+            if BLEConnection.UART_SERVICE_UUID.lower() in adv.service_uuids:
+                if self.device_bonded(device):
+                    if self.target_bt_addr == None and self.target_name == None:
+                        if device.name.startswith("RNode "):
+                            return True
+
+                    if self.target_bt_addr == None or (device.address != None and device.address == self.target_bt_addr):
+                        if self.target_name == None or (device.name != None and device.name == self.target_name):
+                            return True
+
+                else:
+                    if self.target_bt_addr != None and device.address == self.target_bt_addr:
+                        RNS.log(f"Can't connect to target device {self.target_bt_addr} over BLE, device is not bonded", RNS.LOG_ERROR)
+                    
+                    elif self.target_name != None and device.name == self.target_name:
+                        RNS.log(f"Can't connect to target device {self.target_name} over BLE, device is not bonded", RNS.LOG_ERROR)
+
+            return False
+
+        device = None
+        try:
+            device = self.asyncio.run(self.bleak.BleakScanner.find_device_by_filter(device_filter, timeout=self.scan_timeout))
+        except Exception as e:
+            RNS.log(f"Error while finding BLE device for {self.owner}: {e}", RNS.LOG_ERROR)
+            self.should_run = False
+
+        return device
+
+    def device_bonded(self, device):
+        try:
+            if hasattr(device, "details"):
+                if "props" in device.details and "Bonded" in device.details["props"]:
+                    if device.details["props"]["Bonded"] == True:
+                        return True
+        
+        except Exception as e:
+            RNS.log(f"Error while determining device bond status for {device}, the contained exception was: {e}", RNS.LOG_ERROR)
+
+        return False

RNS/Interfaces/RNodeMultiInterface.py

@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 from time import sleep
 import sys
 import threading
@@ -33,6 +33,8 @@ class KISS():
     FESC            = 0xDB
     TFEND           = 0xDC
     TFESC           = 0xDD
+
+    CMD_DATA = 0x00
     
     CMD_UNKNOWN     = 0xFE
     CMD_FREQUENCY   = 0x01
@@ -70,7 +72,7 @@ class KISS():
     CMD_INT1_DATA   = 0x10
     CMD_INT2_DATA   = 0x20
     CMD_INT3_DATA   = 0x70
-    CMD_INT4_DATA   = 0x80
+    CMD_INT4_DATA   = 0x75
     CMD_INT5_DATA   = 0x90
     CMD_INT6_DATA   = 0xA0
     CMD_INT7_DATA   = 0xB0
@@ -79,19 +81,6 @@ class KISS():
     CMD_INT10_DATA  = 0xE0
     CMD_INT11_DATA  = 0xF0
 
-    CMD_SEL_INT0    = 0x1E
-    CMD_SEL_INT1    = 0x1F
-    CMD_SEL_INT2    = 0x2F
-    CMD_SEL_INT3    = 0x7F
-    CMD_SEL_INT4    = 0x8F
-    CMD_SEL_INT5    = 0x9F
-    CMD_SEL_INT6    = 0xAF
-    CMD_SEL_INT7    = 0xBF
-    CMD_SEL_INT8    = 0xCF
-    CMD_SEL_INT9    = 0xDF
-    CMD_SEL_INT10   = 0xEF
-    CMD_SEL_INT11   = 0xFF
-
     DETECT_REQ      = 0x73
     DETECT_RESP     = 0x46
     
@@ -106,6 +95,7 @@ class KISS():
 
     PLATFORM_AVR   = 0x90
     PLATFORM_ESP32 = 0x80
+    PLATFORM_NRF52 = 0x70
 
     SX127X    = 0x00
     SX1276    = 0x01
@@ -115,33 +105,7 @@ class KISS():
     SX128X    = 0x20
     SX1280    = 0x21
 
-    def int_data_cmd_to_index(int_data_cmd):
-        if int_data_cmd == KISS.CMD_INT0_DATA:
-            return 0
-        elif int_data_cmd == KISS.CMD_INT1_DATA:
-            return 1
-        elif int_data_cmd == KISS.CMD_INT2_DATA:
-            return 2
-        elif int_data_cmd == KISS.CMD_INT3_DATA:
-            return 3
-        elif int_data_cmd == KISS.CMD_INT4_DATA:
-            return 4
-        elif int_data_cmd == KISS.CMD_INT5_DATA:
-            return 5
-        elif int_data_cmd == KISS.CMD_INT6_DATA:
-            return 6
-        elif int_data_cmd == KISS.CMD_INT7_DATA:
-            return 7
-        elif int_data_cmd == KISS.CMD_INT8_DATA:
-            return 8
-        elif int_data_cmd == KISS.CMD_INT9_DATA:
-            return 9
-        elif int_data_cmd == KISS.CMD_INT10_DATA:
-            return 10
-        elif int_data_cmd == KISS.CMD_INT11_DATA:
-            return 11
-        else:
-            return 0
+    CMD_SEL_INT = 0x1F
 
     def interface_type_to_str(interface_type):
         if interface_type == KISS.SX126X or interface_type == KISS.SX1262:
@@ -162,21 +126,22 @@ class KISS():
 
 class RNodeMultiInterface(Interface):
     MAX_CHUNK = 32768
+    DEFAULT_IFAC_SIZE = 8
 
     CALLSIGN_MAX_LEN    = 32
 
     REQUIRED_FW_VER_MAJ = 1
-    REQUIRED_FW_VER_MIN = 73
+    REQUIRED_FW_VER_MIN = 74
 
     RECONNECT_WAIT = 5
 
     MAX_SUBINTERFACES = 11
 
-    def __init__(self, owner, name, port, subint_config, id_interval = None, id_callsign = None):
+    def __init__(self, owner, configuration):
         if RNS.vendor.platformutils.is_android():
             raise SystemError("Invalid interface type. The Android-specific RNode interface must be used on Android")
 
-        import importlib
+        import importlib.util
         if importlib.util.find_spec('serial') != None:
             import serial
         else:
@@ -186,6 +151,77 @@ class RNodeMultiInterface(Interface):
 
         super().__init__()
 
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+
+        count = 0
+        enabled_count = 0
+
+        # Count how many interfaces are in the file
+        for subinterface in c:
+            # if the retrieved entry is not a string, it must be a dictionary, which is what we want
+            if isinstance(c[subinterface], dict):
+                count += 1
+
+        # Count how many interfaces are enabled to allow for appropriate matrix sizing
+        for subinterface in c:
+            if isinstance(c[subinterface], dict):
+                subinterface_config = c[subinterface]
+                if (("interface_enabled" in subinterface_config) and subinterface_config.as_bool("interface_enabled") == True) or (("enabled" in c) and c.as_bool("enabled") == True):
+                    enabled_count += 1
+
+        # Create an array with a row for each subinterface
+        subint_config = [[None for x in range(11)] for y in range(enabled_count)]
+        subint_index = 0
+
+        for subinterface in c:
+            if isinstance(c[subinterface], dict):
+                subinterface_config = c[subinterface]
+                if (("interface_enabled" in subinterface_config) and subinterface_config.as_bool("interface_enabled") == True) or (("enabled" in c) and c.as_bool("enabled") == True):
+                    subint_vport = subinterface_config["vport"] if "vport" in subinterface_config else None
+
+                    subint_config[subint_index][0] = subinterface
+
+                    subint_config[subint_index][1] = subint_vport
+
+                    frequency = int(subinterface_config["frequency"]) if "frequency" in subinterface_config else None
+                    subint_config[subint_index][2] = frequency
+                    bandwidth = int(subinterface_config["bandwidth"]) if "bandwidth" in subinterface_config else None
+                    subint_config[subint_index][3] = bandwidth
+                    txpower = int(subinterface_config["txpower"]) if "txpower" in subinterface_config else None
+                    subint_config[subint_index][4] = txpower
+                    spreadingfactor = int(subinterface_config["spreadingfactor"]) if "spreadingfactor" in subinterface_config else None
+                    subint_config[subint_index][5] = spreadingfactor 
+                    codingrate = int(subinterface_config["codingrate"]) if "codingrate" in subinterface_config else None
+                    subint_config[subint_index][6] = codingrate
+                    flow_control = subinterface_config.as_bool("flow_control") if "flow_control" in subinterface_config else False
+                    subint_config[subint_index][7] = flow_control 
+                    st_alock = float(subinterface_config["airtime_limit_short"]) if "airtime_limit_short" in subinterface_config else None
+                    subint_config[subint_index][8] = st_alock
+                    lt_alock = float(subinterface_config["airtime_limit_long"]) if "airtime_limit_long" in subinterface_config else None
+                    subint_config[subint_index][9] = lt_alock
+
+                    if "outgoing" in subinterface_config and subinterface_config.as_bool("outgoing") == False:
+                        subint_config[subint_index][10] = False
+                    else:
+                        subint_config[subint_index][10] = True
+
+                    subint_index += 1
+
+        # if no subinterfaces are defined
+        if count == 0:
+            raise ValueError("No subinterfaces configured for "+name)
+        # if no subinterfaces are enabled
+        elif enabled_count == 0:
+            raise ValueError("No subinterfaces enabled for "+name)
+
+        id_interval = int(c["id_interval"]) if "id_interval" in c else None
+        id_callsign = c["id_callsign"] if "id_callsign" in c else None
+        port = c["port"] if "port" in c else None
+        
+        if port == None:
+            raise ValueError("No port specified for "+name)
+
         self.HW_MTU = 508
         
         self.clients = 0
@@ -249,6 +285,7 @@ class RNodeMultiInterface(Interface):
         if (not self.validcfg):
             raise ValueError("The configuration for "+str(self)+" contains errors, interface is offline")
 
+    def start(self):
         try:
             self.open_port()
 
@@ -297,7 +334,7 @@ class RNodeMultiInterface(Interface):
             RNS.log("Could not detect device for "+str(self), RNS.LOG_ERROR)
             self.serial.close()
         else:
-            if self.platform == KISS.PLATFORM_ESP32:
+            if self.platform == KISS.PLATFORM_ESP32 or self.platform == KISS.PLATFORM_NRF52:
                 self.display = True
 
         RNS.log("Serial port "+self.port+" is now open")
@@ -323,8 +360,8 @@ class RNodeMultiInterface(Interface):
                         lt_alock=subint[9]
                 )
 
-                interface.OUT = self.OUT
-                interface.IN  = self.IN
+                interface.OUT = subint[10]
+                interface.IN  = True
                 
                 interface.announce_rate_target = self.announce_rate_target
                 interface.mode = self.mode
@@ -402,11 +439,10 @@ class RNodeMultiInterface(Interface):
         c4 = frequency & 0xFF
         data = KISS.escape(bytes([c1])+bytes([c2])+bytes([c3])+bytes([c4]))
 
-        kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_FREQUENCY])+data+bytes([KISS.FEND])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_FREQUENCY])+data+bytes([KISS.FEND])
         written = self.serial.write(kiss_command)
         if written != len(kiss_command):
             raise IOError("An IO error occurred while configuring frequency for "+str(self))
-        self.selected_index = interface.index
 
     def setBandwidth(self, bandwidth, interface):
         c1 = bandwidth >> 24
@@ -415,35 +451,31 @@ class RNodeMultiInterface(Interface):
         c4 = bandwidth & 0xFF
         data = KISS.escape(bytes([c1])+bytes([c2])+bytes([c3])+bytes([c4]))
 
-        kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_BANDWIDTH])+data+bytes([KISS.FEND])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_BANDWIDTH])+data+bytes([KISS.FEND])
         written = self.serial.write(kiss_command)
         if written != len(kiss_command):
             raise IOError("An IO error occurred while configuring bandwidth for "+str(self))
-        self.selected_index = interface.index
 
     def setTXPower(self, txpower, interface):
         txp = txpower.to_bytes(1, byteorder="big", signed=True)
-        kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_TXPOWER])+txp+bytes([KISS.FEND])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_TXPOWER])+txp+bytes([KISS.FEND])
         written = self.serial.write(kiss_command)
         if written != len(kiss_command):
             raise IOError("An IO error occurred while configuring TX power for "+str(self))
-        self.selected_index = interface.index
 
     def setSpreadingFactor(self, sf, interface):
         sf = bytes([sf])
-        kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_SF])+sf+bytes([KISS.FEND])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_SF])+sf+bytes([KISS.FEND])
         written = self.serial.write(kiss_command)
         if written != len(kiss_command):
             raise IOError("An IO error occurred while configuring spreading factor for "+str(self))
-        self.selected_index = interface.index
 
     def setCodingRate(self, cr, interface):
         cr = bytes([cr])
-        kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_CR])+cr+bytes([KISS.FEND])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_CR])+cr+bytes([KISS.FEND])
         written = self.serial.write(kiss_command)
         if written != len(kiss_command):
             raise IOError("An IO error occurred while configuring coding rate for "+str(self))
-        self.selected_index = interface.index
 
     def setSTALock(self, st_alock, interface):
         if st_alock != None:
@@ -452,11 +484,10 @@ class RNodeMultiInterface(Interface):
             c2 = at & 0xFF
             data = KISS.escape(bytes([c1])+bytes([c2]))
 
-            kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_ST_ALOCK])+data+bytes([KISS.FEND])
+            kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_ST_ALOCK])+data+bytes([KISS.FEND])
             written = self.serial.write(kiss_command)
             if written != len(kiss_command):
                 raise IOError("An IO error occurred while configuring short-term airtime limit for "+str(self))
-            self.selected_index = interface.index
 
     def setLTALock(self, lt_alock, interface):
         if lt_alock != None:
@@ -465,19 +496,17 @@ class RNodeMultiInterface(Interface):
             c2 = at & 0xFF
             data = KISS.escape(bytes([c1])+bytes([c2]))
 
-            kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_LT_ALOCK])+data+bytes([KISS.FEND])
+            kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_LT_ALOCK])+data+bytes([KISS.FEND])
             written = self.serial.write(kiss_command)
             if written != len(kiss_command):
                 raise IOError("An IO error occurred while configuring long-term airtime limit for "+str(self))
-            self.selected_index = interface.index
 
     def setRadioState(self, state, interface):
         #self.state = state
-        kiss_command = bytes([KISS.FEND])+bytes([interface.sel_cmd])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_RADIO_STATE])+bytes([state])+bytes([KISS.FEND])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_RADIO_STATE])+bytes([state])+bytes([KISS.FEND])
         written = self.serial.write(kiss_command)
         if written != len(kiss_command):
             raise IOError("An IO error occurred while configuring radio state for "+str(self))
-        self.selected_index = interface.index
 
     def validate_firmware(self):
         if (self.maj_version >= RNodeMultiInterface.REQUIRED_FW_VER_MAJ):
@@ -492,13 +521,13 @@ class RNodeMultiInterface(Interface):
         RNS.log("Please update your RNode firmware with rnodeconf from https://github.com/markqvist/Reticulum/RNS/Utilities/rnodeconf.py")
         RNS.panic()
 
-    def processOutgoing(self, data, interface = None):
+    def process_outgoing(self, data, interface = None):
         if interface is None:
             # do nothing if RNS tries to transmit on this interface directly
             pass
         else:
             data    = KISS.escape(data)
-            frame   = bytes([0xc0])+bytes([interface.data_cmd])+data+bytes([0xc0])
+            frame   = bytes([KISS.FEND])+bytes([KISS.CMD_SEL_INT])+bytes([interface.index])+bytes([KISS.FEND])+bytes([KISS.FEND])+bytes([KISS.CMD_DATA])+data+bytes([KISS.FEND])
 
             written = self.serial.write(frame)
             self.txb += len(data)
@@ -527,21 +556,9 @@ class RNodeMultiInterface(Interface):
                     last_read_ms = int(time.time()*1000)
 
                     if (in_frame and byte == KISS.FEND and
-                            (command == KISS.CMD_INT0_DATA or
-                            command == KISS.CMD_INT1_DATA or
-                            command == KISS.CMD_INT2_DATA or
-                            command == KISS.CMD_INT3_DATA or
-                            command == KISS.CMD_INT4_DATA or
-                            command == KISS.CMD_INT5_DATA or
-                            command == KISS.CMD_INT6_DATA or
-                            command == KISS.CMD_INT7_DATA or
-                            command == KISS.CMD_INT8_DATA or
-                            command == KISS.CMD_INT9_DATA or
-                            command == KISS.CMD_INT10_DATA or
-                            command == KISS.CMD_INT11_DATA)):
+                            (command == KISS.CMD_DATA)):
                         in_frame = False
-                        self.subinterfaces[KISS.int_data_cmd_to_index(command)].processIncoming(data_buffer)
-                        self.selected_index = KISS.int_data_cmd_to_index(command)
+                        self.subinterfaces[self.selected_index].process_incoming(data_buffer)
                         data_buffer = b""
                         command_buffer = b""
                     elif (byte == KISS.FEND):
@@ -606,6 +623,9 @@ class RNodeMultiInterface(Interface):
                                     RNS.log(str(self.subinterfaces[self.selected_index])+" Radio reporting bandwidth is "+str(self.subinterfaces[self.selected_index].r_bandwidth/1000.0)+" KHz", RNS.LOG_DEBUG)
                                     self.subinterfaces[self.selected_index].updateBitrate()
 
+                        elif (command == KISS.CMD_SEL_INT):
+                            self.selected_index = byte
+
                         elif (command == KISS.CMD_TXPOWER):
                             txp = byte - 256 if byte > 127 else byte
                             self.subinterfaces[self.selected_index].r_txpower = txp
@@ -819,7 +839,7 @@ class RNodeMultiInterface(Interface):
                                 for interface in self.subinterfaces:
                                     if interface != 0 and interface.online:
                                         interface_available = True
-                                        self.subinterfaces[interface.index].processOutgoing(self.id_callsign)
+                                        self.subinterfaces[interface.index].process_outgoing(self.id_callsign)
 
                                 if interface_available:
                                     RNS.log("Interface "+str(self)+" is transmitting beacon data on all subinterfaces: "+str(self.id_callsign.decode("utf-8")), RNS.LOG_DEBUG)
@@ -907,7 +927,7 @@ class RNodeSubInterface(Interface):
         if RNS.vendor.platformutils.is_android():
             raise SystemError("Invalid interface type. The Android-specific RNode interface must be used on Android")
 
-        import importlib
+        import importlib.util
         if importlib.util.find_spec('serial') != None:
             import serial
         else:
@@ -917,51 +937,9 @@ class RNodeSubInterface(Interface):
 
         super().__init__()
         
-        if index == 0:
-            sel_cmd = KISS.CMD_SEL_INT0
-            data_cmd= KISS.CMD_INT0_DATA
-        elif index == 1:
-            sel_cmd = KISS.CMD_SEL_INT1
-            data_cmd= KISS.CMD_INT1_DATA
-        elif index == 2:
-            sel_cmd = KISS.CMD_SEL_INT2
-            data_cmd= KISS.CMD_INT2_DATA
-        elif index == 3:
-            sel_cmd = KISS.CMD_SEL_INT3
-            data_cmd= KISS.CMD_INT3_DATA
-        elif index == 4:
-            sel_cmd = KISS.CMD_SEL_INT4
-            data_cmd= KISS.CMD_INT4_DATA
-        elif index == 5:
-            sel_cmd = KISS.CMD_SEL_INT5
-            data_cmd= KISS.CMD_INT5_DATA
-        elif index == 6:
-            sel_cmd = KISS.CMD_SEL_INT6
-            data_cmd= KISS.CMD_INT6_DATA
-        elif index == 7:
-            sel_cmd = KISS.CMD_SEL_INT7
-            data_cmd= KISS.CMD_INT7_DATA
-        elif index == 8:
-            sel_cmd = KISS.CMD_SEL_INT8
-            data_cmd= KISS.CMD_INT8_DATA
-        elif index == 9:
-            sel_cmd = KISS.CMD_SEL_INT9
-            data_cmd= KISS.CMD_INT9_DATA
-        elif index == 10:
-            sel_cmd = KISS.CMD_SEL_INT10
-            data_cmd= KISS.CMD_INT10_DATA
-        elif index == 11:
-            sel_cmd = KISS.CMD_SEL_INT11
-            data_cmd= KISS.CMD_INT11_DATA
-        else:
-            sel_cmd = KISS.CMD_SEL_INT0
-            data_cmd= KISS.CMD_INT0_DATA
-
         self.owner       = owner
         self.name        = name
         self.index       = index
-        self.sel_cmd     = sel_cmd
-        self.data_cmd    = data_cmd
         self.interface_type= interface_type
         self.flow_control= flow_control
         self.online      = False
@@ -1006,6 +984,11 @@ class RNodeSubInterface(Interface):
         self.parent_interface = parent_interface
         self.announce_rate_target = None
 
+        self.mode = None
+        self.announce_cap = None
+        self.bitrate = None
+        self.ifac_size = None
+
         # add this interface to the subinterfaces array
         self.parent_interface.subinterfaces[index] = self
 
@@ -1120,13 +1103,13 @@ class RNodeSubInterface(Interface):
         except:
             self.bitrate = 0
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)
         self.owner.inbound(data, self)
         self.r_stat_rssi = None
         self.r_stat_snr = None
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         if self.online:
             if self.interface_ready:
                 if self.flow_control:
@@ -1138,7 +1121,7 @@ class RNodeSubInterface(Interface):
                     if self.parent_interface.first_tx == None:
                         self.parent_interface.first_tx = time.time()
                 self.txb += len(data)
-                self.parent_interface.processOutgoing(data, self)
+                self.parent_interface.process_outgoing(data, self)
             else:
                 self.queue(data)
 
@@ -1150,7 +1133,7 @@ class RNodeSubInterface(Interface):
         if len(self.packet_queue) > 0:
             data = self.packet_queue.pop(0)
             self.interface_ready = True
-            self.processOutgoing(data)
+            self.process_outgoing(data)
         elif len(self.packet_queue) == 0:
             self.interface_ready = True
 

RNS/Interfaces/SerialInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 from time import sleep
 import sys
 import threading
@@ -42,6 +42,7 @@ class HDLC():
 
 class SerialInterface(Interface):
     MAX_CHUNK = 32768
+    DEFAULT_IFAC_SIZE = 8
 
     owner    = None
     port     = None
@@ -51,8 +52,8 @@ class SerialInterface(Interface):
     stopbits = None
     serial   = None
 
-    def __init__(self, owner, name, port, speed, databits, parity, stopbits):
-        import importlib
+    def __init__(self, owner, configuration):
+        import importlib.util
         if importlib.util.find_spec('serial') != None:
             import serial
         else:
@@ -62,6 +63,17 @@ class SerialInterface(Interface):
 
         super().__init__()
 
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        port = c["port"] if "port" in c else None
+        speed = int(c["speed"]) if "speed" in c else 9600
+        databits = int(c["databits"]) if "databits" in c else 8
+        parity = c["parity"] if "parity" in c else "N"
+        stopbits = int(c["stopbits"]) if "stopbits" in c else 1
+
+        if port == None:
+            raise ValueError("No port specified for serial interface")
+
         self.HW_MTU = 564
         
         self.pyserial = serial
@@ -121,12 +133,12 @@ class SerialInterface(Interface):
         RNS.log("Serial port "+self.port+" is now open", RNS.LOG_VERBOSE)
 
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)            
         self.owner.inbound(data, self)
 
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         if self.online:
             data = bytes([HDLC.FLAG])+HDLC.escape(data)+bytes([HDLC.FLAG])
             written = self.serial.write(data)
@@ -149,7 +161,7 @@ class SerialInterface(Interface):
 
                     if (in_frame and byte == HDLC.FLAG):
                         in_frame = False
-                        self.processIncoming(data_buffer)
+                        self.process_incoming(data_buffer)
                     elif (byte == HDLC.FLAG):
                         in_frame = True
                         data_buffer = b""

RNS/Interfaces/TCPInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 import socketserver
 import threading
 import platform
@@ -30,6 +30,9 @@ import sys
 import os
 import RNS
 
+class TCPInterface():
+    HW_MTU            = 262144
+
 class HDLC():
     FLAG              = 0x7E
     ESC               = 0x7D
@@ -58,8 +61,13 @@ class KISS():
 class ThreadingTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
     pass
 
+class ThreadingTCP6Server(socketserver.ThreadingMixIn, socketserver.TCPServer):
+    address_family = socket.AF_INET6
+
 class TCPClientInterface(Interface):
     BITRATE_GUESS = 10*1000*1000
+    DEFAULT_IFAC_SIZE = 16
+    AUTOCONFIGURE_MTU = True
 
     RECONNECT_WAIT = 5
     RECONNECT_MAX_TRIES = None
@@ -78,11 +86,21 @@ class TCPClientInterface(Interface):
     I2P_PROBE_INTERVAL = 9
     I2P_PROBES = 5
 
-    def __init__(self, owner, name, target_ip=None, target_port=None, connected_socket=None, max_reconnect_tries=None, kiss_framing=False, i2p_tunneled = False, connect_timeout = None):
+    def __init__(self, owner, configuration, connected_socket=None):
         super().__init__()
+
+        c = Interface.get_config_obj(configuration)
+        name = c["name"]
+        target_ip = c["target_host"] if "target_host" in c and c["target_host"] != None else None
+        target_port = int(c["target_port"]) if "target_port" in c and c["target_host"] != None else None
+        kiss_framing = False
+        if "kiss_framing" in c and c.as_bool("kiss_framing") == True:
+            kiss_framing = True
+        i2p_tunneled = c.as_bool("i2p_tunneled") if "i2p_tunneled" in c else False
+        connect_timeout = c.as_int("connect_timeout") if "connect_timeout" in c else None
+        max_reconnect_tries = c.as_int("max_reconnect_tries") if "max_reconnect_tries" in c else None
         
-        self.HW_MTU = 1064
-        
+        self.HW_MTU           = TCPInterface.HW_MTU
         self.IN               = True
         self.OUT              = False
         self.socket           = None
@@ -177,19 +195,21 @@ class TCPClientInterface(Interface):
             self.socket.setsockopt(socket.IPPROTO_TCP, TCP_KEEPIDLE, int(TCPClientInterface.I2P_PROBE_AFTER))
         
     def detach(self):
+        self.online = False
         if self.socket != None:
             if hasattr(self.socket, "close"):
                 if callable(self.socket.close):
-                    RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
                     self.detached = True
                     
                     try:
-                        self.socket.shutdown(socket.SHUT_RDWR)
+                        if self.socket != None:
+                            self.socket.shutdown(socket.SHUT_RDWR)
                     except Exception as e:
                         RNS.log("Error while shutting down socket for "+str(self)+": "+str(e))
 
                     try:
-                        self.socket.close()
+                        if self.socket != None:
+                            self.socket.close()
                     except Exception as e:
                         RNS.log("Error while closing socket for "+str(self)+": "+str(e))
 
@@ -200,10 +220,14 @@ class TCPClientInterface(Interface):
             if initial:
                 RNS.log("Establishing TCP connection for "+str(self)+"...", RNS.LOG_DEBUG)
 
-            self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            address_info = socket.getaddrinfo(self.target_ip, self.target_port, proto=socket.IPPROTO_TCP)[0]
+            address_family = address_info[0]
+            target_address = address_info[4]
+
+            self.socket = socket.socket(address_family, socket.SOCK_STREAM)
             self.socket.settimeout(TCPClientInterface.INITIAL_CONNECT_TIMEOUT)
             self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
-            self.socket.connect((self.target_ip, self.target_port))
+            self.socket.connect(target_address)
             self.socket.settimeout(None)
             self.online  = True
 
@@ -265,15 +289,16 @@ class TCPClientInterface(Interface):
             RNS.log("Attempt to reconnect on a non-initiator TCP interface. This should not happen.", RNS.LOG_ERROR)
             raise IOError("Attempt to reconnect on a non-initiator TCP interface")
 
-    def processIncoming(self, data):
-        self.rxb += len(data)
-        if hasattr(self, "parent_interface") and self.parent_interface != None:
-            self.parent_interface.rxb += len(data)
-                    
-        self.owner.inbound(data, self)
+    def process_incoming(self, data):
+        if self.online and not self.detached:
+            self.rxb += len(data)
+            if hasattr(self, "parent_interface") and self.parent_interface != None:
+                self.parent_interface.rxb += len(data)
+                        
+            self.owner.inbound(data, self)
 
-    def processOutgoing(self, data):
-        if self.online:
+    def process_outgoing(self, data):
+        if self.online and not self.detached:
             # while self.writing:
             #     time.sleep(0.01)
 
@@ -301,22 +326,23 @@ class TCPClientInterface(Interface):
         try:
             in_frame = False
             escape = False
+            frame_buffer = b""
+            data_in = b""
             data_buffer = b""
-            command = KISS.CMD_UNKNOWN
 
             while True:
-                data_in = self.socket.recv(4096)
+                if self.socket: data_in = self.socket.recv(4096)
+                else: data_in = b""
                 if len(data_in) > 0:
-                    pointer = 0
-                    while pointer < len(data_in):
-                        byte = data_in[pointer]
-                        pointer += 1
-
-                        if self.kiss_framing:
-                            # Read loop for KISS framing
+                    if self.kiss_framing:
+                        # Read loop for KISS framing
+                        pointer = 0
+                        while pointer < len(data_in):
+                            byte = data_in[pointer]
+                            pointer += 1
                             if (in_frame and byte == KISS.FEND and command == KISS.CMD_DATA):
                                 in_frame = False
-                                self.processIncoming(data_buffer)
+                                self.process_incoming(data_buffer)
                             elif (byte == KISS.FEND):
                                 in_frame = True
                                 command = KISS.CMD_UNKNOWN
@@ -339,25 +365,26 @@ class TCPClientInterface(Interface):
                                             escape = False
                                         data_buffer = data_buffer+bytes([byte])
 
-                        else:
-                            # Read loop for HDLC framing
-                            if (in_frame and byte == HDLC.FLAG):
-                                in_frame = False
-                                self.processIncoming(data_buffer)
-                            elif (byte == HDLC.FLAG):
-                                in_frame = True
-                                data_buffer = b""
-                            elif (in_frame and len(data_buffer) < self.HW_MTU):
-                                if (byte == HDLC.ESC):
-                                    escape = True
+                    else:
+                        # Read loop for standard HDLC framing
+                        frame_buffer += data_in
+                        flags_remaining = True
+                        while flags_remaining:
+                            frame_start = frame_buffer.find(HDLC.FLAG)
+                            if frame_start != -1:
+                                frame_end = frame_buffer.find(HDLC.FLAG, frame_start+1)
+                                if frame_end != -1:
+                                    frame = frame_buffer[frame_start+1:frame_end]
+                                    frame = frame.replace(bytes([HDLC.ESC, HDLC.FLAG ^ HDLC.ESC_MASK]), bytes([HDLC.FLAG]))
+                                    frame = frame.replace(bytes([HDLC.ESC, HDLC.ESC  ^ HDLC.ESC_MASK]), bytes([HDLC.ESC]))
+                                    if len(frame) > RNS.Reticulum.HEADER_MINSIZE:
+                                        self.process_incoming(frame)
+                                    frame_buffer = frame_buffer[frame_end:]
                                 else:
-                                    if (escape):
-                                        if (byte == HDLC.FLAG ^ HDLC.ESC_MASK):
-                                            byte = HDLC.FLAG
-                                        if (byte == HDLC.ESC  ^ HDLC.ESC_MASK):
-                                            byte = HDLC.ESC
-                                        escape = False
-                                    data_buffer = data_buffer+bytes([byte])
+                                    flags_remaining = False
+                            else:
+                                flags_remaining = False
+
                 else:
                     self.online = False
                     if self.initiator and not self.detached:
@@ -394,7 +421,8 @@ class TCPClientInterface(Interface):
         self.IN = False
 
         if hasattr(self, "parent_interface") and self.parent_interface != None:
-            self.parent_interface.clients -= 1
+            while self in self.parent_interface.spawned_interfaces:
+                self.parent_interface.spawned_interfaces.remove(self)
 
         if self in RNS.Transport.interfaces:
             if not self.initiator:
@@ -402,31 +430,80 @@ class TCPClientInterface(Interface):
 
 
     def __str__(self):
-        return "TCPInterface["+str(self.name)+"/"+str(self.target_ip)+":"+str(self.target_port)+"]"
+        if ":" in self.target_ip:
+            ip_str = f"[{self.target_ip}]"
+        else:
+            ip_str = f"{self.target_ip}"
+
+        return "TCPInterface["+str(self.name)+"/"+ip_str+":"+str(self.target_port)+"]"
 
 
 class TCPServerInterface(Interface):
-    BITRATE_GUESS      = 10*1000*1000
+    BITRATE_GUESS     = 10_000_000
+    DEFAULT_IFAC_SIZE = 16
+    AUTOCONFIGURE_MTU = True
 
     @staticmethod
-    def get_address_for_if(name):
-        import RNS.vendor.ifaddr.niwrapper as netinfo
+    def get_address_for_if(name, bind_port, prefer_ipv6=False):
+        from RNS.Interfaces import netinfo
         ifaddr = netinfo.ifaddresses(name)
-        return ifaddr[netinfo.AF_INET][0]["addr"]
+        if len(ifaddr) < 1:
+            raise SystemError(f"No addresses available on specified kernel interface \"{name}\" for TCPServerInterface to bind to")
+
+        if (prefer_ipv6 or not netinfo.AF_INET in ifaddr) and netinfo.AF_INET6 in ifaddr:
+            bind_ip = ifaddr[netinfo.AF_INET6][0]["addr"]
+            if bind_ip.lower().startswith("fe80::"):
+                # We'll need to add the interface as scope for link-local addresses
+                return TCPServerInterface.get_address_for_host(f"{bind_ip}%{name}", bind_port, prefer_ipv6)
+            else:
+                return TCPServerInterface.get_address_for_host(bind_ip, bind_port, prefer_ipv6)
+        elif netinfo.AF_INET in ifaddr:
+            bind_ip = ifaddr[netinfo.AF_INET][0]["addr"]
+            return (bind_ip, bind_port)
+        else:
+            raise SystemError(f"No addresses available on specified kernel interface \"{name}\" for TCPServerInterface to bind to")
 
     @staticmethod
-    def get_broadcast_for_if(name):
-        import RNS.vendor.ifaddr.niwrapper as netinfo
-        ifaddr = netinfo.ifaddresses(name)
-        return ifaddr[netinfo.AF_INET][0]["broadcast"]
+    def get_address_for_host(name, bind_port, prefer_ipv6=False):
+        address_infos = socket.getaddrinfo(name, bind_port, proto=socket.IPPROTO_TCP)
+        address_info  = address_infos[0]
+        for entry in address_infos:
+            if prefer_ipv6 and entry[0] == socket.AF_INET6:
+                address_info = entry; break
+            elif not prefer_ipv6 and entry[0] == socket.AF_INET:
+                address_info = entry; break
 
-    def __init__(self, owner, name, device=None, bindip=None, bindport=None, i2p_tunneled=False):
+        if address_info[0] == socket.AF_INET6:
+            return (name, bind_port, address_info[4][2], address_info[4][3])
+        elif address_info[0] == socket.AF_INET:
+            return (name, bind_port)
+        else:
+            raise SystemError(f"No suitable kernel interface available for address \"{name}\" for TCPServerInterface to bind to")
+
+
+    @property
+    def clients(self):
+        return len(self.spawned_interfaces)
+
+    def __init__(self, owner, configuration):
         super().__init__()
 
-        self.HW_MTU = 1064
+        c            = Interface.get_config_obj(configuration)
+        name         = c["name"]
+        device       = c["device"] if "device" in c else None
+        port         = int(c["port"]) if "port" in c else None
+        bindip       = c["listen_ip"] if "listen_ip" in c else None
+        bindport     = int(c["listen_port"]) if "listen_port" in c else None
+        i2p_tunneled = c.as_bool("i2p_tunneled") if "i2p_tunneled" in c else False
+        prefer_ipv6  = c.as_bool("prefer_ipv6") if "prefer_ipv6" in c else False
+
+        if port != None:
+            bindport = port
+
+        self.HW_MTU = TCPInterface.HW_MTU
 
         self.online = False
-        self.clients = 0
+        self.spawned_interfaces = []
         
         self.IN  = True
         self.OUT = False
@@ -436,24 +513,41 @@ class TCPServerInterface(Interface):
         self.i2p_tunneled = i2p_tunneled
         self.mode         = RNS.Interfaces.Interface.Interface.MODE_FULL
 
-        if device != None:
-            bindip = TCPServerInterface.get_address_for_if(device)
-
-        if (bindip != None and bindport != None):
-            self.receives = True
-            self.bind_ip = bindip
+        if bindport == None:
+            raise SystemError(f"No TCP port configured for interface \"{name}\"")
+        else:
             self.bind_port = bindport
 
+        bind_address = None
+        if device != None:
+            bind_address = TCPServerInterface.get_address_for_if(device, self.bind_port, prefer_ipv6)
+        else:
+            if bindip == None:
+                raise SystemError(f"No TCP bind IP configured for interface \"{name}\"")
+            bind_address = TCPServerInterface.get_address_for_host(bindip, self.bind_port, prefer_ipv6)
+
+        if bind_address != None:
+            self.receives = True
+            self.bind_ip = bind_address[0]
+
             def handlerFactory(callback):
                 def createHandler(*args, **keys):
                     return TCPInterfaceHandler(callback, *args, **keys)
                 return createHandler
 
             self.owner = owner
-            address = (self.bind_ip, self.bind_port)
 
-            ThreadingTCPServer.allow_reuse_address = True
-            self.server = ThreadingTCPServer(address, handlerFactory(self.incoming_connection))
+            if len(bind_address) == 4:
+                try:
+                    ThreadingTCP6Server.allow_reuse_address = True
+                    self.server = ThreadingTCP6Server(bind_address, handlerFactory(self.incoming_connection))
+                except Exception as e:
+                    RNS.log(f"Error while binding IPv6 socket for interface, the contained exception was: {e}", RNS.LOG_ERROR)
+                    raise SystemError("Could not bind IPv6 socket for interface. Please check the specified \"listen_ip\" configuration option")
+            else:
+                ThreadingTCPServer.allow_reuse_address = True
+                self.server = ThreadingTCPServer(bind_address, handlerFactory(self.incoming_connection))
+                self.server.daemon_threads = True
 
             self.bitrate = TCPServerInterface.BITRATE_GUESS
 
@@ -463,17 +557,20 @@ class TCPServerInterface(Interface):
 
             self.online = True
 
+        else:
+            raise SystemError("Insufficient parameters to create TCP listener")
 
     def incoming_connection(self, handler):
         RNS.log("Accepting incoming TCP connection", RNS.LOG_VERBOSE)
-        interface_name = "Client on "+self.name
-        spawned_interface = TCPClientInterface(self.owner, interface_name, target_ip=None, target_port=None, connected_socket=handler.request, i2p_tunneled=self.i2p_tunneled)
+        spawned_configuration = {"name": "Client on "+self.name, "target_host": None, "target_port": None, "i2p_tunneled": self.i2p_tunneled}
+        spawned_interface = TCPClientInterface(self.owner, spawned_configuration, connected_socket=handler.request)
         spawned_interface.OUT = self.OUT
         spawned_interface.IN  = self.IN
         spawned_interface.target_ip = handler.client_address[0]
         spawned_interface.target_port = str(handler.client_address[1])
         spawned_interface.parent_interface = self
         spawned_interface.bitrate = self.bitrate
+        spawned_interface.optimise_mtu()
         
         spawned_interface.ifac_size = self.ifac_size
         spawned_interface.ifac_netname = self.ifac_netname
@@ -503,7 +600,9 @@ class TCPServerInterface(Interface):
         spawned_interface.online = True
         RNS.log("Spawned new TCPClient Interface: "+str(spawned_interface), RNS.LOG_VERBOSE)
         RNS.Transport.interfaces.append(spawned_interface)
-        self.clients += 1
+        while spawned_interface in self.spawned_interfaces:
+            self.spawned_interfaces.remove(spawned_interface)
+        self.spawned_interfaces.append(spawned_interface)
         spawned_interface.read_loop()
 
     def received_announce(self, from_spawned=False):
@@ -512,18 +611,19 @@ class TCPServerInterface(Interface):
     def sent_announce(self, from_spawned=False):
         if from_spawned: self.oa_freq_deque.append(time.time())
 
-    def processOutgoing(self, data):
+    def process_outgoing(self, data):
         pass
 
-
     def detach(self):
+        self.detached = True
+        self.online = False
         if self.server != None:
             if hasattr(self.server, "shutdown"):
                 if callable(self.server.shutdown):
                     try:
                         RNS.log("Detaching "+str(self), RNS.LOG_DEBUG)
                         self.server.shutdown()
-                        self.detached = True
+                        self.server.server_close()
                         self.server = None
 
                     except Exception as e:
@@ -531,7 +631,12 @@ class TCPServerInterface(Interface):
 
 
     def __str__(self):
-        return "TCPServerInterface["+self.name+"/"+self.bind_ip+":"+str(self.bind_port)+"]"
+        if ":" in self.bind_ip:
+            ip_str = f"[{self.bind_ip}]"
+        else:
+            ip_str = f"{self.bind_ip}"
+
+        return "TCPServerInterface["+self.name+"/"+ip_str+":"+str(self.bind_port)+"]"
 
 
 class TCPInterfaceHandler(socketserver.BaseRequestHandler):

RNS/Interfaces/UDPInterface.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,7 +20,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-from .Interface import Interface
+from RNS.Interfaces.Interface import Interface
 import socketserver
 import threading
 import socket
@@ -31,22 +31,38 @@ import RNS
 
 class UDPInterface(Interface):
     BITRATE_GUESS = 10*1000*1000
+    DEFAULT_IFAC_SIZE = 16
 
     @staticmethod
     def get_address_for_if(name):
-        import RNS.vendor.ifaddr.niwrapper as netinfo
+        from RNS.Interfaces import netinfo
         ifaddr = netinfo.ifaddresses(name)
         return ifaddr[netinfo.AF_INET][0]["addr"]
 
     @staticmethod
     def get_broadcast_for_if(name):
-        import RNS.vendor.ifaddr.niwrapper as netinfo
+        from RNS.Interfaces import netinfo
         ifaddr = netinfo.ifaddresses(name)
         return ifaddr[netinfo.AF_INET][0]["broadcast"]
 
-    def __init__(self, owner, name, device=None, bindip=None, bindport=None, forwardip=None, forwardport=None):
+    def __init__(self, owner, configuration):
         super().__init__()
 
+        c           = Interface.get_config_obj(configuration)
+        name        = c["name"]
+        device      = c["device"] if "device" in c else None
+        port        = int(c["port"]) if "port" in c else None
+        bindip      = c["listen_ip"] if "listen_ip" in c else None
+        bindport    = int(c["listen_port"]) if "listen_port" in c else None
+        forwardip   = c["forward_ip"] if "forward_ip" in c else None
+        forwardport = int(c["forward_port"]) if "forward_port" in c else None
+
+        if port != None:
+            if bindport == None:
+                bindport = port
+            if forwardport == None:
+                forwardport = port
+
         self.HW_MTU = 1064
 
         self.IN  = True
@@ -75,7 +91,7 @@ class UDPInterface(Interface):
             self.owner = owner
             address = (self.bind_ip, self.bind_port)
             socketserver.UDPServer.address_family = socket.AF_INET
-            self.server = socketserver.UDPServer(address, handlerFactory(self.processIncoming))
+            self.server = socketserver.UDPServer(address, handlerFactory(self.process_incoming))
 
             thread = threading.Thread(target=self.server.serve_forever)
             thread.daemon = True
@@ -89,11 +105,11 @@ class UDPInterface(Interface):
             self.forward_port = forwardport
 
 
-    def processIncoming(self, data):
+    def process_incoming(self, data):
         self.rxb += len(data)
         self.owner.inbound(data, self)
 
-    def processOutgoing(self,data):
+    def process_outgoing(self,data):
         try:
             udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
             udp_socket.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)

RNS/Interfaces/__init__.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -23,6 +23,10 @@
 import os
 import glob
 import RNS.Interfaces.Android
+import RNS.Interfaces.util
+import RNS.Interfaces.util.netinfo as netinfo
 
-modules = glob.glob(os.path.dirname(__file__)+"/*.py")
-__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]
+py_modules  = glob.glob(os.path.dirname(__file__)+"/*.py")
+pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
+modules     = py_modules+pyc_modules
+__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))

RNS/Interfaces/util/__init__.py

@@ -0,0 +1,7 @@
+import os
+import glob
+
+py_modules  = glob.glob(os.path.dirname(__file__)+"/*.py")
+pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
+modules     = py_modules+pyc_modules
+__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))

RNS/Interfaces/util/netinfo.py

@@ -0,0 +1,325 @@
+# MIT License
+#
+# Copyright (c) 2025 Mark Qvist
+# Copyright (c) 2014 Stefan C. Mueller
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import os
+import socket
+import ipaddress
+import platform
+import ctypes.util
+import collections
+from typing import List, Iterable, Optional, Tuple, Union
+
+AF_INET6 = socket.AF_INET6.value
+AF_INET = socket.AF_INET.value
+
+def interfaces() -> List[str]:
+    adapters = get_adapters(include_unconfigured=True)
+    return [a.name for a in adapters]
+
+def interface_names_to_indexes() -> dict:
+    adapters = get_adapters(include_unconfigured=True)
+    results = {}
+    for adapter in adapters:
+        results[adapter.name] = adapter.index
+    return results
+
+def interface_name_to_nice_name(ifname) -> str:
+    try:
+        adapters = get_adapters(include_unconfigured=True)
+        for adapter in adapters:
+            if adapter.name == ifname:
+                if hasattr(adapter, "nice_name"):
+                    return adapter.nice_name
+
+    except: return None
+    return None
+
+def ifaddresses(ifname) -> dict:
+    adapters = get_adapters(include_unconfigured=True)
+    ifa = {}
+    for a in adapters:
+        if a.name == ifname:
+            ipv4s = []
+            ipv6s = []
+            for ip in a.ips:
+                t = {}
+                if ip.is_IPv4:
+                    net = ipaddress.ip_network(str(ip.ip)+"/"+str(ip.network_prefix), strict=False)
+                    t["addr"] = ip.ip
+                    t["prefix"] = ip.network_prefix
+                    t["broadcast"] = str(net.broadcast_address)
+                    ipv4s.append(t)
+                if ip.is_IPv6:
+                    t["addr"] = ip.ip[0]
+                    ipv6s.append(t)
+
+            if len(ipv4s) > 0: ifa[AF_INET] = ipv4s
+            if len(ipv6s) > 0: ifa[AF_INET6] = ipv6s
+
+    return ifa
+
+def get_adapters(include_unconfigured=False):
+    if os.name == "posix": return _get_adapters_posix(include_unconfigured=include_unconfigured)
+    elif os.name == "nt":  return _get_adapters_win(include_unconfigured=include_unconfigured)
+    else: raise RuntimeError(f"Unsupported Operating System: {os.name}")
+
+class Adapter(object):
+    def __init__(self, name: str, nice_name: str, ips: List["IP"], index: Optional[int] = None) -> None:
+        self.name = name
+        self.nice_name = nice_name
+        self.ips = ips
+        self.index = index
+
+    def __repr__(self) -> str:
+        return "Adapter(name={name}, nice_name={nice_name}, ips={ips}, index={index})".format(
+            name=repr(self.name), nice_name=repr(self.nice_name), ips=repr(self.ips), index=repr(self.index))
+
+_IPv4Address = str
+_IPv6Address = Tuple[str, int, int]
+class IP(object):
+    def __init__(self, ip: Union[_IPv4Address, _IPv6Address], network_prefix: int, nice_name: str) -> None:
+        self.ip = ip
+        self.network_prefix = network_prefix
+        self.nice_name = nice_name
+
+    @property
+    def is_IPv4(self) -> bool: return not isinstance(self.ip, tuple)
+
+    @property
+    def is_IPv6(self) -> bool: return isinstance(self.ip, tuple)
+
+    def __repr__(self) -> str:
+        return "IP(ip={ip}, network_prefix={network_prefix}, nice_name={nice_name})".format(ip=repr(self.ip), network_prefix=repr(self.network_prefix), nice_name=repr(self.nice_name))
+
+if platform.system() == "Darwin" or "BSD" in platform.system():
+    class sockaddr(ctypes.Structure):
+        _fields_ = [
+            ("sa_len", ctypes.c_uint8),
+            ("sa_familiy", ctypes.c_uint8),
+            ("sa_data", ctypes.c_uint8 * 14)]
+
+    class sockaddr_in(ctypes.Structure):
+        _fields_ = [
+            ("sa_len", ctypes.c_uint8),
+            ("sa_familiy", ctypes.c_uint8),
+            ("sin_port", ctypes.c_uint16),
+            ("sin_addr", ctypes.c_uint8 * 4),
+            ("sin_zero", ctypes.c_uint8 * 8)]
+
+    class sockaddr_in6(ctypes.Structure):
+        _fields_ = [
+            ("sa_len", ctypes.c_uint8),
+            ("sa_familiy", ctypes.c_uint8),
+            ("sin6_port", ctypes.c_uint16),
+            ("sin6_flowinfo", ctypes.c_uint32),
+            ("sin6_addr", ctypes.c_uint8 * 16),
+            ("sin6_scope_id", ctypes.c_uint32)]
+
+else:
+    class sockaddr(ctypes.Structure):  # type: ignore
+        _fields_ = [("sa_familiy", ctypes.c_uint16), ("sa_data", ctypes.c_uint8 * 14)]
+
+    class sockaddr_in(ctypes.Structure):  # type: ignore
+        _fields_ = [
+            ("sin_familiy", ctypes.c_uint16),
+            ("sin_port", ctypes.c_uint16),
+            ("sin_addr", ctypes.c_uint8 * 4),
+            ("sin_zero", ctypes.c_uint8 * 8)]
+
+    class sockaddr_in6(ctypes.Structure):  # type: ignore
+        _fields_ = [
+            ("sin6_familiy", ctypes.c_uint16),
+            ("sin6_port", ctypes.c_uint16),
+            ("sin6_flowinfo", ctypes.c_uint32),
+            ("sin6_addr", ctypes.c_uint8 * 16),
+            ("sin6_scope_id", ctypes.c_uint32)]
+
+def sockaddr_to_ip(sockaddr_ptr: "ctypes.pointer[sockaddr]") -> Optional[Union[_IPv4Address, _IPv6Address]]:
+    if sockaddr_ptr:
+        if sockaddr_ptr[0].sa_familiy == socket.AF_INET:
+            ipv4 = ctypes.cast(sockaddr_ptr, ctypes.POINTER(sockaddr_in))
+            ippacked = bytes(bytearray(ipv4[0].sin_addr))
+            ip = str(ipaddress.ip_address(ippacked))
+            return ip
+        elif sockaddr_ptr[0].sa_familiy == socket.AF_INET6:
+            ipv6 = ctypes.cast(sockaddr_ptr, ctypes.POINTER(sockaddr_in6))
+            flowinfo = ipv6[0].sin6_flowinfo
+            ippacked = bytes(bytearray(ipv6[0].sin6_addr))
+            ip = str(ipaddress.ip_address(ippacked))
+            scope_id = ipv6[0].sin6_scope_id
+            return (ip, flowinfo, scope_id)
+    return None
+
+
+def ipv6_prefixlength(address: ipaddress.IPv6Address) -> int:
+    prefix_length = 0
+    for i in range(address.max_prefixlen):
+        if int(address) >> i & 1: prefix_length = prefix_length + 1
+    return prefix_length
+
+if os.name == "posix":
+    class ifaddrs(ctypes.Structure): pass
+    ifaddrs._fields_ = [
+        ("ifa_next", ctypes.POINTER(ifaddrs)),
+        ("ifa_name", ctypes.c_char_p),
+        ("ifa_flags", ctypes.c_uint),
+        ("ifa_addr", ctypes.POINTER(sockaddr)),
+        ("ifa_netmask", ctypes.POINTER(sockaddr)),]
+
+    libc = ctypes.CDLL(ctypes.util.find_library("socket" if os.uname()[0] == "SunOS" else "c"), use_errno=True) # type: ignore
+
+    def _get_adapters_posix(include_unconfigured: bool = False) -> Iterable[Adapter]:
+        addr0 = addr = ctypes.POINTER(ifaddrs)()
+        retval = libc.getifaddrs(ctypes.byref(addr))
+        if retval != 0:
+            eno = ctypes.get_errno()
+            raise OSError(eno, os.strerror(eno))
+
+        ips = collections.OrderedDict()
+
+        def add_ip(adapter_name: str, ip: Optional[IP]) -> None:
+            if adapter_name not in ips:
+                index = None  # type: Optional[int]
+                try:
+                    index = socket.if_nametoindex(adapter_name) # type: ignore
+                except (OSError, AttributeError): pass
+                ips[adapter_name] = Adapter(adapter_name, adapter_name, [], index=index)
+            if ip is not None:
+                ips[adapter_name].ips.append(ip)
+
+        while addr:
+            name = addr[0].ifa_name.decode(encoding="UTF-8")
+            ip_addr = sockaddr_to_ip(addr[0].ifa_addr)
+            if ip_addr:
+                if addr[0].ifa_netmask and not addr[0].ifa_netmask[0].sa_familiy:
+                    addr[0].ifa_netmask[0].sa_familiy = addr[0].ifa_addr[0].sa_familiy
+                netmask = sockaddr_to_ip(addr[0].ifa_netmask)
+                if isinstance(netmask, tuple):
+                    netmaskStr = str(netmask[0])
+                    prefixlen = ipv6_prefixlength(ipaddress.IPv6Address(netmaskStr))
+                else:
+                    assert netmask is not None, f"sockaddr_to_ip({addr[0].ifa_netmask}) returned None"
+                    netmaskStr = str("0.0.0.0/" + netmask)
+                    prefixlen = ipaddress.IPv4Network(netmaskStr).prefixlen
+                ip = IP(ip_addr, prefixlen, name)
+                add_ip(name, ip)
+            else:
+                if include_unconfigured:
+                    add_ip(name, None)
+            addr = addr[0].ifa_next
+
+        libc.freeifaddrs(addr0)
+        return ips.values()
+
+elif os.name == "nt":
+    from ctypes import wintypes
+    NO_ERROR = 0
+    ERROR_BUFFER_OVERFLOW = 111
+    MAX_ADAPTER_NAME_LENGTH = 256
+    MAX_ADAPTER_DESCRIPTION_LENGTH = 128
+    MAX_ADAPTER_ADDRESS_LENGTH = 8
+    AF_UNSPEC = 0
+
+    class SOCKET_ADDRESS(ctypes.Structure): _fields_ = [("lpSockaddr", ctypes.POINTER(sockaddr)), ("iSockaddrLength", wintypes.INT)]
+    class IP_ADAPTER_UNICAST_ADDRESS(ctypes.Structure): pass
+    IP_ADAPTER_UNICAST_ADDRESS._fields_ = [
+        ("Length", wintypes.ULONG),
+        ("Flags", wintypes.DWORD),
+        ("Next", ctypes.POINTER(IP_ADAPTER_UNICAST_ADDRESS)),
+        ("Address", SOCKET_ADDRESS),
+        ("PrefixOrigin", ctypes.c_uint),
+        ("SuffixOrigin", ctypes.c_uint),
+        ("DadState", ctypes.c_uint),
+        ("ValidLifetime", wintypes.ULONG),
+        ("PreferredLifetime", wintypes.ULONG),
+        ("LeaseLifetime", wintypes.ULONG),
+        ("OnLinkPrefixLength", ctypes.c_uint8)]
+
+    class IP_ADAPTER_ADDRESSES(ctypes.Structure): pass
+    IP_ADAPTER_ADDRESSES._fields_ = [
+        ("Length", wintypes.ULONG),
+        ("IfIndex", wintypes.DWORD),
+        ("Next", ctypes.POINTER(IP_ADAPTER_ADDRESSES)),
+        ("AdapterName", ctypes.c_char_p),
+        ("FirstUnicastAddress", ctypes.POINTER(IP_ADAPTER_UNICAST_ADDRESS)),
+        ("FirstAnycastAddress", ctypes.c_void_p),
+        ("FirstMulticastAddress", ctypes.c_void_p),
+        ("FirstDnsServerAddress", ctypes.c_void_p),
+        ("DnsSuffix", ctypes.c_wchar_p),
+        ("Description", ctypes.c_wchar_p),
+        ("FriendlyName", ctypes.c_wchar_p)]
+
+    iphlpapi = ctypes.windll.LoadLibrary("Iphlpapi") # type: ignore
+
+    def _enumerate_interfaces_of_adapter_win(nice_name: str, address: IP_ADAPTER_UNICAST_ADDRESS) -> Iterable[IP]:
+        # Iterate through linked list and fill list
+        addresses = [] # type: List[IP_ADAPTER_UNICAST_ADDRESS]
+        while True:
+            addresses.append(address)
+            if not address.Next: break
+            address = address.Next[0]
+
+        for address in addresses:
+            ip = sockaddr_to_ip(address.Address.lpSockaddr)
+            assert ip is not None, f"sockaddr_to_ip({address.Address.lpSockaddr}) returned None"
+            network_prefix = address.OnLinkPrefixLength
+            yield IP(ip, network_prefix, nice_name)
+
+    def _get_adapters_win(include_unconfigured: bool = False) -> Iterable[Adapter]:
+        addressbuffersize = wintypes.ULONG(15 * 1024)
+        retval = ERROR_BUFFER_OVERFLOW
+        while retval == ERROR_BUFFER_OVERFLOW:
+            addressbuffer = ctypes.create_string_buffer(addressbuffersize.value)
+            retval = iphlpapi.GetAdaptersAddresses(
+                wintypes.ULONG(AF_UNSPEC),
+                wintypes.ULONG(0),
+                None,
+                ctypes.byref(addressbuffer),
+                ctypes.byref(addressbuffersize))
+
+        if retval != NO_ERROR:
+            raise ctypes.WinError() # type: ignore
+
+        # Iterate through adapters and fill array
+        address_infos = []  # type: List[IP_ADAPTER_ADDRESSES]
+        address_info = IP_ADAPTER_ADDRESSES.from_buffer(addressbuffer)
+        while True:
+            address_infos.append(address_info)
+            if not address_info.Next: break
+            address_info = address_info.Next[0]
+
+        # Iterate through unicast addresses
+        result = [] # type: List[Adapter]
+        for adapter_info in address_infos:
+            name = adapter_info.AdapterName.decode()
+            nice_name = adapter_info.Description
+            index = adapter_info.IfIndex
+
+            if adapter_info.FirstUnicastAddress:
+                ips = _enumerate_interfaces_of_adapter_win(adapter_info.FriendlyName, adapter_info.FirstUnicastAddress[0])
+                ips = list(ips)
+                result.append(Adapter(name, nice_name, ips, index=index))
+            
+            elif include_unconfigured: result.append(Adapter(name, nice_name, [], index=index))
+
+        return result

RNS/Link.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2024 Mark Qvist / unsigned.io and contributors.
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -21,18 +21,18 @@
 # SOFTWARE.
 
 from RNS.Cryptography import X25519PrivateKey, X25519PublicKey, Ed25519PrivateKey, Ed25519PublicKey
-from RNS.Cryptography import Fernet
+from RNS.Cryptography import Token
 from RNS.Channel import Channel, LinkChannelOutlet
 
 from time import sleep
 from .vendor import umsgpack as umsgpack
 import threading
 import inspect
+import struct
 import math
 import time
 import RNS
 
-
 class LinkCallbacks:
     def __init__(self):
         self.link_established = None
@@ -61,15 +61,16 @@ class Link:
     ECPUBSIZE         = 32+32
     KEYSIZE           = 32
 
-    MDU = math.floor((RNS.Reticulum.MTU-RNS.Reticulum.IFAC_MIN_SIZE-RNS.Reticulum.HEADER_MINSIZE-RNS.Identity.FERNET_OVERHEAD)/RNS.Identity.AES128_BLOCKSIZE)*RNS.Identity.AES128_BLOCKSIZE - 1
+    MDU = math.floor((RNS.Reticulum.MTU-RNS.Reticulum.IFAC_MIN_SIZE-RNS.Reticulum.HEADER_MINSIZE-RNS.Identity.TOKEN_OVERHEAD)/RNS.Identity.AES128_BLOCKSIZE)*RNS.Identity.AES128_BLOCKSIZE - 1
 
     ESTABLISHMENT_TIMEOUT_PER_HOP = RNS.Reticulum.DEFAULT_PER_HOP_TIMEOUT
     """
     Timeout for link establishment in seconds per hop to destination.
     """
 
-    TRAFFIC_TIMEOUT_MIN_MS = 5
-    TRAFFIC_TIMEOUT_FACTOR = 6
+    LINK_MTU_SIZE            = 3
+    TRAFFIC_TIMEOUT_MIN_MS   = 5
+    TRAFFIC_TIMEOUT_FACTOR   = 6
     KEEPALIVE_TIMEOUT_FACTOR = 4
     """
     RTT timeout factor used in link timeout calculation.
@@ -106,16 +107,46 @@ class Link:
     ACCEPT_ALL  = 0x02
     resource_strategies = [ACCEPT_NONE, ACCEPT_APP, ACCEPT_ALL]
 
+    @staticmethod
+    def mtu_bytes(mtu):
+        return struct.pack(">I", mtu & 0xFFFFFF)[1:]
+
+    @staticmethod
+    def mtu_from_lr_packet(packet):
+        if len(packet.data) == Link.ECPUBSIZE+Link.LINK_MTU_SIZE:
+            return (packet.data[Link.ECPUBSIZE] << 16) + (packet.data[Link.ECPUBSIZE+1] << 8) + (packet.data[Link.ECPUBSIZE+2])
+        else:
+            return None
+
+    @staticmethod
+    def mtu_from_lp_packet(packet):
+        if len(packet.data) == RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2+Link.LINK_MTU_SIZE:
+            mtu_bytes = packet.data[RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2:RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2+Link.LINK_MTU_SIZE]
+            return (mtu_bytes[0] << 16) + (mtu_bytes[1] << 8) + (mtu_bytes[2])
+        else:
+            return None
+
     @staticmethod
     def validate_request(owner, data, packet):
-        if len(data) == (Link.ECPUBSIZE):
+        if len(data) == Link.ECPUBSIZE or len(data) == Link.ECPUBSIZE+Link.LINK_MTU_SIZE:
             try:
                 link = Link(owner = owner, peer_pub_bytes=data[:Link.ECPUBSIZE//2], peer_sig_pub_bytes=data[Link.ECPUBSIZE//2:Link.ECPUBSIZE])
                 link.set_link_id(packet)
+
+                if len(data) == Link.ECPUBSIZE+Link.LINK_MTU_SIZE:
+                    RNS.log("Link request includes MTU signalling", RNS.LOG_DEBUG) # TODO: Remove debug
+                    try:
+                        link.mtu = Link.mtu_from_lr_packet(packet) or Reticulum.MTU
+                    except Exception as e:
+                        RNS.trace_exception(e)
+                        link.mtu = RNS.Reticulum.MTU
+
+                link.update_mdu()
                 link.destination = packet.destination
                 link.establishment_timeout = Link.ESTABLISHMENT_TIMEOUT_PER_HOP * max(1, packet.hops) + Link.KEEPALIVE
                 link.establishment_cost += len(packet.raw)
-                RNS.log("Validating link request "+RNS.prettyhexrep(link.link_id), RNS.LOG_VERBOSE)
+                RNS.log(f"Validating link request {RNS.prettyhexrep(link.link_id)}", RNS.LOG_DEBUG)
+                RNS.log(f"Link MTU configured to {RNS.prettysize(link.mtu)}", RNS.LOG_EXTREME)
                 RNS.log(f"Establishment timeout is {RNS.prettytime(link.establishment_timeout)} for incoming link request "+RNS.prettyhexrep(link.link_id), RNS.LOG_EXTREME)
                 link.handshake()
                 link.attached_interface = packet.receiving_interface
@@ -123,18 +154,18 @@ class Link:
                 link.request_time = time.time()
                 RNS.Transport.register_link(link)
                 link.last_inbound = time.time()
+                link.__update_phy_stats(packet, force_update=True)
                 link.start_watchdog()
-                
-                RNS.log("Incoming link request "+str(link)+" accepted", RNS.LOG_DEBUG)
+
+                RNS.log("Incoming link request "+str(link)+" accepted on "+str(link.attached_interface), RNS.LOG_DEBUG)
                 return link
 
             except Exception as e:
-                RNS.log("Validating link request failed", RNS.LOG_VERBOSE)
-                RNS.log("exc: "+str(e))
+                RNS.log(f"Validating link request failed: {e}", RNS.LOG_VERBOSE)
                 return None
 
         else:
-            RNS.log("Invalid link request payload size, dropping request", RNS.LOG_DEBUG)
+            RNS.log(f"Invalid link request payload size of {len(data)} bytes, dropping request", RNS.LOG_DEBUG)
             return None
 
 
@@ -142,10 +173,14 @@ class Link:
         if destination != None and destination.type != RNS.Destination.SINGLE:
             raise TypeError("Links can only be established to the \"single\" destination type")
         self.rtt = None
+        self.mtu = RNS.Reticulum.MTU
         self.establishment_cost = 0
         self.establishment_rate = None
+        self.expected_rate = None
         self.callbacks = LinkCallbacks()
         self.resource_strategy = Link.ACCEPT_NONE
+        self.last_resource_window = None
+        self.last_resource_eifr = None
         self.outgoing_resources = []
         self.incoming_resources = []
         self.pending_requests   = []
@@ -188,7 +223,7 @@ class Link:
             self.prv     = X25519PrivateKey.generate()
             self.sig_prv = Ed25519PrivateKey.generate()
 
-        self.fernet  = None
+        self.token  = None
         
         self.pub = self.prv.public_key()
         self.pub_bytes = self.pub.public_bytes()
@@ -208,8 +243,13 @@ class Link:
         if closed_callback != None:
             self.set_link_closed_callback(closed_callback)
 
-        if (self.initiator):
-            self.request_data = self.pub_bytes+self.sig_pub_bytes
+        if self.initiator:
+            link_mtu = b""
+            nh_hw_mtu = RNS.Transport.next_hop_interface_hw_mtu(destination.hash)
+            if RNS.Reticulum.link_mtu_discovery() and nh_hw_mtu:
+                link_mtu = Link.mtu_bytes(nh_hw_mtu)
+                RNS.log(f"Signalling link MTU of {RNS.prettysize(nh_hw_mtu)} for link", RNS.LOG_DEBUG) # TODO: Remove debug
+            self.request_data = self.pub_bytes+self.sig_pub_bytes+link_mtu
             self.packet = RNS.Packet(destination, self.request_data, packet_type=RNS.Packet.LINKREQUEST)
             self.packet.pack()
             self.establishment_cost += len(self.packet.raw)
@@ -233,8 +273,17 @@ class Link:
         if not hasattr(self.peer_pub, "curve"):
             self.peer_pub.curve = Link.CURVE
 
+    @staticmethod
+    def link_id_from_lr_packet(packet):
+        hashable_part = packet.get_hashable_part()
+        if len(packet.data) > Link.ECPUBSIZE:
+            diff = len(packet.data) - Link.ECPUBSIZE
+            hashable_part = hashable_part[:-diff]
+
+        return RNS.Identity.truncated_hash(hashable_part)
+
     def set_link_id(self, packet):
-        self.link_id = packet.getTruncatedHash()
+        self.link_id = Link.link_id_from_lr_packet(packet)
         self.hash = self.link_id
 
     def handshake(self):
@@ -253,10 +302,14 @@ class Link:
 
 
     def prove(self):
-        signed_data = self.link_id+self.pub_bytes+self.sig_pub_bytes
+        mtu_bytes = b""
+        if self.mtu != RNS.Reticulum.MTU:
+            mtu_bytes = Link.mtu_bytes(self.mtu)
+
+        signed_data = self.link_id+self.pub_bytes+self.sig_pub_bytes+mtu_bytes
         signature = self.owner.identity.sign(signed_data)
 
-        proof_data = signature+self.pub_bytes
+        proof_data = signature+self.pub_bytes+mtu_bytes
         proof = RNS.Packet(self, proof_data, packet_type=RNS.Packet.PROOF, context=RNS.Packet.LRPROOF)
         proof.send()
         self.establishment_cost += len(proof.raw)
@@ -279,6 +332,14 @@ class Link:
     def validate_proof(self, packet):
         try:
             if self.status == Link.PENDING:
+                mtu_bytes = b""
+                confirmed_mtu = None
+                if len(packet.data) == RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2+Link.LINK_MTU_SIZE:
+                    confirmed_mtu = Link.mtu_from_lp_packet(packet)
+                    mtu_bytes = Link.mtu_bytes(confirmed_mtu)
+                    packet.data = packet.data[:RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2]
+                    RNS.log(f"Destination confirmed link MTU of {RNS.prettysize(confirmed_mtu)}", RNS.LOG_DEBUG) # TODO: Remove debug
+
                 if self.initiator and len(packet.data) == RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2:
                     peer_pub_bytes = packet.data[RNS.Identity.SIGLENGTH//8:RNS.Identity.SIGLENGTH//8+Link.ECPUBSIZE//2]
                     peer_sig_pub_bytes = self.destination.identity.get_public_key()[Link.ECPUBSIZE//2:Link.ECPUBSIZE]
@@ -286,7 +347,7 @@ class Link:
                     self.handshake()
 
                     self.establishment_cost += len(packet.raw)
-                    signed_data = self.link_id+self.peer_pub_bytes+self.peer_sig_pub_bytes
+                    signed_data = self.link_id+self.peer_pub_bytes+self.peer_sig_pub_bytes+mtu_bytes
                     signature = packet.data[:RNS.Identity.SIGLENGTH//8]
                     
                     if self.destination.identity.validate(signature, signed_data):
@@ -296,11 +357,13 @@ class Link:
                         self.rtt = time.time() - self.request_time
                         self.attached_interface = packet.receiving_interface
                         self.__remote_identity = self.destination.identity
+                        self.mtu = confirmed_mtu or RNS.Reticulum.MTU
+                        self.update_mdu()
                         self.status = Link.ACTIVE
                         self.activated_at = time.time()
                         self.last_proof = self.activated_at
                         RNS.Transport.activate_link(self)
-                        RNS.log("Link "+str(self)+" established with "+str(self.destination)+", RTT is "+str(round(self.rtt, 3))+"s", RNS.LOG_VERBOSE)
+                        RNS.log("Link "+str(self)+" established with "+str(self.destination)+", RTT is "+str(round(self.rtt, 3))+"s", RNS.LOG_DEBUG)
                         
                         if self.rtt != None and self.establishment_cost != None and self.rtt > 0 and self.establishment_cost > 0:
                             self.establishment_rate = self.establishment_cost/self.rtt
@@ -309,6 +372,7 @@ class Link:
                         rtt_packet = RNS.Packet(self, rtt_data, context=RNS.Packet.LRRTT)
                         rtt_packet.send()
                         self.had_outbound()
+                        self.__update_phy_stats(packet)
 
                         if self.callbacks.link_established != None:
                             thread = threading.Thread(target=self.callbacks.link_established, args=(self,))
@@ -360,7 +424,7 @@ class Link:
         if timeout == None:
             timeout = self.rtt * self.traffic_timeout_factor + RNS.Resource.RESPONSE_MAX_GRACE_TIME*1.125
 
-        if len(packed_request) <= Link.MDU:
+        if len(packed_request) <= self.mdu:
             request_packet   = RNS.Packet(self, packed_request, RNS.Packet.DATA, context = RNS.Packet.REQUEST)
             packet_receipt   = request_packet.send()
 
@@ -394,6 +458,10 @@ class Link:
             )
 
 
+    def update_mdu(self):
+        self.mdu = self.mtu - RNS.Reticulum.HEADER_MAXSIZE - RNS.Reticulum.IFAC_MIN_SIZE
+        self.mdu = math.floor((self.mtu-RNS.Reticulum.IFAC_MIN_SIZE-RNS.Reticulum.HEADER_MINSIZE-RNS.Identity.TOKEN_OVERHEAD)/RNS.Identity.AES128_BLOCKSIZE)*RNS.Identity.AES128_BLOCKSIZE - 1
+
     def rtt_packet(self, packet):
         try:
             measured_rtt = time.time() - self.request_time
@@ -435,19 +503,28 @@ class Link:
         """
         :returns: The physical layer *Received Signal Strength Indication* if available, otherwise ``None``. Physical layer statistics must be enabled on the link for this method to return a value.
         """
-        return self.rssi
+        if self.__track_phy_stats:
+            return self.rssi
+        else:
+            return None
 
     def get_snr(self):
         """
         :returns: The physical layer *Signal-to-Noise Ratio* if available, otherwise ``None``. Physical layer statistics must be enabled on the link for this method to return a value.
         """
-        return self.rssi
+        if self.__track_phy_stats:
+            return self.snr
+        else:
+            return None
 
     def get_q(self):
         """
         :returns: The physical layer *Link Quality* if available, otherwise ``None``. Physical layer statistics must be enabled on the link for this method to return a value.
         """
-        return self.rssi
+        if self.__track_phy_stats:
+            return self.q
+        else:
+            return None
 
     def get_establishment_rate(self):
         """
@@ -458,6 +535,33 @@ class Link:
         else:
             return None
 
+    def get_mtu(self):
+        """
+        :returns: The MTU of an established link.
+        """
+        if self.status == Link.ACTIVE:
+            return self.mtu
+        else:
+            return None
+
+    def get_mdu(self):
+        """
+        :returns: The packet MDU of an established link.
+        """
+        if self.status == Link.ACTIVE:
+            return self.mdu
+        else:
+            return None
+
+    def get_expected_rate(self):
+        """
+        :returns: The packet expected in-flight data rate of an established link.
+        """
+        if self.status == Link.ACTIVE:
+            return self.expected_rate
+        else:
+            return None
+
     def get_salt(self):
         return self.link_id
 
@@ -640,9 +744,14 @@ class Link:
 
                 sleep(sleep_time)
 
+                if not self.__track_phy_stats:
+                    self.rssi = None
+                    self.snr  = None
+                    self.q    = None
 
-    def __update_phy_stats(self, packet, query_shared = True):
-        if self.__track_phy_stats:
+
+    def __update_phy_stats(self, packet, query_shared = True, force_update = False):
+        if self.__track_phy_stats or force_update:
             if query_shared:
                 reticulum = RNS.Reticulum.get_instance()
                 if packet.rssi == None: packet.rssi = reticulum.get_packet_rssi(packet.packet_hash)
@@ -694,7 +803,7 @@ class Link:
                     if response != None:
                         packed_response = umsgpack.packb([request_id, response])
 
-                        if len(packed_response) <= Link.MDU:
+                        if len(packed_response) <= self.mdu:
                             RNS.Packet(self, packed_response, RNS.Packet.DATA, context = RNS.Packet.RESPONSE).send()
                         else:
                             response_resource = RNS.Resource(packed_response, self, request_id = request_id, is_response = True)
@@ -762,7 +871,7 @@ class Link:
         self.watchdog_lock = True
         if not self.status == Link.CLOSED and not (self.initiator and packet.context == RNS.Packet.KEEPALIVE and packet.data == bytes([0xFF])):
             if packet.receiving_interface != self.attached_interface:
-                RNS.log("Link-associated packet received on unexpected interface! Someone might be trying to manipulate your communication!", RNS.LOG_ERROR)
+                RNS.log(f"Link-associated packet received on unexpected interface {packet.receiving_interface} instead of {self.attached_interface}! Someone might be trying to manipulate your communication!", RNS.LOG_ERROR)
             else:
                 self.last_inbound = time.time()
                 if packet.context != RNS.Packet.KEEPALIVE:
@@ -778,6 +887,8 @@ class Link:
                         plaintext = self.decrypt(packet.data)
                         packet.ratchet_id = self.link_id
                         if plaintext != None:
+                            self.__update_phy_stats(packet, query_shared=True)
+
                             if self.callbacks.packet != None:
                                 thread = threading.Thread(target=self.callbacks.packet, args=(plaintext, packet))
                                 thread.daemon = True
@@ -785,19 +896,15 @@ class Link:
                             
                             if self.destination.proof_strategy == RNS.Destination.PROVE_ALL:
                                 packet.prove()
-                                should_query = True
 
                             elif self.destination.proof_strategy == RNS.Destination.PROVE_APP:
                                 if self.destination.callbacks.proof_requested:
                                     try:
                                         if self.destination.callbacks.proof_requested(packet):
                                             packet.prove()
-                                            should_query = True
                                     except Exception as e:
                                         RNS.log("Error while executing proof request callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
 
-                            self.__update_phy_stats(packet, query_shared=should_query)
-
                     elif packet.context == RNS.Packet.LINKIDENTIFY:
                         plaintext = self.decrypt(packet.data)
                         if plaintext != None:
@@ -882,6 +989,8 @@ class Link:
                                         resource_advertisement.link = self
                                         if self.callbacks.resource(resource_advertisement):
                                             RNS.Resource.accept(packet, self.callbacks.resource_concluded)
+                                        else:
+                                            RNS.Resource.reject(packet)
                                     except Exception as e:
                                         RNS.log("Error while executing resource accept callback from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
                             elif self.resource_strategy == Link.ACCEPT_ALL:
@@ -927,6 +1036,15 @@ class Link:
                                 if resource_hash == resource.hash:
                                     resource.cancel()
 
+                    elif packet.context == RNS.Packet.RESOURCE_RCL:
+                        plaintext = self.decrypt(packet.data)
+                        if plaintext != None:
+                            self.__update_phy_stats(packet)
+                            resource_hash = plaintext[:RNS.Identity.HASHLENGTH//8]
+                            for resource in self.outgoing_resources:
+                                if resource_hash == resource.hash:
+                                    resource._rejected()
+
                     elif packet.context == RNS.Packet.KEEPALIVE:
                         if not self.initiator and packet.data == bytes([0xFF]):
                             keepalive_packet = RNS.Packet(self, bytes([0xFE]), context=RNS.Packet.KEEPALIVE)
@@ -966,14 +1084,14 @@ class Link:
 
     def encrypt(self, plaintext):
         try:
-            if not self.fernet:
+            if not self.token:
                 try:
-                    self.fernet = Fernet(self.derived_key)
+                    self.token = Token(self.derived_key)
                 except Exception as e:
-                    RNS.log("Could not instantiate Fernet while performin encryption on link "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+                    RNS.log("Could not instantiate token while performing encryption on link "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
                     raise e
 
-            return self.fernet.encrypt(plaintext)
+            return self.token.encrypt(plaintext)
 
         except Exception as e:
             RNS.log("Encryption on link "+str(self)+" failed. The contained exception was: "+str(e), RNS.LOG_ERROR)
@@ -982,10 +1100,10 @@ class Link:
 
     def decrypt(self, ciphertext):
         try:
-            if not self.fernet:
-                self.fernet = Fernet(self.derived_key)
+            if not self.token:
+                self.token = Token(self.derived_key)
                 
-            return self.fernet.decrypt(ciphertext)
+            return self.token.decrypt(ciphertext)
 
         except Exception as e:
             RNS.log("Decryption failed on link "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
@@ -1062,10 +1180,15 @@ class Link:
         self.callbacks.remote_identified = callback
 
     def resource_concluded(self, resource):
+        concluded_at = time.time()
         if resource in self.incoming_resources:
+            self.last_resource_window = resource.window
+            self.last_resource_eifr = resource.eifr
             self.incoming_resources.remove(resource)
+            self.expected_rate = (resource.size*8)/(max(concluded_at-resource.started_transferring, 0.0001))
         if resource in self.outgoing_resources:
             self.outgoing_resources.remove(resource)
+            self.expected_rate = (resource.size*8)/(max(concluded_at-resource.started_transferring, 0.0001))
 
     def set_resource_strategy(self, resource_strategy):
         """
@@ -1092,6 +1215,12 @@ class Link:
 
         return False
 
+    def get_last_resource_window(self):
+        return self.last_resource_window
+
+    def get_last_resource_eifr(self):
+        return self.last_resource_eifr
+
     def cancel_outgoing_resource(self, resource):
         if resource in self.outgoing_resources:
             self.outgoing_resources.remove(resource)

RNS/Packet.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2024 Mark Qvist / unsigned.io and contributors.
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -95,7 +95,7 @@ class Packet:
     # With an MTU of 500, the maximum of data we can
     # send in a single encrypted packet is given by
     # the below calculation; 383 bytes.
-    ENCRYPTED_MDU  = math.floor((RNS.Reticulum.MDU-RNS.Identity.FERNET_OVERHEAD-RNS.Identity.KEYSIZE//16)/RNS.Identity.AES128_BLOCKSIZE)*RNS.Identity.AES128_BLOCKSIZE - 1
+    ENCRYPTED_MDU  = math.floor((RNS.Reticulum.MDU-RNS.Identity.TOKEN_OVERHEAD-RNS.Identity.KEYSIZE//16)/RNS.Identity.AES128_BLOCKSIZE)*RNS.Identity.AES128_BLOCKSIZE - 1
     """
     The maximum size of the payload data in a single encrypted packet 
     """
@@ -106,6 +106,11 @@ class Packet:
 
     TIMEOUT_PER_HOP = RNS.Reticulum.DEFAULT_PER_HOP_TIMEOUT
 
+    __slots__  = "hops", "header", "header_type", "packet_type", "transport_type", "context", "context_flag", "destination"
+    __slots__ += "transport_id", "data", "flags", "raw", "packed", "sent", "create_receipt", "receipt", "fromPacked", "MTU"
+    __slots__ += "sent_at", "packet_hash", "ratchet_id", "attached_interface", "receiving_interface", "rssi", "snr", "q"
+    __slots__ += "ciphertext", "plaintext", "destination_hash", "destination_type", "link", "map_hash"
+
     def __init__(self, destination, data, packet_type = DATA, context = NONE, transport_type = RNS.Transport.BROADCAST,
                  header_type = HEADER_1, transport_id = None, attached_interface = None, create_receipt = True, context_flag=FLAG_UNSET):
 
@@ -137,7 +142,11 @@ class Packet:
             self.fromPacked     = True
             self.create_receipt = False
 
-        self.MTU         = RNS.Reticulum.MTU
+        if destination and destination.type == RNS.Destination.LINK:
+            self.MTU     = destination.mtu
+        else:
+            self.MTU     = RNS.Reticulum.MTU
+
         self.sent_at     = None
         self.packet_hash = None
         self.ratchet_id  = None
@@ -262,7 +271,11 @@ class Packet:
         if not self.sent:
             if self.destination.type == RNS.Destination.LINK:
                 if self.destination.status == RNS.Link.CLOSED:
-                    raise IOError("Attempt to transmit over a closed link")
+                    RNS.log("Attempt to transmit over a closed link, dropping packet", RNS.LOG_DEBUG)
+                    self.sent = False
+                    self.receipt = None
+                    return False
+
                 else:
                     self.destination.last_outbound = time.time()
                     self.destination.tx += 1
@@ -341,6 +354,33 @@ class Packet:
 
         return hashable_part
 
+    def get_rssi(self):
+        """
+        :returns: The physical layer *Received Signal Strength Indication* if available, otherwise ``None``.
+        """
+        if self.rssi != None:
+            return self.rssi
+        else:
+            return reticulum.get_packet_rssi(self.packet_hash)
+            
+    def get_snr(self):
+        """
+        :returns: The physical layer *Signal-to-Noise Ratio* if available, otherwise ``None``.
+        """
+        if self.snr != None:
+            return self.snr
+        else:
+            return reticulum.get_packet_snr(self.packet_hash)
+
+    def get_q(self):
+        """
+        :returns: The physical layer *Link Quality* if available, otherwise ``None``.
+        """
+        if self.q != None:
+            return self.q
+        else:
+            return reticulum.get_packet_q(self.packet_hash)
+
 class ProofDestination:
     def __init__(self, packet):
         self.hash = packet.get_hash()[:RNS.Reticulum.TRUNCATED_HASHLENGTH//8];
@@ -453,7 +493,7 @@ class PacketReceipt:
             # This is an explicit proof
             proof_hash = proof[:RNS.Identity.HASHLENGTH//8]
             signature = proof[RNS.Identity.HASHLENGTH//8:RNS.Identity.HASHLENGTH//8+RNS.Identity.SIGLENGTH//8]
-            if proof_hash == self.hash:
+            if proof_hash == self.hash and hasattr(self.destination, "identity") and self.destination.identity != None:
                 proof_valid = self.destination.identity.validate(signature, self.hash)
                 if proof_valid:
                     self.status = PacketReceipt.DELIVERED
@@ -474,6 +514,10 @@ class PacketReceipt:
                 return False
         elif len(proof) == PacketReceipt.IMPL_LENGTH:
             # This is an implicit proof
+
+            if not hasattr(self.destination, "identity"):
+                return False
+
             if self.destination.identity == None:
                 return False
 

RNS/Resolver.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal

RNS/Resource.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors.
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors.
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -54,6 +54,9 @@ class Resource:
     # The maximum window size for transfers on slow links
     WINDOW_MAX_SLOW      = 10
 
+    # The maximum window size for transfers on very slow links
+    WINDOW_MAX_VERY_SLOW = 4
+
     # The maximum window size for transfers on fast links
     WINDOW_MAX_FAST      = 75
     
@@ -65,12 +68,22 @@ class Resource:
     # rounds, the fast link window size will be allowed.
     FAST_RATE_THRESHOLD  = WINDOW_MAX_SLOW - WINDOW - 2
 
+    # If the very slow rate is sustained for this many request
+    # rounds, window will be capped to the very slow limit.
+    VERY_SLOW_RATE_THRESHOLD = 2
+
     # If the RTT rate is higher than this value,
     # the max window size for fast links will be used.
     # The default is 50 Kbps (the value is stored in
     # bytes per second, hence the "/ 8").
     RATE_FAST            = (50*1000) / 8
 
+    # If the RTT rate is lower than this value,
+    # the window size will be capped at .
+    # The default is 50 Kbps (the value is stored in
+    # bytes per second, hence the "/ 8").
+    RATE_VERY_SLOW       = (2*1000) / 8
+
     # The minimum allowed flexibility of the window size.
     # The difference between window_max and window_min
     # will never be smaller than this value.
@@ -105,6 +118,7 @@ class Resource:
 
     PART_TIMEOUT_FACTOR           = 4
     PART_TIMEOUT_FACTOR_AFTER_RTT = 2
+    PROOF_TIMEOUT_FACTOR          = 3
     MAX_RETRIES                   = 16
     MAX_ADV_RETRIES               = 4
     SENDER_GRACE_TIME             = 10.0
@@ -127,6 +141,19 @@ class Resource:
     COMPLETE        = 0x06
     FAILED          = 0x07
     CORRUPT         = 0x08
+    REJECTED        = 0x00
+
+    @staticmethod
+    def reject(advertisement_packet):
+        try:
+            adv = ResourceAdvertisement.unpack(advertisement_packet.plaintext)
+            resource_hash = adv.h
+            reject_packet = RNS.Packet(advertisement_packet.link, resource_hash, context=RNS.Packet.RESOURCE_RCL)
+            reject_packet.send()
+
+        except Exception as e:
+            RNS.log(f"An error ocurred while rejecting advertised resource: {e}", RNS.LOG_ERROR)
+            RNS.trace_exception(e)
 
     @staticmethod
     def accept(advertisement_packet, callback=None, progress_callback = None, request_id = None):
@@ -136,32 +163,33 @@ class Resource:
             resource = Resource(None, advertisement_packet.link, request_id = request_id)
             resource.status = Resource.TRANSFERRING
 
-            resource.flags               = adv.f
-            resource.size                = adv.t
-            resource.total_size          = adv.d
-            resource.uncompressed_size   = adv.d
-            resource.hash                = adv.h
-            resource.original_hash       = adv.o
-            resource.random_hash         = adv.r
-            resource.hashmap_raw         = adv.m
-            resource.encrypted           = True if resource.flags & 0x01 else False
-            resource.compressed          = True if resource.flags >> 1 & 0x01 else False
-            resource.initiator           = False
+            resource.flags                = adv.f
+            resource.size                 = adv.t
+            resource.total_size           = adv.d
+            resource.uncompressed_size    = adv.d
+            resource.hash                 = adv.h
+            resource.original_hash        = adv.o
+            resource.random_hash          = adv.r
+            resource.hashmap_raw          = adv.m
+            resource.encrypted            = True if resource.flags & 0x01 else False
+            resource.compressed           = True if resource.flags >> 1 & 0x01 else False
+            resource.initiator            = False
             resource.callback             = callback
-            resource.__progress_callback = progress_callback
-            resource.total_parts         = int(math.ceil(resource.size/float(Resource.SDU)))
-            resource.received_count      = 0
-            resource.outstanding_parts   = 0
-            resource.parts               = [None] * resource.total_parts
-            resource.window              = Resource.WINDOW
-            resource.window_max          = Resource.WINDOW_MAX_SLOW
-            resource.window_min          = Resource.WINDOW_MIN
-            resource.window_flexibility  = Resource.WINDOW_FLEXIBILITY
-            resource.last_activity       = time.time()
+            resource.__progress_callback  = progress_callback
+            resource.total_parts          = int(math.ceil(resource.size/float(resource.sdu)))
+            resource.received_count       = 0
+            resource.outstanding_parts    = 0
+            resource.parts                = [None] * resource.total_parts
+            resource.window               = Resource.WINDOW
+            resource.window_max           = Resource.WINDOW_MAX_SLOW
+            resource.window_min           = Resource.WINDOW_MIN
+            resource.window_flexibility   = Resource.WINDOW_FLEXIBILITY
+            resource.last_activity        = time.time()
+            resource.started_transferring = resource.last_activity
 
-            resource.storagepath         = RNS.Reticulum.resourcepath+"/"+resource.original_hash.hex()
-            resource.segment_index       = adv.i
-            resource.total_segments      = adv.l
+            resource.storagepath          = RNS.Reticulum.resourcepath+"/"+resource.original_hash.hex()
+            resource.segment_index        = adv.i
+            resource.total_segments       = adv.l
             if adv.l > 1:
                 resource.split = True
             else:
@@ -172,6 +200,13 @@ class Resource:
             resource.waiting_for_hmu = False
             resource.receiving_part = False
             resource.consecutive_completed_height = -1
+
+            previous_window = resource.link.get_last_resource_window()
+            previous_eifr   = resource.link.get_last_resource_eifr()
+            if previous_window:
+                resource.window = previous_window
+            if previous_eifr:
+                resource.previous_eifr = previous_eifr
             
             if not resource.link.has_incoming_resource(resource):
                 resource.link.register_incoming_resource(resource)
@@ -220,7 +255,6 @@ class Resource:
                 data_size = os.stat(data.name).st_size
 
             self.total_size = data_size
-            self.grand_total_parts = math.ceil(data_size/Resource.SDU)
 
             if data_size <= Resource.MAX_EFFICIENT_SIZE:
                 self.total_segments = 1
@@ -241,7 +275,6 @@ class Resource:
 
         elif isinstance(data, bytes):
             data_size = len(data)
-            self.grand_total_parts = math.ceil(data_size/Resource.SDU)
             self.total_size  = data_size
             
             resource_data = data
@@ -259,6 +292,10 @@ class Resource:
 
         self.status = Resource.NONE
         self.link = link
+        if self.link.mtu:
+            self.sdu = self.link.mtu - RNS.Reticulum.HEADER_MAXSIZE - RNS.Reticulum.IFAC_MIN_SIZE
+        else:
+            self.sdu = link.mdu or Resource.SDU
         self.max_retries = Resource.MAX_RETRIES
         self.max_adv_retries = Resource.MAX_ADV_RETRIES
         self.retries_left = self.max_retries
@@ -274,9 +311,15 @@ class Resource:
         self.req_sent = 0
         self.req_resp_rtt_rate = 0
         self.rtt_rxd_bytes_at_part_req = 0
+        self.req_data_rtt_rate = 0
+        self.eifr = None
+        self.previous_eifr = None
         self.fast_rate_rounds = 0
+        self.very_slow_rate_rounds = 0
         self.request_id = request_id
+        self.started_transferring = None
         self.is_response = is_response
+        self.auto_compress = auto_compress
 
         self.req_hashlist = []
         self.receiver_min_consecutive_height = 0
@@ -293,9 +336,9 @@ class Resource:
 
             compression_began = time.time()
             if (auto_compress and len(self.uncompressed_data) <= Resource.AUTO_COMPRESS_MAX_SIZE):
-                RNS.log("Compressing resource data...", RNS.LOG_DEBUG)
+                RNS.log("Compressing resource data...", RNS.LOG_EXTREME)
                 self.compressed_data   = bz2.compress(self.uncompressed_data)
-                RNS.log("Compression completed in "+str(round(time.time()-compression_began, 3))+" seconds", RNS.LOG_DEBUG)
+                RNS.log("Compression completed in "+str(round(time.time()-compression_began, 3))+" seconds", RNS.LOG_EXTREME)
             else:
                 self.compressed_data   = self.uncompressed_data
 
@@ -304,7 +347,7 @@ class Resource:
 
             if (self.compressed_size < self.uncompressed_size and auto_compress):
                 saved_bytes = len(self.uncompressed_data) - len(self.compressed_data)
-                RNS.log("Compression saved "+str(saved_bytes)+" bytes, sending compressed", RNS.LOG_DEBUG)
+                RNS.log("Compression saved "+str(saved_bytes)+" bytes, sending compressed", RNS.LOG_EXTREME)
 
                 self.data  = b""
                 self.data += RNS.Identity.get_random_hash()[:Resource.RANDOM_HASH_SIZE]
@@ -322,7 +365,7 @@ class Resource:
                 self.compressed = False
                 self.compressed_data = None
                 if auto_compress:
-                    RNS.log("Compression did not decrease size, sending uncompressed", RNS.LOG_DEBUG)
+                    RNS.log("Compression did not decrease size, sending uncompressed", RNS.LOG_EXTREME)
 
             # Resources handle encryption directly to
             # make optimal use of packet MTU on an entire
@@ -333,12 +376,13 @@ class Resource:
 
             self.size = len(self.data)
             self.sent_parts = 0
-            hashmap_entries = int(math.ceil(self.size/float(Resource.SDU)))
+            hashmap_entries = int(math.ceil(self.size/float(self.sdu)))
+            self.total_parts = hashmap_entries
                 
             hashmap_ok = False
             while not hashmap_ok:
                 hashmap_computation_began = time.time()
-                RNS.log("Starting resource hashmap computation with "+str(hashmap_entries)+" entries...", RNS.LOG_DEBUG)
+                RNS.log("Starting resource hashmap computation with "+str(hashmap_entries)+" entries...", RNS.LOG_EXTREME)
 
                 self.random_hash       = RNS.Identity.get_random_hash()[:Resource.RANDOM_HASH_SIZE]
                 self.hash = RNS.Identity.full_hash(data+self.random_hash)
@@ -354,11 +398,11 @@ class Resource:
                 self.hashmap = b""
                 collision_guard_list = []
                 for i in range(0,hashmap_entries):
-                    data = self.data[i*Resource.SDU:(i+1)*Resource.SDU]
+                    data = self.data[i*self.sdu:(i+1)*self.sdu]
                     map_hash = self.get_map_hash(data)
 
                     if map_hash in collision_guard_list:
-                        RNS.log("Found hash collision in resource map, remapping...", RNS.LOG_VERBOSE)
+                        RNS.log("Found hash collision in resource map, remapping...", RNS.LOG_DEBUG)
                         hashmap_ok = False
                         break
                     else:
@@ -374,7 +418,7 @@ class Resource:
                         self.hashmap += part.map_hash
                         self.parts.append(part)
 
-                RNS.log("Hashmap computation concluded in "+str(round(time.time()-hashmap_computation_began, 3))+" seconds", RNS.LOG_DEBUG)
+                RNS.log("Hashmap computation concluded in "+str(round(time.time()-hashmap_computation_began, 3))+" seconds", RNS.LOG_EXTREME)
                 
             if advertise:
                 self.advertise()
@@ -428,12 +472,13 @@ class Resource:
         try:
             self.advertisement_packet.send()
             self.last_activity = time.time()
+            self.started_transferring = self.last_activity
             self.adv_sent = self.last_activity
             self.rtt = None
             self.status = Resource.ADVERTISED
             self.retries_left = self.max_adv_retries
             self.link.register_outgoing_resource(self)
-            RNS.log("Sent resource advertisement for "+RNS.prettyhexrep(self.hash), RNS.LOG_DEBUG)
+            RNS.log("Sent resource advertisement for "+RNS.prettyhexrep(self.hash), RNS.LOG_EXTREME)
         except Exception as e:
             RNS.log("Could not advertise resource, the contained exception was: "+str(e), RNS.LOG_ERROR)
             self.cancel()
@@ -441,6 +486,23 @@ class Resource:
 
         self.watchdog_job()
 
+    def update_eifr(self):
+        if self.rtt == None:
+            rtt = self.link.rtt
+        else:
+            rtt = self.rtt
+
+        if self.req_data_rtt_rate != 0:
+            expected_inflight_rate = self.req_data_rtt_rate*8
+        else:
+            if self.previous_eifr != None:
+                expected_inflight_rate = self.previous_eifr
+            else:
+                expected_inflight_rate = self.link.establishment_cost*8 / rtt
+
+        self.eifr = expected_inflight_rate
+        if self.link: self.link.expected_rate = self.eifr
+
     def watchdog_job(self):
         thread = threading.Thread(target=self.__watchdog_job)
         thread.daemon = True
@@ -455,7 +517,6 @@ class Resource:
                 sleep(0.025)
 
             sleep_time = None
-
             if self.status == Resource.ADVERTISED:
                 sleep_time = (self.adv_sent+self.timeout+Resource.PROCESSING_GRACE)-time.time()
                 if sleep_time < 0:
@@ -479,17 +540,19 @@ class Resource:
 
             elif self.status == Resource.TRANSFERRING:
                 if not self.initiator:
-
-                    if self.rtt == None:
-                        rtt = self.link.rtt
-                    else:
-                        rtt = self.rtt
-
-                    window_remaining = self.outstanding_parts
-
                     retries_used = self.max_retries - self.retries_left
                     extra_wait = retries_used * Resource.PER_RETRY_DELAY
-                    sleep_time = self.last_activity + (rtt*(self.part_timeout_factor+window_remaining)) + Resource.RETRY_GRACE_TIME + extra_wait - time.time()
+
+                    self.update_eifr()
+                    expected_tof_remaining = (self.outstanding_parts*self.sdu*8)/self.eifr
+
+                    if self.req_resp_rtt_rate != 0:
+                        sleep_time = self.last_activity + self.part_timeout_factor*expected_tof_remaining + Resource.RETRY_GRACE_TIME + extra_wait - time.time()
+                    else:
+                        sleep_time = self.last_activity + self.part_timeout_factor*((3*self.sdu)/self.eifr) + Resource.RETRY_GRACE_TIME + extra_wait - time.time()
+                    
+                    # RNS.log(f"EIFR {RNS.prettyspeed(self.eifr)}, ETOF {RNS.prettyshorttime(expected_tof_remaining)} ", RNS.LOG_DEBUG, pt=True)
+                    # RNS.log(f"Resource ST {RNS.prettyshorttime(sleep_time)}, RTT {RNS.prettyshorttime(self.rtt or self.link.rtt)}, {self.outstanding_parts} left", RNS.LOG_DEBUG, pt=True)
                     
                     if sleep_time < 0:
                         if self.retries_left > 0:
@@ -519,6 +582,10 @@ class Resource:
                         sleep_time = 0.001
 
             elif self.status == Resource.AWAITING_PROOF:
+                # Decrease timeout factor since proof packets are
+                # significantly smaller than full req/resp roundtrip
+                self.timeout_factor = Resource.PROOF_TIMEOUT_FACTOR
+
                 sleep_time = self.last_part_sent + (self.rtt*self.timeout_factor+self.sender_grace_time) - time.time()
                 if sleep_time < 0:
                     if self.retries_left <= 0:
@@ -535,8 +602,11 @@ class Resource:
                         self.last_part_sent = time.time()
                         sleep_time = 0.001
 
+            elif self.status == Resource.REJECTED:
+                sleep_time = 0.001
+
             if sleep_time == 0:
-                RNS.log("Warning! Link watchdog sleep time of 0!", RNS.LOG_WARNING)
+                RNS.log("Warning! Link watchdog sleep time of 0!", RNS.LOG_DEBUG)
             if sleep_time == None or sleep_time < 0:
                 RNS.log("Timing error, cancelling resource transfer.", RNS.LOG_ERROR)
                 self.cancel()
@@ -610,6 +680,7 @@ class Resource:
                 proof_data = self.hash+proof
                 proof_packet = RNS.Packet(self.link, proof_data, packet_type=RNS.Packet.PROOF, context=RNS.Packet.RESOURCE_PRF)
                 proof_packet.send()
+                RNS.Transport.cache(proof_packet, force_cache=True)
             except Exception as e:
                 RNS.log("Could not send proof packet, cancelling resource", RNS.LOG_DEBUG)
                 RNS.log("The contained exception was: "+str(e), RNS.LOG_DEBUG)
@@ -628,6 +699,7 @@ class Resource:
             request_id = self.request_id,
             is_response = self.is_response,
             advertise = False,
+            auto_compress = self.auto_compress,
         )
 
     def validate_proof(self, proof_data):
@@ -751,6 +823,7 @@ class Resource:
 
                         if rtt != 0:
                             self.req_data_rtt_rate = req_transferred/rtt
+                            self.update_eifr()
                             self.rtt_rxd_bytes_at_part_req = self.rtt_rxd_bytes
 
                             if self.req_data_rtt_rate > Resource.RATE_FAST and self.fast_rate_rounds < Resource.FAST_RATE_THRESHOLD:
@@ -759,6 +832,12 @@ class Resource:
                                 if self.fast_rate_rounds == Resource.FAST_RATE_THRESHOLD:
                                     self.window_max = Resource.WINDOW_MAX_FAST
 
+                            if self.fast_rate_rounds == 0 and self.req_data_rtt_rate < Resource.RATE_VERY_SLOW and self.very_slow_rate_rounds < Resource.VERY_SLOW_RATE_THRESHOLD:
+                                self.very_slow_rate_rounds += 1
+
+                                if self.very_slow_rate_rounds == Resource.VERY_SLOW_RATE_THRESHOLD:
+                                    self.window_max = Resource.WINDOW_MAX_VERY_SLOW
+
                     self.request_next()
             else:
                 self.receiving_part = False
@@ -900,6 +979,7 @@ class Resource:
 
             if self.sent_parts == len(self.parts):
                 self.status = Resource.AWAITING_PROOF
+                self.retries_left = 3
 
             if self.__progress_callback != None:
                 try:
@@ -931,6 +1011,18 @@ class Resource:
                 except Exception as e:
                     RNS.log("Error while executing callbacks on resource cancel from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
 
+    def _rejected(self):
+        if self.status < Resource.COMPLETE:
+            if self.initiator:
+                self.status = Resource.REJECTED
+                self.link.cancel_outgoing_resource(self)
+                if self.callback != None:
+                    try:
+                        self.link.resource_concluded(self)
+                        self.callback(self)
+                    except Exception as e:
+                        RNS.log("Error while executing callbacks on resource reject from "+str(self)+". The contained exception was: "+str(e), RNS.LOG_ERROR)
+
     def set_callback(self, callback):
         self.callback = callback
 
@@ -943,21 +1035,68 @@ class Resource:
         """
         if self.status == RNS.Resource.COMPLETE and self.segment_index == self.total_segments:
             return 1.0
+        
         elif self.initiator:
-            self.processed_parts  = (self.segment_index-1)*math.ceil(Resource.MAX_EFFICIENT_SIZE/Resource.SDU)
-            self.processed_parts += self.sent_parts
-            self.progress_total_parts = float(self.grand_total_parts)
-        else:
-            self.processed_parts  = (self.segment_index-1)*math.ceil(Resource.MAX_EFFICIENT_SIZE/Resource.SDU)            
-            self.processed_parts += self.received_count
-            if self.split:
-                self.progress_total_parts = float(math.ceil(self.total_size/Resource.SDU))
-            else:
+            if not self.split:
+                self.processed_parts = self.sent_parts
                 self.progress_total_parts = float(self.total_parts)
 
+            else:
+                is_last_segment = self.segment_index != self.total_segments
+                total_segments = self.total_segments
+                processed_segments = self.segment_index-1
+
+                current_segment_parts = self.total_parts
+                max_parts_per_segment = math.ceil(Resource.MAX_EFFICIENT_SIZE/self.sdu)
+
+                previously_processed_parts = processed_segments*max_parts_per_segment
+
+                if current_segment_parts < max_parts_per_segment:
+                    current_segment_factor = max_parts_per_segment / current_segment_parts
+                else:
+                    current_segment_factor = 1
+
+                self.processed_parts = previously_processed_parts + self.sent_parts*current_segment_factor
+                self.progress_total_parts = self.total_segments*max_parts_per_segment
+
+        else:
+            if not self.split:
+                self.processed_parts = self.received_count
+                self.progress_total_parts = float(self.total_parts)
+
+            else:
+                is_last_segment = self.segment_index != self.total_segments
+                total_segments = self.total_segments
+                processed_segments = self.segment_index-1
+
+                current_segment_parts = self.total_parts
+                max_parts_per_segment = math.ceil(Resource.MAX_EFFICIENT_SIZE/self.sdu)
+
+                previously_processed_parts = processed_segments*max_parts_per_segment
+
+                if current_segment_parts < max_parts_per_segment:
+                    current_segment_factor = max_parts_per_segment / current_segment_parts
+                else:
+                    current_segment_factor = 1
+
+                self.processed_parts = previously_processed_parts + self.received_count*current_segment_factor
+                self.progress_total_parts = self.total_segments*max_parts_per_segment
+
+
         progress = min(1.0, self.processed_parts / self.progress_total_parts)
         return progress
 
+    def get_segment_progress(self):
+        if self.status == RNS.Resource.COMPLETE and self.segment_index == self.total_segments:
+            return 1.0
+        elif self.initiator:
+            processed_parts = self.sent_parts
+        else:
+            processed_parts = self.received_count
+
+        progress = min(1.0, processed_parts / self.total_parts)
+        return progress
+
     def get_transfer_size(self):
         """
         :returns: The number of bytes needed to transfer the resource.

RNS/Utilities/__init__.py

@@ -23,5 +23,7 @@
 import os
 import glob
 
-modules = glob.glob(os.path.dirname(__file__)+"/*.py")
-__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]
+py_modules  = glob.glob(os.path.dirname(__file__)+"/*.py")
+pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
+modules     = py_modules+pyc_modules
+__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))

RNS/Utilities/rncp.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -34,17 +34,26 @@ from RNS._version import __version__
 APP_NAME = "rncp"
 allow_all = False
 allow_fetch = False
+fetch_auto_compress = True
 fetch_jail = None
+save_path = None
+show_phy_rates = False
 allowed_identity_hashes = []
 
 REQ_FETCH_NOT_ALLOWED = 0xF0
 
+es = " "
+erase_str = "\33[2K\r"
+
 def listen(configdir, verbosity = 0, quietness = 0, allowed = [], display_identity = False,
-           limit = None, disable_auth = None, fetch_allowed = False, jail = None, announce = False):
-    global allow_all, allow_fetch, allowed_identity_hashes, fetch_jail
+           limit = None, disable_auth = None, fetch_allowed = False, no_compress=False,
+           jail = None, save = None, announce = False):
+
+    global allow_all, allow_fetch, allowed_identity_hashes, fetch_jail, save_path, fetch_auto_compress
     from tempfile import TemporaryFile
 
     allow_fetch = fetch_allowed
+    fetch_auto_compress = not no_compress
     identity = None
     if announce < 0:
         announce = False
@@ -56,6 +65,20 @@ def listen(configdir, verbosity = 0, quietness = 0, allowed = [], display_identi
         fetch_jail = os.path.abspath(os.path.expanduser(jail))
         RNS.log("Restricting fetch requests to paths under \""+fetch_jail+"\"", RNS.LOG_VERBOSE)
 
+    if save != None:
+        sp = os.path.abspath(os.path.expanduser(save))
+        if os.path.isdir(sp):
+            if os.access(sp, os.W_OK):
+                save_path = sp
+            else:
+                RNS.log("Output directory not writable", RNS.LOG_ERROR)
+                RNS.exit(4)
+        else:
+            RNS.log("Output directory not found", RNS.LOG_ERROR)
+            RNS.exit(3)
+
+        RNS.log("Saving received files in \""+save_path+"\"", RNS.LOG_VERBOSE)
+
     identity_path = RNS.Reticulum.identitypath+"/"+APP_NAME
     if os.path.isfile(identity_path):
         identity = RNS.Identity.from_file(identity_path)                
@@ -70,7 +93,7 @@ def listen(configdir, verbosity = 0, quietness = 0, allowed = [], display_identi
     if display_identity:
         print("Identity     : "+str(identity))
         print("Listening on : "+RNS.prettyhexrep(destination.hash))
-        exit(0)
+        RNS.exit(0)
 
     if disable_auth:
         allow_all = True
@@ -120,20 +143,25 @@ def listen(configdir, verbosity = 0, quietness = 0, allowed = [], display_identi
                         raise ValueError("Invalid destination entered. Check your input.")
                 except Exception as e:
                     print(str(e))
-                    exit(1)
+                    RNS.exit(1)
 
     if len(allowed_identity_hashes) < 1 and not disable_auth:
         print("Warning: No allowed identities configured, rncp will not accept any files!")
 
     def fetch_request(path, data, request_id, link_id, remote_identity, requested_at):
-        global allow_fetch, fetch_jail
+        global allow_fetch, fetch_jail, fetch_auto_compress
         if not allow_fetch:
             return REQ_FETCH_NOT_ALLOWED
 
-        file_path = os.path.abspath(os.path.expanduser(data))
         if fetch_jail:
-            if not file_path.startswith(jail):
+            if data.startswith(fetch_jail+"/"):
+                data = data.replace(fetch_jail+"/", "")
+            file_path = os.path.abspath(os.path.expanduser(f"{fetch_jail}/{data}"))
+            if not file_path.startswith(fetch_jail+"/"):
+                RNS.log(f"Disallowing fetch request for {file_path} outside of fetch jail {fetch_jail}", RNS.LOG_WARNING)
                 return REQ_FETCH_NOT_ALLOWED
+        else:
+            file_path = os.path.abspath(os.path.expanduser(f"{data}"))
 
         target_link = None
         for link in RNS.Transport.active_links:
@@ -154,21 +182,27 @@ def listen(configdir, verbosity = 0, quietness = 0, allowed = [], display_identi
 
                 if filename_len > 0xFFFF:
                     print("Filename exceeds max size, cannot send")
-                    exit(1)
+                    RNS.exit(1)
 
                 temp_file.write(filename_len.to_bytes(2, "big"))
                 temp_file.write(filename_bytes)
                 temp_file.write(real_file.read())
                 temp_file.seek(0)
 
-                fetch_resource = RNS.Resource(temp_file, target_link)
+                fetch_resource = RNS.Resource(temp_file, target_link, auto_compress=fetch_auto_compress)
                 return True
             else:
                 return None
 
 
     destination.set_link_established_callback(client_link_established)
-    destination.register_request_handler("fetch_file", response_generator=fetch_request, allow=RNS.Destination.ALLOW_LIST, allowed_list=allowed_identity_hashes)
+    if allow_fetch:
+        if allow_all:
+            RNS.log("Allowing unauthenticated fetch requests", RNS.LOG_WARNING)
+            destination.register_request_handler("fetch_file", response_generator=fetch_request, allow=RNS.Destination.ALLOW_ALL)
+        else:
+            destination.register_request_handler("fetch_file", response_generator=fetch_request, allow=RNS.Destination.ALLOW_LIST, allowed_list=allowed_identity_hashes)
+
     print("rncp listening on "+RNS.prettyhexrep(destination.hash))
 
     if announce >= 0:
@@ -227,6 +261,7 @@ def receive_resource_started(resource):
     print("Starting resource transfer "+RNS.prettyhexrep(resource.hash)+id_str)
 
 def receive_resource_concluded(resource):
+    global save_path
     if resource.status == RNS.Resource.COMPLETE:
         print(str(resource)+" completed")
 
@@ -235,12 +270,20 @@ def receive_resource_concluded(resource):
             filename = resource.data.read(filename_len).decode("utf-8")
 
             counter = 0
-            saved_filename = filename
-            while os.path.isfile(saved_filename):
+            if save_path:
+                saved_filename = os.path.abspath(os.path.expanduser(save_path+"/"+filename))
+                if not saved_filename.startswith(save_path+"/"):
+                    RNS.log(f"Invalid save path {saved_filename}, ignoring", RNS.LOG_ERROR)
+                    return
+            else:
+                saved_filename = filename
+
+            full_save_path = saved_filename
+            while os.path.isfile(full_save_path):
                 counter += 1
-                saved_filename = filename+"."+str(counter)
+                full_save_path = saved_filename+"."+str(counter)
             
-            file = open(saved_filename, "wb")
+            file = open(full_save_path, "wb")
             file.write(resource.data.read())
             file.close()
 
@@ -254,33 +297,59 @@ resource_done = False
 current_resource = None
 stats = []
 speed = 0.0
+phy_speed = 0.0
+phy_got_total = 0
 def sender_progress(resource):
     stats_max = 32
-    global current_resource, stats, speed, resource_done
+    global current_resource, stats, speed, phy_speed, phy_got_total, resource_done
     current_resource = resource
+    
     now = time.time()
-    got = current_resource.get_progress()*current_resource.total_size
-    entry = [now, got]
+    got = current_resource.get_progress()*current_resource.get_data_size()
+    phy_got = current_resource.get_segment_progress()*current_resource.get_transfer_size()
+    
+    entry = [now, got, phy_got]
     stats.append(entry)
+
     while len(stats) > stats_max:
         stats.pop(0)
 
     span = now - stats[0][0]
     if span == 0:
         speed = 0
+        phy_speed = 0
+    
     else:
         diff = got - stats[0][1]
         speed = diff/span
 
+        phy_diff = phy_got - stats[0][2]
+        if phy_diff > 0:
+            phy_speed = phy_diff/span
+            # phy_got_total += phy_diff
+
     if resource.status < RNS.Resource.COMPLETE:
         resource_done = False
     else:
         resource_done = True
 
 link = None
-def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False):
-    global current_resource, resource_done, link, speed
+def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False, phy_rates=False, save=None):
+    global current_resource, resource_done, link, speed, show_phy_rates, save_path
     targetloglevel = 3+verbosity-quietness
+    show_phy_rates = phy_rates
+
+    if save:
+        sp = os.path.abspath(os.path.expanduser(save))
+        if os.path.isdir(sp):
+            if os.access(sp, os.W_OK):
+                save_path = sp
+            else:
+                RNS.log("Output directory not writable", RNS.LOG_ERROR)
+                RNS.exit(4)
+        else:
+            RNS.log("Output directory not found", RNS.LOG_ERROR)
+            RNS.exit(3)
 
     try:
         dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
@@ -292,7 +361,7 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
             raise ValueError("Invalid destination entered. Check your input.")
     except Exception as e:
         print(str(e))
-        exit(1)
+        RNS.exit(1)
 
     reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
 
@@ -301,7 +370,7 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
         identity = RNS.Identity.from_file(identity_path)
         if identity == None:
             RNS.log("Could not load identity for rncp. The identity file at \""+str(identity_path)+"\" may be corrupt or unreadable.", RNS.LOG_ERROR)
-            exit(2)
+            RNS.exit(2)
     else:
         identity = None
 
@@ -315,7 +384,7 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
         if silent:
             print("Path to "+RNS.prettyhexrep(destination_hash)+" requested")
         else:
-            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
+            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=es)
         sys.stdout.flush()
 
     i = 0
@@ -332,13 +401,13 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
         if silent:
             print("Path not found")
         else:
-            print("\r                                                            \rPath not found")
-        exit(1)
+            print(f"{erase_str}Path not found")
+        RNS.exit(1)
     else:
         if silent:
             print("Establishing link with "+RNS.prettyhexrep(destination_hash))
         else:
-            print("\r                                                            \rEstablishing link with "+RNS.prettyhexrep(destination_hash)+"  ", end=" ")
+            print(f"{erase_str}Establishing link with "+RNS.prettyhexrep(destination_hash)+"  ", end=es)
 
     listener_identity = RNS.Identity.recall(destination_hash)
     listener_destination = RNS.Destination(
@@ -361,13 +430,13 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
         if silent:
             print("Could not establish link with "+RNS.prettyhexrep(destination_hash))
         else:
-            print("\r                                                            \rCould not establish link with "+RNS.prettyhexrep(destination_hash))
-        exit(1)
+            print(f"{erase_str}Could not establish link with "+RNS.prettyhexrep(destination_hash))
+        RNS.exit(1)
     else:
         if silent:
             print("Requesting file from remote...")
         else:
-            print("\r                                                            \rRequesting file from remote  ", end=" ")
+            print(f"{erase_str}Requesting file from remote  ", end=es)
 
     link.identify(identity)
 
@@ -376,6 +445,7 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
     resource_resolved = False
     resource_status = "unrequested"
     current_resource = None
+    current_transfer_started = None
     def request_response(request_receipt):
         nonlocal request_resolved, request_status
         if request_receipt.response == False:
@@ -395,25 +465,33 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
         request_resolved = True
 
     def fetch_resource_started(resource):
-        nonlocal resource_status
+        nonlocal resource_status, current_transfer_started
         current_resource = resource
         current_resource.progress_callback(sender_progress)
         resource_status = "started"
+        if not current_transfer_started: current_transfer_started = time.time()
 
     def fetch_resource_concluded(resource):
         nonlocal resource_resolved, resource_status
+        global save_path
         if resource.status == RNS.Resource.COMPLETE:
             if resource.total_size > 4:
                 filename_len = int.from_bytes(resource.data.read(2), "big")
                 filename = resource.data.read(filename_len).decode("utf-8")
 
                 counter = 0
-                saved_filename = filename
-                while os.path.isfile(saved_filename):
+                if save_path:
+                    saved_filename = os.path.abspath(os.path.expanduser(save_path+"/"+filename))
+
+                else:
+                    saved_filename = filename
+
+                full_save_path = saved_filename
+                while os.path.isfile(full_save_path):
                     counter += 1
-                    saved_filename = filename+"."+str(counter)
-                
-                file = open(saved_filename, "wb")
+                    full_save_path = saved_filename+"."+str(counter)
+ 
+                file = open(full_save_path, "wb")
                 file.write(resource.data.read())
                 file.close()
                 resource_status = "completed"
@@ -442,31 +520,31 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
             i = (i+1)%len(syms)
 
     if request_status == "fetch_not_allowed":
-        if not silent: print("\r                                                            \r", end="")
+        if not silent: print(f"{erase_str}", end="")
         print("Fetch request failed, fetching the file "+str(file)+" was not allowed by the remote")
         link.teardown()
         time.sleep(0.15)
-        exit(0)
+        RNS.exit(0)
     elif request_status == "not_found":
-        if not silent: print("\r                                                            \r", end="")
+        if not silent: print(f"{erase_str}", end="")
         print("Fetch request failed, the file "+str(file)+" was not found on the remote")
         link.teardown()
         time.sleep(0.15)
-        exit(0)
+        RNS.exit(0)
     elif request_status == "remote_error":
-        if not silent: print("\r                                                            \r", end="")
+        if not silent: print(f"{erase_str}", end="")
         print("Fetch request failed due to an error on the remote system")
         link.teardown()
         time.sleep(0.15)
-        exit(0)
+        RNS.exit(0)
     elif request_status == "unknown":
-        if not silent: print("\r                                                            \r", end="")
+        if not silent: print(f"{erase_str}", end="")
         print("Fetch request failed due to an unknown error (probably not authorised)")
         link.teardown()
         time.sleep(0.15)
-        exit(0)
+        RNS.exit(0)
     elif request_status == "found":
-        if not silent: print("\r                                                            \r", end="")
+        if not silent: print(f"{erase_str}", end="")
 
     while not resource_resolved:
         if not silent:
@@ -474,40 +552,53 @@ def fetch(configdir, verbosity = 0, quietness = 0, destination = None, file = No
             if current_resource:
                 prg = current_resource.get_progress()
                 percent = round(prg * 100.0, 1)
-                stat_str = str(percent)+"% - " + size_str(int(prg*current_resource.total_size)) + " of " + size_str(current_resource.total_size) + " - " +size_str(speed, "b")+"ps"
-                if prg != 1.0:
-                    print("\r                                                                                  \rTransferring file "+syms[i]+" "+stat_str, end=" ")
+                if show_phy_rates:
+                    pss = size_str(phy_speed, "b")
+                    phy_str = f" ({pss}ps at physical layer)"
                 else:
-                    print("\r                                                                                  \rTransfer complete  "+stat_str, end=" ")
+                    phy_str = ""
+                ps = size_str(int(prg*current_resource.total_size))
+                ts = size_str(current_resource.total_size)
+                ss = size_str(speed, "b")
+                stat_str = f"{percent}% - {ps} of {ts} - {ss}ps{phy_str}"
+                if prg != 1.0:
+                    print(f"{erase_str}Transferring file {syms[i]} {stat_str}", end=es)
+                else:
+                    end_time = time.time(); delta_time = end_time - current_transfer_started
+                    speed = current_resource.total_size/delta_time; dt_str = RNS.prettytime(delta_time)
+                    ss = size_str(speed, "b")
+                    stat_str = f"{percent}% - {ps} of {ts} in {dt_str} - {ss}ps{phy_str}"
+                    print(f"{erase_str}Transfer complete  {stat_str}", end=es)
             else:
-                print("\r                                                                                  \rWaiting for transfer to start "+syms[i]+" ", end=" ")
+                print(f"{erase_str}Waiting for transfer to start {syms[i]} ", end=es)
             sys.stdout.flush()
             i = (i+1)%len(syms)
 
-    if current_resource.status != RNS.Resource.COMPLETE:
+    if not current_resource or current_resource.status != RNS.Resource.COMPLETE:
         if silent:
             print("The transfer failed")
         else:
-            print("\r                                                            \rThe transfer failed")
-        exit(1)
+            print(f"{erase_str}The transfer failed")
+        RNS.exit(1)
     else:
         if silent:
             print(str(file)+" fetched from "+RNS.prettyhexrep(destination_hash))
         else:
-            #print("\r                                                                                  \r"+str(file)+" fetched from "+RNS.prettyhexrep(destination_hash))
             print("\n"+str(file)+" fetched from "+RNS.prettyhexrep(destination_hash))
         link.teardown()
-        time.sleep(0.15)
-        exit(0)
+        time.sleep(0.1)
+        RNS.exit(0)
 
     link.teardown()
-    exit(0)
+    time.sleep(0.1)
+    RNS.exit(0)
 
 
-def send(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False):
-    global current_resource, resource_done, link, speed
+def send(configdir, verbosity = 0, quietness = 0, destination = None, file = None, timeout = RNS.Transport.PATH_REQUEST_TIMEOUT, silent=False, phy_rates=False, no_compress=False):
+    global current_resource, resource_done, link, speed, show_phy_rates, phy_got_total, phy_speed
     from tempfile import TemporaryFile
     targetloglevel = 3+verbosity-quietness
+    show_phy_rates = phy_rates
 
     try:
         dest_len = (RNS.Reticulum.TRUNCATED_HASHLENGTH//8)*2
@@ -519,13 +610,13 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
             raise ValueError("Invalid destination entered. Check your input.")
     except Exception as e:
         print(str(e))
-        exit(1)
+        RNS.exit(1)
 
     
     file_path = os.path.expanduser(file)
     if not os.path.isfile(file_path):
         print("File not found")
-        exit(1)
+        sys.exit(1)
 
     temp_file = TemporaryFile()
     real_file = open(file_path, "rb")
@@ -534,16 +625,16 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
 
     if filename_len > 0xFFFF:
         print("Filename exceeds max size, cannot send")
-        exit(1)
+        RNS.exit(1)
     else:
-        print("Preparing file...", end=" ")
+        print("Preparing file...", end=es)
 
     temp_file.write(filename_len.to_bytes(2, "big"))
     temp_file.write(filename_bytes)
     temp_file.write(real_file.read())
     temp_file.seek(0)
 
-    print("\r                                                            \r", end="")
+    print(f"{erase_str}", end="")
 
     reticulum = RNS.Reticulum(configdir=configdir, loglevel=targetloglevel)
 
@@ -552,7 +643,7 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
         identity = RNS.Identity.from_file(identity_path)
         if identity == None:
             RNS.log("Could not load identity for rncp. The identity file at \""+str(identity_path)+"\" may be corrupt or unreadable.", RNS.LOG_ERROR)
-            exit(2)
+            RNS.exit(2)
     else:
         identity = None
 
@@ -566,7 +657,7 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
         if silent:
             print("Path to "+RNS.prettyhexrep(destination_hash)+" requested")
         else:
-            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=" ")
+            print("Path to "+RNS.prettyhexrep(destination_hash)+" requested  ", end=es)
         sys.stdout.flush()
 
     i = 0
@@ -583,13 +674,13 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
         if silent:
             print("Path not found")
         else:
-            print("\r                                                            \rPath not found")
-        exit(1)
+            print(f"{erase_str}Path not found")
+        RNS.exit(1)
     else:
         if silent:
             print("Establishing link with "+RNS.prettyhexrep(destination_hash))
         else:
-            print("\r                                                            \rEstablishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=" ")
+            print(f"{erase_str}Establishing link with "+RNS.prettyhexrep(destination_hash)+" ", end=es)
 
     receiver_identity = RNS.Identity.recall(destination_hash)
     receiver_destination = RNS.Destination(
@@ -612,22 +703,25 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
         if silent:
             print("Link establishment with "+RNS.prettyhexrep(destination_hash)+" timed out")
         else:
-            print("\r                                                            \rLink establishment with "+RNS.prettyhexrep(destination_hash)+" timed out")
-        exit(1)
+            print(f"{erase_str}Link establishment with "+RNS.prettyhexrep(destination_hash)+" timed out")
+        RNS.exit(1)
     elif not RNS.Transport.has_path(destination_hash):
         if silent:
             print("No path found to "+RNS.prettyhexrep(destination_hash))
         else:
-            print("\r                                                            \rNo path found to "+RNS.prettyhexrep(destination_hash))
-        exit(1)
+            print(f"{erase_str}No path found to "+RNS.prettyhexrep(destination_hash))
+        RNS.exit(1)
     else:
         if silent:
             print("Advertising file resource...")
         else:
-            print("\r                                                            \rAdvertising file resource  ", end=" ")
+            print(f"{erase_str}Advertising file resource  ", end=es)
 
     link.identify(identity)
-    resource = RNS.Resource(temp_file, link, callback = sender_progress, progress_callback = sender_progress)
+    auto_compress = True
+    if no_compress:
+        auto_compress = False
+    resource = RNS.Resource(temp_file, link, callback = sender_progress, progress_callback = sender_progress, auto_compress = auto_compress)
     current_resource = resource
 
     while resource.status < RNS.Resource.TRANSFERRING:
@@ -637,28 +731,38 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
             sys.stdout.flush()
             i = (i+1)%len(syms)
 
+    resource_started_at = time.time()
     
     if resource.status > RNS.Resource.COMPLETE:
         if silent:
             print("File was not accepted by "+RNS.prettyhexrep(destination_hash))
         else:
-            print("\r                                                            \rFile was not accepted by "+RNS.prettyhexrep(destination_hash))
-        exit(1)
+            print(f"{erase_str}File was not accepted by "+RNS.prettyhexrep(destination_hash))
+        RNS.exit(1)
     else:
         if silent:
             print("Transferring file...")
         else:
-            print("\r                                                            \rTransferring file  ", end=" ")
+            print(f"{erase_str}Transferring file  ", end=es)
 
     def progress_update(i, done=False):
         time.sleep(0.1)
         prg = current_resource.get_progress()
         percent = round(prg * 100.0, 1)
-        stat_str = str(percent)+"% - " + size_str(int(prg*current_resource.total_size)) + " of " + size_str(current_resource.total_size) + " - " +size_str(speed, "b")+"ps"
-        if not done:
-            print("\r                                                                                  \rTransferring file "+syms[i]+" "+stat_str, end=" ")
+        if show_phy_rates and not resource_done:
+            pss = size_str(phy_speed, "b")
+            phy_str = f" ({pss}ps at physical layer)"
         else:
-            print("\r                                                                                  \rTransfer complete  "+stat_str, end=" ")
+            phy_str = ""
+        es = "  "
+        cs = size_str(int(prg*current_resource.total_size))
+        ts = size_str(current_resource.total_size)
+        ss = size_str(speed, "b")
+        stat_str = f"{percent}% - {cs} of {ts} - {ss}ps{phy_str}"
+        if not done:
+            print(f"{erase_str}Transferring file "+syms[i]+" "+stat_str, end=es)
+        else:
+            print(f"{erase_str}Transfer complete  "+stat_str, end=es)
         sys.stdout.flush()
         i = (i+1)%len(syms)
         return i
@@ -667,6 +771,11 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
         if not silent:
             i = progress_update(i)
 
+    resource_concluded_at = time.time()
+    transfer_time = resource_concluded_at - resource_started_at
+    speed = current_resource.total_size/transfer_time
+    # phy_speed = phy_got_total/transfer_time
+
     if not silent:
         i = progress_update(i, done=True)
 
@@ -674,19 +783,18 @@ def send(configdir, verbosity = 0, quietness = 0, destination = None, file = Non
         if silent:
             print("The transfer failed")
         else:
-            print("\r                                                            \rThe transfer failed")
-        exit(1)
+            print(f"{erase_str}The transfer failed")
+        RNS.exit(1)
     else:
         if silent:
             print(str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
         else:
-            # print("\r                                                                                  \r"+str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
             print("\n"+str(file_path)+" copied to "+RNS.prettyhexrep(destination_hash))
         link.teardown()
         time.sleep(0.25)
         real_file.close()
         temp_file.close()
-        exit(0)
+        RNS.exit(0)
 
 def main():
     try:
@@ -698,14 +806,17 @@ def main():
         parser.add_argument('-q', '--quiet', action='count', default=0, help="decrease verbosity")
         parser.add_argument("-S", '--silent', action='store_true', default=False, help="disable transfer progress output")
         parser.add_argument("-l", '--listen', action='store_true', default=False, help="listen for incoming transfer requests")
+        parser.add_argument("-C", '--no-compress', action='store_true', default=False, help="disable automatic compression")
         parser.add_argument("-F", '--allow-fetch', action='store_true', default=False, help="allow authenticated clients to fetch files")
         parser.add_argument("-f", '--fetch', action='store_true', default=False, help="fetch file from remote listener instead of sending")
         parser.add_argument("-j", "--jail", metavar="path", action="store", default=None, help="restrict fetch requests to specified path", type=str)
+        parser.add_argument("-s", "--save", metavar="path", action="store", default=None, help="save received files in specified path", type=str)
         parser.add_argument("-b", action='store', metavar="seconds", default=-1, help="announce interval, 0 to only announce at startup", type=int)
-        parser.add_argument('-a', metavar="allowed_hash", dest="allowed", action='append', help="allow this identity", type=str)
+        parser.add_argument('-a', metavar="allowed_hash", dest="allowed", action='append', help="allow this identity (or add in ~/.rncp/allowed_identities)", type=str)
         parser.add_argument('-n', '--no-auth', action='store_true', default=False, help="accept requests from anyone")
         parser.add_argument('-p', '--print-identity', action='store_true', default=False, help="print identity and destination info and exit")
         parser.add_argument("-w", action="store", metavar="seconds", type=float, help="sender timeout before giving up", default=RNS.Transport.PATH_REQUEST_TIMEOUT)
+        parser.add_argument('-P', '--phy-rates', action='store_true', default=False, help="display physical layer transfer rates")
         # parser.add_argument("--limit", action="store", metavar="files", type=float, help="maximum number of files to accept", default=None)
         parser.add_argument("--version", action="version", version="rncp {version}".format(version=__version__))
         
@@ -718,7 +829,9 @@ def main():
                 quietness=args.quiet,
                 allowed = args.allowed,
                 fetch_allowed = args.allow_fetch,
+                no_compress = args.no_compress,
                 jail = args.jail,
+                save = args.save,
                 display_identity=args.print_identity,
                 # limit=args.limit,
                 disable_auth=args.no_auth,
@@ -735,6 +848,8 @@ def main():
                     file = args.file,
                     timeout = args.w,
                     silent = args.silent,
+                    phy_rates = args.phy_rates,
+                    save = args.save,
                 )
             else:
                 print("")
@@ -750,6 +865,8 @@ def main():
                 file = args.file,
                 timeout = args.w,
                 silent = args.silent,
+                phy_rates = args.phy_rates,
+                no_compress = args.no_compress,
             )
 
         else:
@@ -763,7 +880,7 @@ def main():
             resource.cancel()
         if link != None:
             link.teardown()
-        exit()
+        RNS.exit()
 
 def size_str(num, suffix='B'):
     units = ['','K','M','G','T','P','E','Z']

RNS/Utilities/rnid.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2023 Mark Qvist / unsigned.io
+# Copyright (c) 2023-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -63,7 +63,7 @@ def main():
         # parser.add_argument("file", nargs="?", default=None, help="input file path", type=str)
 
         parser.add_argument("--config", metavar="path", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
-        parser.add_argument("-i", "--identity", metavar="identity", action="store", default=None, help="hexadecimal Reticulum Destination hash or path to Identity file", type=str)
+        parser.add_argument("-i", "--identity", metavar="identity", action="store", default=None, help="hexadecimal Reticulum identity or destination hash, or path to Identity file", type=str)
         parser.add_argument("-g", "--generate", metavar="file", action="store", default=None, help="generate a new Identity")
         parser.add_argument("-m", "--import", dest="import_str", metavar="identity_data", action="store", default=None, help="import Reticulum identity in hex, base32 or base64 format", type=str)
         parser.add_argument("-x", "--export", action="store_true", default=None, help="export identity to hex, base32 or base64 format")
@@ -205,29 +205,32 @@ def main():
             if len(identity_str) == RNS.Reticulum.TRUNCATED_HASHLENGTH//8*2 and not os.path.isfile(identity_str):
                 # Try recalling Identity from hex-encoded hash
                 try:
-                    destination_hash = bytes.fromhex(identity_str)
-                    identity = RNS.Identity.recall(destination_hash)
+                    ident_hash = bytes.fromhex(identity_str)
+                    identity = RNS.Identity.recall(ident_hash) or RNS.Identity.recall(ident_hash, from_identity_hash=True)
 
                     if identity == None:
                         if not args.request:
-                            RNS.log("Could not recall Identity for "+RNS.prettyhexrep(destination_hash)+".", RNS.LOG_ERROR)
+                            RNS.log("Could not recall Identity for "+RNS.prettyhexrep(ident_hash)+".", RNS.LOG_ERROR)
                             RNS.log("You can query the network for unknown Identities with the -R option.", RNS.LOG_ERROR)
                             exit(5)
                         else:
-                            RNS.Transport.request_path(destination_hash)
+                            RNS.Transport.request_path(ident_hash)
                             def spincheck():
-                                return RNS.Identity.recall(destination_hash) != None
-                            spin(spincheck, "Requesting unknown Identity for "+RNS.prettyhexrep(destination_hash), args.t)
+                                return RNS.Identity.recall(ident_hash) != None
+                            spin(spincheck, "Requesting unknown Identity for "+RNS.prettyhexrep(ident_hash), args.t)
 
                             if not spincheck():
                                 RNS.log("Identity request timed out", RNS.LOG_ERROR)
                                 exit(6)
                             else:
-                                identity = RNS.Identity.recall(destination_hash)
-                                RNS.log("Received Identity "+str(identity)+" for destination "+RNS.prettyhexrep(destination_hash)+" from the network")
+                                identity = RNS.Identity.recall(ident_hash)
+                                RNS.log("Received Identity "+str(identity)+" for destination "+RNS.prettyhexrep(ident_hash)+" from the network")
 
                     else:
-                        RNS.log("Recalled Identity "+str(identity)+" for destination "+RNS.prettyhexrep(destination_hash))
+                        ident_str = str(identity)
+                        hash_str  = RNS.prettyhexrep(ident_hash)
+                        if ident_str == hash_str: RNS.log(f"Recalled Identity {ident_str}")
+                        else:                     RNS.log(f"Recalled Identity {ident_str} for destination {hash_str}")
 
 
                 except Exception as e:
@@ -286,6 +289,7 @@ def main():
                                 destination = RNS.Destination(identity, RNS.Destination.IN, RNS.Destination.SINGLE, app_name, *aspects)
                                 RNS.log("Created destination "+str(destination))
                                 RNS.log("Announcing destination "+RNS.prettyhexrep(destination.hash))
+                                time.sleep(1.1)
                                 destination.announce()
                                 time.sleep(0.25)
                                 exit(0)

RNS/Utilities/rnir.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2023 Mark Qvist / unsigned.io
+# Copyright (c) 2023-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal

RNS/Utilities/rnodeconf.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2018-2022 Mark Qvist - unsigned.io/rnode
+# Copyright (c) 2018-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -41,7 +41,7 @@ import RNS
 RNS.logtimefmt      = "%H:%M:%S"
 RNS.compact_log_fmt = True
 
-program_version = "2.1.3"
+program_version = "2.4.0"
 eth_addr = "0xFDabC71AC4c0C78C95aDDDe3B4FA19d6273c5E73"
 btc_addr = "35G9uWVzrpJJibzUwpNUQGQNFzLirhrYAH"
 xmr_addr = "87HcDx6jRSkMQ9nPRd5K9hGGpZLn2s7vWETjMaVM5KfV4TD36NcYa8J8WSxhTSvBzzFpqDwp2fg5GX2moZ7VAP9QMZCZGET"
@@ -81,9 +81,14 @@ class KISS():
     CMD_BLINK       = 0x30
     CMD_RANDOM      = 0x40
     CMD_DISP_INT    = 0x45
+    CMD_NP_INT      = 0x65
     CMD_DISP_ADR    = 0x63
+    CMD_DISP_BLNK   = 0x64
+    CMD_DISP_ROT    = 0x67
+    CMD_DISP_RCND   = 0x68
     CMD_BT_CTRL     = 0x46
     CMD_BT_PIN      = 0x62
+    CMD_DIS_IA      = 0x69
     CMD_BOARD       = 0x47
     CMD_PLATFORM    = 0x48
     CMD_MCU         = 0x49
@@ -119,67 +124,86 @@ class KISS():
         return data
 
 class ROM():
-    PLATFORM_AVR   = 0x90
-    PLATFORM_ESP32 = 0x80
-    PLATFORM_NRF52 = 0x70
+    PLATFORM_AVR        = 0x90
+    PLATFORM_ESP32      = 0x80
+    PLATFORM_NRF52      = 0x70
 
-    MCU_1284P      = 0x91
-    MCU_2560       = 0x92
-    MCU_ESP32      = 0x81
-    MCU_NRF52      = 0x71
+    MCU_1284P           = 0x91
+    MCU_2560            = 0x92
+    MCU_ESP32           = 0x81
+    MCU_NRF52           = 0x71
 
-    PRODUCT_RNODE  = 0x03
-    MODEL_A1       = 0xA1
-    MODEL_A6       = 0xA6
-    MODEL_A4       = 0xA4
-    MODEL_A9       = 0xA9
-    MODEL_A3       = 0xA3
-    MODEL_A8       = 0xA8
-    MODEL_A2       = 0xA2
-    MODEL_A7       = 0xA7
+    PRODUCT_RNODE       = 0x03
+    MODEL_A1            = 0xA1
+    MODEL_A6            = 0xA6
+    MODEL_A4            = 0xA4
+    MODEL_A9            = 0xA9
+    MODEL_A3            = 0xA3
+    MODEL_A8            = 0xA8
+    MODEL_A2            = 0xA2
+    MODEL_A7            = 0xA7
+    MODEL_A5            = 0xA5
+    MODEL_AA            = 0xAA
+    MODEL_AC            = 0xAC
 
-    PRODUCT_T32_10 = 0xB2
-    MODEL_BA       = 0xBA
-    MODEL_BB       = 0xBB
+    PRODUCT_T32_10      = 0xB2
+    MODEL_BA            = 0xBA
+    MODEL_BB            = 0xBB
 
-    PRODUCT_T32_20 = 0xB0
-    MODEL_B3       = 0xB3
-    MODEL_B8       = 0xB8
+    PRODUCT_T32_20      = 0xB0
+    MODEL_B3            = 0xB3
+    MODEL_B8            = 0xB8
 
-    PRODUCT_T32_21 = 0xB1
-    MODEL_B4       = 0xB4
-    MODEL_B9       = 0xB9
-    MODEL_B4_TCXO  = 0x04 # The TCXO model codes are only used here to select the
-    MODEL_B9_TCXO  = 0x09 # correct firmware, actual model codes in firmware is
-                          # still 0xB4 and 0xB9.
+    PRODUCT_T32_21      = 0xB1
+    MODEL_B4            = 0xB4
+    MODEL_B9            = 0xB9
+    MODEL_B4_TCXO       = 0x04 # The TCXO model codes are only used here to select the correct firmware,
+    MODEL_B9_TCXO       = 0x09 # actual model codes in firmware is still 0xB4 and 0xB9.
+    PRODUCT_H32_V2      = 0xC0
+    MODEL_C4            = 0xC4
+    MODEL_C9            = 0xC9
 
-    PRODUCT_H32_V2 = 0xC0
-    MODEL_C4       = 0xC4
-    MODEL_C9       = 0xC9
+    PRODUCT_H32_V3      = 0xC1
+    MODEL_C5            = 0xC5
+    MODEL_CA            = 0xCA
 
-    PRODUCT_H32_V3 = 0xC1
-    MODEL_C5       = 0xC5
-    MODEL_CA       = 0xCA
+    PRODUCT_TBEAM       = 0xE0
+    MODEL_E4            = 0xE4
+    MODEL_E9            = 0xE9
+    MODEL_E3            = 0xE3
+    MODEL_E8            = 0xE8
 
-    PRODUCT_TBEAM  = 0xE0
-    MODEL_E4       = 0xE4
-    MODEL_E9       = 0xE9
-    MODEL_E3       = 0xE3
-    MODEL_E8       = 0xE8
+    PRODUCT_TBEAM_S_V1  = 0xEA
+    MODEL_DB            = 0xDB
+    MODEL_DC            = 0xDC
 
-    PRODUCT_RAK4631 = 0x10
-    MODEL_11       = 0x11
-    MODEL_12       = 0x12
-    MODEL_13       = 0x13
-    MODEL_14       = 0x14
-    PRODUCT_OPENCOM_XL = 0x20
-    MODEL_21       = 0x21
+    PRODUCT_TDECK       = 0xD0
+    MODEL_D4            = 0xD4
+    MODEL_D9            = 0xD9
 
+    PRODUCT_RAK4631     = 0x10
+    MODEL_11            = 0x11
+    MODEL_12            = 0x12
+    MODEL_13            = 0x13
+    MODEL_14            = 0x14
 
-    PRODUCT_TECHO  = 0x15
-    MODEL_T4       = 0x16
-    MODEL_T9       = 0x17
+    PRODUCT_OPENCOM_XL  = 0x20
+    MODEL_21            = 0x21
+
+    PRODUCT_TECHO       = 0x15
+    MODEL_16            = 0x16
+    MODEL_17            = 0x17
+
+    PRODUCT_HELTEC_T114 = 0xC2
+    BOARD_HELTEC_T114   = 0x3C
+    MODEL_C6            = 0xC6 # Heltec Mesh Node T114, 470-510 MHz (HT-n5262-LF)
+    MODEL_C7            = 0xC7 # Heltec Mesh Node T114, 863-928 MHz (HT-n5262-HF)
     
+    PRODUCT_XIAO_S3     = 0xEB
+    BOARD_XIAO_S3       = 0x3E
+    MODEL_DE            = 0xDE # Xiao ESP32S3 with Wio-SX1262 module, 433 MHz
+    MODEL_DD            = 0xDD # Xiao ESP32S3 with Wio-SX1262 module, 868 MHz
+
     PRODUCT_HMBRW  = 0xF0
     MODEL_FF       = 0xFF
     MODEL_FE       = 0xFE
@@ -199,12 +223,24 @@ class ROM():
     ADDR_CONF_FREQ = 0xA3
     ADDR_CONF_OK   = 0xA7
 
+    ADDR_CONF_BT   = 0xB0
+    ADDR_CONF_DSET = 0xB1
+    ADDR_CONF_DINT = 0xB2
+    ADDR_CONF_DADR = 0xB3
+    ADDR_CONF_DBLK = 0xB4
+    ADDR_CONF_DROT = 0xB8
+    ADDR_CONF_PSET = 0xB5
+    ADDR_CONF_PINT = 0xB6
+    ADDR_CONF_BSET = 0xB7
+    ADDR_CONF_DIA  = 0xB9
+
     INFO_LOCK_BYTE = 0x73
     CONF_OK_BYTE   = 0x73
 
     BOARD_RNODE         = 0x31
     BOARD_HMBRW         = 0x32
     BOARD_TBEAM         = 0x33
+    BOARD_TDECK         = 0x3B
     BOARD_HUZZAH32      = 0x34
     BOARD_GENERIC_ESP32 = 0x35
     BOARD_LORA32_V2_0   = 0x36
@@ -212,13 +248,15 @@ class ROM():
     BOARD_TECHO         = 0x43
     BOARD_RAK4631       = 0x51
 
-    MANUAL_FLASH_MODELS = [MODEL_A1, MODEL_A6]
+    MANUAL_FLASH_MODELS = []
 
 mapped_product = ROM.PRODUCT_RNODE
 products = {
     ROM.PRODUCT_RNODE:  "RNode",
     ROM.PRODUCT_HMBRW:  "Hombrew RNode",
     ROM.PRODUCT_TBEAM:  "LilyGO T-Beam",
+    ROM.PRODUCT_TBEAM_S_V1:"LilyGO T-Beam Supreme",
+    ROM.PRODUCT_TDECK:  "LilyGO T-Deck",
     ROM.PRODUCT_T32_10: "LilyGO LoRa32 v1.0",
     ROM.PRODUCT_T32_20: "LilyGO LoRa32 v2.0",
     ROM.PRODUCT_T32_21: "LilyGO LoRa32 v2.1",
@@ -227,6 +265,8 @@ products = {
     ROM.PRODUCT_TECHO:  "LilyGO T-Echo",
     ROM.PRODUCT_RAK4631: "RAK4631",
     ROM.PRODUCT_OPENCOM_XL: "openCom XL",
+    ROM.PRODUCT_HELTEC_T114: "Heltec Mesh Node T114",
+    ROM.PRODUCT_XIAO_S3: "Seeed XIAO ESP32S3 Wio-SX1262",
 }
 
 platforms = {
@@ -247,6 +287,9 @@ models = {
     0xA9: [820000000, 1020000000, 17, "820 - 1020 MHz", "rnode_firmware.hex", "SX1276"],
     0xA1: [410000000, 525000000, 22, "410 - 525 MHz", "rnode_firmware_t3s3.zip", "SX1268"],
     0xA6: [820000000, 1020000000, 22, "820 - 960 MHz", "rnode_firmware_t3s3.zip", "SX1262"],
+    0xA5: [410000000, 525000000, 17, "410 - 525 MHz", "rnode_firmware_t3s3_sx127x.zip", "SX1278"],
+    0xAA: [820000000, 1020000000, 17, "820 - 960 MHz", "rnode_firmware_t3s3_sx127x.zip", "SX1276"],
+    0xAC: [2400000000, 2500000000, 20, "2.4 - 2.5 GHz", "rnode_firmware_t3s3_sx1280_pa.zip", "SX1280"],
     0xA2: [410000000, 525000000, 17, "410 - 525 MHz", "rnode_firmware_ng21.zip", "SX1278"],
     0xA7: [820000000, 1020000000, 17, "820 - 1020 MHz", "rnode_firmware_ng21.zip", "SX1276"],
     0xA3: [410000000, 525000000, 17, "410 - 525 MHz", "rnode_firmware_ng20.zip", "SX1278"],
@@ -261,19 +304,27 @@ models = {
     0xBB: [850000000, 950000000, 17, "850 - 950 MHz", "rnode_firmware_lora32v10.zip", "SX1276"],
     0xC4: [420000000, 520000000, 17, "420 - 520 MHz", "rnode_firmware_heltec32v2.zip", "SX1278"],
     0xC9: [850000000, 950000000, 17, "850 - 950 MHz", "rnode_firmware_heltec32v2.zip", "SX1276"],
-    0xC5: [470000000, 510000000, 21, "470 - 510 MHz", "rnode_firmware_heltec32v3.zip", "SX1262"],
-    0xCA: [863000000, 928000000, 21, "863 - 928 MHz", "rnode_firmware_heltec32v3.zip", "SX1262"],
+    0xC5: [420000000, 520000000, 22, "420 - 520 MHz", "rnode_firmware_heltec32v3.zip", "SX1268"],
+    0xCA: [850000000, 950000000, 22, "850 - 950 MHz", "rnode_firmware_heltec32v3.zip", "SX1262"],
+    0xC6: [420000000, 520000000, 22, "420 - 520 MHz", "rnode_firmware_heltec_t114.zip", "SX1268"],
+    0xC7: [850000000, 950000000, 22, "850 - 950 MHz", "rnode_firmware_heltec_t114.zip", "SX1262"],
     0xE4: [420000000, 520000000, 17, "420 - 520 MHz", "rnode_firmware_tbeam.zip", "SX1278"],
     0xE9: [850000000, 950000000, 17, "850 - 950 MHz", "rnode_firmware_tbeam.zip", "SX1276"],
+    0xD4: [420000000, 520000000, 22, "420 - 520 MHz", "rnode_firmware_tdeck.zip", "SX1268"],
+    0xD9: [850000000, 950000000, 22, "850 - 950 MHz", "rnode_firmware_tdeck.zip", "SX1262"],
+    0xDB: [420000000, 520000000, 22, "420 - 520 MHz", "rnode_firmware_tbeam_supreme.zip", "SX1268"],
+    0xDC: [850000000, 950000000, 22, "850 - 950 MHz", "rnode_firmware_tbeam_supreme.zip", "SX1262"],
     0xE3: [420000000, 520000000, 22, "420 - 520 MHz", "rnode_firmware_tbeam_sx1262.zip", "SX1268"],
     0xE8: [850000000, 950000000, 22, "850 - 950 MHz", "rnode_firmware_tbeam_sx1262.zip", "SX1262"],
     0x11: [430000000, 510000000, 22, "430 - 510 MHz", "rnode_firmware_rak4631.zip", "SX1262"],
     0x12: [779000000, 928000000, 22, "779 - 928 MHz", "rnode_firmware_rak4631.zip", "SX1262"],
-    0x11: [430000000, 510000000, 22, "430 - 510 MHz", "rnode_firmware_rak4631_sx1280.zip", "SX1262 + SX1280"],
-    0x12: [779000000, 928000000, 22, "779 - 928 MHz", "rnode_firmware_rak4631_sx1280.zip", "SX1262 + SX1280"],
+    0x13: [430000000, 510000000, 22, "430 - 510 MHz", "rnode_firmware_rak4631_sx1280.zip", "SX1262 + SX1280"],
+    0x14: [779000000, 928000000, 22, "779 - 928 MHz", "rnode_firmware_rak4631_sx1280.zip", "SX1262 + SX1280"],
     0x16: [779000000, 928000000, 22, "430 - 510 Mhz", "rnode_firmware_techo.zip", "SX1262"],
     0x17: [779000000, 928000000, 22, "779 - 928 Mhz", "rnode_firmware_techo.zip", "SX1262"],
     0x21: [820000000, 960000000, 22, "820 - 960 MHz", "rnode_firmware_opencom_xl.zip", "SX1262 + SX1280"],
+    0xDE: [420000000, 520000000, 22, "420 - 520 MHz", "rnode_firmware_xiao_esp32s3.zip", "SX1262"],
+    0xDD: [850000000, 950000000, 22, "850 - 950 MHz", "rnode_firmware_xiao_esp32s3.zip", "SX1262"],
     0xFE: [100000000, 1100000000, 17, "(Band capabilities unknown)", None, "Unknown"],
     0xFF: [100000000, 1100000000, 14, "(Band capabilities unknown)", None, "Unknown"],
 }
@@ -637,6 +688,44 @@ class RNode():
         if written != len(kiss_command):
             raise IOError("An IO error occurred while sending display intensity command to device")
 
+    def set_display_blanking(self, blanking_timeout):
+        data = bytes([blanking_timeout & 0xFF])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_DISP_BLNK])+data+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("An IO error occurred while sending display blanking timeout command to device")
+
+    def set_display_rotation(self, rotation):
+        data = bytes([rotation & 0xFF])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_DISP_ROT])+data+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("An IO error occurred while sending display rotation command to device")
+
+    def recondition_display(self):
+        data = bytes([0x01])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_DISP_RCND])+data+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("An IO error occurred while sending display recondition command to device")
+
+    def set_disable_interference_avoidance(self, ia_disabled):
+        if ia_disabled:
+            data = bytes([0x01])
+        else:
+            data = bytes([0x00])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_DIS_IA])+data+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("An IO error occurred while sending interference avoidance configuration command to device")
+
+    def set_neopixel_intensity(self, intensity):
+        data = bytes([intensity & 0xFF])
+        kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_NP_INT])+data+bytes([KISS.FEND])
+        written = self.serial.write(kiss_command)
+        if written != len(kiss_command):
+            raise IOError("An IO error occurred while sending NeoPixel intensity command to device")
+
     def set_display_address(self, address):
         data = bytes([address & 0xFF])
         kiss_command = bytes([KISS.FEND])+bytes([KISS.CMD_DISP_ADR])+data+bytes([KISS.FEND])
@@ -1113,9 +1202,10 @@ def ensure_firmware_file(fw_filename):
                 RNS.log("The selected firmware for this board is version "+selected_version)
 
             else:
-                RNS.log("Online firmware version check was disabled, but no firmware version specified for install.")
-                RNS.log("use the --fw-version option to manually specify a version.")
-                graceful_exit(98)
+                if selected_version == None:
+                    RNS.log("Online firmware version check was disabled, but no firmware version specified for install.")
+                    RNS.log("use the --fw-version option to manually specify a version.")
+                    graceful_exit(98)
 
             # if custom firmware url, use it
             if fw_url != None:
@@ -1159,6 +1249,7 @@ def ensure_firmware_file(fw_filename):
                         pass
                     else:
                         RNS.log("")
+                        RNS.log(f"Firmware hash {file_hash} but should be {selected_hash}, possibly due to download corruption.")
                         RNS.log("Firmware corrupt. Try clearing the local firmware cache with: rnodeconf --clear-cache")
                         graceful_exit(96)
 
@@ -1261,7 +1352,12 @@ def main():
         parser.add_argument("-p", "--bluetooth-pair", action="store_true", help="Put device into bluetooth pairing mode")
 
         parser.add_argument("-D", "--display", action="store", metavar="i", type=int, default=None, help="Set display intensity (0-255)")
+        parser.add_argument("-t", "--timeout", action="store", metavar="s", type=int, default=None, help="Set display timeout in seconds, 0 to disable")
+        parser.add_argument("-R", "--rotation", action="store", metavar="rotation", type=int, default=None, help="Set display rotation, valid values are 0 through 3")
         parser.add_argument("--display-addr", action="store", metavar="byte", type=str, default=None, help="Set display address as hex byte (00 - FF)")
+        parser.add_argument("--recondition-display", action="store_true", help="Start display reconditioning")
+
+        parser.add_argument("--np", action="store", metavar="i", type=int, default=None, help="Set NeoPixel intensity (0-255)")
 
         parser.add_argument("--freq", action="store", metavar="Hz", type=int, default=None, help="Frequency in Hz for TNC mode")
         parser.add_argument("--bw", action="store", metavar="Hz", type=int, default=None, help="Bandwidth in Hz for TNC mode")
@@ -1269,6 +1365,11 @@ def main():
         parser.add_argument("--sf", action="store", metavar="factor", type=int, default=None, help="Spreading factor for TNC mode (7 - 12)")
         parser.add_argument("--cr", action="store", metavar="rate", type=int, default=None, help="Coding rate for TNC mode (5 - 8)")
 
+        parser.add_argument("-x", "--ia-enable", action="store_true", help="Enable interference avoidance")
+        parser.add_argument("-X", "--ia-disable", action="store_true", help="Disable interference avoidance")
+
+        parser.add_argument("-c", "--config", action="store_true", help="Print device configuration")
+
         parser.add_argument("--eeprom-backup", action="store_true", help="Backup EEPROM to file")
         parser.add_argument("--eeprom-dump", action="store_true", help="Dump EEPROM to console")
         parser.add_argument("--eeprom-wipe", action="store_true", help="Unlock and wipe EEPROM")
@@ -1282,7 +1383,7 @@ def main():
         parser.add_argument("-r", "--rom", action="store_true", help="Bootstrap EEPROM without flashing firmware")
         parser.add_argument("-k", "--key", action="store_true", help="Generate a new signing key and exit") # 
         parser.add_argument("-S", "--sign", action="store_true", help="Display public part of signing key")
-        parser.add_argument("-H", "--firmware-hash", action="store", help="Display installed firmware hash")
+        parser.add_argument("-H", "--firmware-hash", action="store", help="Set installed firmware hash")
         parser.add_argument("-K", "--get-target-firmware-hash", action="store_true", help=argparse.SUPPRESS) # Get target firmware hash from device
         parser.add_argument("-L", "--get-firmware-hash", action="store_true", help=argparse.SUPPRESS) # Get calculated firmware hash from device
         parser.add_argument("--platform", action="store", metavar="platform", type=str, default=None, help="Platform specification for device bootstrap")
@@ -1602,28 +1703,32 @@ def main():
                 
             print("")
             print("What kind of device is this?\n")
-            print("[1] A specific kind of RNode")
-            print("       .")
-            print("      / \\   Select this option if you have an RNode of a specific")
-            print("       |    type, built from a recipe or bought from a vendor.")
+            print("       |   Select this option if you have an RNode of a specific")
+            print("      \\ /  type, built from a recipe or bought from a vendor.")
+            print("       '")
+            print("[1]  A specific kind of RNode")
             print("")
-            print("[2] Homebrew RNode")
-            print("       .")
-            print("      / \\   Select this option if you have put toghether an RNode")
-            print("       |    of your own design, or if you are prototyping one.")
+            print("       |   Select this option if you have put toghether an RNode")
+            print("      \\ /  of your own design, or if you are prototyping one.")
+            print("       '")
+            print("[2]  Homebrew RNode")
             print("")
-            print("[3] LilyGO LoRa32 v2.1 (aka T3 v1.6 / T3 v1.6.1)")
-            print("[4] LilyGO LoRa32 v2.0")
-            print("[5] LilyGO LoRa32 v1.0")
-            print("[6] LilyGO T-Beam")
-            print("[7] Heltec LoRa32 v2")
-            print("[8] Heltec LoRa32 v3")
-            print("[9] LilyGO LoRa T3S3")
+            print("       |   Select one of these options if you want to easily turn")
+            print("      \\ /  a supported development board into an RNode.")
+            print("       '")
+            print("[3]  LilyGO LoRa32 v2.1 (aka T3 v1.6 / T3 v1.6.1)")
+            print("[4]  LilyGO LoRa32 v2.0")
+            print("[5]  LilyGO LoRa32 v1.0")
+            print("[6]  LilyGO T-Beam")
+            print("[7]  Heltec LoRa32 v2")
+            print("[8]  Heltec LoRa32 v3")
+            print("[9]  LilyGO LoRa T3S3")
             print("[10] RAK4631")
             print("[11] LilyGo T-Echo")
-            print("       .")
-            print("      / \\   Select one of these options if you want to easily turn")
-            print("       |    a supported development board into an RNode.")
+            print("[12] LilyGO T-Beam Supreme")
+            print("[13] LilyGO T-Deck")
+            print("[14] Heltec T114")
+            print("[15] Seeed XIAO ESP32S3 Wio-SX1262")
             print("")
             print("---------------------------------------------------------------------------")
             print("\nEnter the number that matches your device type:\n? ", end="")
@@ -1632,7 +1737,7 @@ def main():
             try:
                 c_dev = int(input())
                 c_mod = False
-                if c_dev < 1 or c_dev > 11:
+                if c_dev < 1 or c_dev > 15:
                     raise ValueError()
                 elif c_dev == 1:
                     selected_product = ROM.PRODUCT_RNODE
@@ -1669,6 +1774,38 @@ def main():
                     print("who would like to experiment with it. Hit enter to continue.")
                     print("---------------------------------------------------------------------------")
                     input()
+                elif c_dev == 12:
+                    selected_product = ROM.PRODUCT_TBEAM_S_V1
+                    clear()
+                    print("")
+                    print("---------------------------------------------------------------------------")
+                    print("                       T-Beam Supreme RNode Installer")
+                    print("")
+                    print("The RNode firmware can currently be installed on T-Beam Supreme devices")
+                    print("using the SX1262 and SX1268 transceiver chips.")
+                    print("")
+                    print("Important! Using RNode firmware on T-Beam devices should currently be")
+                    print("considered experimental. It is not intended for production or critical use.")
+                    print("The currently supplied firmware is provided AS-IS as a courtesey to those")
+                    print("who would like to experiment with it. Hit enter to continue.")
+                    print("---------------------------------------------------------------------------")
+                    input()
+                elif c_dev == 13:
+                    selected_product = ROM.PRODUCT_TDECK
+                    clear()
+                    print("")
+                    print("---------------------------------------------------------------------------")
+                    print("                            T-Deck RNode Installer")
+                    print("")
+                    print("The RNode firmware can currently be installed on T-Deck devices using the")
+                    print("SX1262 and SX1268 transceiver chips.")
+                    print("")
+                    print("Important! Using RNode firmware on T-Beam devices should currently be")
+                    print("considered experimental. It is not intended for production or critical use.")
+                    print("The currently supplied firmware is provided AS-IS as a courtesey to those")
+                    print("who would like to experiment with it. Hit enter to continue.")
+                    print("---------------------------------------------------------------------------")
+                    input()
                 elif c_dev == 4:
                     selected_product = ROM.PRODUCT_T32_20
                     clear()
@@ -1739,8 +1876,6 @@ def main():
                     print("Important! Using RNode firmware on T3S3 devices should currently be")
                     print("considered experimental. It is not intended for production or critical use.")
                     print("")
-                    print("Please note that Bluetooth is currently not implemented on this board.")
-                    print("")
                     print("The currently supplied firmware is provided AS-IS as a courtesey to those")
                     print("who would like to experiment with it. Hit enter to continue.")
                     print("---------------------------------------------------------------------------")
@@ -1755,8 +1890,6 @@ def main():
                     print("Important! Using RNode firmware on Heltec devices should currently be")
                     print("considered experimental. It is not intended for production or critical use.")
                     print("")
-                    print("Please note that Bluetooth is currently not implemented on this board.")
-                    print("")
                     print("The currently supplied firmware is provided AS-IS as a courtesey to those")
                     print("who would like to experiment with it. Hit enter to continue.")
                     print("---------------------------------------------------------------------------")
@@ -1787,6 +1920,33 @@ def main():
                     print("who would like to experiment with it. Hit enter to continue.")
                     print("---------------------------------------------------------------------------")
                     input()
+                elif c_dev == 14:
+                    selected_product = ROM.PRODUCT_HELTEC_T114
+                    clear()
+                    print("")
+                    print("---------------------------------------------------------------------------")
+                    print("                      Heltec T114 RNode Installer")
+                    print("")
+                    print("Important! Using RNode firmware on Heltec T114 devices should currently be")
+                    print("considered experimental. It is not intended for production or critical use.")
+                    print("The currently supplied firmware is provided AS-IS as a courtesey to those")
+                    print("who would like to experiment with it. Hit enter to continue.")
+                    print("---------------------------------------------------------------------------")
+                    input()
+                elif c_dev == 15:
+                    selected_product = ROM.PRODUCT_XIAO_S3
+                    clear()
+                    print("")
+                    print("---------------------------------------------------------------------------")
+                    print("                 SeeedStudio XIAO esp32s3 wio RNode Installer")
+                    print("")
+                    print("Important! Using RNode firmware on SeeedStudio XIAO/wio devices should currently be")
+                    print("considered experimental. It is not intended for production or critical use.")
+                    print("The currently supplied firmware is provided AS-IS as a courtesey to those")
+                    print("who would like to experiment with it. Hit enter to continue.")
+                    print("---------------------------------------------------------------------------")
+                    input()
+
             except Exception as e:
                 print("That device type does not exist, exiting now.")
                 graceful_exit()
@@ -1889,22 +2049,39 @@ def main():
                         print("That model does not exist, exiting now.")
                         graceful_exit()
                 else:
-                    print("\nWhat model is this T3S3?\n")
-                    print("[1] 410 - 525 MHz (with SX1268 chip)")
-                    print("[2] 820 - 1020 MHz (with SX1262 chip)")
+                    print("\nWhat band is this T3S3 for?\n")
+                    print("[1] 433 MHz         (with SX1278 chip)")
+                    print("[2] 868/915/923 MHz (with SX1276 chip)")
+                    print("");
+                    print("[3] 433 MHz         (with SX1268 chip)")
+                    print("[4] 868/915/923 MHz (with SX1262 chip)")
+                    print("");
+                    print("[5] 2.4 GHz         (with SX1280 chip and PA)")
                     print("\n? ", end="")
                     try:
                         c_model = int(input())
-                        if c_model < 1 or c_model > 2:
+                        if c_model < 1 or c_model > 5:
                             raise ValueError()
                         elif c_model == 1:
-                            selected_model = ROM.MODEL_A1
+                            selected_model = ROM.MODEL_A5
                             selected_mcu = ROM.MCU_ESP32
                             selected_platform = ROM.PLATFORM_ESP32
                         elif c_model == 2:
+                            selected_model = ROM.MODEL_AA
+                            selected_mcu = ROM.MCU_ESP32
+                            selected_platform = ROM.PLATFORM_ESP32
+                        elif c_model == 3:
+                            selected_model = ROM.MODEL_A1
+                            selected_mcu = ROM.MCU_ESP32
+                            selected_platform = ROM.PLATFORM_ESP32
+                        elif c_model == 4:
                             selected_model = ROM.MODEL_A6
                             selected_mcu = ROM.MCU_ESP32
                             selected_platform = ROM.PLATFORM_ESP32
+                        elif c_model == 5:
+                            selected_model = ROM.MODEL_AC
+                            selected_mcu = ROM.MCU_ESP32
+                            selected_platform = ROM.PLATFORM_ESP32
                     except Exception as e:
                         print("That model does not exist, exiting now.")
                         graceful_exit()
@@ -1938,6 +2115,46 @@ def main():
                     print("That band does not exist, exiting now.")
                     graceful_exit()
 
+            elif selected_product == ROM.PRODUCT_TBEAM_S_V1:
+                selected_mcu = ROM.MCU_ESP32
+                print("\nWhat band is this T-Beam Supreme for?\n")
+                print("[1] 433 MHz         (with SX1268 chip)")
+                print("[2] 868/915/923 MHz (with SX1262 chip)")
+                print("\n? ", end="")
+                try:
+                    c_model = int(input())
+                    if c_model < 1 or c_model > 2:
+                        raise ValueError()
+                    elif c_model == 1:
+                        selected_model = ROM.MODEL_DB
+                        selected_platform = ROM.PLATFORM_ESP32
+                    elif c_model == 2:
+                        selected_model = ROM.MODEL_DC
+                        selected_platform = ROM.PLATFORM_ESP32
+                except Exception as e:
+                    print("That band does not exist, exiting now.")
+                    graceful_exit()
+
+            elif selected_product == ROM.PRODUCT_TDECK:
+                selected_mcu = ROM.MCU_ESP32
+                print("\nWhat band is this T-Deck for?\n")
+                print("[1] 433 MHz         (with SX1268 chip)")
+                print("[2] 868/915/923 MHz (with SX1262 chip)")
+                print("\n? ", end="")
+                try:
+                    c_model = int(input())
+                    if c_model < 1 or c_model > 2:
+                        raise ValueError()
+                    elif c_model == 1:
+                        selected_model = ROM.MODEL_D4
+                        selected_platform = ROM.PLATFORM_ESP32
+                    elif c_model == 2:
+                        selected_model = ROM.MODEL_D9
+                        selected_platform = ROM.PLATFORM_ESP32
+                except Exception as e:
+                    print("That band does not exist, exiting now.")
+                    graceful_exit()
+
             elif selected_product == ROM.PRODUCT_T32_10:
                 selected_mcu = ROM.MCU_ESP32
                 print("\nWhat band is this LoRa32 for?\n")
@@ -2052,6 +2269,47 @@ def main():
                 except Exception as e:
                     print("That band does not exist, exiting now.")
                     exit()
+            
+            elif selected_product == ROM.PRODUCT_HELTEC_T114:
+                selected_mcu = ROM.MCU_NRF52
+                print("\nWhat band is this Heltec T114 for?\n")
+                print("[1] 433 MHz")
+                print("[2] 868 MHz")
+                print("[3] 915 MHz")
+                print("[4] 923 MHz")
+                try:
+                    c_model = int(input())
+                    if c_model < 1 or c_model > 4:
+                        raise ValueError()
+                    elif c_model == 1:
+                        selected_model = ROM.MODEL_C6
+                        selected_platform = ROM.PLATFORM_NRF52
+                    elif c_model > 1:
+                        selected_model = ROM.MODEL_C7
+                        selected_platform = ROM.PLATFORM_NRF52
+                except Exception as e:
+                    print("That band does not exist, exiting now.")
+                    exit()
+
+            elif selected_product == ROM.PRODUCT_XIAO_S3:
+                selected_mcu = ROM.MCU_ESP32
+                print("\nWhat band is this XIAO esp32s3 wio module for?\n")
+                print("[1] 433 MHz")
+                print("[2] 868 MHz")
+                try:
+                    c_model = int(input())
+                    if c_model < 1 or c_model > 2:
+                        raise ValueError()
+                    elif c_model == 1:
+                        selected_model = ROM.MODEL_DE
+                        selected_platform = ROM.PLATFORM_ESP32
+                    elif c_model == 2:
+                        selected_model = ROM.MODEL_DD
+                        selected_platform = ROM.PLATFORM_ESP32
+                except Exception as e:
+                    print("That band does not exist, exiting now.")
+                    exit()
+
             elif selected_product == ROM.PRODUCT_RAK4631:
                 selected_mcu = ROM.MCU_NRF52
                 print("\nWhat band is this RAK4631 for?\n")
@@ -2083,13 +2341,13 @@ def main():
                 print("\n? ", end="")
                 try:
                     c_model = int(input())
-                    if c_model < 1 or c_model > 1:
+                    if c_model < 1 or c_model > 4:
                         raise ValueError()
                     elif c_model == 1:
-                        selected_model = ROM.MODEL_T4
+                        selected_model = ROM.MODEL_16
                         selected_platform = ROM.PLATFORM_NRF52
                     elif c_model > 1:
-                        selected_model = ROM.MODEL_T9
+                        selected_model = ROM.MODEL_17
                         selected_platform = ROM.PLATFORM_NRF52
                 except Exception as e:
                     print("That band does not exist, exiting now.")
@@ -2606,6 +2864,7 @@ def main():
                             "0xe000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_heltec32v3.boot_app0",
                             "0x0",  UPD_DIR+"/"+selected_version+"/rnode_firmware_heltec32v3.bootloader",
                             "0x10000", UPD_DIR+"/"+selected_version+"/rnode_firmware_heltec32v3.bin",
+                            "0x210000",UPD_DIR+"/"+selected_version+"/console_image.bin",
                             "0x8000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_heltec32v3.partitions",
                         ]
                     elif fw_filename == "rnode_firmware_featheresp32.zip":
@@ -2770,6 +3029,96 @@ def main():
                             "0x210000",UPD_DIR+"/"+selected_version+"/console_image.bin",
                             "0x8000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3.partitions",
                         ]
+                    elif fw_filename == "rnode_firmware_t3s3_sx127x.zip":
+                        return [
+                            sys.executable, flasher,
+                            "--chip", "esp32s3",
+                            "--port", args.port,
+                            "--baud", args.baud_flash,
+                            "--before", "default_reset",
+                            "--after", "hard_reset",
+                            "write_flash", "-z",
+                            "--flash_mode", "dio",
+                            "--flash_freq", "80m",
+                            "--flash_size", "4MB",
+                            "0xe000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3_sx127x.boot_app0",
+                            "0x0",  UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3_sx127x.bootloader",
+                            "0x10000", UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3_sx127x.bin",
+                            "0x210000",UPD_DIR+"/"+selected_version+"/console_image.bin",
+                            "0x8000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3_sx127x.partitions",
+                        ]
+                    elif fw_filename == "rnode_firmware_t3s3_sx1280_pa.zip":
+                        return [
+                            sys.executable, flasher,
+                            "--chip", "esp32s3",
+                            "--port", args.port,
+                            "--baud", args.baud_flash,
+                            "--before", "default_reset",
+                            "--after", "hard_reset",
+                            "write_flash", "-z",
+                            "--flash_mode", "dio",
+                            "--flash_freq", "80m",
+                            "--flash_size", "4MB",
+                            "0xe000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3_sx1280_pa.boot_app0",
+                            "0x0",  UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3_sx1280_pa.bootloader",
+                            "0x10000", UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3_sx1280_pa.bin",
+                            "0x210000",UPD_DIR+"/"+selected_version+"/console_image.bin",
+                            "0x8000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_t3s3_sx1280_pa.partitions",
+                        ]
+                    elif fw_filename == "rnode_firmware_tbeam_supreme.zip":
+                        return [
+                            sys.executable, flasher,
+                            "--chip", "esp32s3",
+                            "--port", args.port,
+                            "--baud", args.baud_flash,
+                            "--before", "default_reset",
+                            "--after", "hard_reset",
+                            "write_flash", "-z",
+                            "--flash_mode", "dio",
+                            "--flash_freq", "80m",
+                            "--flash_size", "4MB",
+                            "0xe000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_tbeam_supreme.boot_app0",
+                            "0x0",  UPD_DIR+"/"+selected_version+"/rnode_firmware_tbeam_supreme.bootloader",
+                            "0x10000", UPD_DIR+"/"+selected_version+"/rnode_firmware_tbeam_supreme.bin",
+                            "0x210000",UPD_DIR+"/"+selected_version+"/console_image.bin",
+                            "0x8000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_tbeam_supreme.partitions",
+                        ]
+                    elif fw_filename == "rnode_firmware_tdeck.zip":
+                        return [
+                            sys.executable, flasher,
+                            "--chip", "esp32s3",
+                            "--port", args.port,
+                            "--baud", args.baud_flash,
+                            "--before", "default_reset",
+                            "--after", "hard_reset",
+                            "write_flash", "-z",
+                            "--flash_mode", "dio",
+                            "--flash_freq", "80m",
+                            "--flash_size", "4MB",
+                            "0xe000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_tdeck.boot_app0",
+                            "0x0",  UPD_DIR+"/"+selected_version+"/rnode_firmware_tdeck.bootloader",
+                            "0x10000", UPD_DIR+"/"+selected_version+"/rnode_firmware_tdeck.bin",
+                            "0x210000",UPD_DIR+"/"+selected_version+"/console_image.bin",
+                            "0x8000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_tdeck.partitions",
+                        ]
+                    elif fw_filename == "rnode_firmware_xiao_esp32s3.zip":
+                        return [
+                            sys.executable, flasher,
+                            "--chip", "esp32s3",
+                            "--port", args.port,
+                            "--baud", args.baud_flash,
+                            "--before", "default_reset",
+                            "--after", "hard_reset",
+                            "write_flash", "-z",
+                            "--flash_mode", "dio",
+                            "--flash_freq", "80m",
+                            "--flash_size", "8MB",
+                            "0xe000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_xiao_esp32s3.boot_app0",
+                            "0x0",  UPD_DIR+"/"+selected_version+"/rnode_firmware_xiao_esp32s3.bootloader",
+                            "0x10000", UPD_DIR+"/"+selected_version+"/rnode_firmware_xiao_esp32s3.bin",
+                            "0x210000",UPD_DIR+"/"+selected_version+"/console_image.bin",
+                            "0x8000",  UPD_DIR+"/"+selected_version+"/rnode_firmware_xiao_esp32s3.partitions",
+                        ]
                     elif fw_filename == "extracted_rnode_firmware.zip":
                         return [
                             sys.executable, flasher,
@@ -2883,12 +3232,12 @@ def main():
                                 if args.platform == ROM.PLATFORM_ESP32:
                                     wants_fw_provision = True
                                     RNS.log("Waiting for ESP32 reset...")
-                                    time.sleep(7)
+                                    time.sleep(8)
                                 if args.platform == ROM.PLATFORM_NRF52:
                                     wants_fw_provision = True
                                     RNS.log("Waiting for NRF52 reset...")
                                     # Don't need to wait as long this time.
-                                    time.sleep(5)
+                                    time.sleep(6)
                             else:
                                 RNS.log("Error from flasher ("+str(flash_status)+") while writing.")
                                 RNS.log("Some boards have trouble flashing at high speeds, and you can")
@@ -3140,6 +3489,63 @@ def main():
                     RNS.log("Firmware update file not found")
                     graceful_exit()
 
+            if args.config:
+                eeprom_reserved = 200
+                if rnode.platform == ROM.PLATFORM_ESP32:
+                    eeprom_size = 296
+                elif rnode.platform == ROM.PLATFORM_NRF52:
+                    eeprom_size = 296
+                else:
+                    eeprom_size = 4096
+
+                eeprom_offset = eeprom_size-eeprom_reserved
+                def ea(a):
+                    return a+eeprom_offset
+                ec_bt   = rnode.eeprom[ROM.ADDR_CONF_BT]
+                ec_dint = rnode.eeprom[ROM.ADDR_CONF_DINT]
+                ec_dadr = rnode.eeprom[ROM.ADDR_CONF_DADR]
+                ec_dblk = rnode.eeprom[ROM.ADDR_CONF_DBLK]
+                ec_drot = rnode.eeprom[ROM.ADDR_CONF_DROT]
+                ec_pset = rnode.eeprom[ROM.ADDR_CONF_PSET]
+                ec_pint = rnode.eeprom[ROM.ADDR_CONF_PINT]
+                ec_bset = rnode.eeprom[ROM.ADDR_CONF_BSET]
+                ec_dia  = rnode.eeprom[ROM.ADDR_CONF_DIA]
+                print("\nDevice configuration:")
+                if ec_bt == 0x73:
+                    print(f"  Bluetooth              : Enabled")
+                else:
+                    print(f"  Bluetooth              : Disabled")
+                if ec_dia == 0x00:
+                    print(f"  Interference avoidance : Enabled")
+                else:
+                    print(f"  Interference avoidance : Disabled")
+                print(    f"  Display brightness     : {ec_dint}")
+                if ec_dadr == 0xFF:
+                    print(f"  Display address        : Default")
+                else:
+                    print(f"  Display address        : {RNS.hexrep(ec_dadr, delimit=False)}")
+                if ec_bset == 0x73 and ec_dblk != 0x00:
+                    print(f"  Display blanking       : {ec_dblk}s")
+                else:
+                    print(f"  Display blanking       : Disabled")
+                if ec_drot != 0xFF:
+                    if ec_drot == 0x00:
+                        rstr = "Landscape"
+                    if ec_drot == 0x01:
+                        rstr = "Portrait"
+                    if ec_drot == 0x02:
+                        rstr = "Landscape 180"
+                    if ec_drot == 0x03:
+                        rstr = "Portrait 180"
+                    print(f"  Display rotation       : {rstr}")
+                else:
+                    print(f"  Display rotation       : Default")
+                if ec_pset == 0x73:
+                    print(f"  Neopixel Intensity     : {ec_pint}")
+                print("")
+
+                graceful_exit()
+
             if args.eeprom_dump:
                 RNS.log("EEPROM contents:")
                 RNS.log(RNS.hexrep(rnode.eeprom))
@@ -3168,6 +3574,52 @@ def main():
                 RNS.log("Setting display intensity to "+str(di))
                 rnode.set_display_intensity(di)
 
+            if isinstance(args.timeout, int):
+                di = args.timeout
+                if di < 0:
+                    di = 0
+                if di > 255:
+                    di = 255
+                if di == 0:
+                    RNS.log("Disabling display blanking")
+                else:
+                    RNS.log("Setting display timeout to "+str(di))
+                rnode.set_display_blanking(di)
+
+            if isinstance(args.rotation, int):
+                dr = args.rotation
+                if dr < 0:
+                    dr = 0
+                if dr > 3:
+                    dr = 3
+
+                RNS.log("Setting display rotation to "+str(dr))
+                rnode.set_display_rotation(dr)
+
+            if isinstance(args.recondition_display, bool):
+                if args.recondition_display:
+                    RNS.log("Starting display reconditioning")
+                    rnode.recondition_display()
+
+            if isinstance(args.ia_enable, bool):
+                if args.ia_enable:
+                    RNS.log("Enabling interference avoidance")
+                    rnode.set_disable_interference_avoidance(False)
+
+            if isinstance(args.ia_disable, bool):
+                if args.ia_disable:
+                    RNS.log("Disabling interference avoidance")
+                    rnode.set_disable_interference_avoidance(True)
+
+            if isinstance(args.np, int):
+                di = args.np
+                if di < 0:
+                    di = 0
+                if di > 255:
+                    di = 255
+                RNS.log("Setting NeoPixel intensity to "+str(di))
+                rnode.set_neopixel_intensity(di)
+
             if isinstance(args.display_addr, str):
                 set_addr = False
                 try:
@@ -3280,7 +3732,7 @@ def main():
                         elif rnode.platform == ROM.PLATFORM_NRF52:
                             rnode_serial.close()
                             RNS.log("Waiting for NRF52 reset...")
-                            time.sleep(14)
+                            time.sleep(18)
                             selected_port = None
                             ports = list_ports.comports()
                             for port in ports:
@@ -3491,7 +3943,9 @@ def main():
                             if rnode.platform == ROM.PLATFORM_ESP32:
                                 rnode.hard_reset()
                                 RNS.log("Waiting for ESP32 reset...")
-                                time.sleep(6.5)
+                                time.sleep(7)
+                                if selected_model in [ROM.MODEL_AC, ROM.MODEL_A6, ROM.MODEL_A1, ROM.MODEL_AA, ROM.MODEL_A5]:
+                                    time.sleep(5)
 
                             elif rnode.platform == ROM.PLATFORM_NRF52:
                                 # Wait a few seconds before hard resetting.
@@ -3510,7 +3964,7 @@ def main():
 
                                 # Give plenty of time for to allow for
                                 # potential e-ink display refresh too.
-                                time.sleep(14)
+                                time.sleep(20)
 
                                 # After the hard reset, the port number will
                                 # change. We need to find the new port number,

RNS/Utilities/rnpath.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal

RNS/Utilities/rnprobe.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal

RNS/Utilities/rnsd.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -29,7 +29,7 @@ import time
 from RNS._version import __version__
 
 
-def program_setup(configdir, verbosity = 0, quietness = 0, service = False):
+def program_setup(configdir, verbosity = 0, quietness = 0, service = False, interactive=False):
     targetverbosity = verbosity-quietness
 
     if service:
@@ -42,10 +42,14 @@ def program_setup(configdir, verbosity = 0, quietness = 0, service = False):
     if reticulum.is_connected_to_shared_instance:
         RNS.log("Started rnsd version {version} connected to another shared local instance, this is probably NOT what you want!".format(version=__version__), RNS.LOG_WARNING)
     else:
+        # TODO: Rethink why this was added
+        # if RNS.Reticulum.get_instance().shared_instance_interface:
+        #     RNS.Reticulum.get_instance().shared_instance_interface.server.daemon_threads = True
         RNS.log("Started rnsd version {version}".format(version=__version__), RNS.LOG_NOTICE)
 
-    while True:
-        time.sleep(1)
+    if interactive: import code; code.interact(local=globals())
+    else:
+        while True: time.sleep(1)
 
 def main():
     try:
@@ -54,6 +58,7 @@ def main():
         parser.add_argument('-v', '--verbose', action='count', default=0)
         parser.add_argument('-q', '--quiet', action='count', default=0)
         parser.add_argument('-s', '--service', action='store_true', default=False, help="rnsd is running as a service and should log to file")
+        parser.add_argument('-i', '--interactive', action='store_true', default=False, help="drop into interactive shell after initialisation")
         parser.add_argument("--exampleconfig", action='store_true', default=False, help="print verbose configuration example to stdout and exit")
         parser.add_argument("--version", action="version", version="rnsd {version}".format(version=__version__))
         
@@ -68,7 +73,7 @@ def main():
         else:
             configarg = None
 
-        program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet, service=args.service)
+        program_setup(configdir = configarg, verbosity=args.verbose, quietness=args.quiet, service=args.service, interactive=args.interactive)
 
     except KeyboardInterrupt:
         print("")
@@ -294,6 +299,23 @@ loglevel = 4
     # Serial port for the device
     port = /dev/ttyUSB0
 
+    # It is also possible to use BLE devices
+    # instead of wired serial ports. The
+    # target RNode must be paired with the
+    # host device before connecting. BLE
+    # devices can be connected by name,
+    # BLE MAC address or by any available.
+    
+    # Connect to specific device by name
+    # port = ble://RNode 3B87
+
+    # Or by BLE MAC address
+    # port = ble://F4:12:73:29:4E:89
+
+    # Or connect to the first available,
+    # paired device
+    # port = ble://
+
     # Set frequency to 867.2 MHz
     frequency = 867200000
 

RNS/Utilities/rnstatus.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -135,8 +135,26 @@ def get_remote_status(destination_hash, include_lstats, identity, no_output=Fals
 
 def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=False, astats=False,
                   lstats=False, sorting=None, sort_reverse=False, remote=None, management_identity=None,
-                  remote_timeout=RNS.Transport.PATH_REQUEST_TIMEOUT):
-    reticulum = RNS.Reticulum(configdir = configdir, loglevel = 3+verbosity)
+                  remote_timeout=RNS.Transport.PATH_REQUEST_TIMEOUT, must_exit=True, rns_instance=None, traffic_totals=False):
+    
+    if remote:
+        require_shared = False
+    else:
+        require_shared = True
+
+    try:
+        if rns_instance:
+            reticulum = rns_instance
+            must_exit = False
+        else:
+            reticulum = RNS.Reticulum(configdir=configdir, loglevel=3+verbosity, require_shared_instance=require_shared)
+
+    except Exception as e:
+        print("No shared RNS instance available to get status from")
+        if must_exit:
+            exit(1)
+        else:
+            return
 
     link_count = None
     stats = None
@@ -164,7 +182,10 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
                     
         except Exception as e:
             print(str(e))
-            exit(20)
+            if must_exit:
+                exit(20)
+            else:
+                return
 
     else:
         if lstats:
@@ -193,7 +214,10 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
                                     i[k] = RNS.hexrep(i[k], delimit=False)
 
             print(json.dumps(stats))
-            exit()
+            if must_exit:
+                exit()
+            else:
+                return
 
         interfaces = stats["interfaces"]
         if sorting != None and isinstance(sorting, str):
@@ -204,6 +228,10 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
                 interfaces.sort(key=lambda i: i["rxb"], reverse=not sort_reverse)
             if sorting == "tx":
                 interfaces.sort(key=lambda i: i["txb"], reverse=not sort_reverse)
+            if sorting == "rxs":
+                interfaces.sort(key=lambda i: i["rxs"], reverse=not sort_reverse)
+            if sorting == "txs":
+                interfaces.sort(key=lambda i: i["txs"], reverse=not sort_reverse)
             if sorting == "traffic":
                 interfaces.sort(key=lambda i: i["rxb"]+i["txb"], reverse=not sort_reverse)
             if sorting == "announces" or sorting == "announce":
@@ -222,6 +250,8 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
             if dispall or not (
                 name.startswith("LocalInterface[") or
                 name.startswith("TCPInterface[Client") or
+                name.startswith("BackboneInterface[Client on") or
+                name.startswith("AutoInterfacePeer[") or
                 name.startswith("I2PInterfacePeer[Connected peer") or
                 (name.startswith("I2PInterface[") and ("i2p_connectable" in ifstat and ifstat["i2p_connectable"] == False))
                 ):
@@ -292,12 +322,26 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
                         if "bitrate" in ifstat and ifstat["bitrate"] != None:
                             print("    Rate      : {ss}".format(ss=speed_str(ifstat["bitrate"])))
 
+                        if "noise_floor" in ifstat:
+                            if ifstat["noise_floor"] != None:
+                                print("    Noise Fl. : {nfl} dBm".format(nfl=str(ifstat["noise_floor"])))
+                            else:
+                                print("    Noise Fl. : Unknown")
+
+                        if "battery_percent" in ifstat and ifstat["battery_percent"] != None:
+                            try:
+                                bpi = int(ifstat["battery_percent"])
+                                bss = ifstat["battery_state"]
+                                print(f"    Battery   : {bpi}% ({bss})")
+                            except:
+                                pass
+
                         if "airtime_short" in ifstat and "airtime_long" in ifstat:
                             print("    Airtime   : {ats}% (15s), {atl}% (1h)".format(ats=str(ifstat["airtime_short"]),atl=str(ifstat["airtime_long"])))
                         
                         if "channel_load_short" in ifstat and "channel_load_long" in ifstat:
-                            print("    Ch.Load   : {ats}% (15s), {atl}% (1h)".format(ats=str(ifstat["channel_load_short"]),atl=str(ifstat["channel_load_long"])))
-                        
+                            print("    Ch. Load  : {ats}% (15s), {atl}% (1h)".format(ats=str(ifstat["channel_load_short"]),atl=str(ifstat["channel_load_long"])))
+
                         if "peers" in ifstat and ifstat["peers"] != None:
                             print("    Peers     : {np} reachable".format(np=ifstat["peers"]))
 
@@ -329,7 +373,21 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
                             print("    Announces : {iaf}↑".format(iaf=RNS.prettyfrequency(ifstat["outgoing_announce_frequency"])))
                             print("                {iaf}↓".format(iaf=RNS.prettyfrequency(ifstat["incoming_announce_frequency"])))
 
-                        print("    Traffic   : {txb}↑\n                {rxb}↓".format(rxb=size_str(ifstat["rxb"]), txb=size_str(ifstat["txb"])))
+                        rxb_str = "↓"+RNS.prettysize(ifstat["rxb"])
+                        txb_str = "↑"+RNS.prettysize(ifstat["txb"])
+                        strdiff = len(rxb_str)-len(txb_str)
+                        if strdiff > 0:
+                            txb_str += " "*strdiff
+                        elif strdiff < 0:
+                            rxb_str += " "*-strdiff
+
+                        rxstat = rxb_str
+                        txstat = txb_str
+                        if "rxs" in ifstat and "txs" in ifstat:
+                            rxstat += "  "+RNS.prettyspeed(ifstat["rxs"])
+                            txstat += "  "+RNS.prettyspeed(ifstat["txs"])
+                        
+                        print(f"    Traffic   : {txstat}\n                {rxstat}")
 
         lstr = ""
         if link_count != None and lstats:
@@ -339,6 +397,19 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
             else:
                 lstr = f" {link_count} entr{ms} in link table"
 
+        if traffic_totals:
+            rxb_str = "↓"+RNS.prettysize(stats["rxb"])
+            txb_str = "↑"+RNS.prettysize(stats["txb"])
+            strdiff = len(rxb_str)-len(txb_str)
+            if strdiff > 0:
+                txb_str += " "*strdiff
+            elif strdiff < 0:
+                rxb_str += " "*-strdiff
+
+            rxstat  = rxb_str+"  "+RNS.prettyspeed(stats["rxs"])
+            txstat  = txb_str+"  "+RNS.prettyspeed(stats["txs"])
+            print(f"\n Totals       : {txstat}\n                {rxstat}")
+
         if "transport_id" in stats and stats["transport_id"] != None:
             print("\n Transport Instance "+RNS.prettyhexrep(stats["transport_id"])+" running")
             if "probe_responder" in stats and stats["probe_responder"] != None:
@@ -356,9 +427,12 @@ def program_setup(configdir, dispall=False, verbosity=0, name_filter=None, json=
             print("Could not get RNS status")
         else:
             print("Could not get RNS status from remote transport instance "+RNS.prettyhexrep(identity_hash))
-        exit(1)
+        if must_exit:
+            exit(2)
+        else:
+            return
 
-def main():
+def main(must_exit=True, rns_instance=None):
     try:
         parser = argparse.ArgumentParser(description="Reticulum Network Stack Status")
         parser.add_argument("--config", action="store", default=None, help="path to alternative Reticulum config directory", type=str)
@@ -388,11 +462,19 @@ def main():
             default=False,
         )
         
+        parser.add_argument(
+            "-t",
+            "--totals",
+            action="store_true",
+            help="display traffic totals",
+            default=False,
+        )
+        
         parser.add_argument(
             "-s",
             "--sort",
             action="store",
-            help="sort interfaces by [rate, traffic, rx, tx, announces, arx, atx, held]",
+            help="sort interfaces by [rate, traffic, rx, tx, rxs, txs, announces, arx, atx, held]",
             default=None,
             type=str
         )
@@ -464,11 +546,17 @@ def main():
             remote=args.R,
             management_identity=args.i,
             remote_timeout=args.w,
+            must_exit=must_exit,
+            rns_instance=rns_instance,
+            traffic_totals=args.totals,
         )
 
     except KeyboardInterrupt:
         print("")
-        exit()
+        if must_exit:
+            exit()
+        else:
+            return
 
 def speed_str(num, suffix='bps'):
     units = ['','k','M','G','T','P','E','Z']

RNS/Utilities/rnx.py

@@ -2,7 +2,7 @@
 
 # MIT License
 #
-# Copyright (c) 2016-2022 Mark Qvist / unsigned.io
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -28,8 +28,8 @@ import argparse
 import shlex
 import time
 import sys
-import tty
 import os
+#import tty
 
 from RNS._version import __version__
 

RNS/__init__.py

@@ -1,6 +1,6 @@
 # MIT License
 #
-# Copyright (c) 2016-2023 Mark Qvist / unsigned.io and contributors
+# Copyright (c) 2016-2025 Mark Qvist / unsigned.io and contributors
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -24,6 +24,7 @@ import os
 import sys
 import glob
 import time
+import datetime
 import random
 import threading
 
@@ -43,9 +44,16 @@ from .Resource import Resource, ResourceAdvertisement
 from .Cryptography import HKDF
 from .Cryptography import Hashes
 
-modules = glob.glob(os.path.dirname(__file__)+"/*.py")
-__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]
+py_modules  = glob.glob(os.path.dirname(__file__)+"/*.py")
+pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
+modules     = py_modules+pyc_modules
+__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))
 
+import importlib.util
+if importlib.util.find_spec("cython"): import cython; compiled = cython.compiled
+else: compiled = False
+
+LOG_NONE     = -1
 LOG_CRITICAL = 0
 LOG_ERROR    = 1
 LOG_WARNING  = 2
@@ -57,13 +65,16 @@ LOG_EXTREME  = 7
 
 LOG_STDOUT   = 0x91
 LOG_FILE     = 0x92
+LOG_CALLBACK = 0x93
 
 LOG_MAXSIZE  = 5*1024*1024
 
 loglevel        = LOG_NOTICE
 logfile         = None
 logdest         = LOG_STDOUT
+logcall         = None
 logtimefmt      = "%Y-%m-%d %H:%M:%S"
+logtimefmt_p    = "%H:%M:%S.%f"
 compact_log_fmt = False
 
 instance_random = random.Random()
@@ -75,21 +86,21 @@ logging_lock = threading.Lock()
 
 def loglevelname(level):
     if (level == LOG_CRITICAL):
-        return "Critical"
+        return "[Critical]"
     if (level == LOG_ERROR):
-        return "Error"
+        return "[Error]   "
     if (level == LOG_WARNING):
-        return "Warning"
+        return "[Warning] "
     if (level == LOG_NOTICE):
-        return "Notice"
+        return "[Notice]  "
     if (level == LOG_INFO):
-        return "Info"
+        return "[Info]    "
     if (level == LOG_VERBOSE):
-        return "Verbose"
+        return "[Verbose] "
     if (level == LOG_DEBUG):
-        return "Debug"
+        return "[Debug]   "
     if (level == LOG_EXTREME):
-        return "Extra"
+        return "[Extra]   "
     
     return "Unknown"
 
@@ -104,40 +115,53 @@ def timestamp_str(time_s):
     timestamp = time.localtime(time_s)
     return time.strftime(logtimefmt, timestamp)
 
-def log(msg, level=3, _override_destination = False):
+def precise_timestamp_str(time_s):
+    return datetime.datetime.now().strftime(logtimefmt_p)[:-3]
+
+def log(msg, level=3, _override_destination = False, pt=False):
+    if loglevel == LOG_NONE: return
     global _always_override_destination, compact_log_fmt
     msg = str(msg)
     if loglevel >= level:
-        if not compact_log_fmt:
-            logstring = "["+timestamp_str(time.time())+"] ["+loglevelname(level)+"] "+msg
+        if pt:
+            logstring = "["+precise_timestamp_str(time.time())+"] "+loglevelname(level)+" "+msg
         else:
-            logstring = "["+timestamp_str(time.time())+"] "+msg
+            if not compact_log_fmt:
+                logstring = "["+timestamp_str(time.time())+"] "+loglevelname(level)+" "+msg
+            else:
+                logstring = "["+timestamp_str(time.time())+"] "+msg
 
-        logging_lock.acquire()
+        with logging_lock:
+            if (logdest == LOG_STDOUT or _always_override_destination or _override_destination):
+                if not threading.main_thread().is_alive(): return
+                else: print(logstring)
 
-        if (logdest == LOG_STDOUT or _always_override_destination or _override_destination):
-            print(logstring)
-            logging_lock.release()
+            elif (logdest == LOG_FILE and logfile != None):
+                try:
+                    file = open(logfile, "a")
+                    file.write(logstring+"\n")
+                    file.close()
+                    
+                    if os.path.getsize(logfile) > LOG_MAXSIZE:
+                        prevfile = logfile+".1"
+                        if os.path.isfile(prevfile):
+                            os.unlink(prevfile)
+                        os.rename(logfile, prevfile)
 
-        elif (logdest == LOG_FILE and logfile != None):
-            try:
-                file = open(logfile, "a")
-                file.write(logstring+"\n")
-                file.close()
-                
-                if os.path.getsize(logfile) > LOG_MAXSIZE:
-                    prevfile = logfile+".1"
-                    if os.path.isfile(prevfile):
-                        os.unlink(prevfile)
-                    os.rename(logfile, prevfile)
+                except Exception as e:
+                    _always_override_destination = True
+                    log("Exception occurred while writing log message to log file: "+str(e), LOG_CRITICAL)
+                    log("Dumping future log events to console!", LOG_CRITICAL)
+                    log(msg, level)
 
-                logging_lock.release()
-            except Exception as e:
-                logging_lock.release()
-                _always_override_destination = True
-                log("Exception occurred while writing log message to log file: "+str(e), LOG_CRITICAL)
-                log("Dumping future log events to console!", LOG_CRITICAL)
-                log(msg, level)
+            elif logdest == LOG_CALLBACK:
+                try:
+                    logcall(logstring)
+                except Exception as e:
+                    _always_override_destination = True
+                    log("Exception occurred while calling external log handler: "+str(e), LOG_CRITICAL)
+                    log("Dumping future log events to console!", LOG_CRITICAL)
+                    log(msg, level)
                 
 
 def rand():
@@ -218,6 +242,11 @@ def prettydistance(m, suffix="m"):
     return "%.2f %s%s" % (num, last_unit, suffix)
 
 def prettytime(time, verbose=False, compact=False):
+    neg = False
+    if time < 0:
+        time = abs(time)
+        neg = True
+
     days = int(time // (24 * 3600))
     time = time % (24 * 3600)
     hours = int(time // 3600)
@@ -268,10 +297,17 @@ def prettytime(time, verbose=False, compact=False):
     if tstr == "":
         return "0s"
     else:
-        return tstr
+        if not neg:
+            return tstr
+        else:
+            return f"-{tstr}"
 
 def prettyshorttime(time, verbose=False, compact=False):
+    neg = False
     time = time*1e6
+    if time < 0:
+        time = abs(time)
+        neg = True
     
     seconds = int(time // 1e6); time %= 1e6
     milliseconds = int(time // 1e3); time %= 1e3
@@ -315,7 +351,10 @@ def prettyshorttime(time, verbose=False, compact=False):
     if tstr == "":
         return "0us"
     else:
-        return tstr
+        if not neg:
+            return tstr
+        else:
+            return f"-{tstr}"
 
 def phyparams():
     print("Required Physical Layer MTU : "+str(Reticulum.MTU)+" bytes")
@@ -329,97 +368,169 @@ def phyparams():
 def panic():
     os._exit(255)
 
-def exit():
-    print("")
-    sys.exit(0)
+exit_called = False
+def exit(code=0):
+    global exit_called
+    if not exit_called:
+        exit_called = True
+        Reticulum.exit_handler()
+        os._exit(code)
 
+class Profiler:
+    _ran = False
+    profilers = {}
+    tags = {}
 
-profiler_ran = False
-profiler_tags = {}
-def profiler(tag=None, capture=False, super_tag=None):
-    global profiler_ran, profiler_tags
-    try:
-        thread_ident = threading.get_ident()
-
-        if capture:
-            end = time.perf_counter()
-            if tag in profiler_tags and thread_ident in profiler_tags[tag]["threads"]:
-                if profiler_tags[tag]["threads"][thread_ident]["current_start"] != None:
-                    begin = profiler_tags[tag]["threads"][thread_ident]["current_start"]
-                    profiler_tags[tag]["threads"][thread_ident]["current_start"] = None
-                    profiler_tags[tag]["threads"][thread_ident]["captures"].append(end-begin)
-                    if not profiler_ran:
-                        profiler_ran = True
-
+    @staticmethod
+    def get_profiler(tag=None, super_tag=None):
+        if tag in Profiler.profilers:
+            return Profiler.profilers[tag]
         else:
-            if not tag in profiler_tags:
-                profiler_tags[tag] = {"threads": {}, "super": super_tag}
-            if not thread_ident in profiler_tags[tag]["threads"]:
-                profiler_tags[tag]["threads"][thread_ident] = {"current_start": None, "captures": []}
+            profiler = Profiler(tag, super_tag)
+            Profiler.profilers[tag] = profiler
+            return profiler
 
-            profiler_tags[tag]["threads"][thread_ident]["current_start"] = time.perf_counter()
+    def __init__(self, tag=None, super_tag=None):
+        self.paused = False
+        self.pause_time = 0
+        self.pause_started = None
+        self.tag = tag
+        self.super_tag = super_tag
+        if self.super_tag in Profiler.profilers:
+            self.super_profiler = Profiler.profilers[self.super_tag]
+            self.pause_super = self.super_profiler.pause
+            self.resume_super = self.super_profiler.resume
+        else:
+            def noop(self=None):
+                pass
+            self.super_profiler = None
+            self.pause_super = noop
+            self.resume_super = noop
 
-    except Exception as e:
-        trace_exception(e)
+    def __enter__(self):
+        self.pause_super()
+        tag = self.tag
+        super_tag = self.super_tag
+        thread_ident = threading.get_ident()
+        if not tag in Profiler.tags:
+            Profiler.tags[tag] = {"threads": {}, "super": super_tag}
+        if not thread_ident in Profiler.tags[tag]["threads"]:
+            Profiler.tags[tag]["threads"][thread_ident] = {"current_start": None, "captures": []}
 
-def profiler_results():
-    from statistics import mean, median, stdev
-    results = {}
-    
-    for tag in profiler_tags:
-        tag_captures = []
-        tag_entry = profiler_tags[tag]
+        Profiler.tags[tag]["threads"][thread_ident]["current_start"] = time.perf_counter()
+        self.resume_super()
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        self.pause_super()
+        tag = self.tag
+        super_tag = self.super_tag
+        end = time.perf_counter() - self.pause_time
+        self.pause_time = 0
+        thread_ident = threading.get_ident()
+        if tag in Profiler.tags and thread_ident in Profiler.tags[tag]["threads"]:
+            if Profiler.tags[tag]["threads"][thread_ident]["current_start"] != None:
+                begin = Profiler.tags[tag]["threads"][thread_ident]["current_start"]
+                Profiler.tags[tag]["threads"][thread_ident]["current_start"] = None
+                Profiler.tags[tag]["threads"][thread_ident]["captures"].append(end-begin)
+                if not Profiler._ran:
+                    Profiler._ran = True
+        self.resume_super()
+
+    def pause(self, pause_started=None):
+        if not self.paused:
+            self.paused = True
+            self.pause_started = pause_started or time.perf_counter()
+            self.pause_super(self.pause_started)
+
+    def resume(self):
+        if self.paused:
+            self.pause_time += time.perf_counter() - self.pause_started
+            self.paused = False
+            self.resume_super()
+
+    @staticmethod
+    def ran():
+        return Profiler._ran
+
+    @staticmethod
+    def results():
+        from statistics import mean, median, stdev
+        results = {}
         
-        for thread_ident in tag_entry["threads"]:
-            thread_entry = tag_entry["threads"][thread_ident]
-            thread_captures = thread_entry["captures"]
-            sample_count = len(thread_captures)
+        for tag in Profiler.tags:
+            tag_captures = []
+            tag_entry = Profiler.tags[tag]
             
-            if sample_count > 2:
-                thread_results = {
-                    "count": sample_count,
-                    "mean": mean(thread_captures),
-                    "median": median(thread_captures),
-                    "stdev": stdev(thread_captures)
+            for thread_ident in tag_entry["threads"]:
+                thread_entry = tag_entry["threads"][thread_ident]
+                thread_captures = thread_entry["captures"]
+                sample_count = len(thread_captures)
+                
+                if sample_count > 1:
+                    thread_results = {
+                        "count": sample_count,
+                        "mean": mean(thread_captures),
+                        "median": median(thread_captures),
+                        "stdev": stdev(thread_captures)
+                    }
+                elif sample_count == 1:
+                    thread_results = {
+                        "count": sample_count,
+                        "mean": mean(thread_captures),
+                        "median": median(thread_captures),
+                        "stdev": None
+                    }
+
+                tag_captures.extend(thread_captures)
+
+            sample_count = len(tag_captures)
+            if sample_count > 1:
+                tag_results = {
+                    "name": tag,
+                    "super": tag_entry["super"],
+                    "count": len(tag_captures),
+                    "mean": mean(tag_captures),
+                    "median": median(tag_captures),
+                    "stdev": stdev(tag_captures)
+                }
+            elif sample_count == 1:
+                tag_results = {
+                    "name": tag,
+                    "super": tag_entry["super"],
+                    "count": len(tag_captures),
+                    "mean": mean(tag_captures),
+                    "median": median(tag_captures),
+                    "stdev": None
                 }
-
-            tag_captures.extend(thread_captures)
-
-        sample_count = len(tag_captures)
-        if sample_count > 2:
-            tag_results = {
-                "name": tag,
-                "super": tag_entry["super"],
-                "count": len(tag_captures),
-                "mean": mean(tag_captures),
-                "median": median(tag_captures),
-                "stdev": stdev(tag_captures)
-            }
 
             results[tag] = tag_results
 
-    def print_results_recursive(tag, results, level=0):
-        print_tag_results(tag, level+1)
+        def print_results_recursive(tag, results, level=0):
+            print_tag_results(tag, level+1)
 
+            for tag_name in results:
+                sub_tag = results[tag_name]
+                if sub_tag["super"] == tag["name"]:
+                    print_results_recursive(sub_tag, results, level=level+1)
+
+
+        def print_tag_results(tag, level):
+            ind = "  "*level
+            name = tag["name"]; count = tag["count"]
+            mean = tag["mean"]; median = tag["median"]; stdev = tag["stdev"]
+            print(    f"{ind}{name}")
+            print(    f"{ind}  Samples  : {count}")
+            if stdev != None:
+                print(f"{ind}  Mean     : {prettyshorttime(mean)}")
+                print(f"{ind}  Median   : {prettyshorttime(median)}")
+                print(f"{ind}  St.dev.  : {prettyshorttime(stdev)}")
+            print(    f"{ind}  Total    : {prettyshorttime(mean*count)}")
+            print("")
+
+        print("\nProfiler results:\n")
         for tag_name in results:
-            sub_tag = results[tag_name]
-            if sub_tag["super"] == tag["name"]:
-                print_results_recursive(sub_tag, results, level=level+1)
+            tag = results[tag_name]
+            if tag["super"] == None:
+                print_results_recursive(tag, results)
 
-
-    def print_tag_results(tag, level):
-        ind = "  "*level
-        name = tag["name"]; count = tag["count"]
-        mean = tag["mean"]; tag["median"]; stdev = tag["stdev"]
-        print(f"{ind}{name}")
-        print(f"{ind}  Samples : {count}")
-        print(f"{ind}  Mean    : {prettyshorttime(mean)}")
-        print(f"{ind}  Median  : {prettyshorttime(median)}")
-        print(f"{ind}  St.dev. : {prettyshorttime(stdev)}")
-        print("")
-
-    print("\nProfiler results:\n")
-    for tag_name in results:
-        tag = results[tag_name]
-        if tag["super"] == None:
-            print_results_recursive(tag, results)
+profile = Profiler.get_profiler

RNS/_version.py

@@ -1 +1 @@
-__version__ = "0.7.8"
+__version__ = "0.9.4"

RNS/vendor/__init__.py

@@ -1,5 +1,7 @@
 import os
 import glob
 
-modules = glob.glob(os.path.dirname(__file__)+"/*.py")
-__all__ = [ os.path.basename(f)[:-3] for f in modules if not f.endswith('__init__.py')]
+py_modules  = glob.glob(os.path.dirname(__file__)+"/*.py")
+pyc_modules = glob.glob(os.path.dirname(__file__)+"/*.pyc")
+modules     = py_modules+pyc_modules
+__all__ = list(set([os.path.basename(f).replace(".pyc", "").replace(".py", "") for f in modules if not (f.endswith("__init__.py") or f.endswith("__init__.pyc"))]))

RNS/vendor/configobj.py

@@ -19,8 +19,7 @@ import sys
 
 from codecs import BOM_UTF8, BOM_UTF16, BOM_UTF16_BE, BOM_UTF16_LE
 
-import RNS.vendor.six as six
-__version__ = '5.0.6'
+__version__ = '5.0.9'
 
 # imported lazily to avoid startup performance hit if it isn't used
 compiler = None
@@ -121,10 +120,6 @@ OPTION_DEFAULTS = {
     'write_empty_values': False,
 }
 
-# this could be replaced if six is used for compatibility, or there are no
-# more assertions about items being a string
-
-
 def getObj(s):
     global compiler
     if compiler is None:
@@ -553,11 +548,11 @@ class Section(dict):
         """Fetch the item and do string interpolation."""
         val = dict.__getitem__(self, key)
         if self.main.interpolation: 
-            if isinstance(val, six.string_types):
+            if isinstance(val, str):
                 return self._interpolate(key, val)
             if isinstance(val, list):
                 def _check(entry):
-                    if isinstance(entry, six.string_types):
+                    if isinstance(entry, str):
                         return self._interpolate(key, entry)
                     return entry
                 new = [_check(entry) for entry in val]
@@ -580,7 +575,7 @@ class Section(dict):
         ``unrepr`` must be set when setting a value to a dictionary, without
         creating a new sub-section.
         """
-        if not isinstance(key, six.string_types):
+        if not isinstance(key, str):
             raise ValueError('The key "%s" is not a string.' % key)
         
         # add the comment
@@ -614,11 +609,11 @@ class Section(dict):
             if key not in self:
                 self.scalars.append(key)
             if not self.main.stringify:
-                if isinstance(value, six.string_types):
+                if isinstance(value, str):
                     pass
                 elif isinstance(value, (list, tuple)):
                     for entry in value:
-                        if not isinstance(entry, six.string_types):
+                        if not isinstance(entry, str):
                             raise TypeError('Value is not a string "%s".' % entry)
                 else:
                     raise TypeError('Value is not a string "%s".' % value)
@@ -959,7 +954,7 @@ class Section(dict):
             return False
         else:
             try:
-                if not isinstance(val, six.string_types):
+                if not isinstance(val, str):
                     # TODO: Why do we raise a KeyError here?
                     raise KeyError()
                 else:
@@ -1230,7 +1225,7 @@ class ConfigObj(Section):
         
         
     def _load(self, infile, configspec):
-        if isinstance(infile, six.string_types):
+        if isinstance(infile, str):
             self.filename = infile
             if os.path.isfile(infile):
                 with open(infile, 'rb') as h:
@@ -1298,7 +1293,7 @@ class ConfigObj(Section):
                         break
                 break
 
-        assert all(isinstance(line, six.string_types) for line in content), repr(content)
+        assert all(isinstance(line, str) for line in content), repr(content)
         content = [line.rstrip('\r\n') for line in content]
             
         self._parse(content)
@@ -1403,7 +1398,7 @@ class ConfigObj(Section):
         else:
             line = infile
 
-        if isinstance(line, six.text_type):
+        if isinstance(line, str):
             # it's already decoded and there's no need to do anything
             # else, just use the _decode utility method to handle
             # listifying appropriately
@@ -1448,7 +1443,7 @@ class ConfigObj(Section):
         
         # No encoding specified - so we need to check for UTF8/UTF16
         for BOM, (encoding, final_encoding) in list(BOMS.items()):
-            if not isinstance(line, six.binary_type) or not line.startswith(BOM):
+            if not isinstance(line, bytes) or not line.startswith(BOM):
                 # didn't specify a BOM, or it's not a bytestring
                 continue
             else:
@@ -1464,9 +1459,9 @@ class ConfigObj(Section):
                     else:
                         infile = newline
                     # UTF-8
-                    if isinstance(infile, six.text_type):
+                    if isinstance(infile, str):
                         return infile.splitlines(True)
-                    elif isinstance(infile, six.binary_type):
+                    elif isinstance(infile, bytes):
                         return infile.decode('utf-8').splitlines(True)
                     else:
                         return self._decode(infile, 'utf-8')
@@ -1474,12 +1469,8 @@ class ConfigObj(Section):
                 return self._decode(infile, encoding)
             
 
-        if six.PY2 and isinstance(line, str):
-            # don't actually do any decoding, since we're on python 2 and
-            # returning a bytestring is fine
-            return self._decode(infile, None)
         # No BOM discovered and no encoding specified, default to UTF-8
-        if isinstance(infile, six.binary_type):
+        if isinstance(infile, bytes):
             return infile.decode('utf-8').splitlines(True)
         else:
             return self._decode(infile, 'utf-8')
@@ -1487,7 +1478,7 @@ class ConfigObj(Section):
 
     def _a_to_u(self, aString):
         """Decode ASCII strings to unicode if a self.encoding is specified."""
-        if isinstance(aString, six.binary_type) and self.encoding:
+        if isinstance(aString, bytes) and self.encoding:
             return aString.decode(self.encoding)
         else:
             return aString
@@ -1499,9 +1490,9 @@ class ConfigObj(Section):
         
         if is a string, it also needs converting to a list.
         """
-        if isinstance(infile, six.string_types):
+        if isinstance(infile, str):
             return infile.splitlines(True)
-        if isinstance(infile, six.binary_type):
+        if isinstance(infile, bytes):
             # NOTE: Could raise a ``UnicodeDecodeError``
             if encoding:
                 return infile.decode(encoding).splitlines(True)
@@ -1510,7 +1501,7 @@ class ConfigObj(Section):
 
         if encoding:
             for i, line in enumerate(infile):
-                if isinstance(line, six.binary_type):
+                if isinstance(line, bytes):
                     # NOTE: The isinstance test here handles mixed lists of unicode/string
                     # NOTE: But the decode will break on any non-string values
                     # NOTE: Or could raise a ``UnicodeDecodeError``
@@ -1520,7 +1511,7 @@ class ConfigObj(Section):
 
     def _decode_element(self, line):
         """Decode element to unicode if necessary."""
-        if isinstance(line, six.binary_type) and self.default_encoding:
+        if isinstance(line, bytes) and self.default_encoding:
             return line.decode(self.default_encoding)
         else:
             return line
@@ -1532,7 +1523,7 @@ class ConfigObj(Section):
         Used by ``stringify`` within validate, to turn non-string values
         into strings.
         """
-        if not isinstance(value, six.string_types):
+        if not isinstance(value, str):
             # intentially 'str' because it's just whatever the "normal"
             # string type is for the python version we're dealing with
             return str(value)
@@ -1786,7 +1777,7 @@ class ConfigObj(Section):
                 return self._quote(value[0], multiline=False) + ','
             return ', '.join([self._quote(val, multiline=False)
                 for val in value])
-        if not isinstance(value, six.string_types):
+        if not isinstance(value, str):
             if self.stringify:
                 # intentially 'str' because it's just whatever the "normal"
                 # string type is for the python version we're dealing with
@@ -2111,7 +2102,7 @@ class ConfigObj(Section):
         if not output.endswith(newline):
             output += newline
 
-        if isinstance(output, six.binary_type):
+        if isinstance(output, bytes):
             output_bytes = output
         else:
             output_bytes = output.encode(self.encoding or
@@ -2170,7 +2161,7 @@ class ConfigObj(Section):
             if preserve_errors:
                 # We do this once to remove a top level dependency on the validate module
                 # Which makes importing configobj faster
-                from validate import VdtMissingValue
+                from configobj.validate import VdtMissingValue
                 self._vdtMissingValue = VdtMissingValue
                 
             section = self
@@ -2353,7 +2344,7 @@ class ConfigObj(Section):
         This method raises a ``ReloadError`` if the ConfigObj doesn't have
         a filename attribute pointing to a file.
         """
-        if not isinstance(self.filename, six.string_types):
+        if not isinstance(self.filename, str):
             raise ReloadError()
 
         filename = self.filename
@@ -2480,4 +2471,4 @@ def get_extra_values(conf, _prepend=()):
     return out
 
 
-"""*A programming language is a medium of expression.* - Paul Graham"""
+"""*A programming language is a medium of expression.* - Paul Graham"""

RNS/vendor/ifaddr/__init__.py

@@ -1,33 +0,0 @@
-# Copyright (c) 2014 Stefan C. Mueller
-
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to
-# deal in the Software without restriction, including without limitation the
-# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-# sell copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-# IN THE SOFTWARE.
-
-
-import os
-
-from RNS.vendor.ifaddr._shared import Adapter, IP
-
-if os.name == "nt":
-    from RNS.vendor.ifaddr._win32 import get_adapters
-elif os.name == "posix":
-    from RNS.vendor.ifaddr._posix import get_adapters
-else:
-    raise RuntimeError("Unsupported Operating System: %s" % os.name)
-
-__all__ = ['Adapter', 'IP', 'get_adapters']

RNS/vendor/ifaddr/_posix.py

@@ -1,93 +0,0 @@
-# Copyright (c) 2014 Stefan C. Mueller
-
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to
-# deal in the Software without restriction, including without limitation the
-# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-# sell copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-# IN THE SOFTWARE.
-
-
-import os
-import ctypes.util
-import ipaddress
-import collections
-import socket
-
-from typing import Iterable, Optional
-
-import RNS.vendor.ifaddr._shared as shared
-
-class ifaddrs(ctypes.Structure):
-    pass
-
-
-ifaddrs._fields_ = [
-    ('ifa_next', ctypes.POINTER(ifaddrs)),
-    ('ifa_name', ctypes.c_char_p),
-    ('ifa_flags', ctypes.c_uint),
-    ('ifa_addr', ctypes.POINTER(shared.sockaddr)),
-    ('ifa_netmask', ctypes.POINTER(shared.sockaddr)),
-]
-
-libc = ctypes.CDLL(ctypes.util.find_library("socket" if os.uname()[0] == "SunOS" else "c"), use_errno=True)  # type: ignore
-
-
-def get_adapters(include_unconfigured: bool = False) -> Iterable[shared.Adapter]:
-
-    addr0 = addr = ctypes.POINTER(ifaddrs)()
-    retval = libc.getifaddrs(ctypes.byref(addr))
-    if retval != 0:
-        eno = ctypes.get_errno()
-        raise OSError(eno, os.strerror(eno))
-
-    ips = collections.OrderedDict()
-
-    def add_ip(adapter_name: str, ip: Optional[shared.IP]) -> None:
-        if adapter_name not in ips:
-            index = None  # type: Optional[int]
-            try:
-                # Mypy errors on this when the Windows CI runs:
-                #     error: Module has no attribute "if_nametoindex"
-                index = socket.if_nametoindex(adapter_name)  # type: ignore
-            except (OSError, AttributeError):
-                pass
-            ips[adapter_name] = shared.Adapter(adapter_name, adapter_name, [], index=index)
-        if ip is not None:
-            ips[adapter_name].ips.append(ip)
-
-    while addr:
-        name = addr[0].ifa_name.decode(encoding='UTF-8')
-        ip_addr = shared.sockaddr_to_ip(addr[0].ifa_addr)
-        if ip_addr:
-            if addr[0].ifa_netmask and not addr[0].ifa_netmask[0].sa_familiy:
-                addr[0].ifa_netmask[0].sa_familiy = addr[0].ifa_addr[0].sa_familiy
-            netmask = shared.sockaddr_to_ip(addr[0].ifa_netmask)
-            if isinstance(netmask, tuple):
-                netmaskStr = str(netmask[0])
-                prefixlen = shared.ipv6_prefixlength(ipaddress.IPv6Address(netmaskStr))
-            else:
-                assert netmask is not None, f'sockaddr_to_ip({addr[0].ifa_netmask}) returned None'
-                netmaskStr = str('0.0.0.0/' + netmask)
-                prefixlen = ipaddress.IPv4Network(netmaskStr).prefixlen
-            ip = shared.IP(ip_addr, prefixlen, name)
-            add_ip(name, ip)
-        else:
-            if include_unconfigured:
-                add_ip(name, None)
-        addr = addr[0].ifa_next
-
-    libc.freeifaddrs(addr0)
-
-    return ips.values()

RNS/vendor/ifaddr/_shared.py

@@ -1,198 +0,0 @@
-# Copyright (c) 2014 Stefan C. Mueller
-
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to
-# deal in the Software without restriction, including without limitation the
-# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-# sell copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-# IN THE SOFTWARE.
-
-
-import ctypes
-import socket
-import ipaddress
-import platform
-
-from typing import List, Optional, Tuple, Union
-
-class Adapter(object):
-    """
-    Represents a network interface device controller (NIC), such as a
-    network card. An adapter can have multiple IPs.
-
-    On Linux aliasing (multiple IPs per physical NIC) is implemented
-    by creating 'virtual' adapters, each represented by an instance
-    of this class. Each of those 'virtual' adapters can have both
-    a IPv4 and an IPv6 IP address.
-    """
-
-    def __init__(self, name: str, nice_name: str, ips: List['IP'], index: Optional[int] = None) -> None:
-
-        #: Unique name that identifies the adapter in the system.
-        #: On Linux this is of the form of `eth0` or `eth0:1`, on
-        #: Windows it is a UUID in string representation, such as
-        #: `{846EE342-7039-11DE-9D20-806E6F6E6963}`.
-        self.name = name
-
-        #: Human readable name of the adpater. On Linux this
-        #: is currently the same as :attr:`name`. On Windows
-        #: this is the name of the device.
-        self.nice_name = nice_name
-
-        #: List of :class:`ifaddr.IP` instances in the order they were
-        #: reported by the system.
-        self.ips = ips
-
-        #: Adapter index as used by some API (e.g. IPv6 multicast group join).
-        self.index = index
-
-    def __repr__(self) -> str:
-        return "Adapter(name={name}, nice_name={nice_name}, ips={ips}, index={index})".format(
-            name=repr(self.name), nice_name=repr(self.nice_name), ips=repr(self.ips), index=repr(self.index)
-        )
-
-
-# Type of an IPv4 address (a string in "xxx.xxx.xxx.xxx" format)
-_IPv4Address = str
-
-# Type of an IPv6 address (a three-tuple `(ip, flowinfo, scope_id)`)
-_IPv6Address = Tuple[str, int, int]
-
-
-class IP(object):
-    """
-    Represents an IP address of an adapter.
-    """
-
-    def __init__(self, ip: Union[_IPv4Address, _IPv6Address], network_prefix: int, nice_name: str) -> None:
-
-        #: IP address. For IPv4 addresses this is a string in
-        #: "xxx.xxx.xxx.xxx" format. For IPv6 addresses this
-        #: is a three-tuple `(ip, flowinfo, scope_id)`, where
-        #: `ip` is a string in the usual collon separated
-        #: hex format.
-        self.ip = ip
-
-        #: Number of bits of the IP that represent the
-        #: network. For a `255.255.255.0` netmask, this
-        #: number would be `24`.
-        self.network_prefix = network_prefix
-
-        #: Human readable name for this IP.
-        #: On Linux is this currently the same as the adapter name.
-        #: On Windows this is the name of the network connection
-        #: as configured in the system control panel.
-        self.nice_name = nice_name
-
-    @property
-    def is_IPv4(self) -> bool:
-        """
-        Returns `True` if this IP is an IPv4 address and `False`
-        if it is an IPv6 address.
-        """
-        return not isinstance(self.ip, tuple)
-
-    @property
-    def is_IPv6(self) -> bool:
-        """
-        Returns `True` if this IP is an IPv6 address and `False`
-        if it is an IPv4 address.
-        """
-        return isinstance(self.ip, tuple)
-
-    def __repr__(self) -> str:
-        return "IP(ip={ip}, network_prefix={network_prefix}, nice_name={nice_name})".format(
-            ip=repr(self.ip), network_prefix=repr(self.network_prefix), nice_name=repr(self.nice_name)
-        )
-
-
-if platform.system() == "Darwin" or "BSD" in platform.system():
-
-    # BSD derived systems use marginally different structures
-    # than either Linux or Windows.
-    # I still keep it in `shared` since we can use
-    # both structures equally.
-
-    class sockaddr(ctypes.Structure):
-        _fields_ = [
-            ('sa_len', ctypes.c_uint8),
-            ('sa_familiy', ctypes.c_uint8),
-            ('sa_data', ctypes.c_uint8 * 14),
-        ]
-
-    class sockaddr_in(ctypes.Structure):
-        _fields_ = [
-            ('sa_len', ctypes.c_uint8),
-            ('sa_familiy', ctypes.c_uint8),
-            ('sin_port', ctypes.c_uint16),
-            ('sin_addr', ctypes.c_uint8 * 4),
-            ('sin_zero', ctypes.c_uint8 * 8),
-        ]
-
-    class sockaddr_in6(ctypes.Structure):
-        _fields_ = [
-            ('sa_len', ctypes.c_uint8),
-            ('sa_familiy', ctypes.c_uint8),
-            ('sin6_port', ctypes.c_uint16),
-            ('sin6_flowinfo', ctypes.c_uint32),
-            ('sin6_addr', ctypes.c_uint8 * 16),
-            ('sin6_scope_id', ctypes.c_uint32),
-        ]
-
-else:
-
-    class sockaddr(ctypes.Structure):  # type: ignore
-        _fields_ = [('sa_familiy', ctypes.c_uint16), ('sa_data', ctypes.c_uint8 * 14)]
-
-    class sockaddr_in(ctypes.Structure):  # type: ignore
-        _fields_ = [
-            ('sin_familiy', ctypes.c_uint16),
-            ('sin_port', ctypes.c_uint16),
-            ('sin_addr', ctypes.c_uint8 * 4),
-            ('sin_zero', ctypes.c_uint8 * 8),
-        ]
-
-    class sockaddr_in6(ctypes.Structure):  # type: ignore
-        _fields_ = [
-            ('sin6_familiy', ctypes.c_uint16),
-            ('sin6_port', ctypes.c_uint16),
-            ('sin6_flowinfo', ctypes.c_uint32),
-            ('sin6_addr', ctypes.c_uint8 * 16),
-            ('sin6_scope_id', ctypes.c_uint32),
-        ]
-
-
-def sockaddr_to_ip(sockaddr_ptr: 'ctypes.pointer[sockaddr]') -> Optional[Union[_IPv4Address, _IPv6Address]]:
-    if sockaddr_ptr:
-        if sockaddr_ptr[0].sa_familiy == socket.AF_INET:
-            ipv4 = ctypes.cast(sockaddr_ptr, ctypes.POINTER(sockaddr_in))
-            ippacked = bytes(bytearray(ipv4[0].sin_addr))
-            ip = str(ipaddress.ip_address(ippacked))
-            return ip
-        elif sockaddr_ptr[0].sa_familiy == socket.AF_INET6:
-            ipv6 = ctypes.cast(sockaddr_ptr, ctypes.POINTER(sockaddr_in6))
-            flowinfo = ipv6[0].sin6_flowinfo
-            ippacked = bytes(bytearray(ipv6[0].sin6_addr))
-            ip = str(ipaddress.ip_address(ippacked))
-            scope_id = ipv6[0].sin6_scope_id
-            return (ip, flowinfo, scope_id)
-    return None
-
-
-def ipv6_prefixlength(address: ipaddress.IPv6Address) -> int:
-    prefix_length = 0
-    for i in range(address.max_prefixlen):
-        if int(address) >> i & 1:
-            prefix_length = prefix_length + 1
-    return prefix_length

RNS/vendor/ifaddr/_win32.py

@@ -1,145 +0,0 @@
-# Copyright (c) 2014 Stefan C. Mueller
-
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to
-# deal in the Software without restriction, including without limitation the
-# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-# sell copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
-# IN THE SOFTWARE.
-
-
-import ctypes
-from ctypes import wintypes
-from typing import Iterable, List
-
-import RNS.vendor.ifaddr._shared as shared
-
-NO_ERROR = 0
-ERROR_BUFFER_OVERFLOW = 111
-MAX_ADAPTER_NAME_LENGTH = 256
-MAX_ADAPTER_DESCRIPTION_LENGTH = 128
-MAX_ADAPTER_ADDRESS_LENGTH = 8
-AF_UNSPEC = 0
-
-
-class SOCKET_ADDRESS(ctypes.Structure):
-    _fields_ = [('lpSockaddr', ctypes.POINTER(shared.sockaddr)), ('iSockaddrLength', wintypes.INT)]
-
-
-class IP_ADAPTER_UNICAST_ADDRESS(ctypes.Structure):
-    pass
-
-
-IP_ADAPTER_UNICAST_ADDRESS._fields_ = [
-    ('Length', wintypes.ULONG),
-    ('Flags', wintypes.DWORD),
-    ('Next', ctypes.POINTER(IP_ADAPTER_UNICAST_ADDRESS)),
-    ('Address', SOCKET_ADDRESS),
-    ('PrefixOrigin', ctypes.c_uint),
-    ('SuffixOrigin', ctypes.c_uint),
-    ('DadState', ctypes.c_uint),
-    ('ValidLifetime', wintypes.ULONG),
-    ('PreferredLifetime', wintypes.ULONG),
-    ('LeaseLifetime', wintypes.ULONG),
-    ('OnLinkPrefixLength', ctypes.c_uint8),
-]
-
-
-class IP_ADAPTER_ADDRESSES(ctypes.Structure):
-    pass
-
-
-IP_ADAPTER_ADDRESSES._fields_ = [
-    ('Length', wintypes.ULONG),
-    ('IfIndex', wintypes.DWORD),
-    ('Next', ctypes.POINTER(IP_ADAPTER_ADDRESSES)),
-    ('AdapterName', ctypes.c_char_p),
-    ('FirstUnicastAddress', ctypes.POINTER(IP_ADAPTER_UNICAST_ADDRESS)),
-    ('FirstAnycastAddress', ctypes.c_void_p),
-    ('FirstMulticastAddress', ctypes.c_void_p),
-    ('FirstDnsServerAddress', ctypes.c_void_p),
-    ('DnsSuffix', ctypes.c_wchar_p),
-    ('Description', ctypes.c_wchar_p),
-    ('FriendlyName', ctypes.c_wchar_p),
-]
-
-
-iphlpapi = ctypes.windll.LoadLibrary("Iphlpapi")  # type: ignore
-
-
-def enumerate_interfaces_of_adapter(
-    nice_name: str, address: IP_ADAPTER_UNICAST_ADDRESS
-) -> Iterable[shared.IP]:
-
-    # Iterate through linked list and fill list
-    addresses = []  # type: List[IP_ADAPTER_UNICAST_ADDRESS]
-    while True:
-        addresses.append(address)
-        if not address.Next:
-            break
-        address = address.Next[0]
-
-    for address in addresses:
-        ip = shared.sockaddr_to_ip(address.Address.lpSockaddr)
-        assert ip is not None, f'sockaddr_to_ip({address.Address.lpSockaddr}) returned None'
-        network_prefix = address.OnLinkPrefixLength
-        yield shared.IP(ip, network_prefix, nice_name)
-
-
-def get_adapters(include_unconfigured: bool = False) -> Iterable[shared.Adapter]:
-
-    # Call GetAdaptersAddresses() with error and buffer size handling
-
-    addressbuffersize = wintypes.ULONG(15 * 1024)
-    retval = ERROR_BUFFER_OVERFLOW
-    while retval == ERROR_BUFFER_OVERFLOW:
-        addressbuffer = ctypes.create_string_buffer(addressbuffersize.value)
-        retval = iphlpapi.GetAdaptersAddresses(
-            wintypes.ULONG(AF_UNSPEC),
-            wintypes.ULONG(0),
-            None,
-            ctypes.byref(addressbuffer),
-            ctypes.byref(addressbuffersize),
-        )
-    if retval != NO_ERROR:
-        raise ctypes.WinError()  # type: ignore
-
-    # Iterate through adapters fill array
-    address_infos = []  # type: List[IP_ADAPTER_ADDRESSES]
-    address_info = IP_ADAPTER_ADDRESSES.from_buffer(addressbuffer)
-    while True:
-        address_infos.append(address_info)
-        if not address_info.Next:
-            break
-        address_info = address_info.Next[0]
-
-    # Iterate through unicast addresses
-    result = []  # type: List[shared.Adapter]
-    for adapter_info in address_infos:
-
-        # We don't expect non-ascii characters here, so encoding shouldn't matter
-        name = adapter_info.AdapterName.decode()
-        nice_name = adapter_info.Description
-        index = adapter_info.IfIndex
-
-        if adapter_info.FirstUnicastAddress:
-            ips = enumerate_interfaces_of_adapter(
-                adapter_info.FriendlyName, adapter_info.FirstUnicastAddress[0]
-            )
-            ips = list(ips)
-            result.append(shared.Adapter(name, nice_name, ips, index=index))
-        elif include_unconfigured:
-            result.append(shared.Adapter(name, nice_name, [], index=index))
-
-    return result

RNS/vendor/ifaddr/niwrapper.py

@@ -1,57 +0,0 @@
-import ipaddress
-import RNS.vendor.ifaddr
-import socket
-
-from typing import List
-
-AF_INET6 = socket.AF_INET6.value
-AF_INET = socket.AF_INET.value
-
-def interfaces() -> List[str]:
-    adapters = RNS.vendor.ifaddr.get_adapters(include_unconfigured=True)
-    return [a.name for a in adapters]
-
-def interface_names_to_indexes() -> dict:
-    adapters = RNS.vendor.ifaddr.get_adapters(include_unconfigured=True)
-    results = {}
-    for adapter in adapters:
-        results[adapter.name] = adapter.index
-    return results
-
-def interface_name_to_nice_name(ifname) -> str:
-    try:
-        adapters = RNS.vendor.ifaddr.get_adapters(include_unconfigured=True)
-        for adapter in adapters:
-            if adapter.name == ifname:
-                if hasattr(adapter, "nice_name"):
-                    return adapter.nice_name
-    except:
-        return None
-
-    return None
-
-def ifaddresses(ifname) -> dict:
-    adapters = RNS.vendor.ifaddr.get_adapters(include_unconfigured=True)
-    ifa = {}
-    for a in adapters:
-        if a.name == ifname:
-            ipv4s = []
-            ipv6s = []
-            for ip in a.ips:
-                t = {}
-                if ip.is_IPv4:
-                    net = ipaddress.ip_network(str(ip.ip)+"/"+str(ip.network_prefix), strict=False)
-                    t["addr"] = ip.ip
-                    t["prefix"] = ip.network_prefix
-                    t["broadcast"] = str(net.broadcast_address)
-                    ipv4s.append(t)
-                if ip.is_IPv6:
-                    t["addr"] = ip.ip[0]
-                    ipv6s.append(t)
-
-            if len(ipv4s) > 0:
-                ifa[AF_INET] = ipv4s
-            if len(ipv6s) > 0:
-                ifa[AF_INET6] = ipv6s
-
-    return ifa

RNS/vendor/platformutils.py

@@ -1,42 +1,39 @@
 def get_platform():
     from os import environ
-    if "ANDROID_ARGUMENT" in environ:
-        return "android"
-    elif "ANDROID_ROOT" in environ:
-        return "android"
+    if "ANDROID_ARGUMENT" in environ: return "android"
+    elif "ANDROID_ROOT" in environ:   return "android"
     else:
         import sys
         return sys.platform
 
 def is_linux():
-    if get_platform() == "linux":
-        return True
-    else:
-        return False
+    if get_platform() == "linux": return True
+    else: return False
 
 def is_darwin():
-    if get_platform() == "darwin":
-        return True
-    else:
-        return False
+    if get_platform() == "darwin": return True
+    else: return False
 
 def is_android():
-    if get_platform() == "android":
-        return True
-    else:
-        return False
+    if get_platform() == "android": return True
+    else: return False
 
 def is_windows():
-    if str(get_platform()).startswith("win"):
-        return True
-    else:
-        return False
+    if str(get_platform()).startswith("win"): return True
+    else: return False
+
+def use_epoll():
+    if is_linux() or is_android(): return True
+    else: return False
+
+def use_af_unix():
+    if is_linux() or is_android(): return True
+    else: return False
 
 def platform_checks():
     if is_windows():
         import sys
-        if sys.version_info.major >= 3 and sys.version_info.minor >= 8:
-            pass
+        if sys.version_info.major >= 3 and sys.version_info.minor >= 8: pass
         else:
             import RNS
             RNS.log("On Windows, Reticulum requires Python 3.8 or higher.", RNS.LOG_ERROR)
@@ -45,7 +42,5 @@ def platform_checks():
 
 def cryptography_old_api():
     import cryptography
-    if cryptography.__version__ == "2.8":
-        return True
-    else:
-        return False
+    if cryptography.__version__ == "2.8": return True
+    else: return False

RNS/vendor/six.py

@@ -1,998 +0,0 @@
-# Copyright (c) 2010-2020 Benjamin Peterson
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in all
-# copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-
-"""Utilities for writing code that runs on Python 2 and 3"""
-
-from __future__ import absolute_import
-
-import functools
-import itertools
-import operator
-import sys
-import types
-
-__author__ = "Benjamin Peterson <benjamin@python.org>"
-__version__ = "1.16.0"
-
-
-# Useful for very coarse version differentiation.
-PY2 = sys.version_info[0] == 2
-PY3 = sys.version_info[0] == 3
-PY34 = sys.version_info[0:2] >= (3, 4)
-
-if PY3:
-    string_types = str,
-    integer_types = int,
-    class_types = type,
-    text_type = str
-    binary_type = bytes
-
-    MAXSIZE = sys.maxsize
-else:
-    string_types = basestring,
-    integer_types = (int, long)
-    class_types = (type, types.ClassType)
-    text_type = unicode
-    binary_type = str
-
-    if sys.platform.startswith("java"):
-        # Jython always uses 32 bits.
-        MAXSIZE = int((1 << 31) - 1)
-    else:
-        # It's possible to have sizeof(long) != sizeof(Py_ssize_t).
-        class X(object):
-
-            def __len__(self):
-                return 1 << 31
-        try:
-            len(X())
-        except OverflowError:
-            # 32-bit
-            MAXSIZE = int((1 << 31) - 1)
-        else:
-            # 64-bit
-            MAXSIZE = int((1 << 63) - 1)
-        del X
-
-if PY34:
-    from importlib.util import spec_from_loader
-else:
-    spec_from_loader = None
-
-
-def _add_doc(func, doc):
-    """Add documentation to a function."""
-    func.__doc__ = doc
-
-
-def _import_module(name):
-    """Import module, returning the module after the last dot."""
-    __import__(name)
-    return sys.modules[name]
-
-
-class _LazyDescr(object):
-
-    def __init__(self, name):
-        self.name = name
-
-    def __get__(self, obj, tp):
-        result = self._resolve()
-        setattr(obj, self.name, result)  # Invokes __set__.
-        try:
-            # This is a bit ugly, but it avoids running this again by
-            # removing this descriptor.
-            delattr(obj.__class__, self.name)
-        except AttributeError:
-            pass
-        return result
-
-
-class MovedModule(_LazyDescr):
-
-    def __init__(self, name, old, new=None):
-        super(MovedModule, self).__init__(name)
-        if PY3:
-            if new is None:
-                new = name
-            self.mod = new
-        else:
-            self.mod = old
-
-    def _resolve(self):
-        return _import_module(self.mod)
-
-    def __getattr__(self, attr):
-        _module = self._resolve()
-        value = getattr(_module, attr)
-        setattr(self, attr, value)
-        return value
-
-
-class _LazyModule(types.ModuleType):
-
-    def __init__(self, name):
-        super(_LazyModule, self).__init__(name)
-        self.__doc__ = self.__class__.__doc__
-
-    def __dir__(self):
-        attrs = ["__doc__", "__name__"]
-        attrs += [attr.name for attr in self._moved_attributes]
-        return attrs
-
-    # Subclasses should override this
-    _moved_attributes = []
-
-
-class MovedAttribute(_LazyDescr):
-
-    def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
-        super(MovedAttribute, self).__init__(name)
-        if PY3:
-            if new_mod is None:
-                new_mod = name
-            self.mod = new_mod
-            if new_attr is None:
-                if old_attr is None:
-                    new_attr = name
-                else:
-                    new_attr = old_attr
-            self.attr = new_attr
-        else:
-            self.mod = old_mod
-            if old_attr is None:
-                old_attr = name
-            self.attr = old_attr
-
-    def _resolve(self):
-        module = _import_module(self.mod)
-        return getattr(module, self.attr)
-
-
-class _SixMetaPathImporter(object):
-
-    """
-    A meta path importer to import six.moves and its submodules.
-
-    This class implements a PEP302 finder and loader. It should be compatible
-    with Python 2.5 and all existing versions of Python3
-    """
-
-    def __init__(self, six_module_name):
-        self.name = six_module_name
-        self.known_modules = {}
-
-    def _add_module(self, mod, *fullnames):
-        for fullname in fullnames:
-            self.known_modules[self.name + "." + fullname] = mod
-
-    def _get_module(self, fullname):
-        return self.known_modules[self.name + "." + fullname]
-
-    def find_module(self, fullname, path=None):
-        if fullname in self.known_modules:
-            return self
-        return None
-
-    def find_spec(self, fullname, path, target=None):
-        if fullname in self.known_modules:
-            return spec_from_loader(fullname, self)
-        return None
-
-    def __get_module(self, fullname):
-        try:
-            return self.known_modules[fullname]
-        except KeyError:
-            raise ImportError("This loader does not know module " + fullname)
-
-    def load_module(self, fullname):
-        try:
-            # in case of a reload
-            return sys.modules[fullname]
-        except KeyError:
-            pass
-        mod = self.__get_module(fullname)
-        if isinstance(mod, MovedModule):
-            mod = mod._resolve()
-        else:
-            mod.__loader__ = self
-        sys.modules[fullname] = mod
-        return mod
-
-    def is_package(self, fullname):
-        """
-        Return true, if the named module is a package.
-
-        We need this method to get correct spec objects with
-        Python 3.4 (see PEP451)
-        """
-        return hasattr(self.__get_module(fullname), "__path__")
-
-    def get_code(self, fullname):
-        """Return None
-
-        Required, if is_package is implemented"""
-        self.__get_module(fullname)  # eventually raises ImportError
-        return None
-    get_source = get_code  # same as get_code
-
-    def create_module(self, spec):
-        return self.load_module(spec.name)
-
-    def exec_module(self, module):
-        pass
-
-_importer = _SixMetaPathImporter(__name__)
-
-
-class _MovedItems(_LazyModule):
-
-    """Lazy loading of moved objects"""
-    __path__ = []  # mark as package
-
-
-_moved_attributes = [
-    MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
-    MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
-    MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"),
-    MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
-    MovedAttribute("intern", "__builtin__", "sys"),
-    MovedAttribute("map", "itertools", "builtins", "imap", "map"),
-    MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"),
-    MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"),
-    MovedAttribute("getoutput", "commands", "subprocess"),
-    MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"),
-    MovedAttribute("reload_module", "__builtin__", "importlib" if PY34 else "imp", "reload"),
-    MovedAttribute("reduce", "__builtin__", "functools"),
-    MovedAttribute("shlex_quote", "pipes", "shlex", "quote"),
-    MovedAttribute("StringIO", "StringIO", "io"),
-    MovedAttribute("UserDict", "UserDict", "collections"),
-    MovedAttribute("UserList", "UserList", "collections"),
-    MovedAttribute("UserString", "UserString", "collections"),
-    MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
-    MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
-    MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"),
-    MovedModule("builtins", "__builtin__"),
-    MovedModule("configparser", "ConfigParser"),
-    MovedModule("collections_abc", "collections", "collections.abc" if sys.version_info >= (3, 3) else "collections"),
-    MovedModule("copyreg", "copy_reg"),
-    MovedModule("dbm_gnu", "gdbm", "dbm.gnu"),
-    MovedModule("dbm_ndbm", "dbm", "dbm.ndbm"),
-    MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread" if sys.version_info < (3, 9) else "_thread"),
-    MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
-    MovedModule("http_cookies", "Cookie", "http.cookies"),
-    MovedModule("html_entities", "htmlentitydefs", "html.entities"),
-    MovedModule("html_parser", "HTMLParser", "html.parser"),
-    MovedModule("http_client", "httplib", "http.client"),
-    MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"),
-    MovedModule("email_mime_image", "email.MIMEImage", "email.mime.image"),
-    MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"),
-    MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"),
-    MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"),
-    MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
-    MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
-    MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
-    MovedModule("cPickle", "cPickle", "pickle"),
-    MovedModule("queue", "Queue"),
-    MovedModule("reprlib", "repr"),
-    MovedModule("socketserver", "SocketServer"),
-    MovedModule("_thread", "thread", "_thread"),
-    MovedModule("tkinter", "Tkinter"),
-    MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
-    MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
-    MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
-    MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
-    MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
-    MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"),
-    MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
-    MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
-    MovedModule("tkinter_colorchooser", "tkColorChooser",
-                "tkinter.colorchooser"),
-    MovedModule("tkinter_commondialog", "tkCommonDialog",
-                "tkinter.commondialog"),
-    MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
-    MovedModule("tkinter_font", "tkFont", "tkinter.font"),
-    MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
-    MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
-                "tkinter.simpledialog"),
-    MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"),
-    MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"),
-    MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"),
-    MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
-    MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
-    MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
-]
-# Add windows specific modules.
-if sys.platform == "win32":
-    _moved_attributes += [
-        MovedModule("winreg", "_winreg"),
-    ]
-
-for attr in _moved_attributes:
-    setattr(_MovedItems, attr.name, attr)
-    if isinstance(attr, MovedModule):
-        _importer._add_module(attr, "moves." + attr.name)
-del attr
-
-_MovedItems._moved_attributes = _moved_attributes
-
-moves = _MovedItems(__name__ + ".moves")
-_importer._add_module(moves, "moves")
-
-
-class Module_six_moves_urllib_parse(_LazyModule):
-
-    """Lazy loading of moved objects in six.moves.urllib_parse"""
-
-
-_urllib_parse_moved_attributes = [
-    MovedAttribute("ParseResult", "urlparse", "urllib.parse"),
-    MovedAttribute("SplitResult", "urlparse", "urllib.parse"),
-    MovedAttribute("parse_qs", "urlparse", "urllib.parse"),
-    MovedAttribute("parse_qsl", "urlparse", "urllib.parse"),
-    MovedAttribute("urldefrag", "urlparse", "urllib.parse"),
-    MovedAttribute("urljoin", "urlparse", "urllib.parse"),
-    MovedAttribute("urlparse", "urlparse", "urllib.parse"),
-    MovedAttribute("urlsplit", "urlparse", "urllib.parse"),
-    MovedAttribute("urlunparse", "urlparse", "urllib.parse"),
-    MovedAttribute("urlunsplit", "urlparse", "urllib.parse"),
-    MovedAttribute("quote", "urllib", "urllib.parse"),
-    MovedAttribute("quote_plus", "urllib", "urllib.parse"),
-    MovedAttribute("unquote", "urllib", "urllib.parse"),
-    MovedAttribute("unquote_plus", "urllib", "urllib.parse"),
-    MovedAttribute("unquote_to_bytes", "urllib", "urllib.parse", "unquote", "unquote_to_bytes"),
-    MovedAttribute("urlencode", "urllib", "urllib.parse"),
-    MovedAttribute("splitquery", "urllib", "urllib.parse"),
-    MovedAttribute("splittag", "urllib", "urllib.parse"),
-    MovedAttribute("splituser", "urllib", "urllib.parse"),
-    MovedAttribute("splitvalue", "urllib", "urllib.parse"),
-    MovedAttribute("uses_fragment", "urlparse", "urllib.parse"),
-    MovedAttribute("uses_netloc", "urlparse", "urllib.parse"),
-    MovedAttribute("uses_params", "urlparse", "urllib.parse"),
-    MovedAttribute("uses_query", "urlparse", "urllib.parse"),
-    MovedAttribute("uses_relative", "urlparse", "urllib.parse"),
-]
-for attr in _urllib_parse_moved_attributes:
-    setattr(Module_six_moves_urllib_parse, attr.name, attr)
-del attr
-
-Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes
-
-_importer._add_module(Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"),
-                      "moves.urllib_parse", "moves.urllib.parse")
-
-
-class Module_six_moves_urllib_error(_LazyModule):
-
-    """Lazy loading of moved objects in six.moves.urllib_error"""
-
-
-_urllib_error_moved_attributes = [
-    MovedAttribute("URLError", "urllib2", "urllib.error"),
-    MovedAttribute("HTTPError", "urllib2", "urllib.error"),
-    MovedAttribute("ContentTooShortError", "urllib", "urllib.error"),
-]
-for attr in _urllib_error_moved_attributes:
-    setattr(Module_six_moves_urllib_error, attr.name, attr)
-del attr
-
-Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes
-
-_importer._add_module(Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"),
-                      "moves.urllib_error", "moves.urllib.error")
-
-
-class Module_six_moves_urllib_request(_LazyModule):
-
-    """Lazy loading of moved objects in six.moves.urllib_request"""
-
-
-_urllib_request_moved_attributes = [
-    MovedAttribute("urlopen", "urllib2", "urllib.request"),
-    MovedAttribute("install_opener", "urllib2", "urllib.request"),
-    MovedAttribute("build_opener", "urllib2", "urllib.request"),
-    MovedAttribute("pathname2url", "urllib", "urllib.request"),
-    MovedAttribute("url2pathname", "urllib", "urllib.request"),
-    MovedAttribute("getproxies", "urllib", "urllib.request"),
-    MovedAttribute("Request", "urllib2", "urllib.request"),
-    MovedAttribute("OpenerDirector", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"),
-    MovedAttribute("ProxyHandler", "urllib2", "urllib.request"),
-    MovedAttribute("BaseHandler", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"),
-    MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"),
-    MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"),
-    MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"),
-    MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPHandler", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"),
-    MovedAttribute("FileHandler", "urllib2", "urllib.request"),
-    MovedAttribute("FTPHandler", "urllib2", "urllib.request"),
-    MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"),
-    MovedAttribute("UnknownHandler", "urllib2", "urllib.request"),
-    MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"),
-    MovedAttribute("urlretrieve", "urllib", "urllib.request"),
-    MovedAttribute("urlcleanup", "urllib", "urllib.request"),
-    MovedAttribute("URLopener", "urllib", "urllib.request"),
-    MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
-    MovedAttribute("proxy_bypass", "urllib", "urllib.request"),
-    MovedAttribute("parse_http_list", "urllib2", "urllib.request"),
-    MovedAttribute("parse_keqv_list", "urllib2", "urllib.request"),
-]
-for attr in _urllib_request_moved_attributes:
-    setattr(Module_six_moves_urllib_request, attr.name, attr)
-del attr
-
-Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes
-
-_importer._add_module(Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"),
-                      "moves.urllib_request", "moves.urllib.request")
-
-
-class Module_six_moves_urllib_response(_LazyModule):
-
-    """Lazy loading of moved objects in six.moves.urllib_response"""
-
-
-_urllib_response_moved_attributes = [
-    MovedAttribute("addbase", "urllib", "urllib.response"),
-    MovedAttribute("addclosehook", "urllib", "urllib.response"),
-    MovedAttribute("addinfo", "urllib", "urllib.response"),
-    MovedAttribute("addinfourl", "urllib", "urllib.response"),
-]
-for attr in _urllib_response_moved_attributes:
-    setattr(Module_six_moves_urllib_response, attr.name, attr)
-del attr
-
-Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes
-
-_importer._add_module(Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"),
-                      "moves.urllib_response", "moves.urllib.response")
-
-
-class Module_six_moves_urllib_robotparser(_LazyModule):
-
-    """Lazy loading of moved objects in six.moves.urllib_robotparser"""
-
-
-_urllib_robotparser_moved_attributes = [
-    MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"),
-]
-for attr in _urllib_robotparser_moved_attributes:
-    setattr(Module_six_moves_urllib_robotparser, attr.name, attr)
-del attr
-
-Module_six_moves_urllib_robotparser._moved_attributes = _urllib_robotparser_moved_attributes
-
-_importer._add_module(Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"),
-                      "moves.urllib_robotparser", "moves.urllib.robotparser")
-
-
-class Module_six_moves_urllib(types.ModuleType):
-
-    """Create a six.moves.urllib namespace that resembles the Python 3 namespace"""
-    __path__ = []  # mark as package
-    parse = _importer._get_module("moves.urllib_parse")
-    error = _importer._get_module("moves.urllib_error")
-    request = _importer._get_module("moves.urllib_request")
-    response = _importer._get_module("moves.urllib_response")
-    robotparser = _importer._get_module("moves.urllib_robotparser")
-
-    def __dir__(self):
-        return ['parse', 'error', 'request', 'response', 'robotparser']
-
-_importer._add_module(Module_six_moves_urllib(__name__ + ".moves.urllib"),
-                      "moves.urllib")
-
-
-def add_move(move):
-    """Add an item to six.moves."""
-    setattr(_MovedItems, move.name, move)
-
-
-def remove_move(name):
-    """Remove item from six.moves."""
-    try:
-        delattr(_MovedItems, name)
-    except AttributeError:
-        try:
-            del moves.__dict__[name]
-        except KeyError:
-            raise AttributeError("no such move, %r" % (name,))
-
-
-if PY3:
-    _meth_func = "__func__"
-    _meth_self = "__self__"
-
-    _func_closure = "__closure__"
-    _func_code = "__code__"
-    _func_defaults = "__defaults__"
-    _func_globals = "__globals__"
-else:
-    _meth_func = "im_func"
-    _meth_self = "im_self"
-
-    _func_closure = "func_closure"
-    _func_code = "func_code"
-    _func_defaults = "func_defaults"
-    _func_globals = "func_globals"
-
-
-try:
-    advance_iterator = next
-except NameError:
-    def advance_iterator(it):
-        return it.next()
-next = advance_iterator
-
-
-try:
-    callable = callable
-except NameError:
-    def callable(obj):
-        return any("__call__" in klass.__dict__ for klass in type(obj).__mro__)
-
-
-if PY3:
-    def get_unbound_function(unbound):
-        return unbound
-
-    create_bound_method = types.MethodType
-
-    def create_unbound_method(func, cls):
-        return func
-
-    Iterator = object
-else:
-    def get_unbound_function(unbound):
-        return unbound.im_func
-
-    def create_bound_method(func, obj):
-        return types.MethodType(func, obj, obj.__class__)
-
-    def create_unbound_method(func, cls):
-        return types.MethodType(func, None, cls)
-
-    class Iterator(object):
-
-        def next(self):
-            return type(self).__next__(self)
-
-    callable = callable
-_add_doc(get_unbound_function,
-         """Get the function out of a possibly unbound function""")
-
-
-get_method_function = operator.attrgetter(_meth_func)
-get_method_self = operator.attrgetter(_meth_self)
-get_function_closure = operator.attrgetter(_func_closure)
-get_function_code = operator.attrgetter(_func_code)
-get_function_defaults = operator.attrgetter(_func_defaults)
-get_function_globals = operator.attrgetter(_func_globals)
-
-
-if PY3:
-    def iterkeys(d, **kw):
-        return iter(d.keys(**kw))
-
-    def itervalues(d, **kw):
-        return iter(d.values(**kw))
-
-    def iteritems(d, **kw):
-        return iter(d.items(**kw))
-
-    def iterlists(d, **kw):
-        return iter(d.lists(**kw))
-
-    viewkeys = operator.methodcaller("keys")
-
-    viewvalues = operator.methodcaller("values")
-
-    viewitems = operator.methodcaller("items")
-else:
-    def iterkeys(d, **kw):
-        return d.iterkeys(**kw)
-
-    def itervalues(d, **kw):
-        return d.itervalues(**kw)
-
-    def iteritems(d, **kw):
-        return d.iteritems(**kw)
-
-    def iterlists(d, **kw):
-        return d.iterlists(**kw)
-
-    viewkeys = operator.methodcaller("viewkeys")
-
-    viewvalues = operator.methodcaller("viewvalues")
-
-    viewitems = operator.methodcaller("viewitems")
-
-_add_doc(iterkeys, "Return an iterator over the keys of a dictionary.")
-_add_doc(itervalues, "Return an iterator over the values of a dictionary.")
-_add_doc(iteritems,
-         "Return an iterator over the (key, value) pairs of a dictionary.")
-_add_doc(iterlists,
-         "Return an iterator over the (key, [values]) pairs of a dictionary.")
-
-
-if PY3:
-    def b(s):
-        return s.encode("latin-1")
-
-    def u(s):
-        return s
-    unichr = chr
-    import struct
-    int2byte = struct.Struct(">B").pack
-    del struct
-    byte2int = operator.itemgetter(0)
-    indexbytes = operator.getitem
-    iterbytes = iter
-    import io
-    StringIO = io.StringIO
-    BytesIO = io.BytesIO
-    del io
-    _assertCountEqual = "assertCountEqual"
-    if sys.version_info[1] <= 1:
-        _assertRaisesRegex = "assertRaisesRegexp"
-        _assertRegex = "assertRegexpMatches"
-        _assertNotRegex = "assertNotRegexpMatches"
-    else:
-        _assertRaisesRegex = "assertRaisesRegex"
-        _assertRegex = "assertRegex"
-        _assertNotRegex = "assertNotRegex"
-else:
-    def b(s):
-        return s
-    # Workaround for standalone backslash
-
-    def u(s):
-        return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
-    unichr = unichr
-    int2byte = chr
-
-    def byte2int(bs):
-        return ord(bs[0])
-
-    def indexbytes(buf, i):
-        return ord(buf[i])
-    iterbytes = functools.partial(itertools.imap, ord)
-    import StringIO
-    StringIO = BytesIO = StringIO.StringIO
-    _assertCountEqual = "assertItemsEqual"
-    _assertRaisesRegex = "assertRaisesRegexp"
-    _assertRegex = "assertRegexpMatches"
-    _assertNotRegex = "assertNotRegexpMatches"
-_add_doc(b, """Byte literal""")
-_add_doc(u, """Text literal""")
-
-
-def assertCountEqual(self, *args, **kwargs):
-    return getattr(self, _assertCountEqual)(*args, **kwargs)
-
-
-def assertRaisesRegex(self, *args, **kwargs):
-    return getattr(self, _assertRaisesRegex)(*args, **kwargs)
-
-
-def assertRegex(self, *args, **kwargs):
-    return getattr(self, _assertRegex)(*args, **kwargs)
-
-
-def assertNotRegex(self, *args, **kwargs):
-    return getattr(self, _assertNotRegex)(*args, **kwargs)
-
-
-if PY3:
-    exec_ = getattr(moves.builtins, "exec")
-
-    def reraise(tp, value, tb=None):
-        try:
-            if value is None:
-                value = tp()
-            if value.__traceback__ is not tb:
-                raise value.with_traceback(tb)
-            raise value
-        finally:
-            value = None
-            tb = None
-
-else:
-    def exec_(_code_, _globs_=None, _locs_=None):
-        """Execute code in a namespace."""
-        if _globs_ is None:
-            frame = sys._getframe(1)
-            _globs_ = frame.f_globals
-            if _locs_ is None:
-                _locs_ = frame.f_locals
-            del frame
-        elif _locs_ is None:
-            _locs_ = _globs_
-        exec("""exec _code_ in _globs_, _locs_""")
-
-    exec_("""def reraise(tp, value, tb=None):
-    try:
-        raise tp, value, tb
-    finally:
-        tb = None
-""")
-
-
-if sys.version_info[:2] > (3,):
-    exec_("""def raise_from(value, from_value):
-    try:
-        raise value from from_value
-    finally:
-        value = None
-""")
-else:
-    def raise_from(value, from_value):
-        raise value
-
-
-print_ = getattr(moves.builtins, "print", None)
-if print_ is None:
-    def print_(*args, **kwargs):
-        """The new-style print function for Python 2.4 and 2.5."""
-        fp = kwargs.pop("file", sys.stdout)
-        if fp is None:
-            return
-
-        def write(data):
-            if not isinstance(data, basestring):
-                data = str(data)
-            # If the file has an encoding, encode unicode with it.
-            if (isinstance(fp, file) and
-                    isinstance(data, unicode) and
-                    fp.encoding is not None):
-                errors = getattr(fp, "errors", None)
-                if errors is None:
-                    errors = "strict"
-                data = data.encode(fp.encoding, errors)
-            fp.write(data)
-        want_unicode = False
-        sep = kwargs.pop("sep", None)
-        if sep is not None:
-            if isinstance(sep, unicode):
-                want_unicode = True
-            elif not isinstance(sep, str):
-                raise TypeError("sep must be None or a string")
-        end = kwargs.pop("end", None)
-        if end is not None:
-            if isinstance(end, unicode):
-                want_unicode = True
-            elif not isinstance(end, str):
-                raise TypeError("end must be None or a string")
-        if kwargs:
-            raise TypeError("invalid keyword arguments to print()")
-        if not want_unicode:
-            for arg in args:
-                if isinstance(arg, unicode):
-                    want_unicode = True
-                    break
-        if want_unicode:
-            newline = unicode("\n")
-            space = unicode(" ")
-        else:
-            newline = "\n"
-            space = " "
-        if sep is None:
-            sep = space
-        if end is None:
-            end = newline
-        for i, arg in enumerate(args):
-            if i:
-                write(sep)
-            write(arg)
-        write(end)
-if sys.version_info[:2] < (3, 3):
-    _print = print_
-
-    def print_(*args, **kwargs):
-        fp = kwargs.get("file", sys.stdout)
-        flush = kwargs.pop("flush", False)
-        _print(*args, **kwargs)
-        if flush and fp is not None:
-            fp.flush()
-
-_add_doc(reraise, """Reraise an exception.""")
-
-if sys.version_info[0:2] < (3, 4):
-    # This does exactly the same what the :func:`py3:functools.update_wrapper`
-    # function does on Python versions after 3.2. It sets the ``__wrapped__``
-    # attribute on ``wrapper`` object and it doesn't raise an error if any of
-    # the attributes mentioned in ``assigned`` and ``updated`` are missing on
-    # ``wrapped`` object.
-    def _update_wrapper(wrapper, wrapped,
-                        assigned=functools.WRAPPER_ASSIGNMENTS,
-                        updated=functools.WRAPPER_UPDATES):
-        for attr in assigned:
-            try:
-                value = getattr(wrapped, attr)
-            except AttributeError:
-                continue
-            else:
-                setattr(wrapper, attr, value)
-        for attr in updated:
-            getattr(wrapper, attr).update(getattr(wrapped, attr, {}))
-        wrapper.__wrapped__ = wrapped
-        return wrapper
-    _update_wrapper.__doc__ = functools.update_wrapper.__doc__
-
-    def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS,
-              updated=functools.WRAPPER_UPDATES):
-        return functools.partial(_update_wrapper, wrapped=wrapped,
-                                 assigned=assigned, updated=updated)
-    wraps.__doc__ = functools.wraps.__doc__
-
-else:
-    wraps = functools.wraps
-
-
-def with_metaclass(meta, *bases):
-    """Create a base class with a metaclass."""
-    # This requires a bit of explanation: the basic idea is to make a dummy
-    # metaclass for one level of class instantiation that replaces itself with
-    # the actual metaclass.
-    class metaclass(type):
-
-        def __new__(cls, name, this_bases, d):
-            if sys.version_info[:2] >= (3, 7):
-                # This version introduced PEP 560 that requires a bit
-                # of extra care (we mimic what is done by __build_class__).
-                resolved_bases = types.resolve_bases(bases)
-                if resolved_bases is not bases:
-                    d['__orig_bases__'] = bases
-            else:
-                resolved_bases = bases
-            return meta(name, resolved_bases, d)
-
-        @classmethod
-        def __prepare__(cls, name, this_bases):
-            return meta.__prepare__(name, bases)
-    return type.__new__(metaclass, 'temporary_class', (), {})
-
-
-def add_metaclass(metaclass):
-    """Class decorator for creating a class with a metaclass."""
-    def wrapper(cls):
-        orig_vars = cls.__dict__.copy()
-        slots = orig_vars.get('__slots__')
-        if slots is not None:
-            if isinstance(slots, str):
-                slots = [slots]
-            for slots_var in slots:
-                orig_vars.pop(slots_var)
-        orig_vars.pop('__dict__', None)
-        orig_vars.pop('__weakref__', None)
-        if hasattr(cls, '__qualname__'):
-            orig_vars['__qualname__'] = cls.__qualname__
-        return metaclass(cls.__name__, cls.__bases__, orig_vars)
-    return wrapper
-
-
-def ensure_binary(s, encoding='utf-8', errors='strict'):
-    """Coerce **s** to six.binary_type.
-
-    For Python 2:
-      - `unicode` -> encoded to `str`
-      - `str` -> `str`
-
-    For Python 3:
-      - `str` -> encoded to `bytes`
-      - `bytes` -> `bytes`
-    """
-    if isinstance(s, binary_type):
-        return s
-    if isinstance(s, text_type):
-        return s.encode(encoding, errors)
-    raise TypeError("not expecting type '%s'" % type(s))
-
-
-def ensure_str(s, encoding='utf-8', errors='strict'):
-    """Coerce *s* to `str`.
-
-    For Python 2:
-      - `unicode` -> encoded to `str`
-      - `str` -> `str`
-
-    For Python 3:
-      - `str` -> `str`
-      - `bytes` -> decoded to `str`
-    """
-    # Optimization: Fast return for the common case.
-    if type(s) is str:
-        return s
-    if PY2 and isinstance(s, text_type):
-        return s.encode(encoding, errors)
-    elif PY3 and isinstance(s, binary_type):
-        return s.decode(encoding, errors)
-    elif not isinstance(s, (text_type, binary_type)):
-        raise TypeError("not expecting type '%s'" % type(s))
-    return s
-
-
-def ensure_text(s, encoding='utf-8', errors='strict'):
-    """Coerce *s* to six.text_type.
-
-    For Python 2:
-      - `unicode` -> `unicode`
-      - `str` -> `unicode`
-
-    For Python 3:
-      - `str` -> `str`
-      - `bytes` -> decoded to `str`
-    """
-    if isinstance(s, binary_type):
-        return s.decode(encoding, errors)
-    elif isinstance(s, text_type):
-        return s
-    else:
-        raise TypeError("not expecting type '%s'" % type(s))
-
-
-def python_2_unicode_compatible(klass):
-    """
-    A class decorator that defines __unicode__ and __str__ methods under Python 2.
-    Under Python 3 it does nothing.
-
-    To support Python 2 and 3 with a single code base, define a __str__ method
-    returning text and apply this decorator to the class.
-    """
-    if PY2:
-        if '__str__' not in klass.__dict__:
-            raise ValueError("@python_2_unicode_compatible cannot be applied "
-                             "to %s because it doesn't define __str__()." %
-                             klass.__name__)
-        klass.__unicode__ = klass.__str__
-        klass.__str__ = lambda self: self.__unicode__().encode('utf-8')
-    return klass
-
-
-# Complete the moves implementation.
-# This code is at the end of this module to speed up module loading.
-# Turn this module into a package.
-__path__ = []  # required for PEP 302 and PEP 451
-__package__ = __name__  # see PEP 366 @ReservedAssignment
-if globals().get("__spec__") is not None:
-    __spec__.submodule_search_locations = []  # PEP 451 @UndefinedVariable
-# Remove other six meta path importers, since they cause problems. This can
-# happen if six is removed from sys.modules and then reloaded. (Setuptools does
-# this for some reason.)
-if sys.meta_path:
-    for i, importer in enumerate(sys.meta_path):
-        # Here's some real nastiness: Another "instance" of the six module might
-        # be floating around. Therefore, we can't use isinstance() to check for
-        # the six meta path importer, since the other six instance will have
-        # inserted an importer with different class.
-        if (type(importer).__name__ == "_SixMetaPathImporter" and
-                importer.name == __name__):
-            del sys.meta_path[i]
-            break
-    del i, importer
-# Finally, add the importer to the meta path import hook.
-sys.meta_path.append(_importer)

Roadmap.md

@@ -14,15 +14,15 @@ This document outlines the currently established development roadmap for Reticul
 ## Currently Active Work Areas
 For each release cycle of Reticulum, improvements and additions from the five [Primary Efforts](#primary-efforts) are selected as active work areas, and can be expected to be included in the upcoming releases within that cycle. While not entirely set in stone for each release cycle, they serve as a pointer of what to expect in the near future.
 
-- The current `0.7.x` release cycle aims at completing
-  - [x] Automatic asynchronous key ratcheting for non-link traffic
-  - [ ] API improvements based on real-world usage and feedback
+- The current `0.8.x` release cycle aims at completing
+  - [ ] Hot-pluggable interface system
+  - [ ] External interface plugins
+  - [ ] Network-wide path balancing and multi-pathing
   - [ ] Expanded hardware support
   - [ ] Overhauling and updating the documentation
   - [ ] Distributed Destination Naming System
-  - [ ] Create a standalone RNS Daemon app for Android
-  - [ ] Network-wide path balancing
-  - [ ] Add automatic retries to all use cases of the `Request` API
+  - [ ] A standalone RNS Daemon app for Android
+  - [ ] Addding automatic retries to all use cases of the `Request` API
   - [ ] Performance and memory optimisations of the Python reference implementation
   - [ ] Fixing bugs discovered while operating Reticulum systems and applications
 
@@ -38,17 +38,9 @@ These efforts are aimed at improving the ease of which Reticulum is understood,
         - Update announce description
         - Add in-depth explanation of the IFAC system
     - Software
-        - Update Sideband screenshots
-        - Update Sideband description
-        - Update NomadNet screenshots
-        - Update Sideband screenshots
-    - Installation
-        - [x] Add a *Reticulum On Raspberry Pi* section
-        - [x] Update *Reticulum On Android* section if necessary
-        - [x] Update Android install documentation.
+        - Update software descriptions and screenshots
     - Communications hardware section
         - Add information about RNode external displays.
-        - [x] Packet radio modems.
         - Possibly add other relevant types here as well.
     - Setup *Best Practices For...* / *Installation Examples* section.
         - Home or office (example)
@@ -68,6 +60,8 @@ These efforts seek to broaden the universality of the Reticulum software and har
 ### Functionality
 These efforts aim to expand and improve the core functionality and reliability of Reticulum.
 
+- Add support for user-supplied external interface drivers
+- Add interface hot-plug and live up/down control to running instances
 - Add automatic retries to all use cases of the `Request` API
 - Network-wide path balancing
 - Distributed Destination Naming System
@@ -85,10 +79,10 @@ These effors seek to make Reticulum easier to use and operate, and to expand the
 ### Interfaceability
 These efforts aim to expand the types of physical and virtual interfaces that Reticulum can natively use to transport data.
 
-- Filesystem interface
 - Plain ESP32 devices (ESP-Now, WiFi, Bluetooth, etc.)
 - More LoRa transceivers
 - AT-compatible modems
+- Filesystem interface
 - Direct SDR Support
 - Optical mediums
 - IR Transceivers
@@ -108,7 +102,7 @@ The Reticulum ecosystem is enriched by several other software and hardware proje
 This section lists, in no particular order, various important efforts that would be beneficial to the goals of Reticulum.
 
 - The [RNode](https://unsigned.io/rnode/) project
-    - [ ] Create a WebUSB-based bootstrapping utility, and integrate this directly into the [RNode Bootstrap Console](#), both on-device, and on an Internet-reachable copy. This will make it much easier to create new RNodes for average users.
+    - [x] Create a WebUSB-based bootstrapping utility, and integrate this directly into the [RNode Bootstrap Console](#), both on-device, and on an Internet-reachable copy. This will make it much easier to create new RNodes for average users.
 
 ## Release History
 

docs/manual/.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 00ed5910a4dea8b0c1214035e7bf2859
+config: b499af51edc22529181ef3b25973fa2b
 tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/manual/_sources/examples.rst.txt

@@ -86,7 +86,7 @@ This example can also be found at `<https://github.com/markqvist/Reticulum/blob/
 Requests & Responses
 ====================
 
-The *Request* example explores sendig requests and receiving responses.
+The *Request* example explores sending requests and receiving responses.
 
 .. literalinclude:: ../../Examples/Request.py
 
@@ -125,4 +125,18 @@ interface to efficiently pass files of any size over a Reticulum :ref:`Link<api-
 
 .. literalinclude:: ../../Examples/Filetransfer.py
 
-This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Filetransfer.py>`_.
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/Filetransfer.py>`_.
+
+.. _example-custominterface:
+
+Custom Interfaces
+=================
+
+The *ExampleInterface* demonstrates creating custom interfaces for Reticulum.
+Any number of custom interfaces can be loaded and utilised by Reticulum, and
+will be fully on-par with natively included interfaces, including all supported
+:ref:`interface modes<interfaces-modes>` and :ref:`common configuration options<interfaces-options>`.
+
+.. literalinclude:: ../../Examples/ExampleInterface.py
+
+This example can also be found at `<https://github.com/markqvist/Reticulum/blob/master/Examples/ExampleInterface.py>`_.