Currently, for compatibility reasons, we implement MSC3245v1 which puts the voice data inside an audio message type. Though at times it seems impractical to deal with a single message type which effectively represents 2 different kinds of messages.
This PR creates a new message type called `VoiceMessageType` which is used whenever we receive an event with `"msgtype": "m.audio"` which also has the `"org.matrix.msc3245.voice": {}` field. This makes it easier to process voice messages as different entities throughout the rest of the codebase.
[MSC3246](https://github.com/matrix-org/matrix-spec-proposals/pull/3246) specifies the waveform as a list of ints because:
> Because floating point numbers are not allowed in Matrix events
Though DSP on audio data is almost always done using their floating point representation.
This PR brings the float<->int rescaling in the `matrix` module so that the application code can always work with float waveform samples.
* Initial support for member suggestion (search and UI)
* Add custom `BottomSheetScaffold` implementation to workaround several scrolling bugs
* Start searching as soon as `@` is typed, add UI following initial designs
* Extract suggestion processing code
* Extract component, add previews, fix tests
* Add tests
* Add exception from kover to the forked bottom sheet code
* Add a feature flag for mentions
- Extract composer & mention suggestions to their composable.
- Extract mentions suggestions processing to its own class.
- Add `MatrixRoom.canTriggerRoomNotification` function.
- Update strings and conditions for displaying the `@room` mention.
---------
Co-authored-by: ElementBot <benoitm+elementbot@element.io>
Shows voice messages in the room summary.
Shows voice messages in the reply context menu and composer.
Show replies to voice messages in the timeline.
(before this PR voice messages were shown the same as audio messages)
Story: https://github.com/vector-im/element-meta/issues/2106
- New `AudioLevelCalculator` that outputs dB0v rescaled to the [0;1] range.
- `VoiceRecorder` now stores the audio levels sampled while recording, then resamples them to 100 samples to use as waveform preview.
- Waveform data is carried all the way as a `List<Float>` and converted to `List<Int>` in the [0;1024] range as per matrix spec only before sending it.
* Update the chat screen UI using `RoomInfo`.
This is specially useful for getting live values for `hasRoomCall`.
* Ensure the first `MatrixRoomInfo` is emitted ASAP
* Try excluding `*Present$present$*` inner functions from kover as separate entities
* Update strings
---------
Co-authored-by: ElementBot <benoitm+elementbot@element.io>
This is in preparation of further changes to the way the audio level is computed and to allow recording and sending of the waveform. The main reasoning behind the change is twofold:
1) We don't need the precision of Double in our context (we just need a rough indication of the changes in audio level to successfully draw a level meter or a waveform in our UI).
2) Performance: It is true that on 64 bit CPUs single operations involving Floats or Doubles take the same amount of time (i.e one clock cycle). But there are other aspects here that vouch in favor of Floats:
- A float takes half the space in memory compared to a double, so when storing long lists of them this can add up.
- On Android O and greater the ART runtime can "vectorize" certain operations on lists and make use of the CPU's SIMD registers which are generally 128 bits. So by using floats 4 of them can fit and be computed at the same time whilst with doubles only 2 will fit halving the throughput.
References:
- https://source.android.com/docs/core/runtime/improvements
- https://www.slideshare.net/linaroorg/automatic-vectorization-in-art-android-runtime-sfo17216
Marco Romano