Commit Graph

76 Commits

Author SHA1 Message Date
Travis Ralston 6b54f83b08 Merge branch 'develop' into travis/voice-messages/large 2021-07-22 04:16:53 -06:00
Travis Ralston b3d14de181 Merge pull request #6435 from matrix-org/travis/voice-messages/duration
Fix duration placeholder parsing for audio files
2021-07-22 03:40:43 -06:00
Travis Ralston 0bb0c8105d Merge pull request #6434 from matrix-org/travis/voice-messages/upload-failed
Handle upload errors in voice messages
2021-07-22 03:40:26 -06:00
Travis Ralston c53ed5be2e Use a MediaElementSourceAudioNode to process large audio files
Fixes https://github.com/vector-im/element-web/issues/18149

See comment block contained within diff.
2021-07-21 17:53:49 -06:00
Travis Ralston 2addca1ca4 Fix duration placeholder parsing for audio files 2021-07-21 17:09:34 -06:00
Travis Ralston 079e75aae9 Handle upload errors in voice messages
Fixes https://github.com/vector-im/element-web/issues/17148

Mostly adding some try/catch blocks, and a generic dialog.
2021-07-21 16:11:10 -06:00
Travis Ralston 3d72b9e227 Render error state for audio components
Fixes https://github.com/vector-im/element-web/issues/17148

Both `AudioPlayer` and `RecordingPlayback` got a fragment added to their DOM structure, to incorporate the rarely seen error message below it. Additionally, a missing try/catch was added to the `Playback` class to handle uncaught decoding issues.

The similarity of the components is tracked in https://github.com/vector-im/element-web/issues/18161
2021-07-21 15:47:52 -06:00
Travis Ralston 45f9f3e13e Name functions better 2021-07-20 23:34:27 -06:00
Travis Ralston a759d61ba0 Play only one audio file at a time
Fixes https://github.com/vector-im/element-web/issues/17439
2021-07-20 23:28:07 -06:00
Travis Ralston 199c80315e Merge pull request #6359 from matrix-org/travis/voice-messages/samplesssss
Increase sample count in voice message thumbnail
2021-07-12 14:18:22 -06:00
Travis Ralston e045aa940e Be smart with readonly 2021-07-12 14:11:24 -06:00
Travis Ralston 79aa205a95 variablize 2021-07-12 14:05:59 -06:00
Travis Ralston 4c98c0bc23 Increase sample count in voice message thumbnail
Fixes https://github.com/vector-im/element-web/issues/17817

Technically requires https://github.com/matrix-org/matrix-react-sdk/pull/6357 for sample sizing.
2021-07-12 14:02:51 -06:00
Travis Ralston 0e2bcb474d delint 2021-07-12 13:52:10 -06:00
Travis Ralston ec0f940ef0 Adjust recording waveform behaviour for voice messages
Fixes https://github.com/vector-im/element-web/issues/17683
2021-07-12 13:48:01 -06:00
Travis Ralston 12d7be2b6a Merge branch 'develop' into travis/new-audio 2021-06-30 13:25:41 -06:00
J. Ryan Stinnett ae0a8b8da4 Auto-fix lint errors 2021-06-29 13:11:58 +01:00
Travis Ralston 964a97d59e Merge remote-tracking branch 'origin/develop' into travis/new-audio 2021-06-28 20:52:44 -06:00
Travis Ralston c16228b8ca Update src/voice/VoiceRecording.ts
Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2021-06-28 01:22:05 -06:00
Travis Ralston cb0d2a2c4f Encrypt the voice message file if needed
Fixes https://github.com/vector-im/element-web/issues/17729

"oops, should have done that"
2021-06-25 13:54:05 -06:00
Travis Ralston ebb6f1b602 Add seeking and notes about clock desync 2021-06-24 20:18:50 -06:00
Travis Ralston dda60949c3 Introduce basic audio playback control 2021-06-24 11:38:35 -06:00
Šimon Brandner 9bceb40820 CallMediaHandler -> MediaDeviceHandler
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
2021-06-23 09:26:33 +02:00
Travis Ralston 55eddae096 Merge pull request #6035 from matrix-org/travis/voicemessages/playback-time
Improve progress bar progression for smaller voice messages
2021-05-14 10:17:35 -06:00
Travis Ralston ddf9027daa Improve progress bar progression for smaller voice messages
Instead of chunking the thing, we'll improve our precision and clock accuracy. Clock accuracy is improved by flagging the "load time" of the clip in the context, which can be about 500ms (or more) off the context's start line. The precision is just a number in the PlaybackWaveform component.
2021-05-13 22:35:43 -06:00
Travis Ralston df7b598ca7 Improve visible waveform for voice messages
This tries to prioritize actual voice to decide the waveform, and clamps noise to zero to ensure the waveform doesn't have a perceptually noisy base.

In theory this better matches the overall voice message content.
2021-05-13 22:20:08 -06:00
Travis Ralston b1cb2b1d93 Fix bad merge 2021-05-11 10:19:09 -06:00
Travis Ralston d8e3458c55 Merge remote-tracking branch 'origin/develop' into travis/voicemessages/resample2 2021-05-11 10:15:37 -06:00
Travis Ralston b007ea81b2 Rescale and smooth playback waveform to better match expectation 2021-05-07 21:06:07 -06:00
Travis Ralston 65f591b69b Make the tests happier
Here we just override the workers because we're not expecting to be able to test them this way. The code paths involved shouldn't be touched.
2021-05-06 22:08:00 -06:00
Travis Ralston f65773ef95 Appease the linter 2021-05-06 21:49:53 -06:00
Travis Ralston b08e47bfe1 Support compatibility points for Safari
Tested on MacOS Big Sur, Safari 14.0.3
2021-05-06 21:43:17 -06:00
Travis Ralston b61fe2f8e6 Improve error recovery when starting a recording
This helps return the microphone access to the user.
2021-05-06 21:43:17 -06:00
Travis Ralston ed43d92579 Match designs more closely with +1 sample 2021-05-04 20:55:35 -06:00
Travis Ralston a94c1a90c1 Update colours and sizing for voice messages
Fixes https://github.com/vector-im/element-web/issues/17162
2021-05-04 20:45:15 -06:00
Travis Ralston 54931cb43e Merge remote-tracking branch 'origin/develop' into travis/voicemessages/timeline 2021-05-04 09:05:04 -06:00
Travis Ralston 7f56ec7bd2 docs 2021-05-03 15:16:50 -06:00
Travis Ralston 0806af0f4d Calculate the real waveform in the Playback class 2021-05-03 15:07:52 -06:00
Travis Ralston 3f4ee9742a Simple rendering of voice messages in the timeline
Fixes https://github.com/vector-im/element-web/issues/17138
2021-05-03 11:41:14 -06:00
Travis Ralston f0ff2fc38d Ensure we capture an absolute maximum amount of audio samples
We say the limit is 2 minutes, not 1m59s, so let's give the user that last frame.
2021-04-27 23:30:54 -06:00
Travis Ralston c2bcdae8a9 Switch global var to the store for easier debugging 2021-04-27 23:04:49 -06:00
Travis Ralston c2d37af1cb Move playback to its own set of classes
This all started with a bug where the clock wouldn't update appropriately, and ended with a whole refactoring to support later playback in the timeline.

Playback and recording instances are now independent, and this applies to the <Playback* /> components as well. Instead of those playback components taking a recording, they take a playback instance which has all the information the components need.

The clock was incredibly difficult to do because of the audio context's time tracking and the source's inability to say where it is at in the buffer/in time. This means we have to track when we started playing the clip so we can capture the audio context's current time, which may be a few seconds by the first time the user hits play. We also track stops so we know when to reset that flag.

Waveform calculations have also been moved into the base component, deduplicating the math a bit.
2021-04-27 22:23:57 -06:00
Travis Ralston 5e646f861c Wire up the send button for voice messages
This fixes a bug where we couldn't upload voice messages because the audio buffer was being read, therefore changing the position of the cursor. When this happened, the upload function would claim that the buffer was empty and could not be read.
2021-04-27 18:59:10 -06:00
Travis Ralston 30e120284d Add simple play/pause controls 2021-04-27 15:02:15 -06:00
Travis Ralston 32e3ce3dea Handle basic state machine of recordings 2021-04-27 15:02:15 -06:00
Travis Ralston 14809dfda7 Misc cleanup 2021-04-22 08:22:31 -06:00
Travis Ralston b519d85127 Update src/voice/RecorderWorklet.ts to use sanity
Co-authored-by: Germain <germain@souquet.com>
2021-04-20 09:32:12 -06:00
Travis Ralston 4f75e2944c Appease the linter 2021-04-19 23:11:41 -06:00
Travis Ralston 61730f2f88 Populate waveform data on voice message event 2021-04-19 23:05:53 -06:00
Travis Ralston 7d9562137e Replace deprecated processor with a worklet 2021-04-19 21:54:08 -06:00