Speech To Text

The Speech To Text plugin listens to your microphone and starts flows when you say a sentence or a particular word. Recognition runs on your computer using OpenAI's Whisper speech model, so your audio isn't sent anywhere. This page describes version 1.3.0, which runs on Windows only. Install it from Manage plugins → Browse. See how nodes work for how to read the node diagrams.

Setup

Plugin pages are hidden from the sidebar until you turn them on; see showing a plugin's pages.

Open SpeechToText in the sidebar.

1. Under Quick Settings, leave Use default microphone on, or turn it off and pick a Microphone Input. The Live Microphone Input meter shows whether it hears you.
2. Pick a Whisper Model. The default, TinyEn, is small and fast and understands English. Larger models, such as BaseEn or SmallEn, are more accurate but slower. Unknown names fall back to TinyEn.
3. Click Save Quick Settings, then Start.

The first time you start with a model, MultiShock downloads it, which can take a while. To start listening automatically, turn on Auto-start listening on app startup. The triggers below only fire while it is listening.

Recognition works on chunks of audio. Segment (ms) sets how long each chunk is (3.5 seconds by default); longer chunks catch whole sentences but take longer to react. Chunks quieter than Silence Threshold are skipped, and results below Min Confidence are ignored. The page lists recent sentences it recognized, which helps when tuning these.

More options, such as the recognition language (Recognition Culture, or auto to detect it) and a custom model file, are under Manage plugins → Configure on SpeechToText.

Sentence Recognized

Fires for each sentence recognized.

Exact compares the filter with the whole sentence.

Word Recognized

Fires once for each word in a recognized sentence.

To react to a keyword, set Word Filter to the word and Match Mode to Exact.

Contributors

Thank you to everyone who has helped improve this documentation.

bwmp