Voxa: Voice typing on Linux with Google's leading-edge AI

Voxa allows you to type on Linux by speaking to your computer.

Basically, the goal was to bring the
Gboard
voice transcription feature to Linux by using the same AI. The following features
are implemented or planned:

• Works on any Linux (X11, Wayland, etc.)
• Spoken Emojis ("woman police officer emoji" = 👮‍♀️)
• Automatic Punctuation (Google AI will infer commas, periods, etc.)
• Spoken Punctuation ("comma", "new paragraph", etc.)
• Endless Transcription
• Automatic Language Detection
• Notification sound when starting and stopping transcription
• Automatic stop after 1 minute not speaking
• User configuration these above features ↑

The following features are being considered but not yet planned:

• Speech detection; start/stop transcription stream based on audio
  level or local speech detection (to save money)
• User manual translations
• support Gboard "clear" and "delete" commands
• Linux distribution packaging

Pricing

Voxa is free and open source software, but the AI costs
money. Transcription costs $0.016/minute, but you get $300 of free
Google cloud credit with new accounts, so transcription is essentially
free for 90 days.

Installation

TODO: packaging

For now, you must build from source using the instructions below

Google Cloud setup

TODO: add instructions to set up Google Cloud and set an API key.

Usage

./build/voxa

Runs Voxa in streaming keyboard emulation mode. Voxa will act as a keyboard and type the transcription as you speak. Make sure you switch to the window where you want to collect the text before speaking. Remember that Voxa is actively typing as an emulated keyboard, so you can't click around while it transcribes!

Voxa will play a start sound when it is connected and streaming audio to the Google API and a stop sound when the stream is disconnected. You can disable these sounds with VOXA_SILENT=true. Note that the API does not support continuous streaming for more than 5 minutes, so every 5 minutes Voxa will automatically reconnect.

Languages

Supported languages are listed here. By default, the primary language is en-US. Change the primary language by setting the VOXA_PRIMARY_LANGUAGE environment variable. Add up to three alternative languages by setting the VOXA_ALT_LANGUAGES environment variable.

For instance:

VOXA_PRIMARY_LANGUAGE="da-DK" VOXA_ALT_LANGUAGES="cmn-Hans-CN,ja-JP,es-MX" ./build/voxa

This will run Voxa with the primary language set to Danish, but you can speak Mandarin Chinese, Japanese, or Spanish.

Build instructions

1. git clone --recurse-submodules --shallow-submodules https://github.com/wrycode/vtt.git

2. Install development and runtime dependencies for the main code: CMake, GCC, Catch2, ICU, just, Gstreamer, probably a few more I've forgotten

3. Install dependencies for the Cloud Speech-to-Text API C++ Client Library: https://github.com/googleapis/google-cloud-cpp/blob/main/doc/packaging.md#required-libraries

On Arch Linux, all the dependencies can be installed with:

• sudo pacman -S cmake abseil-cpp grpc curl nlohmann-json just icu catch2
• manually install google-crc32c from the AUR

4. Run just to build and compile

TODO debugging

• GST_DEBUG=true
• TODO set up logging

Enabling virtual keyboard

Keys are typed using the uinput kernel interface, exposed in /dev/uinput. The program needs to have permissions to write to /dev/uinput. The easiest way is to chmod /dev/uinput, but there is probably a smarter way with udev.

Unicode characters

For languages other than English, you must have IBus. It may be enabled/installed already. You can test if it is installed by typing Ctrl+Shift+U (then release), then 2+7+0+5. You should see: ✅