When Apps Talk Back: the Text-To-Speech Plugin

As any mobile developer knows, once you start a mobile app project, the strength of the project often depends on the solidity of the plugins you need to integrate. I’ve personally never worked on an app that didn’t have at least five plugins to add custom, special mobile-friendly features. Some plugins offer layers of complex functionality - the Firebase plugin, for example, is as complex as Firebase itself, allowing the developer to choose which parts of the plugin to integrate: Admob, file storage, push notifications, remote config, etc. But some plugins work so seamlessly out of the box that they are a dream to configure and use. The text-to-speech plugin, initially created by Matthew Knight and then taken over and enhanced by Developer Expert Brad Martin, is just such a plugin.

Why would I need to use a text-to-speech plugin in my apps? Well, a great use case is if you are targeting the elderly, who have trouble reading text on mobile - a simple button push, and a voice reads the text out loud to you. Another would be if you are creating an e-book - I have had requests to have a read-aloud functionality in my own Snappy Squirrel e-book series. Yet another is my current use case: a recipe app, for use when you don’t want to be cooking and scrolling down in a recipe at the same time with chocolate-covered fingers. Instead, a button tap will allow the ingredients, method, and tool list to be read aloud.

Integrating the plugin is easy. The plugin page offers an example and demo in vanilla NativeScript, but I have done it as well with Angular and NativeScript: 

Install the plugin in your app:

tns plugin add nativescript-texttospeech

Then, import the plugin reference where you need it. I use it in my recipe-detail component:

...
//import the plugin
import { TNSTextToSpeech, SpeakOptions } from 'nativescript-texttospeech';
...
@Component({
    ...
})
export class RecipeDetailComponent implements OnInit {
    ...
    //set a default boolean for a button to animate when the tts plugin 'speaks'
    isSpeaking: boolean = false;
    ...
    private TTS: TNSTextToSpeech;
    constructor(
        ...
    ) {
        ...
        this.TTS = new TNSTextToSpeech();
    }
    ...
    //on button press, the text visible on the screen is spoken aloud
    speak(text: string){
        this.isSpeaking = true;
        let speakOptions: SpeakOptions = {
            text: text,
            speakRate: 0.5,
            //language: "fr",
            finishedCallback: (() => {
                this.isSpeaking = false;
            })  
        }
        this.TTS.speak(speakOptions);
    }
}

Once the plugin is installed and integrated, the amusing tinkering can begin. Various SpeakOptions are provided so that you can customize the playback. Tweak the speak rate, volume, and pitch to your liking, noting that emulators and platforms have varying results. I found that setting the speakRate to the recommended ’1.0’ led to a “chipmunks” version of my recipe, but I can imagine that a slider to increase the volume for some users would be very useful:

text: string ** required **
queue?: boolean = false
pitch?: number = 1.0
speakRate?: number = 1.0
volume?: number = 1.0
language?: string = default system language
finishedCallback?: Function

Of particular interest is the ability to set a language. If your target audience is in France, for example, you might want to set the language to its appropriate ISO 639-1 language code (such as ‘fr’ for French). I found that by feeding English-language text into the TTS plugin with the language code set to French produced bizarrely funny results, like a person with no knowledge of inflection attempting to read Le Monde. 

Conversely, if I switched my emulator’s base language to French, and then fed the plugin text written in English, it sounded like an unfortunate French speaker strangling on English words. The best results, of course, occurred when you switch your phone’s default language to the target language, and then feed the plugin a localized string. Thus my app, when speaking French using a French text, sounded quite marvelous, as if Paul Bocuse had discovered how to make pizza bagels and explained it all to me on my emulator. Lovely!



Switch the device's base language in the emulator's settings pane

As the plugin documentation notes, use this language switching capability with care - but if you know that your userbase’s phones will all be switched to a particular idiom, by all means use it! Doubling the text-to-speech plugin with an i18n plugin that detects the user’s phone language of choice would be amazing: https://github.com/rborn/nativescript-i18n.

Here’s the final result of a QuickNoms recipe tool list, spoken aloud using this plugin: