“Ok Google, use my Flutter app!”

“Ok Google, use my Flutter app!”

In this post I would like to show you how I integrated Google Assistant/Google Now with my WeightTracker app written in Flutter (works only on Android).

First, let’s see what we are trying to accomplish:


(I had to type in the message because recording tool was capturing mic input, so Google Assistant couldn’t hear what I was saying)

How to achieve this? Let’s walk through it!

1. Let your app catch the intent.

Google Assistant actions are based on Voice Actions. If we want to integrate your app with it we need to pick one from the list (you can find list of Voice Actions here). After we choose an action (for me it was Take a note) we can add a proper intent-filter to our AndroidManifest.xml:

<activity android:name=".MainActivity"
          android:launchMode="singleTop"
          android:theme="@android:style/Theme.Black.NoTitleBar"
          android:configChanges="orientation|keyboardHidden|keyboard|screenSize|locale|layoutDirection"
          android:hardwareAccelerated="true"
          android:windowSoftInputMode="adjustResize">
    <intent-filter>
        <action android:name="android.intent.action.MAIN"/>
        <category android:name="android.intent.category.LAUNCHER"/>
    </intent-filter>
    <intent-filter>
        <action android:name="com.google.android.gms.actions.CREATE_NOTE"/>
        <category android:name="android.intent.category.DEFAULT"/>
        <data android:mimeType="text/plain"/>
    </intent-filter>
</activity>

2. Handle the intent.

When our app is able to catch the intent, we need to handle it in MainActivity. For my Note intent it looks like this:

public class MainActivity extends FlutterActivity {
    private String savedNote;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        GeneratedPluginRegistrant.registerWith(this);
        Intent intent = getIntent();
        String action = intent.getAction();
        String type = intent.getType();

        if (NoteIntents.ACTION_CREATE_NOTE.equals(action) && type != null) {
            if ("text/plain".equals(type)) {
                savedNote = intent.getStringExtra(Intent.EXTRA_TEXT)
            }
        }
    }
}

What happens here is if we get a ACTION_CREATE_NOTE intent and it is a text, we simply extract the value and store it in private field.

3. Expose note to Dart

Since we cannot explicitly tell our Flutter app that we have a note, we can only return it when we are asked for it. To do that all we have to do is add new MethodChannel in MainActivity:

@Override
protected void onCreate(Bundle savedInstanceState) {
  ...
  new MethodChannel(getFlutterView(), "app.channel.shared.data")
        .setMethodCallHandler(new MethodChannel.MethodCallHandler() {
            @Override
            public void onMethodCall(MethodCall methodCall, MethodChannel.Result result) {
                if (methodCall.method.contentEquals("getSavedNote")) {
                    result.success(savedNote);
                    savedNote = null;
                }
            }
        });
}

When MethodChannel “getSavedNote” is being invoked, we return savedNote and then set it to null.

4. Ask for data

You can do it in many ways, I stayed with approach proposed on Flutter page.

Inside onInit function, I simply invoke MethodChannel method for saved note:

Future<double> _getSavedWeightNote() async {
  String sharedData = await const MethodChannel('app.channel.shared.data')
      .invokeMethod("getSavedNote");
  if (sharedData != null) {
    int firstIndex = sharedData.indexOf(new RegExp("[0-9]"));
    int lastIndex = sharedData.lastIndexOf(new RegExp("[0-9]"));
    if (firstIndex != -1) {
      String number = sharedData.substring(firstIndex, lastIndex + 1);
      double num = double.parse(number, (error) => null);
      return num;
    }
  }
  return null;
}

If there is any note saved I try to find a number there (I know it’s not perfect) and then return it. According to our app architecture we can store that value in database, use it in setState or do whatever we want.

When it comes to coding we’re all done. However saying “Ok google, save a note my weight is 80” or just “Ok google, note to self 80” may not work. We have to choose our app as a default one to use it. To do that with Google Assistant you need to say “Save a note”  and then press default app’s icon and choose WeightTracker (present on the right).

And that’s it! Enjoy!

Interesting facts:

  • You can say “save a note blabla in Evernote” and it will pick Evernote. However if you say “save a note blabla in MyApp” it won’t work. Don’t ask me why.
  • If you call fetching function in onInit function, it won’t get invoked if you say “ok google…” while your app is in the foreground.
  • There is no option to create your own voice actions 🙁
  • If you download WeightTracker from Google Play and set it as default noting app, you should be able to use it with Google Assistant/Google Now. If not, please tell me!

If you would like to do it by your own, here are important  materials: Voice Actions, Handling Intents.

If you are interested in full code, here are links to repository, MainActivity class, onInit method and note handling method. (I am using Redux architecture)

If you have any questions, feel free to ask! 🙂
Cheers 🙂

6 thoughts on ““Ok Google, use my Flutter app!”

  1. Hej,

    czy jestes zainteresowany internship/ junior w UK jako software engineer. Jestem leadem Android Team w The App Business. Jednoczesnie jestem autorem Flutter publication na Medium.

  2. Hi, thank you so much for sharing.
    About the caveats you mentioned:

    -“If you call fetching function in onInit function” –
    well, if you have Java calling Flutter instead of the reverse, this will be resolved. I understand it’s possible. Was this an approach you tried? Did you find any issues with it?

    -“There is no option to create your own voice actions” – is this a general restriction, or just using your method? I know Android offers custom actions, where’s the issue implementing these using Flutter?

    -Do you see any issues trying to implement a 2-way conversation with the user? In your example you only allow a 1-way call from the user, no questions back…

      1. OK, we’ll learn together 🙂
        1) Platform Channels seem to support the Java-to-Dart call direction.
        See https://flutter.io/platform-channels/ , search for “can also be sent in the reverse direction”.
        2&3) Starting API23, Android supports custom actions + 2-way conversations,
        see https://youtu.be/OW1A4XFRuyc?t=2m22s
        and https://developers.google.com/voice-actions/interaction/

        This is just the info, I haven’t tried anything yet 🙂 Have fun, and please, do share.

Leave a Reply

Your email address will not be published.