Integrating a Chatbot: Dialogflow

Testing your Dialogflow agent in standalone

Now we want to integrate Dialogflow in our Android application.

In order to do that, we will create a project with two modules:

  • “app”, an Android Application that uses the QiSDK
  • “data”, a java library that will contain all our calls to Dialogflow

This enforces Separation of concerns - the app module doesn’t need to know anything about Dialogflow, and the “data” module doesn’t need to know anything about Android.

And this will enable us to try the Dialogflow calls in isolation, without needing to install anything on Pepper’s tablet.

Create the project

Create a new android studio project (with Kotlin, API 23).

screenshot of "create project"

Robotify it, as usual (File > New > Robot Application).

Create a new module (File > New > Module) of type “Java Library”, called “data”; with a class called DialogflowDataSource (also make sure to update the package name):

screenshot of data module creation

Add the necessary libraries to this "data" module’s build.gradle’s dependencies:

implementation 'com.google.cloud:google-cloud-dialogflow:0.92.0-alpha'
implementation 'com.google.http-client:google-http-client:1.29.1'
implementation 'junit:junit:4.12'

Note: here we use the v2 API; which is the latest (as of May 2019). Using the earlier APIs is not recommended (and would require a few changes elsewhere in the code). See “Migrating to Dialogflow Enterprise Edition”.

These were the latest versions at the time of writing, but you can check for newer versions here: google-cloud-dialogflow and google-http-client.

Add the Dialogflow code

Now convert your newly generated DialogflowDataSource class to Kotlin (with right click on the file), and put this code in it:

import com.google.api.gax.core.FixedCredentialsProvider
import com.google.auth.oauth2.ServiceAccountCredentials
import com.google.cloud.dialogflow.v2.*
import java.io.InputStream

class DialogflowDataSource constructor(credentialsStream : InputStream) {
   private val credentials : ServiceAccountCredentials
       = ServiceAccountCredentials.fromStream(credentialsStream)

   fun detectIntentTexts(
       text: String,
       sessionId: String,
       languageCode: String
   ): String? {
       val sessionsSettings = SessionsSettings.newBuilder()
           .setCredentialsProvider(FixedCredentialsProvider.create(credentials))
           .build()
       SessionsClient.create(sessionsSettings).use { sessionsClient ->
           val session = SessionName.of(credentials.projectId, sessionId)
           val textInput = TextInput.newBuilder()
               .setText(text).setLanguageCode(languageCode)
           val queryInput = QueryInput
               .newBuilder().setText(textInput).build()
           val response = sessionsClient.detectIntent(session, queryInput)
           return response.queryResult.fulfillmentText
       }
   }
}

Here we’ve wrapped the Dialogflow library into a simple method that takes a string as input, and returns an input.

We also need to pass:

  • A session ID, to represent a given conversation session (this is not important now, because our dialog doesn’t require memory)
  • A language code (“en-US” in our case)

Set up authentication

Now add a "raw" directory to your app module's resources, and put the “credentials.json” file you downloaded earlier in it:

where credentials is

If you are working under source control, you usually want to exclude this from your source control.

Try it in the Kotlin REPL

Note: if you are not very comfortable with the Kotlin REPL you can skip this part; it can be useful for trying things out but is not very user-friendly.

Before you start integrating this further into the application, check that it works!

Open the Kotlin REPL (Read-Evaluate-Print Loop): go in Tool > Kotlin > REPL, and try calling this class directly (you will need to pass the full path to the credentials.json file):

import com.softbankrobotics.jokeswithdialogflow.data.DialogflowDataSource
import java.io.File
val stream = File("(absolute project path path)/app/src/main/res/raw/credentials.json").inputStream()
val dataSource = DialogflowDataSource(stream)
dataSource.detectIntentTexts("Are you waterproof?", "my-test-session", "en-US")

Hit alt+Enter to execute it, and if all works fine, you should get something like this:

res2: kotlin.String? = No, please keep me inside!

(if not, you might be offered a “build module data and restart”; try that)

You can then use this REPL to test other commands, like asking for a joke.

It is better to try out your code while it’s still very simple, so that if something doesn’t work it’s easier to troubleshoot (it could be a problem with the credentials, or with your Service Account’s rights)

Create a simple unit test

Now let’s turn this casual test into a formal unit test. In your “data” module, add a directory for tests (right click on your “data” module -> new directory -> “src/test/java”), and inside it, create a file “com/softbankrobotics/data/DialogflowDataSourceUnitTest.kt"

In this file, put:

package com.softbankrobotics.jokeswithdialogflow.data

import org.junit.Test
import org.junit.Assert.*
import java.io.File

class DialogflowDataSourceUnitTest {
   @Test
   fun simpleIntent_hasAnswer() {
       val stream = File("../app/src/main/res/raw/credentials.json").inputStream()
       val dataSource = DialogflowDataSource(stream)
       val sessionId = "my-unittest-session"
       val languageCode = "en-US"
       val result = dataSource.detectIntentTexts("Are you waterproof?",
           sessionId, languageCode)
       assertEquals("No, please keep my inside!", result)
   }
}

Your “Android” view should look like this:

screenhots of module structure

(If it doesn’t, double-check the paths; note that the directory is called “test” even if the Android view shows “tests”)

Right-click on the file and run it, it should pass:

screenhots of successful tests

Having tests like this will help you quickly make sure that everything is still working fine after you change something.

Create an android unit test

Now let’s make a similar test, but now running in an android environment. Things work a bit differently in android.

This time we will be working in our “app” module; add these to it's build.gradle file:

android {
  // ... (other Android parameters)
  packagingOptions {
       exclude 'META-INF/INDEX.LIST' 
       exclude 'META-INF/DEPENDENCIES'

   }
}

dependencies {
    // ... (all of the other dependencies)
    implementation project(':data')
    implementation 'io.grpc:grpc-okhttp:1.19.0'
}

Create a DialogflowOnAndroidTest kotlin class in your app’s “androidTest” directory (there should already be an ExampleInstrumentedTest there).

import android.support.test.InstrumentationRegistry
import android.support.test.runner.AndroidJUnit4
import com.softbankrobotics.jokeswithdialogflow.data.DialogflowDataSource
import org.junit.Assert
import org.junit.Test
import org.junit.runner.RunWith

@RunWith(AndroidJUnit4::class)
class DialogflowOnAndroidTest {
   @Test
   fun simpleIntent_hasAnswer() {
       val appContext = InstrumentationRegistry.getTargetContext()
       val credentials = appContext.resources.openRawResource(R.raw.credentials)
       val dataSource = DialogflowDataSource(credentials)
       val sessionId = "my-andoid-unittest-session"
       val languageCode = "en-US"
       val result = dataSource.detectIntentTexts("Are you waterproof?",sessionId, languageCode)
       Assert.assertEquals("No, please keep my inside!", result)
   }
}

Note that the logic is the same as previously, but now we’re getting the file as an android resource. Internally, the call to Dialogflow servers will also work differently.

If you run this tests, some errors you might encounter:

  • "More than one file was found with OS independent path" (depending of your exact version of dialogflow); in which case you need to add those to the gradle's `packagingOptions.
  • The internet permission is required - make sure you robotified the application (see the beginning of this step)
  • (There may be some specific errors depending on the exact version of gradle and of the Dialogflow library)

Once the tests pass, you should get something like this:

screenhots of successful tests

You now have two kinds of tests to run:

  • A pure Kotlin unit test, that you can run very quickly on your computer
  • An android unit test, that takes a bit longer to run (and requires starting a virtual device or connecting to a real one), but is closer to the true execution context