Integrating a Chatbot: Dialogflow

Adding a QiChatbot for fallback

Now we have a Pepper whose speech is entirely controlled by Dialogflow. How about QiChat?

QiChat has a few advantages over web chatbots:

  • It works offline
  • It supports custom, unusual names (like brand or product names) that may not always be recognized by the Cloud ASR
  • It offers functions to like ^execute and bookmarks that allow more fine control and timing of the robot

Here, we’re mostly interested in the first advantage: we would like to have a fallback when Pepper is offline. This can be something as simple as Pepper saying sorry about the service not being available - which is definitely better than not saying anything.

So let’s create a QiChatbot.

Create a topic

Start by creating a new chat topic called "chat" ( File > New > Chat topic ) - keep it simple:

topic: ~chat()

concept:(tell_me) [
   "tell me"
   "tell"
   "I want"
]

u:(~tell_me a joke)
   Sorry, I don't seem to be able to access the joke database

u:(how do you know all that?)
   Thanks to an advanced Cloud AI Joke Knowledge Base
   # a.k.a a big look-up table

Note that this only handles two inputs:

  • “Tell me a joke” -> we only want to trigger this when Dialogflow didn’t answer
  • “How do you know all that?” -> It’s useful to have one simple trigger that can be used to quickly check whether this topic is indeed loaded.

Add this second chatbot in the Chat action

Modify your main activity’s onFocusGained to add the QiChatbot adapting the previous code with the following:

val credentials = applicationContext.resources.openRawResource(R.raw.credentials)
val dialogflowChatbot = DialogflowChatbot(qiContext, credentials)
val topic = TopicBuilder.with(qiContext).withResource(R.raw.chat).build()
val qiChatbot = QiChatbotBuilder.with(qiContext).withTopic(topic).build()
chat = ChatBuilder.with(qiContext).withChatbot(dialogflowChatbot, qiChatbot).build()
chat.async().run()

The order in which these chatbots are passed to the ChatBuilder is important: when both are capable of providing an answer (for example, to “tell me a joke?”), the first one in this list will answer. Since here we are using qiChatbot as a fallback for when Dialogflow doesn’t have an answer, this is the right order.

Trouble with fallbacks

However, if you test this, you will notice a problem: if you ask “how do you know all that?”, Pepper will not give the answer from our QiChat, but instead, a generic fallback from Dialogflow, like "Sorry, could you say that again?".

Our problem stems from the fact that whatever Dialogflow sends us, we treat it as a “Normal” priority answer.

Dialogflow does provide the information of whether an answer is just a fallback; if you inspect the responses sent by Dialogflow (which you can do using the online Dialogflow console, or using the Kotlin REPL as explained earlier), you will see that fallback responses can contain items like:

fulfillment_messages {
  text {
    text: "Sorry, could you say that again?"
  }
}
intent {
  name: "your-project-id"
  display_name: "Default Fallback Intent"
  is_fallback: true
}
// ... (more information)

So we want to get this is_fallback information, and give it to DialogflowChatbot.

Your current API doesn’t allow that: the Dialogflow-related logic is isolated in a function that just returns a string. Start by defining a new data class TextResponse in your "data" module:

package com.softbankrobotics.jokeswithdialogflow.data

data class TextResponse(
   val text: String,
   val isFallback: Boolean) {
}

Now update the DialogflowDataSource to return this object:

val response = sessionsClient.detectIntent(session, queryInput)
return TextResponse(
   response.queryResult.fulfillmentText,
   response.queryResult.intent.isFallback)

Now that the return type changed, you must update DialogflowDataSourceUnitTest:

val response = dataSource.detectIntentTexts("Are you waterproof?",
   sessionId, languageCode)
assertEquals("No, please keep my inside!", response.text)
assertEquals(false, response.isFallback)

And finally, use this information in DialogflowChatbot:

override fun replyTo(phrase: Phrase, locale: Locale): StandardReplyReaction {
   val input = phrase.text.toString()
   val language = locale.language.toString()
   var answer: TextResponse?= null
   try {
       answer = dataSource.detectIntentTexts(input, dialogflowSessionId, language)
       Log.i(TAG, "Got answer: '$answer'")
   } catch (e: Exception) {
       e.printStackTrace()
   }
   val reaction = if(answer != null) {
       SimpleSayReaction(qiContext, answer.text)
   } else {
       EmptyChatbotReaction(qiContext)
   }
   val priority = if (answer == null || answer.isFallback) {
       ReplyPriority.FALLBACK
   } else {
       ReplyPriority.NORMAL
   }
   return StandardReplyReaction(reaction, priority)
}

This chatbot is now able to create a "fallback" reply reaction when appropriate, in which case the answer from the QiChatbot may be used instead.

Run it

Now you can try running the app: Pepper should now be able to reply correctly even when offline.