Linking QiChat and Code

Asking smart questions: Bookmarks, Variables and Dynamic Concepts

QiChat allows you to easily create simple chatbots. But if we want to improve the conversation with Pepper’s other abilities (animation, navigation, perception...), we will need to link it to the application’s code.

Let’s start by looking at how to control the conversation and its content with the application’s code:

  1. Using Bookmarks to make Pepper proactively ask a specific question
  2. Adding Variables to diversify the speech content
  3. Varying which answers Pepper handles with dynamic concepts

1. Start a project

First, create a new QiChat-based application as seen in Getting Started (create a project, robotify it, register to the QiSDK, etc.).

Your MainActivity class should look end up like this:

const val TAG = "MainActivityMaths"

class MainActivity : RobotActivity(), RobotLifecycleCallbacks {
   override fun onRobotFocusGained(qiContext: QiContext) {
       // This is where you will put your code
   }

   override fun onRobotFocusLost() {
       Log.i(TAG, "Focus lost")
   }

   override fun onRobotFocusRefused(reason: String?) {
       Log.i(TAG, "Focus refused because $reason")
   }

   override fun onCreate(savedInstanceState: Bundle?) {
       super.onCreate(savedInstanceState)
       setContentView(R.layout.activity_main)
       QiSDK.register(this, this)
   }

   override fun onDestroy() {
       super.onDestroy()
       QiSDK.unregister(this, this)
   }
}

Note: this will not compile right away, you will usually need a gradle sync and then use quick-fix (Alt+Enter, or Option+Enter on Mac) on the names in red (RobotActivity, QiContext etc.), to automatically import the right libraries.

Now create a new chat topic called “maths”, with this content:

topic: ~maths()

u:(who are you?)
   I'm Pepper the Maths teacher!

As we advance in this lesson, we will be adding more user rules to this topic file.

Then activate this topic in onRobotFocusGained, by creating three objects (a Topic, a QiChatbot, and a Chat action, see Discovering Qichat and store them as member variables - you will need them later on.

class MainActivity : RobotActivity(), RobotLifecycleCallbacks {
   lateinit var topic : Topic
   lateinit var qiChatbot : QiChatbot
   lateinit var chat : Chat

   override fun onRobotFocusGained(qiContext: QiContext) {
       topic = TopicBuilder.with(qiContext).withResource(R.raw.maths).build()
       qiChatbot = QiChatbotBuilder.with(qiContext).withTopic(topic).build()
       chat = ChatBuilder.with(qiContext).withChatbot(qiChatbot).build()
       // Start the dialogue
       chat.run()
   }

   // ...here is the rest of the code
}

2. Make Pepper spontaneously ask a question

All the dialogue in the Discovering QiChat lesson has been reactive, i.e. Pepper only speaks to answer what users said. Now it’s time to introduce proactive dialogue: when Pepper speaks first.

To do that, we’ll use Proposals and Bookmarks (that you may have already used before), to make Pepper ask a simple question.

proposal: %MULTIPLY_CHALLENGE
Okay, what's 7 times 8?
  u1:(56)
    right!
  u1:(*)
    wrong!

But this time you will trigger them from your code. For that, let’s first create a helper method in MainActivity:

private fun goToBookmark(bookmarkName : String) {
   qiChatbot.goToBookmark (
       topic.bookmarks[bookmarkName],
       AutonomousReactionImportance.HIGH,
       AutonomousReactionValidity.IMMEDIATE)
}

Note that in addition to the bookmark, we have to pass two more parameters to qiChatbot.goToBookmark: the Importance (high or low) and the Validity (immediate or delayable). These help determine what happens if the robot is already speaking:

  • if the Importance is high, the bookmark will be triggered right away (it’s what we want here and what we will want most of the time);
  • If the priority is low and the Validity delayable, the bookmark will be triggered once the robot is done speaking (typical use case: a notification like “Your order is ready”, that can wait a few seconds);
  • If the priority is low and the Validity immediate, the bookmark will not be triggered at all if the robot is already speaking (typical use case: Pepper saying “stop tickling me!” if touched).

Let’s call this method as soon as our Chat action is started:

override fun onRobotFocusGained(qiContext: QiContext) {
   // ...
   chat.addOnStartedListener {  goToBookmark("MULTIPLY_CHALLENGE") }
   // Start the dialogue
   chat.run()
}

When using QiChat, goToBookmark is the standard way of making Pepper proactively say something (a “Say” action will fail anytime a Chat action is already running), usually you don’t need Pepper to say something as soon as the application launches (as done in this simple example), but rather:

  • When Pepper sees someone (and isn’t already engaged in conversation);
  • When someone presses a button on the tablet;
  • To notify a special event (e.g. a reminder).

3. Setting variables from code

Now we have a quiz, but not a very interesting one - it only has a single question. Let’s randomize it a bit.

To do that, let’s replace all our numbers with variables:

proposal: %MULTIPLY_CHALLENGE
Okay, what's, $numberA times, $numberB ?
   u1:($result)
   Right!
   u1:(*)
   Wrong!

Let’s add a function in MainActivity to randomize these variables.

private fun randomize() {
   val numberA = Random.nextInt(3, 9)
   val numberB = Random.nextInt(3, 9)
   val multiplicationResult = numberA * numberB

   qiChatbot.variable("numberA").value = "$numberA"
   qiChatbot.variable("numberB").value = "$numberB"
   qiChatbot.variable("result").value = "$multiplicationResult"
}

Then call it when the app starts:

override fun onRobotFocusGained(qiContext: QiContext) {
   // ...
   randomize()
   // Start the dialogue
   chat.run()
}

Note that we create an object for each variable we want to use, and if there is no variable of that name in your QiChat, the creation of this object will fail (and raise an exception).

You can run the application - Pepper will ask a different question every time we launch the app (if it doesn’t understand, but does understand the earlier “who are you”, you may want to make sure Cloud ASR works on your robot).

4. Dynamic concepts

One issue here is that we don’t distinguish wrong answers (like “58”) from off-topic answers (like “uh wait a minute”).

So, let’s introduce a Dynamic Concept called “wrongNumbers” containing only the wrong answers. It can be used in QiChat like other concepts ( concept:(concept_name)).

dynamic: wrongNumbers

proposal: %MULTIPLY_CHALLENGE
Okay, what's, $numberA times, $numberB ?
   u1:($result)
   Right!
   u1:(~wrongNumbers)
   Wrong!

This dynamic concept can then be accessed as code; add a new member variable to your activity:

lateinit var wrongNumbersSet : EditablePhraseSet

Then, assign the dynamic concept object to it.

override fun onRobotFocusGained(qiContext: QiContext) {
   // ...
   wrongNumbersSet = qiChatbot.dynamicConcept("wrongNumbers")
   randomize()
   // Start the dialogue
   chat.run()
}

Now set this dynamic concept in the randomize function:

private fun randomize() {
   val numberA = Random.nextInt(3, 9)
   val numberB = Random.nextInt(3, 9)
   val multiplicationResult = numberA * numberB

   qiChatbot.variable("numberA").value = "$numberA"
   qiChatbot.variable("numberB").value = "$numberB"
   qiChatbot.variable("result").value = "$multiplicationResult"
   val numberPhrases = ArrayList<Phrase>()
   for (i in 0..100) {
       if (i != multiplicationResult) {
           numberPhrases.add(Phrase("$i"))
       }
   }
   wrongNumbersSet.removePhrases(wrongNumbersSet.phrases)
   wrongNumbersSet.addPhrases(numberPhrases)
}

This should make the interaction less frustrating, as only wrong numbers will be interpreted as wrong answers (not the hesitations nor the off-topic remarks).