Smart Queries: ask Pepper about monsters and get accurate replies

Simple Questions

Now it’s time to plug in dialogue.

Robotify the application (File > New > Robot Application...), register and unregister the application; also, remove the loading of monster data (we’ll put it back later)

class MainActivity : RobotActivity(), RobotLifecycleCallbacks {
  val TAG = "MainActivity"
  private lateinit var imageHolder : LinearLayout
   private val imageLayoutParams = LinearLayout.LayoutParams(LinearLayout.LayoutParams.MATCH_PARENT,
           LinearLayout.LayoutParams.MATCH_PARENT, 1.0f)
   private var chatFuture: Future<Void>? = null

   override fun onCreate(savedInstanceState: Bundle?) {
      imageHolder = findViewById(
      QiSDK.register(this, this)

   override fun onDestroy() {

   override fun onRobotFocusGained(qiContext: QiContext?) {
      Log.w(TAG, "Robot Focus Gained!")
      // TODO: fill in qichat

   override fun onRobotFocusLost() {
      Log.w(TAG, "Robot Focus Lost")

   override fun onRobotFocusRefused(reason: String?) {
      Log.w(TAG, "Robot Focus Refused because $reason")

   fun showMonsters(monstersToShow: List<Monster>) {
      runOnUiThread {
          Log.i(TAG, "Showing ${monstersToShow.size} monsters")
          monstersToShow.forEach { monster ->
              val imageView = ImageView(applicationContext).apply {
                  id = View.generateViewId()
                  layoutParams = imageLayoutParams


Create a topic (File > New > Chat Topic) called “monsters” with a simple query; we’ll make a more complex one later:

topic: ~monsters()
concept:(color) [red blue yellow green orange purple pink brown black grey white]

u:(Show me _~color monsters)
Let me see.
^execute(showMonsters, $color)
Here they are!

To handle this topic, we will create a dedicated class, MonsterDialogue; it will contain:

  • The code to load the dialogue topic
  • The executors

Its only interaction with mainActivity will be through a showMonsterList callback, passed at construction.

Create the MonsterDialogue.kt file and add this:

class MonsterDialogue(qiContext: QiContext,
                     val allMonsters: List<Monster>,
                     val showMonsterList: (List<Monster>) -> Unit) {
   val TAG = "MonsterDialogue"

   private val topic = TopicBuilder.with(qiContext).withResource(R.raw.monsters).build()
   private val qiChatbot = QiChatbotBuilder.with(qiContext).withTopic(topic).build().apply {
       executors = hashMapOf<String, QiChatExecutor>(
               "showMonsters" to ShowMonstersExecutor(qiContext),
               "clearScreen" to ClearScreenExecutor(qiContext),
   private val chat = ChatBuilder.with(qiContext).withChatbot(qiChatbot).build() 
   fun run(): Future<Void> {
       return chat.async().run()

   // Executors

   inner class ClearScreenExecutor(context: QiContext?) : BaseQiChatExecutor(context) {
      override fun runWith(params: MutableList<String>) {
      override fun stop() {}

   inner class ShowMonstersExecutor(context: QiContext?) : BaseQiChatExecutor(context) {
      override fun runWith(params: MutableList<String>) {
          Log.i(TAG, "Showing by color; $params")
          val color = params[0]
          showMonsterList(allMonsters.filter { it.color == color })

      override fun stop() {}


… and then update mainActivity so that:

  • This class is instantiated on onRobotFocusGained
  • The list of monster data is created in onRobotFocusGained
override fun onRobotFocusGained(qiContext: QiContext) {
   Log.w(TAG, "Robot Focus Gained!")
   val monsterData = resources.openRawResource(R.raw.monsters_data)
   val allMonsters = readMonstersJSON(monsterData)
   allMonsters.forEach {
      it.resourceId = resources.getIdentifier(it.filename, "drawable", packageName)
   val monsterDialogue = MonsterDialogue(qiContext, allMonsters, ::showMonsters)

Now is a good time to test this integration on Pepper, and check that Pepper can show monsters of the color you asked him.