Making a multimodal presentation

Part 1: Speech and images

1 Making Pepper speak

Make a small helper function for creating a Say action, and call it from runPresentation:

////////////////////////////
//        helpers         //
////////////////////////////

private fun makeSay(text : String) : Say {
   return SayBuilder.with(qiContext)
           .withText(text)
           .build()
}

////////////////////////
// Presentation logic //
////////////////////////


private fun runPresentation() {

   // Part 1: "Making me talk..."

   makeSay("Okay, so ... making me talk is a first step a bit like ...").run()
   Thread.sleep(200)
   makeSay("a rolling rock ...").run()

}

The content of runPresentation should stay very simple and readable, to keep the logic easy to understand.

Some notes:

  • Here the string is directly hard-coded in the code; normally you would want to keep that in a dedicated strings.xml file.
  • We use Thread.sleep to control the timing a bit more finely

2 Adding an image

Now, let’s put a fullscreen image on Pepper’s tablet. Edit main_activity.xml, so that instead of a simple hello world TextView, it contains a large, centered, ImageView, like this:

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
   xmlns:app="http://schemas.android.com/apk/res-auto"
   xmlns:tools="http://schemas.android.com/tools"
   android:layout_width="match_parent"
   android:layout_height="match_parent"
   tools:context="com.softbankrobotics.app.pepperpresentation.MainActivity">
   <ImageView
       android:id="@+id/splashImageView"
       android:layout_width="wrap_content"
       android:layout_height="wrap_content"
       android:layout_marginStart="8dp"
       android:layout_marginTop="8dp"
       app:layout_constraintBottom_toBottomOf="parent"
       app:layout_constraintEnd_toEndOf="parent"
       app:layout_constraintStart_toStartOf="parent"
       app:layout_constraintTop_toTopOf="parent"
       app:srcCompat="@drawable/scene1" />
</androidx.constraintlayout.widget.ConstraintLayout>

Now add a couple helper functions in MainActivity.kt to set this image:

private fun setImage(resource : Int) {
   runOnUiThread {
       splashImageView.setImageResource(resource)
       splashImageView.visibility = View.VISIBLE
   }
}

private fun clearImage() {
   runOnUiThread {
       splashImageView.visibility = View.GONE
   }
}

Why use helper functions? In this case, it ensures that the code in runPresentation stays short and readable, focused on what Pepper is presenting exactly, and not implementation details like the UI thread and visibility.

3 Putting it all together

You can now do the first part of the presentation, making Pepper speak, and show an image:

////////////////////////
// Presentation logic //
////////////////////////

private fun runPresentation() {

   // Part 1: "Making me talk..."

   setImage(R.drawable.scene1)
   makeSay("Okay, so ... making me talk is a first step a bit like ...").run()
   Thread.sleep(200)
   makeSay("a rolling rock ...").run()
   setImage(R.drawable.scene2)

   // Part 2: "Let me show you..."

   // to do next

}