Pepper is not just a tablet

Pepper is a robot, not a tablet-holder. Pepper and Pepper’s tablet work as a team. Pepper controls the tablet; the tablet does not control Pepper. A robot interface is a new, intuitive way to communicate. It offers one-to-one interaction with the option to use the tablet. Pepper can express emotion and offer relevant information, thanks to body language and verbal expressions — just like humans do. Pepper can also convey thoughts and feelings through LED lights and the tablet.

A. Pepper’s tablet is secondary

https://developers.softbankrobotics.com/sites/default/files/repository/58_rst_pepper/public/source/_build/html/_images/chap_04a.png

Speech should be the primary driver of interaction design; design the tablet content to support the interaction new

WHY?

The most common mistake developers make in designing for Pepper is first designing the graphical tablet assets, and then adding speech and animations. Take full advantage of Pepper’s natural language abilities to create compelling experiences.

HOW?

Imagine that your experience is taking place between two humans, and consider how the tablet might supplement - or detract from - the flow of the interaction.

EXAMPLE

  • Don’t:

    Hello. Here is my menu. Please select a choice on my tablet.

  • Do:

    Hello, please come closer! My name is Pepper - how are you doing today?

Use verbal communication as Pepper’s main method of interacting with users and use the tablet only as a support too

WHY?

Focusing on the tablet as the primary form of communication diminishes Pepper’s ability to interact with people through speech and other human-like means. Solely communicating with Pepper via the tablet lessens the magic and wonder of the experience.

HOW?

The primary goal is to make users forget about the tablet, which should only be used as a backup assistant. The focus should always be on talking to Pepper.

Tip: Don’t hesitate to leave the tablet screen without content most of the time. That way, when something is shown, it catches the eye, but doesn’t prompt the user to tap it for information.

EXAMPLE

Use the tablet for information that is difficult to communicate verbally, such as a long list of choices.

B. Pepper’s tablet is essential in some cases

https://developers.softbankrobotics.com/sites/default/files/repository/58_rst_pepper/public/source/_build/html/_images/chap_04b.png

In the B2B area, the tablet is a key support tool, helping customers understand Pepper’s role inthe store

WHY?

The tablet display is one of the first places users will see information. The tablet improves the user experience by showcasing content that supports Pepper’s dialogue.

HOW?

Define a default tablet layout representative of Pepper with a content-light home screen or an illustration about its content.

EXAMPLE

To show that Pepper is associated with the store, it’s possible to make the image on Pepper’s tablet look like a staff member’s uniform, complete with a Pepper nametag.

The tablet is a key support tool in helping users understand what Pepper is explaining or waiting for

WHY?

The tablet is helpful during interactions because it allows Pepper to display or highlight information, signs, and conversation indicators, in addition to the verbal communication taking place.

HOW?

When Pepper is talking, the tablet can be used to display related content or to highlight some information. The tablet is also powerful to display some non-sayable information like a map or a long list of information.

When Pepper is listening and waiting for a user answer, Pepper’s tablet can show what are the possible options thanks to buttons or explicit vocal commands. In a noisy environment, the tablet is also a very useful fallback when the verbal communication is not easy.

EXAMPLE

For example, in a shop you can use the tablet to display the clothes Pepper is talking about, then some action buttons about the possible options the users can do about them, in order to let the users know what they can say to answer.

Pepper must be able to respond to every user action through Pepper’s voice and tablet

WHY?

A verbal and tablet response should be activated for users at every step of communication so they have the choice to connect with Pepper in whichever way they prefer. Every action should be able to be initiated via voice or touch.

HOW?

Implement a trigger word and a button for every user input. Make sure every button that is labelled and displayed on the tablet has words/sentences that Pepper will actually be listening for. Of course, there are instances, such as typing names and emails, that can only be done on the tablet’s keyboard.

Tip: When you test your application, you should be able to do everything by voice. Then retry it using only the tablet to see if you missed a user input.

EXAMPLE

When you have a “validation” button on the tablet, Pepper also has to be able to understand the voice trigger, “Validation.”

In the B2B area, allow users to exit an application from the tablet

WHY?

In B2C use cases, users can exit an application thanks to the exit gesture on Pepper’s head. But in a B2B setting, users don’t know Pepper or the controls that allow for exiting an application. That’s why an exit feature must be displayed on the tablet for B2B applications only.

HOW?

Add an exit cross icon on the tablet layout. For most applications, the top right corner is an ideal location.

C. Pepper is aware of the tablet

https://developers.softbankrobotics.com/sites/default/files/repository/58_rst_pepper/public/source/_build/html/_images/chap_04c.png

Pepper has to know when the tablet is being used

WHY?

Pepper has to know when the tablet is being used in the same way Pepper is aware when users touch Pepper’s head or a tactile part of Pepper’s body.

HOW?

Take tablet usage into account in every application

EXAMPLE

After a few seconds of the tablet not being used or managed, Pepper could ask, “Are you still here?”.

Guide the user when the focus switches between Pepper’s speech and tablet content new

WHY?

If the user’s attention needs to shift from Pepper to the tablet, Pepper should make that shift obvious. If Pepper stays tracking straight at the users, they may try speaking at moments when interacting by voice is not possible.

HOW?

When Pepper is not able to communicate by voice, Pepper must signal that state by looking away from the user and toward the tablet.

EXAMPLE

When Pepper is asking the users to fill out a form on the tablet, Pepper should:

1 - Verbally invite the user to use the tablet.

2 - Indicate the tablet with the arms and hands.

3 - Look down at the tablet until the user has validated the data on the screen.

During this scene, the robot must not show any listening signs on the LEDs to not encourage the verbal communication during this step.

Pepper must stop moving when the tablet is in use

WHY?

When people have to use the tablet to enter information such as email or phone number, Pepper must stop moving because Pepper may hit the user or make the task of typing difficult.

HOW?

Do not launch specific animations during tablet usage, minimize the hip and torso movements, and freeze Pepper’s arms.

EXAMPLE

When Pepper has to speak while the tablet is being used, deactivate the “Speaking Movement” autonomous ability.

Tip: use EnforceTabletReachability.

D. Pepper and the tablet work as a team

https://developers.softbankrobotics.com/sites/default/files/repository/58_rst_pepper/public/source/_build/html/_images/chap_04d.png

Pepper introduces, assumes, and welcomes any tablet content

WHY?

To show users that they should focus on the tablet, Pepper must present the tablet (through gestures) and look down at it.

HOW?

When Pepper displays something on the tablet and requests user input, Pepper should present the tablet with at least one arm and look down.

EXAMPLE

Use one of the three “Show Tablet” animations from the animation library. You can also create a new animation that’s relevant to what you want to showcase.

Guide the user when the focus switches between Pepper’s speech and tablet content new

WHY?

If the user’s attention needs to shift from Pepper to the tablet, Pepper should make that shift obvious. If Pepper stays tracking straight at the users, they may try speaking at moments when interacting by voice is not possible.

HOW?

When Pepper is not able to communicate by voice, Pepper must signal that state by looking away from the user and toward the tablet.

EXAMPLE

When Pepper is asking the users to fill out a form on the tablet, Pepper should:

1 - Verbally invite the user to use the tablet.

2 - Indicate the tablet with the arms and hands.

3 - Look down at the tablet until the user has validated the data on the screen.

During this scene, the robot must not show any listening signs on the LEDs to not encourage the verbal communication during this step.

Tip: use EnforceTabletReachability.

Pepper and the tablet are in sync: Pepper prioritizes speech but respects the tablet as an alternate mode of interaction new

WHY?

Pepper always tries to interact by voice first, but for the tablet to be useful as a fallback, every action should be triggerable both by voice and by tablet touch. The tablet can make interacting with Pepper more accessible to users with different abilities. Pepper’s tablet can also serve as a shortcut when the users want to skip some content.

HOW?

When a user taps a tablet button during Pepper’s speech, the tap should interrupt Pepper’s speech and allow the user to move on through the flow of interaction.

EXAMPLE

In a menu, while Pepper is verbally listing the different categories, a tap on one of these options should interrupt the menu instructions, open the tapped category, and start narrating the category content.

Use the same wording for what Pepper says verbally and what Pepper displays on the tablet

WHY?

While Pepper controls the tablet, both what Pepper says and what is written on the tablet need to be in sync.

HOW?

Ensure the wording and verbal expressions are the same on the tablet as they are coming from Pepper.

EXAMPLE

In a menu, Pepper has to pronounce the same category’s names displayed on the tablet.

Display items on the tablet in the same order that Pepper verbally lists them

WHY?

It’s easier for the user to recognize items on the tablet if they are listed in the same order as Pepper says them.

HOW?

Keep the same order on the tablet that’s spoken verbally by Pepper.

EXAMPLE

When Pepper lists two choices, like “Blue or Red,” the tablet has to display icons in the same order that they were said.

All content displayed on the tablet must match what Pepper is saying

WHY?

Keep consistency between the topic of conversation and Pepper’s tablet content. The tablet is a support tool for Pepper’s speech.

HOW?

The tablet must be in sync with what Pepper is saying. So, the tablet must never display content that is separate or out of sync with Pepper’s words.

EXAMPLE

For example, do not display anything on the tablet that is separate to what Pepper is talking about.

Take-Aways

4A - Pepper’s tablet is secondary

  • new Speech should be the primary driver of interaction design; design the tablet content to support the interaction.
  • Use verbal communication as Pepper’s main method of interacting with users and use the tablet only as a support tool.

4B - Pepper’s tablet is essential for Pepper in some instances

  • In the B2B area, the tablet is a key support tool in helping users understand Pepper’s role in the store.
  • The tablet is a key support tool in helping users understand what Pepper is explaining or waiting for.
  • Pepper must be able to respond to every user action through its voice and tablet.
  • In the B2B area, allow users to exit an application from the tablet.

4C - Pepper is aware of the tablet

  • Pepper has to know when the tablet is being used.
  • new Guide the user when the focus switches between Pepper’s speech and tablet content.
  • Pepper must stop moving when the tablet is in use.

4D - Pepper and the tablet work as a team

  • Pepper introduces, assumes, and welcomes any tablet content.
  • new Guide the user when the focus switches between Pepper’s speech and tablet content.
  • new Pepper and the tablet are in sync: Pepper prioritizes speech but respects the tablet as an alternate mode of interaction.
  • Use the same wording for what Pepper says verbally and what Pepper displays on the tablet.
  • Display items on the tablet in the same order that Pepper verbally lists them.
  • All content displayed on the tablet must match what Pepper is saying.