Mastering Focus & Robot lifecycle

Understanding the focus

The robot focus is a required state for an Activity to run actions on the robot.

It is guaranteed to be owned by only one Activity at a time: this Activity is said to be the robot focus owner. This ensures that the robot focus owner is the only one to control the robot.

The robot focus is managed by a service which has the responsibility to give this focus to activities. This mechanism implies that an Activity can gain or lose the robot focus at any time.


The robot lifecycle

The QiSDK provides a robot lifecycle that allows any Activity to own the robot focus.

https://developer.softbankrobotics.com/sites/default/files/repository/60_rst_pepper/public/source/_build/html/_images/robot_life_cycle.png

An object wishing to use the QiSDK must implement the RobotLifecycleCallbacks interface. For example, the Activity itself can implement it:

public class MyActivity extends RobotActivity implements RobotLifecycleCallbacks

Also, this object must be registered to the Activity to receive the calls to the RobotLifecycleCallbacks methods.

It may register in the onCreate method:

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    // Register the RobotLifecycleCallbacks to this Activity.
    QiSDK.register(this, this);
}

And unregister in the onDestroy method:

@Override
protected void onDestroy() {
    // Unregister the RobotLifecycleCallbacks for this Activity.
    QiSDK.unregister(this, this);
    super.onDestroy();
}

Important

For simplicity sake, these tutorials recommend registering and unregistering in onCreate and onDestroy of an Activity object.

More complex scenarios may benefit from or require other strategies.

The developer has fine-grained control of when they wish to actually receive the callbacks, by registering and unregistering when needed.

And it is not necessary to implement these callbacks into the Android Activity object, they can be in any object.

Gaining the robot focus

When the Activity gains the Android focus (moves to foreground), it asks for the robot focus.

If the Activity gains the robot focus, the onRobotFocusGained callback is called for each RobotLifecycleCallbacks registered to this Activity:

void onRobotFocusGained(QiContext qiContext);

Warning

This callback runs on a background thread, so you cannot manipulate UI components directly in it. See Get back on UI thread to run code on the UI thread.

It provides a QiContext that allows you to:

  • create actions,
  • create resources for actions,
  • retrieve robot services.

Create an action:

// Create a Say action with a QiContext.
Say say = SayBuilder.with(qiContext)
                    .withText("Hello")
                    .build();

Create a resource:

// Create an Animation with a QiContext.
Animation animation = AnimationBuilder.with(qiContext)
                                      .withResources(R.raw.elephant_a001)
                                      .build();

Get a service:

// Get the HumanAwareness service with a QiContext.
HumanAwareness humanAwareness = qiContext.getHumanAwareness();

In this callback, you can typically run your actions:

Run an action:

// Run an action synchronously.
say.run();

// Run an action asynchronously.
say.async().run();

Losing the robot focus

When the Activity loses the robot focus, the onRobotFocusLost callback is called for each RobotLifecycleCallbacks registered to this Activity:

void onRobotFocusLost();

Warning

This callback runs on a background thread, so you cannot manipulate UI components directly in it. See Get back on UI thread to run code on the UI thread.

For more information on when an Activity can lose the focus, and for best practices, refer to Robot lifecycle and Activity lifecycle and Disabled Robot State case.

Impact on actions

When this callback is called, this Activity cannot run actions on Pepper until it regains the robot focus:

// This will fail if robot focus is lost.
say.run();

Also, if onRobotFocusLost is called while an action is running, the action execution will stop and an exception will be raised:

// This will raise an exception.
say.run();

say.async().run().thenConsume(future -> {
    if (future.isSuccess()) {
        // This will not be called.
    } else if (future.hasError()) {
        // This will be called.
    }
});

Because listeners can be triggered without robot focus, you should remove all listeners in the onRobotFocusLost callback:

// Remove listeners from LookAt.
if (lookAt != null) {
    lookAt.removeAllOnPolicyChangedListeners();
}

Impact on services

The robot services are not impacted when the robot focus is lost.

This means that the Futures created with services will continue their execution and the listeners associated to any service will still be triggered:

// The Future will continue its execution.
Future<List<Human>> humansAroundFuture = humanAwareness.async().getHumansAround();

// The listener will still be triggered.
humanAwareness.addOnHumansAroundChangedListener(listener);

Because listeners can be triggered without robot focus, you should remove all service listeners in the onRobotFocusLost callback:

// Remove listeners from HumanAwareness.
if (humanAwareness != null) {
    humanAwareness.removeAllOnHumansAroundChangedListeners();
}

If a service listener is not removed in the onRobotFocusLost callback, it will be triggered for all your application lifetime.

Refused robot focus

In some cases, the focus can be refused for your Activity, for example if the robot is in Sleep Mode or in Disabled State. The onRobotFocusRefused callback will be called for each RobotLifecycleCallbacks registered to this Activity:

void onRobotFocusRefused(String reason);

The reason of the focus refusal is provided as a parameter.

For more details on best practices, refer to Robot lifecycle and Activity lifecycle and Disabled Robot State case.

Get back on UI thread

You may want to update your UI while in the robot lifecycle callbacks. For example, you could want to update a TextView.

To achieve this, you have 2 common possibilities.

Android-based solution

The Activity class provides the runOnUiThread method to execute a Runnable on the UI thread:

@Override
public void onRobotFocusGained(QiContext qiContext) {

    Say say = SayBuilder.with(qiContext)
                        .withText("Hello")
                        .build();

    // Synchronous call.
    say.run();

    // Update the TextView to notify that the Say action is done.
    runOnUiThread(() -> textView.setText("Done!"));
}

QiSDK-based solution

The QiSDK provides the onUiThread static method to execute a Consumer on the UI thread:

@Override
public void onRobotFocusGained(QiContext qiContext) {

    Say say = SayBuilder.with(qiContext)
                        .withText("Hello")
                        .build();

    // Asynchronous call.
    Future<Void> sayFuture = say.async().run();

    // Update the TextView to notify that the Say action is done.
    sayFuture.andThenConsume(Qi.onUiThread((Consumer<Void>) ignore -> {
            textView.setText("Done!");
    }));
}


Advanced information

Robot lifecycle and Activity lifecycle

The robot lifecycle is not entirely linked to the Activity lifecycle.

However, if you consider two Activity instances A and B, the Activity A will lose the robot focus if:

  • the current Activity changes from A to B,
  • the application is put to the background,
  • the application is killed.

Important

In any case, an Activity that owns the robot focus and becomes not visible (even partially) on the UI will lose the robot focus.

Performance and limitations

There is no guarantee that one of the callbacks onRobotFocusGained or onRobotFocusRefused will be called. In some corner cases such as communication difficulties between the robot and tablet, the RobotLifecycleCallbacks may never be notified of the focus gain or the focus denial. The application should implement its own time-out in order to handle this contingency and avoid waiting indefinitely.

There is no way to explicitly (re)request the robot focus. When the Android Activity focus changes, the right to robot focus ownership implicitly moves with it. Registering to RobotLifecycleCallbacks tells your object if the activity passed to it has the robot focus or not.

RobotLifecycleCallbacks will only be called once for each object’s registration. For example, after receiving onRobotFocusRefused or onRobotFocusLost, it is not possible to receive onRobotFocusGained again, as long as the activity is still entirely visible. To receive further callbacks, it is necessary to register a new object, or unregister and re-register the current object.

Going further in corner cases

startActivity API Usage, and Unexpected Focus Changes

Be diligent of Activity and robot focus lifecycles when using the startActivity API.

Important

Calling a startActivity means onRobotFocusLost for the current activity.

After calling startActivity, a new Activity will be focused and then onRobotFocusLost will be called for the previous activity. So be sure that your onRobotFocusLost callback will not interfere with the new Activity.

Here are some pitfalls to avoid:

  • Be careful when calling startActivity in RobotLifecycleCallbacks such as onRobotFocusLost. This can lead to unexpected application states. For example if onRobotFocusLost had been called as a result of a different Activity focus change, you risk unintentionally taking the Android Activity focus from it.

  • It is usually recommended to not call startActivity from an activity not visible on the screen. Most of the time, it leads to unexpected results. For example, avoid calling startActivity from a background service and make sure to cancel any asynchronous task that lead to an activity launch when the activity that runs it is closed or hidden.

  • Likewise, it is not recommended to call startActivity from Android focus lifecycle callbacks such as onCreate, onStart, onResume, onPause, onStop and onDestroy.

  • Sleep Mode specific case:

    Important

    When Pepper enters Sleep Mode, a black Android Activity is focused. This may appear as unexpected from the perspective of your application.

    Your activity’s onPause method will first be called in this case.

    Then, after the black Activity appears, code registered in your Activity to RobotLifecycleCallbacks will receive the onRobotFocusLost callback.

    While Pepper is still in Sleep Mode, requests for robot focus will be answered with the onRobotFocusRefused callback.

Disabled Robot State case

Pepper’s Disabled State is entered by double-clicking the chest button. Then Pepper enters a rest position with all motors off.

In Disabled State it is not possible for the QiSDK to control Pepper. It cannot receive the robot focus.

Important

Disabled State is not intended to be used in end-user applications as it interferes with the expected flow of Activity and robot focus lifecycles.

As Disabled State can only be entered and exited manually by a knowledgeable user, it is usually not necessary to automatically detect and handle this case from your application.

As noted in Performance and limitations only the newly displayed Activity will request the robot focus. So after exiting Disabled state, it is recommended for the person who disabled the robot to simply restart their application, or change the displayed activity within it. It is not recommended to implement complex logic in an application to handle this automatically, as the user is already present and in control.