Mastering FreeFrames

no_virtual_ robotCannot be tested on an emulated robot, requires a real robot.

Goal

In this tutorial, we will see how to save a specific location and how to make Pepper go to this location, using FreeFrames.

Prerequisites

Before stepping in this tutorial, you should:

Let’s start a new project

  • Start a new project, let’s call it GoToWorldPepper.
  • Robotify it and make sure it implements the QiSDK & the Robot Life Cycle.

For further details, see: Creating a robot application.

Saving the robot location

To save locations, you will need the following fields in your MainActivity:

// Store the saved locations.
private Map<String, FreeFrame> savedLocations = new HashMap<>();
// The QiContext provided by the QiSDK.
private QiContext qiContext;
// Store the Actuation service.
private Actuation actuation;
// Store the Mapping service.
private Mapping mapping;

In the onRobotFocusGained method, add the following code:

// Store the provided QiContext and services.
this.qiContext = qiContext;
actuation = qiContext.getActuation();
mapping = qiContext.getMapping();

In the onRobotFocusLost method, add the following code:

// Remove the QiContext.
qiContext = null;

To save the robot current location, we will retrieve the robot frame, create a FreeFrame at this location and finally store it inside a Map<String, FreeFrame>. Add this method to your MainActivity:

void saveLocation(final String location) {
    // Get the robot frame asynchronously.
    Future<Frame> robotFrameFuture = actuation.async().robotFrame();
    robotFrameFuture.andThenConsume(robotFrame -> {
        // Create a FreeFrame representing the current robot frame.
        FreeFrame locationFrame = mapping.makeFreeFrame();
        Transform transform = TransformBuilder.create().fromXTranslation(0);
        locationFrame.update(robotFrame, transform, 0L);

        // Store the FreeFrame.
        savedLocations.put(location, locationFrame);
    });
}

How to go to a saved location

Now that we know how to save a location, we can make Pepper go to it. We need to get the FreeFrame from the map and execute a GoTo action with it.

Add a GoTo field in your MainActivity:

// Store the GoTo action.
private GoTo goTo;

Add this method to your MainActivity:

void goToLocation(final String location) {
    // Get the FreeFrame from the saved locations.
    FreeFrame freeFrame = savedLocations.get(location);

    // Extract the Frame asynchronously.
    Future<Frame> frameFuture = freeFrame.async().frame();
    frameFuture.andThenCompose(frame -> {
        // Create a GoTo action.
        goTo = GoToBuilder.with(qiContext)
                .withFrame(frame)
                .build();

        // Display text when the GoTo action starts.
        goTo.addOnStartedListener(() -> Log.i(TAG, "Moving..."));

        // Execute the GoTo action asynchronously.
        return goTo.async().run();
    }).thenConsume(future -> {
        if (future.isSuccess()) {
            Log.i(TAG, "Location reached: " + location);
        } else if (future.hasError()) {
            Log.e(TAG, "Go to location error", future.getError());
        }
    });
}

Do not forget to remove this listener on GoTo in the onRobotFocusLost method:

// Remove on started listeners from the GoTo action.
if (goTo != null) {
    goTo.removeAllOnStartedListeners();
}

Testing the functionality

We will implement this functionality using:

  • an EditText to enter the location name,
  • a Button to save the location,
  • a Spinner to display the saved locations and the selected one,
  • a Button to make Pepper move to the selected location.

Modify your activity_main.xml file with the following code:

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:focusableInTouchMode="true"
    tools:context=".MainActivity">

    <Button
        android:id="@+id/save_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="@string/save"
        app:layout_constraintRight_toLeftOf="@+id/editText"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintBottom_toTopOf="@+id/goto_button"
        app:layout_constraintTop_toTopOf="parent" />

    <Button
        android:id="@+id/goto_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="@string/go_to_text"
        app:layout_constraintLeft_toLeftOf="@+id/save_button"
        app:layout_constraintRight_toRightOf="@+id/save_button"
        app:layout_constraintHorizontal_bias="0.0"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/save_button" />

    <EditText
        android:id="@+id/editText"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:ems="10"
        android:inputType="text"
        android:labelFor="@+id/editText"
        android:hint="@string/location"
        tools:text="Location"
        app:layout_constraintBaseline_toBaselineOf="@+id/save_button"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintLeft_toRightOf="@+id/save_button" />

    <Spinner
        android:id="@+id/spinner"
        android:layout_width="0dp"
        android:layout_height="wrap_content"
        app:layout_constraintLeft_toLeftOf="@+id/editText"
        app:layout_constraintRight_toRightOf="@+id/editText"
        app:layout_constraintTop_toTopOf="@+id/goto_button"
        app:layout_constraintBottom_toBottomOf="@+id/goto_button"
        app:layout_constraintHorizontal_bias="0.0" />

</android.support.constraint.ConstraintLayout>

Add the following fields in your MainActivity:

private Button goToButton;
private Button saveButton;
private ArrayAdapter<String> spinnerAdapter;

// Store the selected location.
private String selectedLocation;

And add this code in the onCreate method:

final EditText editText = (EditText) findViewById(R.id.editText);
final Spinner spinner = (Spinner) findViewById(R.id.spinner);

// Save location on save button clicked.
saveButton = (Button) findViewById(R.id.save_button);
saveButton.setOnClickListener(v -> {
    String location = editText.getText().toString();
    editText.setText("");
    // Save location only if new.
    if (!location.isEmpty() && !savedLocations.containsKey(location)) {
        spinnerAdapter.add(location);
        saveLocation(location);
    }
});

// Go to location on go to button clicked.
goToButton = (Button) findViewById(R.id.goto_button);
goToButton.setOnClickListener(v -> {
    if (selectedLocation != null) {
        goToButton.setEnabled(false);
        saveButton.setEnabled(false);
        goToLocation(selectedLocation);
    }
});

// Store location on selection.
spinner.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {
    @Override
    public void onItemSelected(AdapterView<?> parent, View view, int position, long id) {
        selectedLocation = (String) parent.getItemAtPosition(position);
        Log.i(TAG, "onItemSelected: " + selectedLocation);
    }

    @Override
    public void onNothingSelected(AdapterView<?> parent) {
        selectedLocation = null;
        Log.i(TAG, "onNothingSelected");
    }
});

// Setup spinner adapter.
spinnerAdapter = new ArrayAdapter<>(this, android.R.layout.simple_spinner_item, new ArrayList<String>());
spinnerAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
spinner.setAdapter(spinnerAdapter);

When Pepper is ready to save locations, we want to enable the UI elements. Add this method to your MainActivity:

private void waitForInstructions() {
    Log.i(TAG, "Waiting for instructions...");
    runOnUiThread(() -> {
        saveButton.setEnabled(true);
        goToButton.setEnabled(true);
    });
}

Call this method at the end of onRobotFocusGained:

// Store the provided QiContext and services.
this.qiContext = qiContext;
actuation = qiContext.getActuation();
mapping = qiContext.getMapping();

waitForInstructions();

And in the .thenConsume(...) inside the goToLocation method:

if (future.isSuccess()) {
    Log.i(TAG, "Location reached: " + location);
    waitForInstructions();
} else if (future.hasError()) {
    Log.e(TAG, "Go to location error", future.getError());
    waitForInstructions();
}

Let’s try it

github_icon The sources for this tutorial are available on GitHub.

Step Action

Install and run the application.

For further details, see: Running an application.

Choose “Saving locations”.

Enter “Room” in the EditText and click on the “Save” button.

The Spinner now contains the “Room” element.

Open the robot hatch and move him somewhere else. Then close his hatch.

Enter “Kitchen” in the EditText and click on the “Save” button.

The Spinner now contains the “Kitchen” element.

Select “Room” on the Spinner and click on the “Go to” button.

Pepper will go to the “Room” location. When he arrives, the log trace “Location reached: Room” is displayed in the console.

https://developers.softbankrobotics.com/sites/default/files/repository/58_rst_pepper/public/source/_build/html/_images/freeframe.png

You are now able to store locations and make Pepper go to them!