Take a picture of what Pepper is seeing

no_virtual_ robotCannot be tested on an emulated robot, requires a real robot.

Goal

In this tutorial, we will use Pepper’s camera. We will use the camera service, which can be accessed using the Camera interface.

Prerequisites

Before stepping in this tutorial, you should be familiar with the robot focus and robot lifecycle notions.

For further details, see: Mastering Focus & Robot lifecycle.

Let’s start a new project

  • Start a new project, let’s call it TakePicturePepper.
  • Robotify it and make sure it implements the QiSDK & the Robot Life Cycle.

For further details, see: Creating a robot application.

Taking a picture

Put the following code in the onRobotFocusGained method:

// Build the action.
Future<TakePicture> takePictureFuture = TakePictureBuilder.with(qiContext).buildAsync();

Now let’s test it using a button to start the action and an imageView to display the taken picture.

In the activity_main.xml layout file, add the following Button and ImageView:

<ImageView
    android:id="@+id/picture_view"
    android:layout_width="300dp"
    android:layout_height="300dp"
    android:layout_above="@+id/button"
    android:layout_centerInParent="true"  />

<Button
    android:id="@+id/button"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_alignParentBottom="true"
    android:text="Take a picture"/>

Create these fields in the MainActivity:

// The button used to start take picture action.
private Button button;
// An image view used to show the picture.
private ImageView pictureView;
// The QiContext provided by the QiSDK.
private QiContext qiContext;
// TimestampedImage future.
Future<TimestampedImageHandle> timestampedImageHandleFuture;

Put the following code for the onRobotFocusGained and onRobotFocusLost methods:

@Override
public void onRobotFocusGained(QiContext qiContext) {
    // Store the provided QiContext.
    this.qiContext = qiContext;
}

@Override
public void onRobotFocusLost() {
    // Remove the QiContext.
    this.qiContext = null;
}

Find the button and the imageView in the onCreate method:

button = (Button) findViewById(R.id.button);
pictureView = findViewById(R.id.picture_view);

And set the button onClick listener:

// Set the button onClick listener.
button.setOnClickListener(v -> takePicture());

Add a method that runs takePicture Action asynchronously:

public void takePicture() {
    // Check that the Activity owns the focus.
    if (qiContext == null) {
        return;
    }

    // Disable the button.
    button.setEnabled(false);

    Future<TimestampedImageHandle> timestampedImageHandleFuture = takePictureFuture.andThenCompose(takePicture -> {
        Log.i(TAG, "take picture launched!");
        return takePicture.async().run();
    });
}

Running a TakePicture action returns a TimestampedImageHandle object that contains the picture and its timestamp.

The following snippet gets the image data as a ByteBuffer then display it as a Bitmap.

timestampedImageHandleFuture.andThenConsume(timestampedImageHandle -> {
    // Consume take picture action when it's ready
    Log.i(TAG, "Picture taken");
    // get picture
    EncodedImageHandle encodedImageHandle = timestampedImageHandle.getImage();

    EncodedImage encodedImage = encodedImageHandle.getValue();
    Log.i(TAG, "PICTURE RECEIVED!");

    // get the byte buffer and cast it to byte array
    ByteBuffer buffer = encodedImage.getData();
    buffer.rewind();
    final int pictureBufferSize = buffer.remaining();
    final byte[] pictureArray = new byte[pictureBufferSize];
    buffer.get(pictureArray);

    Log.i(TAG, "PICTURE RECEIVED! (" + pictureBufferSize + " Bytes)");
    // display picture
    pictureBitmap = BitmapFactory.decodeByteArray(pictureArray, 0, pictureBufferSize);
    runOnUiThread(() -> pictureView.setImageBitmap(pictureBitmap));
});

Let’s try it

github_icon The sources for this tutorial are available on GitHub.

Step Action

Install and run the application.

For further details, see: Running an application.

Choose “Take a picture”.

Wait for pepper to say the instruction.

Click on take picture button.

When the picture is taken you will see it in the center of the screen.

https://developers.softbankrobotics.com/sites/default/files/repository/58_rst_pepper/public/source/_build/html/_images/takepicture.png

You are now able to take pictures with Pepper!