LabelMe App

LabelMe App

Released: Dec 10, 2012
Version: 1.0
For iPhone and iPad.

LabelMe

The LabelMe App will allow you taking pictures, adding bouding boxes and uploading them your LabelMe online account.

LabelMe App LabelMe App LabelMe App LabelMe App LabelMe App

A quick guide

Step 1: Get the LabelMe App

The LabelMe app is available for free at the App Store

Step 2: Create an account

If you have already a LabelMe account, you can use the same username and password. You can also create a user account from the App and use it at the LabelMe website. Once you create an account you can log in and start taking pictures and store them online.

Step 3: Collect and annotate images with the App

The images you collect will be uploaded to your online LabelMe account.

Step 4: Download the images and annotations

You can download the images from the LabelMe website (collection: iPhoneCollection).

You can also use the LabelMe toolbox to download the images and annotations. Use the matlab function:

APPdownload('yourUserName');

And that's it!


Some more details

The app has 3 sections: Labels Gallery, Detectors Gallery and Settings.

Labels Gallery

The Labels Gallery stores the images that you take and annotate. In this library you can see the images on a grid or a list layout. You can use the gallery view to upload and delete images.

The circle on the top-right corner of each image indicates if the image has been uploaded to the server. A number indicates how many new objects have been added since the last upload.

The gallery only shows the images available on the mobile device.

Image annotation

You can add bounding boxes around objects on each image. Once you have adjusted the bounding box to contain the object, you can add a tag to describe the object. You can use the keyboard or the microphone to tag the annotated object.

Once you upload the image you will be able to visualize the annotations on the LabelMe website. You can also make changes and add new annotations, however online changes will not be reflected on the data stores in the App and might get deleted if the image is uploaded again from the App.

Upload images

Images are uploaded to your account on the labelMe server. The images will be stored into the folder /iPhoneCollection/.

Currently, all the images captured from the App will be stored in a single folder. It is not possible to create collections within the App.

Downloading images

To download the images captured with the app you need to first upload them to the server and then download them from the LabelMe website.

Sharing images

If you share your username with other people, they will be able to download your images and annotations using the LabelMe toolbox.

If you want multiple people to help you collect and annotate images, they can create their own accounts and share the username with you so that you can download the images and annotations.

Delete images

If you delete images from the app, the images and annotations will only be deleted on the mobile device and will still be available on the LabelMe website if they have been uploaded. If you want to delete the images from the server you have to log in into the LabelMe website and delete the images there.

Deleting images on the App will allow you freeing memory without loosing the images that had been uploaded.

If you want to delete images from the server too, you have to do it from the LabelMe website.

Detectors Gallery

The Detectors Gallery shows you the list of the trained detectors. A detector, after trained, let you detect objects through the iOS device camera, in real time.

Detector Gallery

Adding a detector

To create a detector you first have to click the '+' sign on the left top of the detectors gallery.

Then choose the class (or classes) for you would like to create a detector. This classes are extracted from the Labels gallery, so you must have annotated images in order to train a new detector. Click next to continue.

The next screen shows you the available images to train from. This are all the annotated images you have from the Labels Gallery for the selected class. The default settings makes you select all the images. It is important to have a reasonable number of annotated images to train a good detector (see training considerations for more details). Click next to begin with the training.

During the detector training you will see a debug page with information about the training process. This page basically shows statistics about the evolution of the detector training. Wait until the Cancel button changes its name to Done.

when the training it is done, you will be prompted to the detector information page. This page shows you information about the detector and two images representing the detector: one is the average image of all the training set images and the other is a representation of the detector weights in the HOG space. Click the play button to execute it!

Choose class Choose images Training debug Detector details

Executing detectors

To execute detector go to the detail page of it and hit the play button. You will see the iOS device camera and if you point to the trained object it should be detected.

The sliding bar sets de detector accuracy. If you swipe it to the left, more detections (with less accuracy) will be made. The opposite will happen if you swipe it to the right.

The settings button let you show on the screen the current Frames Per Second (FPS) achieved during the detection. Depending on the parameters used for training, you will be able to increase the FPS rate.

The HOG settings let you turn the camera into the Histogram Of Gradients (HOG) space. This is the space used were the detector is trained and it can help you debug and improve the dector. See the N. Dalal and B. Triggs article for more details.

There is also the possibility to execute more than one detector at the same time. To do it, just go to the detectors collection gallery, tap the Edit button and select as many detectors as you would like to execute simultaneously. Then tap the Execute button.

Executing 'cup' detector

Retraining a detector

When a detector is created you can retrain it as many times as you want. This is useful, for example, when you want to improve the detector adding more images and annotations.

To retrain the detector, just go to the detector details and click the train button. You will have to choose again which training set you would like to use. Notice that the selected images that will appear are those that you use in the previous training.

Also you have the undo button to return to the previous trained detector if it has not improved.

Deleting a detector

To delete detectors just click the Edit button in the detectors gallery and select those you would like to delete.

Training considerations

To ensure a good performance on the object detection phase, here are some guidelines to take into account when training the detector.

Settings

There are several settings that can be modified:

Licenses

Please make sure that the images you upload are copyright-free and do not contain inappropriate content. The images that you upload and annotate will be considered part of the public domain and other people might see them. As the goal of LabelMe is to provide a tool for research, the images and annotations are expected to become available to the research comunity without restrictions.

You are free to post your collected database of images and annotations on your own website. For example, you may do this when you release your database with your research publications.

Privacy

Once you upload your images, they will be available online. Even if they are not listed, knowing the URL will be enough to access them. This makes easier to do research with the images collected.

Therefore, do not upload personal material that you do not want others to see

Citation

If you use the dataset or any functions on this website, we would appreciate it if you cite:
B. C. Russell, A. Torralba, K. P. Murphy, W. T. Freeman, LabelMe: a database and web-based tool for image annotation. International Journal of Computer Vision, pages 157-173, Volume 77, Numbers 1-3, May, 2008. (paper.pdf)

Suggestions

Send email to labelme@csail.mit.edu if you have suggestions, find bugs or there are new features that you would like to see in the tool.

Credits

The LabelMe app has been developed by Dolores Blanco, Aina Torralba, David Way, Josep Marc Ming0t and Antonio Torralba.