The PASCAL Visual Object Classes Challenge 2006
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
The development kit provided for the VOC challenge 2006 is available. You can:
Following completion of the challenge, the training/validation sets with complete annotation have now been made available on the database page. Some additional example images are online.
The VOC2006 data includes some images provided by Microsoft Research Cambridge and "flickr". Use of these images must respect the corresponding terms of use:
Details of the contributor of each image can be found in the file "contrib.txt" included in the dataset. Any queries about the use or ownership of the data should be addressed to the organizers.
The second round aims to give an opportunity for groups who participated to submit additional results, and for others who might have been unable to meet the first deadline to participate. The deadline for submission of results for the second round is 11pm GMT, Friday 30th June 2006.
The second round will be run as per the first round. In evaluating the second round, submitted results will be evaluated separately from the first round to give consideration for the additional time available. Comparison of the results obtained between the first and second rounds will follow. As in the first round, ground truth annotation will not be provided for the test data.
On completion of the second round, the test set annotation will be released for researchers wishing to use the data for further evaluation.
We invite submission of results obtained after the completion of the challenge, both by participants and those who did not participate in the challenge. Results will be treated as distinct from the results obtained during the challenge due to the availability of the test data annotation. If you would like your results included in the online summary of results, please contact Mark Everingham, me@comp.leeds.ac.uk.
A technical report summarizing the challenge and results is now available. The results may also be browsed online. Slides presented at the challenge workshop can also be downloaded.
If you make use of the VOC2006 data, please cite the following reference in any publications:
@misc{pascal-voc-2006,
author = "Everingham, M. and Zisserman, A. and Williams, C. K. I. and Van~Gool, L.",
title = "The {PASCAL} {V}isual {O}bject {C}lasses {C}hallenge 2006 {(VOC2006)} {R}esults",
howpublished = "http://www.pascal-network.org/challenges/VOC/voc2006/results.pdf"
}
The goal of this challenge is to recognize objects from a number of visual object classes in realistic scenes (i.e. not pre-segmented objects). It is fundamentally a supervised learning learning problem in that a training set of labelled images is provided. The ten object classes that have been selected are:
There will be two main competitions:
Participants may enter either (or both) of these competitions, and can choose to tackle any (or all) of the ten object classes. The challenge allows for two approaches to each of the competitions:
The intention in the first case is to establish just what level of success can currently be achieved on these problems and by what method; in the second case the intention is to establish which method is most successful given a specified training set.
The training data provided consists of a set of images; each image has an annotation file giving a bounding box and object class label for each object in one of the ten classes present in the image. Note that multiple objects from multiple classes may be present in the same image. Some example images can be viewed online.
Annotation was performed according to a set of guidelines distributed to all annotators. These guidelines can be viewed here.
The data will be made available in two stages; in the first stage, a development kit will be released consisting of training and validation data, plus evaluation software (written in MATLAB). One purpose of the validation set is to demonstrate how the evaluation software works ahead of the competition submission.
In the second stage, the test set will be made available for the actual competition. In contrast to the 2005 VOC challenge, no ground truth for the test data will be released until after the challenge is complete.
Following completion of the challenge, the training/validation sets with complete annotation have now been made available on the database page.
The table below summarizes statistics of the training/validation and test data. The data has been split into 50% for training/validation and 50% for testing. The distributions of images and objects by class are approximately equal across the training/validation and test sets. In total there are 5304 images, containing 9507 annotated objects.
| train | val | trainval | test | |||||
|---|---|---|---|---|---|---|---|---|
| Images | Objects | Images | Objects | Images | Objects | Images | Objects | |
| Bicycle | 127 | 161 | 143 | 162 | 270 | 323 | 268 | 326 |
| Bus | 93 | 118 | 81 | 117 | 174 | 235 | 180 | 233 |
| Car | 271 | 427 | 282 | 427 | 553 | 854 | 544 | 854 |
| Cat | 192 | 214 | 194 | 215 | 386 | 429 | 388 | 429 |
| Cow | 102 | 156 | 104 | 157 | 206 | 313 | 197 | 315 |
| Dog | 189 | 211 | 176 | 211 | 365 | 422 | 370 | 423 |
| Horse | 129 | 164 | 118 | 162 | 247 | 326 | 254 | 324 |
| Motorbike | 118 | 138 | 117 | 137 | 235 | 275 | 234 | 274 |
| Person | 319 | 577 | 347 | 579 | 666 | 1156 | 675 | 1153 |
| Sheep | 119 | 211 | 132 | 210 | 251 | 421 | 238 | 422 |
| Total | 1277 | 2377 | 1341 | 2377 | 2618 | 4754 | 2686 | 4753 |
The VOC2006 data includes some images provided by Microsoft Research Cambridge and "flickr". Use of these images must respect the corresponding terms of use:
For the purposes of the challenge, the identity of the images in the database, e.g. source and name of owner, was obscured. Details of the contributor of each image can be found in the file "contrib.txt" in the final release of the data. Any queries about the use or ownership of the data should be addressed to the organizers.
Participants are expected to submit a single set of results per method employed. Participants who have investigated several algorithms may submit one result per method. Changes in algorithm parameters do not constitute a different method - all parameter tuning must be conducted using the training and validation data alone.
Details of the required file formats for submitted results can be found in the development kit documentation. For each competition you should produce a single text file, named following the examples in the development kit. For example, your results file for competition 1 (classification trained on VOC data) on the "car" class should be named "comp1_cls_test_car.txt"; your results file for competition 4 (detection trained on your own data) on the "car" class should be named "comp4_det_test_car.txt".
If you intend to submit results for several different methods, please place results for each in a separate, clearly named directory. For each method you submit, please include a brief description of the approach used. If you wish to withhold details of the method, e.g. for commercial reasons, please make this clear. Methods submitted without a description will be judged in a separate category of the challenge.
The results files should be collected in a single archive file (tar/zip) and placed on an FTP/HTTP server accessible from outside your institution. Email the URL and any details needed to access the file to Mark Everingham, me@comp.leeds.ac.uk. Please do not send files directly by email.
The main mechanism for dissemination of the results will be the challenge webpage. It is also possible that an overview paper of the results will be produced.
If you make use of the VOC2006 data, please cite the technical report of results, see the results section.