Michel Jansen

VideoTrees

This research on a novel video retrieval user interface, was originally part of my Bachelor Thesis at the University of Twente, where I presented it at the 8th TSC on IT. Currently, it has been accepted, in a revised form, for publication at CBMI2008. Please use the latter in when referring to this work.

Here, you will find information on the original research, as well as the publications.

A video demonstration

Considering this research is about video retrieval, it is only fitting to demonstrate what it is all about with a short video:

Paper title

VideoTrees: Improving the presentation of video surrogates using hierarchy

Abstract

As the amount of available video content increases, so does the need for better ways of browsing all this material. Because the nature of video makes it hard to process, the need arises for adequate surrogates for video that can readily be skimmed and browsed.

In this paper, the effects of the use of hierarchy in a pictorial summary of keyframes are explored, and a novel type of video surrogate is presented: the VideoTree. Moreover, a prototype browser was developed and tested in a preliminary usability study. This showed that users performed better using the VideoTrees browser than using a regular storyboard-based browser. They also found it more flexible, but more difficult to use.

BibTeX citation

@ARTICLE{Jansen2008, 
title={Videotrees: Improving video surrogate presentation using hierarchy}, 
author={Jansen, M. and Heeren, W. and van Dijk, B.}, 
journal={Content-Based Multimedia Indexing, 2008. CBMI 2008. International Workshop on}, 
year={2008}, 
month={June}, 
volume={}, 
number={}, 
pages={560-567}, 
abstract={
As the amount of available video content increases, so does the need for better 
ways of browsing all this material. Because the nature of video makes it hard to 
process, the need arises for adequate surrogates for video that can readily be 
skimmed and browsed. In this paper, the effects of the use of hierarchy in a 
pictorial summary of keyframes are explored, and a novel type of video 
surrogate is presented: the VideoTree. Moreover, a prototype browser was 
developed and tested in a preliminary usability study. This showed that users 
performed better using the VideoTrees browser than using a regular storyboard-
based browser. They also found it more flexible, but more difficult to use.}, 
keywords={
Internet, online front-ends, trees (mathematics), video retrieval, video signal processing
VideoTrees, prototype browser, regular storyboard-based browser, 
video content, video surrogate presentation}, 
doi={10.1109/CBMI.2008.4564997}, 
ISSN={}, }

More information

This paper was first presented at the 8th Twente Student Conference on IT. It has currently been accepted for publication in the proceedings of the Sixth International Workshop on Content-Based Multimedia Indexing (CBMI 2008).

Presentation

The results of this research were presented at a number of occasions. For people who want to review the slides, they have been made available here under the downloads section. Also, one of the presentations was recorded on video. You can review this presentation on line at collegerama. Because it was a presentation specifically for a video recording experiment, it is a little shorter and a little less dynamic than usual ;)

Prototype

To test the effects of hierarchical presentation in video surrogates, a prototype video browser based on VideoTrees was built and compared to a non-hierarchical storyboard browser. Both of these prototypes were implemented in Flash and are available here.

You can play around with a demo of both the VideoTrees prototype and the Storyboard prototype as they were used in the usability study.

The source code is available under the GNU Lesser Public License (LGPL) and can be found in its Subversion repository or from one of the download links below.

Data set

For the usability study, two episodes of the Dutch children’s programme “Willem Wever” were used. These were taken from the TRECVID 2007 reference set and came with a list of previously detected shot boundaries. If you have access to this set, the files are called “BG_2402” and “BG_3273”. These videos were originally provided by the Dutch Institute for Image and Sound (“Nederlands Instituut voor Beeld en Geluid”) and this is where the copyright remains.

To work with this prototype, the original shot boundary detection list was extended to include hierarchical information. This metadata is available as XML. Also part of the data set, is a collection of images, extracted from the video at all of the time indices present in the XML.

For the purpose of reproducing, comparing or extending the research mentioned, the dataset can be downloaded below.

Download

Both the original bachelor thesis paper and the one presented at CBMI 2008 are available here for download in PDF (Portable Document Format). The prototype source code is distributed as a compressed archive.