A Theory and its Applications of Annotation Technology for Advanced Usage of Multimedia Content

PDF
Daisuke YAMAMOTO
Department of Media Science, Graduate School of Information Science, Nagoya University

Abstract

In this paper, we present a new Web-based video annotation system,named iVAS (intelligent Video Annotation Server). Web audiences canassociate any video content on the Internet with their ownannotations. The system analyzes video content in order to acquirecut/shot information and color histograms. And it also automaticallygenerates a Web content for generating and editing annotations. Then,audiences can create annotation data by three methods. The first onehelps the users to create text data such as person/object names, scenedescriptions, and comments interactively. The second methodfacilitates the users associating any video fragments with theirsubjective impression by just clicking a mouse button. And the thirdmethod is evaluation annotation by clicking O-X buttons for each textcomments and messages written by other users. The generated annotationdata are accumulated and managed by an XML database connected withiVAS. We also developed some application systems based on annotationssuch as video retrieval, video simplification, and video-content-basedcommunity support. One of the major advantages of our approach is easyintegration of hand-coded and automatically-generated (such as colorhistograms and cut/shot information) annotations. Additionally, sinceour annotation system is open for public, we must consider somereliability or correctness of annotation data. We also present anautomatic evaluation method of annotation reliability using the users'feedbacks. In the future, these fundamental technologies willcontribute to the formation of new communities centered around onlinevideo content.

1

2

2.1

Fugure1:

Fugure2:

2.2

2.2.1

Fugure3:

2.2.2

2.2.3

2.3

2.3.1

2.3.2

2.3.3

2.3.4

3

3.1

Fugure4:

3.2

3.2.1

Fugure5:

3.2.2

Fugure6:

3.2.3

Fugure7:

3.2.4

3.3

3.3.1

Fugure8:

3.3.2

Fugure9:

3.3.3

$I_{k}(t)=\sum_{S_{k}}N(t_{i},m)$
(1)

Fugure10:

3.4

$e_{k} = s \cdot \frac{g_{k}-a \cdot b_{k}}{g_{k}+a \cdot b_{k}} + t \cdot c_{k}$
(2)
$a = \frac {g_{all}}{ b_{all}}$
(3)
$p=d(G+B) \frac {G - a \cdot B}{G + a \cdot B}$
(4)
$d(x) = 1-\exp(-\tau \cdot x)$
(5)
$r_{k}=(1-d(g_{k}+b_{k})) \cdot p+ d(g_{k}+b_{k}) \cdot e_{k}$
(6)

4

4.1

4.2

Fugure11:

4.3

Fugure13:

4.4

$Dc(P_{j},P_{k})=\int (I_{pj}-I_{nj}) \cdot (I_{pk} - I_{nk})dt$
(7)

4.5

5

5.1

5.2

5.3

5.4

Fugure14:

5.5

Fugure15:

5.6

5.7

6

6.1

6.1.1

6.1.2

Fugure16:

6.2

Fugure17:

6.3

7