Prof. Dr. Werner Kießling (Project leader)
W. T. Balke, T. Birke, M. Wagner

The goal of HERON is image retrieval in multimedia databases.

Project related publications

HERON - Getting the picture right

Finding images in digital libraries or large image archives without detailed knowledge about domain, iconography or the specific kind of objects depicted, is the aim of the interdisciplinary HERON project. HERON is an acronym for 'heraldry online', indicating that the project's first practical application area is about heraldry, the science of coats of arms. Though heraldry represents an important branch of art-historical research, the project's vision is not restricted to the field of art-history, but may serve various applications with need of image databases, as for example medical imaging or collections of trademark and designs. The HERON framework merges leading-edge database technology with professional handling of historical images from the humanities. From 1998 to 2002 HERON was funded for four successful years by the German Research Foundation "Deutsche Forschungsgemeinschaft" (DFG) within the strategic research program "Distributed Processing and Exchange of Digital Documents (V3D2)". This project was carried out by the Institute of Computer Science and the library of the University of Augsburg. Heron was followed by the P-News project in V3D2. Since the focus of P-News is on deep personalization in digital libraries, much of the Heron middleware technology was transferred to P-News.

The HERON system

The HERON prototype for the first time was introduced at the Munich computer trade fair SYSTEMS'99 and later the full system was presented at CeBIT'01. It features an innovative middleware for multimedia queries in large image archives and digital libraries. This middleware is based on industrial standards like IBM DB2 Universal Database with relational extenders for images and text, JDBC and Java. Complex multimedia retrievals may contain queries on image content, e.g. colors, textures or shapes, as well as conventional text-based queries including thesauri searches. These complex multi-feature queries are split into smaller parts that can be handled by the underlying database systems. The ranked result sets of each system can be merged efficiently using a new algorithm that has been proposed for patenting. To guarantee a global distribution of images via the Internet the heterogeneity of current web-browsers, net access and user profiles must be considered. A new optimization algorithm is capable of providing optimal storage schemes for diverse image formats in terms of costs on the database server. It also guarantees a fast delivery of images returned in a variety of qualities and resolutions to the Internet clients. For digitization of large image collections promising techniques for the (semi-) automatic segmentation of image objects have been developed as a basis for advanced shape matching.

The target group for the innovative new techniques developed in the framework of the HERON system are partners for technology transfer and commercialization.

The HERON application

As a first area of application the HERON approach has been tested on heraldry. Though the stylized bearings in medieval coats of arms are part of a limited repository, the exact terminology of heraldry is only known to a small number of experts limiting the use of text-based approaches. Nevertheless heraldry is important for a wide community of scientists and interested persons in connection with general historic researches. The availability of fundamental visual query capabilities is going to reduce the retrieval costs by far while improving the relevance of results. Up to now coats of arms had to be specified using large standard volumes (e.g. Siebmacher: about 150 000 bearings in more than 100 volumes). In the near future this extremely time-consuming task can be done computer aided on digitized collections with a minimum knowledge of both history and computer science. Query by visual example allows convenient queries like: "Retrieve all database objects that look similar to this bearing!". If the user recognizes of a specific object (e.g. a lion), the verbal annotation can be used to support the query and improve the specific result set.

For the digitization standard volumes like "Siebmacher's grosses Wappenbuch" from 1856 and books from the historical library of Oettingen-Wallerstein have been carefully chosen. For the first time it is possible to open these historical collections of emblems and coats of arms to a broader public without any conservatory objections. By distributing images via WWW the possibility of building up virtual, distributed Corpora or libraries becomes feasible and paves the way towards new dimensions of research in the human sciences.

The target group for the HERON application are professional experts and interested persons from the humanities and cultural sciences as well as librarians.