Journal of Applied Science and Engineering

Published by Tamkang University Press

1.30

Impact Factor

2.10

CiteScore

Zijun Yang1 and C.-C. Jay Kuo1

1Integrated Media Systems Center Department of Electrical Engineering-Systems University of Southern California, Los Angeles, CA 90089-2564


 

Received: March 1, 1999
Accepted: September 5, 1999
Publication Date: September 5, 1999

Download Citation: ||https://doi.org/10.6180/jase.1999.2.3.01  


ABSTRACT


Multimedia database management has been intensively studied recently due to the rapid growth of multimedia data and the demand of their access over the Internet. The new member of the MPEG (Moving Picture Expert Group) family, called the “Multimedia Content Description Interface” or MPEG-7 in short, will extend the limited capabilities of proprietary solutions in identifying multimedia contents that exist today. State-of-the-art technologies and systems will be evaluated and merged to specify a standard to describe multimedia contents in such a process. In this review paper, a comprehensive survey of image database management techniques and systems is performed, and the status in the MPEG-7 standardization process is reported. Based on this study, future trends and promising research directions are also predicted.


Keywords: Image content analysis, Feature extraction, Image retrieval, MPEG-7, Descriptors, Description scheme, Description definition language


REFERENCES


  1. [1] Abdel-Mottaleb M., Krishnamachari, S., “Color representation by multiple local histogram,” ISO/IEC/JTC1/ SC29/WG11, pp. 648, Lancaster, UK, Feb. (1999).
  2. [2] ACTS-DICEMAN, ISO/IEC/JTC1/SC29/ WG11, pp. 184, Lancaster, UK, Feb. (1999).
  3. [3] ACTS-DICEMAN, ISO/IEC/JTC1/SC29/ WG11, pp. 186, Lancaster, UK, Feb. (1999).
  4. [4] Aksoy, S. and Haralick, R. M., “Textural features for image database retrieval,” in IEEE CVPR'98 Workshop on Content-Based Access of Image and Video Libraries, (1998).
  5. [5] Alata, O., Cariou, C., Ramannanjarasoa, C. and Najim, M., “Classification of rotated and scaled textures using HMHV spectrum estimation and the Fourier-Mellin Transform,” in IEEE International Conference on Image Processing, (1998).
  6. [6] Bach, J. R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Humphrey, R., Jain, R., and Shu, C. F., “The Virage image search engine: an open framework for image management,” in SPIE Storage and Retrieval for Image and Video Databases V, (1996).
  7. [7] Beek, P. V., Qian, R. and Sezan, I., “Scalable blob histogram descriptor,” ISO/IEC/JTC1/SC29/WG11, pp. 430, Lancaster, UK, Feb. (1999).
  8. [8] Bhanu, B., Qing, S. and Peng, J., “Learning integrated online indexing for image databases,” in IEEE International Conference on Image Processing, (1998).
  9. [9] Bober, M., ISO/IEC/JTC1/SC29/WG11 ,pp. 320, Lancaster, UK, Feb. (1999).
  10. [10] Bouet, M. and Djeraba, C., “Powerful image organization in visual retrieval systems,” in ACM Multimedia'98, Bristol, UK, (1998).
  11. [11] Bowman, C., Danzig, P., Hardy, D., Maber, U., and Schwartz, M., “The Harvest information discovery and access system,” in the 2nd International World Wide Web Conference, (1994).
  12. [12] Caron, C., Challapali, K. and Yan, Y., ISO/IEC/JTC1/SC29/WG11, pp. 646, Lancaster, UK, Feb. (1999).
  13. [13] Carson, C., Belongie, S., Greenspan, H. and Malik, J., “Region-based image querying,” in IEEE CVPR'97 Workshop on Content-Based Access of Image and Video Libraries, (1997).
  14. [14] Carson, C., Belongie, S., Greenspan, H., and Malik, J., “Blobworld: image segmentation using expectation- maximization and its application to image querying,” in submitted to IEEE Trans. on Pattern Analysis and Machine Intelligence, (1998).
  15. [15] Cha, K. H., Lee, H. A., Park, J. D., Ryu P.-M., Chae Y.-S., and Park S.-Y., “Representation of the situational meaning of an image based on domain ontology,” ISO/IEC/JTC1/SC29/WG11 pp. 331, Lancaster, UK, Feb. (1999).
  16. [16] Chakrabarti, S., Dom, B., Kumar, S., Raghavan, P., Rajagopalan, S., Tomkins, A., Gibson, D. and Kleinberg, J., “Mining the Web's link structure,” in IEEE Computer Magazine, Aug. (1999).
  17. [17] Chang, S. F., Smith, J. R. and Meng, H. J., “Exploring image functionalities in WWW applications - development of image/video search and editing engines”, in IEEE International Conference on Image Processing, (1997).
  18. [18] Chang, T. and Kuok, C. C. J., “Texture analysis and classification with tree-structured wavelet transform,” in IEEE Trans. on Image Processing, (1993).
  19. [19] Cieplinski, L., ISO/IEC/JTC1/SC29/ WG11/, Lancaster, UK, Feb. pp. 319, (1999).
  20. [20] “Core experiments on MPEG-7 color and texture descriptors,” ISO/IEC/JTC1/ SC29/WG11 Seoul, Korea, Mar. pp. 2691, (1999).
  21. [21] Cutting, D., Karger, D., Pedersen, J., and Tukey, J., “Scatter/gather: A cluster-based approach to browsing large document collections,” in ACM SIGIR'92, (1992).
  22. [22] Deng, Y., Kenney, C., Moore, M. S. and Manjunath, B. S., “Peer group flltering and perceptual color quantization”, in IEEE International Symposium on Circuits and Systems, (1999).
  23. [23] Deng, Y., Manjunath, B. S., Shin, H. and Choi, Y., “A color descriptor for MPEG-7: variable-bin color histogram,” ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 76 (1999).
  24. [24] “Description of core experiments for MPEG-7 motion/shape”, ISO/IEC JTC1/SC29/WG11N2690, Seoul, Korea, Mar. (1999).
  25. [25] Dimitrova, N., Agnihotri, L., Martino, J. and Elenbaas, H., “Super-histogram for video classification and program,” ISO/IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 641 (1999).
  26. [26] Dimitrova, N., McGee, T., Elenbaas, H. and Martino, J., “Video content management in consumer devices,” in IEEE Trans. on Knowledge and Data Engineering, Nov. (1998).
  27. [27] “Document content description (DCD) for XML”, Submission to W3C, Jul. (1998).
  28. [28] Douglass, R. J., “Description definition language (DDL), knowledge representation language for MPEG-7 DDL,” ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 124 (1999).
  29. [29] Dowe, J., “Content-based retrieval in multimedia imaging,” in SPIE Storage and Retrieval for Image and Video Databases II, (1993).
  30. [30] “Extensible markup language (XML) 1.0,” REC-xml-(1998)0210, W3C Recommendation, Feb. (1998).
  31. [31] Ferman, A. M. and Tekalp, A. M., “Efficient filtering and clustering methods for temporal video segmentation and visual summarization,” in Journal of Visual Communication and Image Representation, Vol. 9,No. 4, pp. 336-351, (1998).
  32. [32] Ferman, A. M., Tekalp, A. M., Mehrotra, R., “Histogram-based color descriptors for multiple frame color characterization,” ISO/IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 529 (1999).
  33. [33] Ganti, V., Gebrke, J. and Ramakrishnan, R., “Mining very large database,” in IEEE Computer Magazine, Aug. (1999).
  34. [34] Gonzalez, R. C., Woods, R. E., Digital Image Processing, Addison Wesley Publishing Company, (1993).
  35. [35] Huang, J., Kumar, S. R. and Mitra, M., “Combining supervised learning with color correlograms for content-based image retrieval,” in ACM Multimedia'97, Seattle, WA, (1997).
  36. [36] Huang, J., Kumar, S. R. and Zabih, R., “An automatic hierarchical image classification scheme,” in ACM Multimedia'98, Bristol, UK, (1998).
  37. [37] Hunter, J., “A proposal for an MPEG-7 description definition language (DDL) ,” ISO/IEC/JTC/SC29/WG11, Lancaster, UK, Feb. pp. 547 (1999).
  38. [38] “IBM Almaden Research Center, Technical summary of color descriptors for MPEG-7,” ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 165 (1999).
  39. [39] Iwayama, M. and Tokunaga, T., “Cluster-based text categorization: A compression of category search strategies,” in ACM SIGIR'95, (1995).
  40. [40] Jung, S., Kim, K., Chun, B. T., Lee, J. Y. and Bae, Y., “Color descriptor by using picture information measure of subregions in video sequence,” ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 549 (1999).
  41. [41] Khotanzad, A. and Hong, Y. H., “Invariant image recognition by Zernike moments,” in IEEE Trans. on Pattern Analysis and Machine Intelligence, May (1990).
  42. [42] Kim, H. J., Lee, J. S., Jun, S. B., Song, J. M., Lee, H. Y., “Descriptor for quantized color using HMMD color model,” ISO/IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 669 (1999).
  43. [43] Kim, J. D. and Kim, H. K., “Shape descriptor based on multi-layer eigen vector,” ISO/IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 517 (1999).
  44. [44] Kim, W. Y. and Kim, Y. S., “A rotation invariant geometric shape descriptor using Zernike moment,” ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 687 (1999).
  45. [45] Kim, Y.-S. and Kim, W.-Y., “Content-based trademark retrieval system using visually salient feature,” in Journal of Image and Vision Computing, Aug. (1998).
  46. [46] Lawrance, S. and Giles, C. L., “Searching the Web: general and scientific information access,” in IEEE Communication Magazine, Jan. (1999).
  47. [47] Lennon, A. and Wan, E., “Dynamic description framework,” ISO/IEC /JTC/SC29/WG11, Lancaster, UK, Feb. pp. 487 (1999).
  48. [48] Li, C. S. and Stone, H. S., “Digital library using next generation internet,” in IEEE Commu-nication Magazine, Jan. (1999).
  49. [49] Lipson, P., Grimson, W. E. L. and Sinha, P., “Configuration based scene classification and image indexing,” in IEEE Conference on Computer Vision and Pattern Recognition, (1997).
  50. [50] Liu, F. and Picard, R., “Periodicity, directionality and randomness: Wold features for image modeling and retrieval,” in IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 18, No. 7, Jul. (1996).
  51. [51] Mahmood, T. S., “Location hashing: an efficient indexing method for object queries in image databases,” ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 144 (1999).
  52. [52] Manjunath, B. S. and Ma, W., “Texture features for browsing and retrieval of image data,” in IEEE Trans. on Pattern Analysis and Machine Intelligence, Aug. (1996).
  53. [53] Medasani, S. and Krishnapuram, R., “Categorization of image databases for efficient retrieval using robust mixture decomposition,” in IEEE CVPR'98 Workshop on Content-Based Access of Images and Video Libraries, (1998).
  54. [54] Minka, T. P. and Picard, R. W., “Interactive learning with a `society of models”, in Pattern Recognition, 30(4), pp. 565-581, Apr. (1997).
  55. [55] Mintzer, F., “Developing digital library of cultural content for Internet access,” in IEEE Communication Magazine, Jan. (1999).
  56. [56] Mottaleb, M. A., “A descriptor for the edges in still images,” ISO/IEC /JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 649 (1999).
  57. [57] “MPEG-7 application document V.8,” ISO/IEC JTC1/SC29/WG11 N2728, Seoul, Korea, Mar. (1999).
  58. [58] “MPEG-7 Context, objectives and technical roadmap V.11,” ISO/IEC JTC1/SC29/WG11 N2729, Seoul, Korea, Mar. (1999).
  59. [59] “MPEG-7 requirements document V.8,” ISO/IEC JTC1/SC29/WG11 N2727, Seoul, Korea, Mar. (1999).
  60. [60] Muller, K. and Ohm, J. R., “Wavelet-based contour descriptor,” ISO/IEC/JTC1/ SC29/WG11, Lancaster, UK, Feb. pp. 567 (1999).
  61. [61] Nevel, A. V., “Texture synthesis via matching first and second order statistics of a wavelet frame decomposition,” in IEEE International Conference on Image Processing, (1998).
  62. [62] Niblack, W., Berber, R., Equitz, W., “Flickner M., Glasman E., Petkovic D., and Yanker P., The QBIC project: querying images by content using color, texture and shape”, in SPIE Storage and Retrieval for Image and Video Database II, Feb. (1993).
  63. [63] Niblack, W., Zhu, X., Hafner, J., Breuel, T., Pondeleon, D., Petkovic, D., Flickner, M., Upfae, L., Nin, S., Sull, S., Dom, B., Yeo, B. L., “Srinivasan S., Zivkovic D., and Penner M., Updates to the QBIC system,” in SPIE Storage and Retrieval for Image and Video Database VI, Jan. (1998). 
  64. [64] Ohm, J. R., Makai, B., ISO/IEC /JTC1/ SC29/WG11 Lancaster, UK, Feb. pp. 563-564 (1999).
  65. [65] Ohm, J. R. and Bunjamin, F., ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 566 (1999).
  66. [66] “Open knowledge base connectivity home page,”
  67. [67] Ostermann, J., Rajendran, R. K., Puri, A., Huang, Q., ISO/IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 472 (1999).
  68. [68] Paek, S., Benitez, A., Chang, S. F., Li, C. S., Smith, J. R., Bergman, L. D., Puri, A., Swain, C. and Ostermann, J., ISO/IEC/JTC1/ SC29/ WG11, Lancaster, UK, Feb. pp. 480 (1999).
  69. [69] Paul, E., Vet, V. and Mars, N. J., “Bottom-up construction of ontologies,” in IEEE Trans. on Knowledge and Data Engineering, (1998).
  70. [70] Pratt, W. K., Digital Image Processing, 2nd edition, a Wiley-Interscience Publication, (1991).
  71. [71] Puri, A., Huang, Q., Smith, J. R., Kim, M. C., Mohan, R., Li, C. S., Bergman, L. D., Eleftheriadis, A., Benetiz, A. B., Fang, Y., Rajendran, R. K. and Chang, S. F., “MPEG multimedia language (MML): a proposal for MPEG-7 DDL,” ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 484 (1999).
  72. [72] Rao, A. R. and Lohse, G. L., “Towards a texture naming system identifying relevant dimensions of texture,” in IEEE Conference on Visualization, Oct. (1993).
  73. [73] Ratan, A. L. and Grimson, W. E. L., “Training templates for scene classification using a few examples,” in IEEE CVPR'97 Workshop on Content-Based Access of Image and Video Libraries, (1997).
  74. [74] “Report of the ad-hoc group on MPEG-7 evaluation logistics,” ISO/IEC/JTC1/SC29/WG11/ MPEG99/ Wxxxx, Lancaster, UK, Feb. (1999).
  75. [75] “Resource description framework (RDF) schema specification,” WD-rdf-schema- (1998)1030, W3C Working Draft, Oct. (1998).
  76. [76] Ricoh Company Ltd., “MINDS's descriptors for still images - spatial edge distribution descriptor,” ISO/IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 102 (1999).
  77. [77] Ricoh Company Ltd., “MINDS's descriptors for still images - spatial texture distribution descriptor,” ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 104 (1999).
  78. [78] Ro, Y. M., Kim, S. Y., You, K. W., Kim, M. and Kim, J., “Texture description using atoms of matching pursuits,” ISO/IEC JTC1/SC29/ WG11, Lancaster, UK, Feb. pp. 612 (1999).
  79. [79] Ro, Y. M., “Matching pursuit: contents featuring and image indexing,” in SPIE Multimedia storage and archiving system III, (1998).
  80. [80] Rui, Y., Huang, T. and Mehrotra, S., “Content-based image retrieval with relevance feedback in MARS,” in IEEE International Conference on Image Processing, pp. 815-818, Oct. (1997).
  81. [81] Rui, Y., Huang, T. S. and Chang, S. F., “Image retrieval: current techniques, promising directions, and open issues,” in Journal of Visual Communication and Image Representation, Vol. 10, No. 1, Mar. (1999).
  82. [82] Salton, G. and Araya, J., “On the use of clustered file organization in information search and retrieval,” in Tech. Rep. TR89-989, Department of Computer Science, Cornell University, (1989).
  83. [83] “Schema for object-oriented XML (SOX) ,” NOTE-SOX-(1998)0930, Submission to W3C, Sep. (1998).
  84. [84] Sharma, R., Pavlovic, V. I. and Huang, T. S., “Toward multimodal human-computer interface,” in Proceedings of the IEEE, May (1998).
  85. [85] Silberschatz, A., Korth, H. F. and Sudarshan, S., Database System Concepts, 3rd Edition, McGraw-Hill Publisher, (1996).
  86. [86] Simoncelli, E. P. and Portilla, J., “Texture characterization via joint statistics of wavelet coefficient magnitudes,” in IEEE International Conference on Image Processing, (1998).
  87. [87] Smith, J. R. and Castelli, V. and Li, C. S., “Adaptive storage and retrieval of large compressed images,” in SPIE Storage and Retrieval for Image and Video Databases VII, Jan. (1999).
  88. [88] Smith, J. R. and Chang, S. F., “An image and video search engine for the World-Wide Web,” in ACM Multimedia'96, (1996).
  89. [89] Smith, J. R. and Chang, S.F., “Joint adaptive space and frequency basis selection,” in IEEE International Conference on Image Processing, Oct. (1997).
  90. [90] Smith, J. R. and Li, C. S., ISO/IEC JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 483 (1999).
  91. [91] Smith, J. R., “Query vector projection access method,” in SPIE Storage and Retrieval for Image and Video Databases VII, Jan. (1999).
  92. [92] “Squire D. M., Learning a similarity-based distance measure for image database organization from human partitionings of an image set,” in SPIE Multimedia Storage and Archiving System III, (1998).
  93. [93] “Synchronized multimedia integration language (SMIL) ,” WD-smil-0202, W3C Working Draft, Feb. (1998).
  94. [94] Szummer, M. and Picard, R. W., “Indoor-outdoor image classification,” in IEEE CVPR'98 Workshop on Content-Based Access of Image and Video Libraries, (1998).
  95. [95] Tabatabai, A., “Color representation for visual objects,” ISO/IEC JTC1/SC29/ WG11, Lancaster, UK, Feb. pp. 576 (1999).
  96. [96] Tamura, H., Mori, S. and Yamawaki, T., “Textural features corresponding to visual perception,” in IEEE Trans. on Sys. Man., and Cyber., Jun. 1978.
  97. [97] Tao, B. and Dickinson, B. W., “Recognition and retrieval of textured images using gradient indexing,” in IEEE International Conference on Image Processing, (1998).
  98. [98] Tektronix Inc., “Normalized contour as a shape descriptor for visual objects,” ISO/IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 579 (1999).
  99. [99] Vellaikal, A. and Kuo, C. C. J., “Hierarchical clustering techniques for image database organization and summarization,” in SPIE Multimedia Storage and Archiving System III, Nov. (1998).
  100. [100] Wan, X. and Kuo, C. C. J., “Image retrieval based on JPEG compressed data,” in SPIE Multimedia Storage and Archiving Systems, Nov. (1996).
  101. [101] Weiss, R., Velez, B., Sheldon, M., Namprempre, C., Szilagyi, P., Duda, A., and Gifford, D., “Hy-Pursuit: A hierarchical network search engine that exploits content-link hypertext clustering,” in ACM Hypertext'96, (1996).
  102. [102] Won, C. S., et al, “Efficient color feature extraction in compressed video,” in SPIE Storage and Retrieval for Image and Video Databases VIII, (1999).
  103. [103] Won, C. S., Park, D. K., Yoo, S. J., Park, S. J., “Generalized image histogram,” ISO/ IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 324 (1999).
  104. [104] Wong, S. and Tjandra, D., “A digital library for biomedical imaging on the Internet,” in IEEE Communication Magazine, Jan. (1999).
  105. [105] Wood, M., Campbell, N. and Thomas, B., “Iterative refinement by relevance feedback in content-based digital image retrieval,” in ACM Multimedia'98, Bristol, UK, (1998).
  106. [106] Wu, P., Ma, W.Y., Manjunath, B. S., Shin, H. and Choi, Y., “Texture descriptor,” ISO/IEC/JTC1/SC29/WG11, Lancaster, UK, Feb. pp. 77 (1999).
  107. [107] Lamdan, Y. and Wolfson, H. J., “Geometric hashing: a general and efficient model-based recognition scheme,” in IEEE Conference on Computer Vision and Pattern Recognition, (1988).
  108. [108] Yang, Z. and Kuo, C. C. J., “A semantic classification and composite indexing approach to robust image retrieval,” in IEEE International Conference on Image Processing, Oct. (1999).
  109. [109] Yang, Z. and Kuo, C. C. J., “Content- based image retrieval via adaptive multi-feature templates,” in SPIE Multimedia Storage and Archiving System IV, Sep. (1999).
  110. [110] Yang, Z. and Kuo, C. C. J., “Intelligent image database indexing and query system,” in SPIE Application of Digital Image Processing XXII, Jul. (1999).
  111. [111] Yang, Z., Wan, X., and Kuo, C. C. J., “Interactive image retrieval: concept, procedure and tools,” in IEEE 32nd Asilomar Conference, Montery, CA, Nov. (1998).
  112. [112] Yu, H. H. and Wolf, W., “Scenic classification methods for image and video databases,” in SPIE Digital Image Storage and Archiving Systems, (1995).