Acta Infologica

SUBMITINFORMATION
EnglishTürkçe

Review Article


DOI :10.26650/acin.765320   IUP :10.26650/acin.765320    Full Text (PDF)

A Survey on Image Super-Resolution with Generative Adversarial Networks

Hürkal Hüsem, Zeynep Orman

Super-resolution is a process to increase image dimensions with a specific upscaling factor while trying to preserve details that matche with the original high-resolution form. Super-resolution can be done with many techniques. But the most effective technique is the one that takes advantage of several neural network designs. Some network designs are more appropriate than others on the specific subject. This study focuses on super resolution studies using Generative Adversarial Network. Many studies use this neural network type to look at various topics such as artificial data production and making the data more meaningful. The key point of this neural network type is having two different sub-networks that try to defeat each other in order to make more realistic results. Performance metrics that measure the quality of a generated image, loss functions used in a neural network and research papers on super-resolution with Generative Adversarial Network are the main domains of this study. 

Keywords: Image Super-Resolution, Generative Adversarial Networks, Resolution Enhancement
DOI :10.26650/acin.765320   IUP :10.26650/acin.765320    Full Text (PDF)

Üretken Çekişmeli Ağlar ile Görsel Çözünürlük Artırımı Üzerine Bir Araştırma

Hürkal Hüsem, Zeynep Orman

Super-resolution is a process to increase image dimensions with a specific upscaling factor while trying to preserve details that matche with the original high-resolution form. Super-resolution can be done with many techniques. But the most effective technique is the one that takes advantage of several neural network designs. Some network designs are more appropriate than others on the specific subject. This study focuses on super resolution studies using Generative Adversarial Network. Many studies use this neural network type to look at various topics such as artificial data production and making the data more meaningful. The key point of this neural network type is having two different sub-networks that try to defeat each other in order to make more realistic results. Performance metrics that measure the quality of a generated image, loss functions used in a neural network and research papers on super-resolution with Generative Adversarial Network are the main domains of this study. 

Keywords: Image Super-Resolution, Generative Adversarial Networks, Resolution Enhancement

PDF View

References

  • Agustsson, E., & Timofte, R. (2017). Ntire 2017 challenge on single image super-resolution: Dataset and study. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops(126-135). google scholar
  • Bai, Y., Zhang, Y., Ding, M., & Ghanem, B. (2018). Finding tiny faces in the wild with generative adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. google scholar
  • Bevilacqua, M., Roumy, A., Guillemot, C., & Alberi-Morel, M. L. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding. Proceedings of the 23rd British Machine Vision Conference (BMVC). google scholar
  • Bin, H., Weihai, C., Xingming, W., & Chun-Liang, L. (2017). High-quality face image generated with conditional boundary equilibrium generative adversarial networks. Pattern Recognition Letters. google scholar
  • Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., & Zelnik-Manor, L. (2018). The 2018 pirm challenge on perceptual image super-resolution. Proceedings of the European Conference on Computer Vision (ECCV). google scholar
  • Bulat, A., & Tzimiropoulos, G. (2017). How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). International Conference on Computer Vision. google scholar
  • Bulat, A., Yang, J., & Tzimiropoulos, G. (2018). To learn image super-resolution, use a gan to learn how to do image degradation first. Proceedings of the European conference on computer vision (ECCV), (pp. 185-200). google scholar
  • Caltech Pedestrian Detection Benchmark. (2019, 12 23). Retrieved from http://www.vision.caltech.edu/Image_Datasets/CaltechPedestrians/ google scholar
  • Dataset, M. -J. (2019, 12 23). Retrieved from http://www.manga109.org/en/ google scholar
  • Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. IEEE Conference on Computer Vision and Pattern Recognition. google scholar
  • Dong, C., Loy, C., & Tang, X. (2016, 12 23). Accelerating the Super-Resolution Convolutional Neural Network. European Conference on Computer Vision (ECCV). google scholar
  • Dosselmann, R., & Yang, X. D. (2005). Existing and emerging image quality metrics. Canadian Conference on Electrical and Computer Engineering. google scholar
  • Gerchberg, R. W. (1974). Super-resolution through error energy reduction. Optica Acta: International Journal of Optics, 21(9), 709-720. google scholar
  • Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. International Conference on Learning Representations. google scholar
  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 2672-2680. google scholar
  • Gotoh, T., & Okutomi, M. (2004). Direct super-resolution and registration using raw CFA images. Computer Vision and Pattern Recognition (CVPR), 2. google scholar
  • Gupta, A., Vedaldi, A., & Zisserman, A. (2016). Synthetic data for text localisation in natural images. IEEE Conference on Computer Vision and Pattern Recognition. google scholar
  • Hradiš, M., Kotera, J., Zemcık, P., & Šroubek, F. (2015). Convolutional neural networks for direct text deblurring. Proceedings of BMVC, 10(2). google scholar
  • Huang, J. B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. google scholar
  • Huynh-Thu, Q., & Ghanbari, M. (2008). Scope of validity of PSNR in image/video quality assessment. 44(13), 800-801. Electronics letters. google scholar
  • ITU-T. (2006). Rec. P.10: Vocabulary for performance and quality of service. google scholar
  • Jaderberg, M., Simonyan, K., Vedaldi, A., & Zisserman, A. (2015). Deep structured output learning for unconstrained text recognition. International Conference on Learning Representations (ICLR). google scholar
  • Johnson, J., Alahi, A., & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. European conference on computer vision. Kaggle - T91 Image Dataset. (2019, 12 23). Retrieved from https://www.kaggle.com/ll01dm/t91-image-dataset google scholar
  • Karatzas, D., Gomez-Bigorda, L., Nicolaou, A., Ghosh, S., Bagdanov, A., Iwamura, M., & Shafait, F. e. (2015). ICDAR 2015 competition on robust reading. International Conference on Document Analysis and Recognition (ICDAR). google scholar
  • Karatzas, D., Shafait, F., Uchida, S., Iwamura, M., i Bigorda, L. G., & Mestre, S. R. (2013). ICDAR 2013 robust reading competition. International Conference on Document Analysis and Recognition. google scholar
  • Kingma, D. P., & Welling, M. (2014). Auto-Encoding Variational Bayes. International Conference on Learning. Representations. google scholar
  • Kingma, D. P., & Welling, M. (2019). An Introduction to Variational Autoencoders. Foundations and Trends® in Machine Learning, 12(4), 307-392. Large-scale CelebFaces Attributes (CelebA) Dataset. (2019, 12 23). Retrieved from http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html google scholar
  • Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., & Shi, W. (2016). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 4681-4690). google scholar
  • Li, J., Liang, X., Wei, Y., Xu, T., Feng, J., & Yan, S. (2017). Perceptual generative adversarial networks for small object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. google scholar
  • Liu, W., Liu, X., Ma, H., & Cheng, P. (2017). Beyond human-level license plate super-resolution with progressive vehicle search and domain priori GAN. Proceedings of the 25th ACM international conference on Multimedia, (pp. 1618-1626). google scholar
  • Liu, X., Liu, W., Mei, T., & Ma, H. (2016). A deep learning-based approach to progressive vehicle re-identification for urban surveillance. European conference on computer vision, (pp. 869-884). google scholar
  • Lucas, S. M., Panaretos, A., Sosa, L., Tang, A., Wong, S., Young, R., . . . Lin, X. (2005). ICDAR 2003 robust reading competitions: entries, results, and future directions. International Journal of Document Analysis and Recognition (IJDAR), 2(105-122), 7. google scholar
  • Ma, W., Pan, Z., Guo, J., & Lei, B. (2018). Super-resolution of remote sensing images based on transferred generative adversarial network. IGARSS 2018- 2018 IEEE International Geoscience and Remote Sensing Symposium. google scholar
  • Mishra, A., Alahari, K., & Jawahar, C. V. (2012). Top-down and bottom-up cues for scene text recognition. IEEE Conference on Computer Vision and Pattern Recognition. google scholar
  • Mjolsness, E. (1985). Neural networks, pattern recognition, and fingerprint hallucination. Diss. California Institute of Technology. google scholar
  • Nasrollahi, K., & Moeslund, T. B. (2014). Super-resolution: a comprehensive survey. Machine vision and applications, 25(6), 1423-1468. google scholar
  • Park, S. J., Son, H., Cho, S., Hong, K. S., & Lee, S. (2018). Srfeat: Single image super-resolution with feature discrimination. European Conference on Computer Vision (ECCV). google scholar
  • Phan, T., Shivakumara, P., Tian, S., & Tan, C. (2013). Recognizing text with perspective distortion in natural scenes. International Conference on Computer Vision. google scholar
  • Protter, M., Elad, M., Takeda, H., & Milanfar, P. (2008). Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Transactions on image processing, 18(1), 36-51. google scholar
  • PSNR. (2020, 7 6). Retrieved 11 23, 2019, from MathWorks: https://www.mathworks.com/help/vision/ref/psnr.html google scholar
  • Risnumawan, A., Shivakumara, P., Chan, C. S., & Tan, C. L. (2014). A robust arbitrary text detection system for natural scene images. Expert Systems with Applications, 18(8027-8048), 41. google scholar
  • Sajjadi, M. S., Scholkopf, B., & Hirsch, M. (2017). Enhancenet: Single image super-resolution through automated texture synthesis. International Conference on Computer Vision (ICCV). google scholar
  • Shi, B., Yang, M., Wang, X., Lyu, P., Yao, C., & Bai, X. (2018). Aster: An attentional scene text recognizer with flexible rectification. IEEE transactions on pattern analysis and machine intelligence, 41(9), 2035-2048. google scholar
  • The Berkeley Segmentation Dataset and Benchmark. (2019, 12 23). Retrieved from https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/ google scholar
  • Timofte, R., Agustsson, E., Van Gool, L., Yang, M. H., & Zhang, L. (2017). Ntire 2017 challenge on single image super-resolution: Methods and results. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 114-125. google scholar
  • Traffic-Sign Detection and Classification in the Wild. (2019, 12 23). Retrieved from https://cg.cs.tsinghua.edu.cn/traffic-sign/ google scholar
  • UC Merced Land Use Dataset. (2019, 12 23). Retrieved from http://weegee.vision.ucmerced.edu/datasets/landuse.html google scholar
  • Wang, K., Babenko, B., & Belongie, S. (2011). End-to-end scene text recognition. International Conference on Computer Vision. google scholar
  • Wang, W., Xie, E., Sun, P., Wang, W., Tian, L., Shen, C., & Luo, P. (2019). TextSR: Content-Aware Text Super-Resolution Guided by Recognition. arXiv preprint. google scholar
  • Wang, X., Yu, K., Dong, C., & Change Loy, C. (2018). Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. google scholar
  • Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., & Change Loy, C. (2018). ESRGAN: Enhanced super-resolution generative adversarial networks. Proceedings of the European Conference on Computer Vision. google scholar
  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612. google scholar
  • Wu, B., Duan, H., Liu, Z., & Sun, G. (2017). Srpgan: Perceptual generative adversarial network for single image super resolution. arXiv preprint. google scholar
  • Xie, Y., Franz, E., Chu, M., & Thuerey, N. (2018). tempoGAN: A temporally coherent, volumetric gan for super-resolution fluid flow. ACM Transactions on Graphics (TOG), 37(4), 1-15. google scholar
  • Xu, X., Sun, D., Pan, J., Zhang, Y., Pfister, H., & Yang, M. H. (2017). Learning to super-resolve blurry face and text images. Proceedings of the IEEE International Conference on Computer Vision. google scholar
  • Yang, S., Luo, P., Loy, C. C., & Tang, X. (2016). WIDER FACE: A Face Detection Benchmark. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). google scholar
  • Zhang, D., Shao, J., Hu, G., & Gao, L. (2017). Sharp and real image super-resolution using generative adversarial network. International Conference on Neural Information Processing, (pp. 217-226). google scholar

Citations

Copy and paste a formatted citation or use one of the options to export in your chosen format


EXPORT



APA

Hüsem, H., & Orman, Z. (2020). A Survey on Image Super-Resolution with Generative Adversarial Networks. Acta Infologica, 4(2), 139-154. https://doi.org/10.26650/acin.765320


AMA

Hüsem H, Orman Z. A Survey on Image Super-Resolution with Generative Adversarial Networks. Acta Infologica. 2020;4(2):139-154. https://doi.org/10.26650/acin.765320


ABNT

Hüsem, H.; Orman, Z. A Survey on Image Super-Resolution with Generative Adversarial Networks. Acta Infologica, [Publisher Location], v. 4, n. 2, p. 139-154, 2020.


Chicago: Author-Date Style

Hüsem, Hürkal, and Zeynep Orman. 2020. “A Survey on Image Super-Resolution with Generative Adversarial Networks.” Acta Infologica 4, no. 2: 139-154. https://doi.org/10.26650/acin.765320


Chicago: Humanities Style

Hüsem, Hürkal, and Zeynep Orman. “A Survey on Image Super-Resolution with Generative Adversarial Networks.” Acta Infologica 4, no. 2 (Apr. 2024): 139-154. https://doi.org/10.26650/acin.765320


Harvard: Australian Style

Hüsem, H & Orman, Z 2020, 'A Survey on Image Super-Resolution with Generative Adversarial Networks', Acta Infologica, vol. 4, no. 2, pp. 139-154, viewed 25 Apr. 2024, https://doi.org/10.26650/acin.765320


Harvard: Author-Date Style

Hüsem, H. and Orman, Z. (2020) ‘A Survey on Image Super-Resolution with Generative Adversarial Networks’, Acta Infologica, 4(2), pp. 139-154. https://doi.org/10.26650/acin.765320 (25 Apr. 2024).


MLA

Hüsem, Hürkal, and Zeynep Orman. “A Survey on Image Super-Resolution with Generative Adversarial Networks.” Acta Infologica, vol. 4, no. 2, 2020, pp. 139-154. [Database Container], https://doi.org/10.26650/acin.765320


Vancouver

Hüsem H, Orman Z. A Survey on Image Super-Resolution with Generative Adversarial Networks. Acta Infologica [Internet]. 25 Apr. 2024 [cited 25 Apr. 2024];4(2):139-154. Available from: https://doi.org/10.26650/acin.765320 doi: 10.26650/acin.765320


ISNAD

Hüsem, Hürkal - Orman, Zeynep. “A Survey on Image Super-Resolution with Generative Adversarial Networks”. Acta Infologica 4/2 (Apr. 2024): 139-154. https://doi.org/10.26650/acin.765320



TIMELINE


Submitted07.07.2020
Accepted17.08.2020
Published Online31.12.2020

LICENCE


Attribution-NonCommercial (CC BY-NC)

This license lets others remix, tweak, and build upon your work non-commercially, and although their new works must also acknowledge you and be non-commercial, they don’t have to license their derivative works on the same terms.


SHARE




Istanbul University Press aims to contribute to the dissemination of ever growing scientific knowledge through publication of high quality scientific journals and books in accordance with the international publishing standards and ethics. Istanbul University Press follows an open access, non-commercial, scholarly publishing.