NIC: Neural Image Coding

Open-source machine learning powered image codec.

View on GitHub

Introduction

NIC aims at exploring the deep neural network based end-to-end solution for learned image compression. It is an on-going work. Current version (NIC-0.1) follow the design of Non-Local Attention optimization and Improved Context modeling-based image compression (NLAIC) [1] with modified Context Model as well as mixed Gaussian probability prediction [2].

Installation

You don't need to manually compile any part of the code. You will need a PyTorch (>=1.3) environment to run the code.
Follow the installation instructions on pytorch.org to install the library.

Usage

Encode

Encode an image with filename INPUT_IMAGE to bit-stream named CODED_BIN with the following command.
You need to specify the model index in the command line to run different models.

$ python inference.py –-encode –i INPUT_IMAGE -o CODED_BIN -m MODEL_INDEX

Decode

Decode from a binary bit-stream CODED_BIN to reconstruct an image named REC_IMAGE with the following command.

$ python inference.py –-decode -i CODEC_BIN -o REC_IMAGE

Performance

Different Methods vs. BPG (Average BD-rate Reduction)

		High Bitrate				Low Bitrate
Methods	Classes	RGB PSNR	RGB MS-SSIM	YUV PSNR	Y MS-SSIM	RGB PSNR	RGB MS-SSIM	YUV PSNR	Y MS-SSIM
VVC	ClassA_6K	-15.1%	-16.0%	-15.9%	-13.1%	-23.6%	-23.1%	-25.1%	-19.0%
	ClassB_4K	-15.3%	-17.2%	-15.9%	-13.6%	-23.7%	-24.7%	-25.3%	-18.8%
	ClassC_2K	-22.4%	-24.1%	-22.6%	-18.9%	-28.8%	-28.2%	-29.6%	-22.8%
	ClassD_Kodak	-19.0%	-18.3%	-20.4%	-14.6%	-23.5%	-20.7%	-27.0%	-16.0%
	overall	-17.9%	-18.9%	-18.7%	-15.1%	-24.9%	-24.2%	-26.8%	-19.1%
NIC-0.1 (MSE)	ClassA_6K	-18.7%	-26.0%	-16.0%	-16.9%	-19.6%	-26.7%	-17.2%	-18.7%
	ClassB_4K	-17.3%	-26.4%	-14.5%	-17.7%	-18.9%	-28.8%	-18.7%	-19.2%
	ClassC_2K	-19.2%	-31.2%	-12.5%	-21.0%	-20.5%	-32.2%	-16.3%	-23.2%
	ClassD_Kodak	-4.2%	-21.5%	-2.6%	-14.0%	-7.3%	-27.7%	-9.8%	-21.8%
	overall	-14.9%	-26.3%	-11.4%	-17.4%	-16.6%	-28.9%	-15.5%	-20.7%
NIC-0.1 (MS-SSIM)	ClassA_6K	32.4%	-53.7%	33.1%	-46.9%	27.9%	-50.1%	33.5%	-45.6%
	ClassB_4K	37.1%	-54.2%	37.3%	-47.6%	37.2%	-51.4%	40.8%	-45.3%
	ClassC_2K	38.2%	-58.3%	52.7%	-48.7%	25.5%	-55.2%	31.7%	-46.8%
	ClassD_Kodak	80.6%	-52.4%	68.3%	-44.4%	69.0%	-50.1%	52.0%	-45.9%
	overall	47.1%	-54.7%	47.8%	-46.9%	39.9%	-51.7%	39.5%	-45.9%

Contributors & Contact

The contributors of this project are listed as follows.
CHEN. Tong, Nanjing University, tong@smail.nju.edu.cn
JIA. Chuanmin, Peking University, cmjia@pku.edu.cn
LIN. Jianping, University of Science and Technology of China, ljp105@mail.ustc.edu.cn
LIU. Feiyang, Wuhan University, liufeiyang@whu.edu.cn
LIU. Haojie, Nanjing University, haojie@smail.nju.edu.cn
MA. Haichuan, University of Science and Technology of China, hcma@mail.ustc.edu.cn
WANG. Yefei, University of Science and Technology of China, wyfei@mail.ustc.edu.cn

Website maintainer
HU. Yueyu, Peking University, huyy@pku.edu.cn

References

[1] Neural Image Compression via Non-Local Attention Optimization and Improved Context Modeling, Tong Chen, Haojie Liu, Zhan Ma, Qiu Shen, Xun Cao, Yao Wang, arXiv preprint arXiv:1910.06244

[2] An End-to-End Joint Learning Scheme of Image Compression and Quality Enhancement with Improved Entropy Minimization, Jooyoung Lee, Seunghyun Cho, Munchurl Kim, arXiv preprint arXiv:1912.12817