论文标题
SuperTran：基于参考的视频变压器，用于实时增强低比特率流
SuperTran: Reference Based Video Transformer for Enhancing Low Bitrate Streams in Real Time
论文作者
论文摘要
这项工作着重于低比特率视频流场景（例如50-200kbps），其中视频质量受到严重损害。我们提出了一个新型的深层生成模型家族，可通过执行超分辨率来增强此类流的感知视频质量，同时还可以消除压缩工件。我们称为SuperTran的模型除了低质量，低分辨率的视频流外，还消耗了单个高质量的高分辨率参考图像。因此，该模型学习了如何借用或复制视觉元素，例如从参考图像中纹理，并从低分辨率流中填写其余细节，以产生感知增强的输出视频。可以在视频会话开始时发送一次参考框，也可以从画廊检索。重要的是，所得的输出具有比仅使用低分辨率输入（例如SuperVegan方法）的方法更好的细节。 SuperTran在云上实时（最多30帧/秒）与标准管道一起工作。
This work focuses on low bitrate video streaming scenarios (e.g. 50 - 200Kbps) where the video quality is severely compromised. We present a family of novel deep generative models for enhancing perceptual video quality of such streams by performing super-resolution while also removing compression artifacts. Our model, which we call SuperTran, consumes as input a single high-quality, high-resolution reference images in addition to the low-quality, low-resolution video stream. The model thus learns how to borrow or copy visual elements like textures from the reference image and fill in the remaining details from the low resolution stream in order to produce perceptually enhanced output video. The reference frame can be sent once at the start of the video session or be retrieved from a gallery. Importantly, the resulting output has substantially better detail than what has been otherwise possible with methods that only use a low resolution input such as the SuperVEGAN method. SuperTran works in real-time (up to 30 frames/sec) on the cloud alongside standard pipelines.
