论文标题

计算时间:野外不可知论的视频重复计数

Counting Out Time: Class Agnostic Video Repetition Counting in the Wild

论文作者

Dwibedi, Debidatta, Aytar, Yusuf, Tompson, Jonathan, Sermanet, Pierre, Zisserman, Andrew

论文摘要

我们提出了一种估计视频中重复操作的时期的方法。该方法的症结在于限制使用时间相似性作为中间表示瓶颈的时期预测模块,从而使概括在野外视频中无法看到。我们使用合成数据集训练这个名为Repnet的模型,该模型是通过对大型视频集合而生成的合成数据集,该模型是通过对长度的短片段进行采样,并使用不同的时期和计数重复它们。合成数据和强大但受约束的模型的这种组合使我们能够以类不足的方式预测周期。我们的模型大大超过了现有周期性(Pertube)和重复计数(QUVA)基准的最先进的状态。我们还收集了一个名为Countix的新挑战性数据集(比现有数据集大的90倍),该数据集捕获了现实世界视频中重复计数的挑战。项目网页:https://sites.google.com/view/repnet。

We present an approach for estimating the period with which an action is repeated in a video. The crux of the approach lies in constraining the period prediction module to use temporal self-similarity as an intermediate representation bottleneck that allows generalization to unseen repetitions in videos in the wild. We train this model, called Repnet, with a synthetic dataset that is generated from a large unlabeled video collection by sampling short clips of varying lengths and repeating them with different periods and counts. This combination of synthetic data and a powerful yet constrained model, allows us to predict periods in a class-agnostic fashion. Our model substantially exceeds the state of the art performance on existing periodicity (PERTUBE) and repetition counting (QUVA) benchmarks. We also collect a new challenging dataset called Countix (~90 times larger than existing datasets) which captures the challenges of repetition counting in real-world videos. Project webpage: https://sites.google.com/view/repnet .

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源