论文标题
一个新型的变压器网络，带有变化的窗户交叉注意，用于时空天气预测
A Novel Transformer Network with Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting
论文作者
论文摘要
地球天文台是一个不断增长的研究领域，可以在短时间预测（现在是一种现已播出的情况）上利用AI的力量。在这项工作中，我们使用视频变压器网络应对天气预报的挑战。视觉变压器体系结构已在各种应用中进行了探索，主要限制是关注的计算复杂性和饥饿的培训。为了解决这些问题，我们建议使用视频Swin-Transformer，再加上专用的增强计划。此外，我们在编码器侧采用逐渐的空间减少，并在解码器上进行了交叉注意。在Weather4cast2021天气预报挑战数据中测试了提出的方法，该数据需要从每小时的天气产品序列预测未来8小时的未来帧（每小时4个）。将数据集归一化为0-1，以促进使用不同数据集的评估指标。该模型在提供训练数据时导致MSE得分为0.4750，在不使用培训数据的情况下转移学习期间为0.4420。
Earth Observatory is a growing research area that can capitalize on the powers of AI for short time forecasting, a Now-casting scenario. In this work, we tackle the challenge of weather forecasting using a video transformer network. Vision transformer architectures have been explored in various applications, with major constraints being the computational complexity of Attention and the data hungry training. To address these issues, we propose the use of Video Swin-Transformer, coupled with a dedicated augmentation scheme. Moreover, we employ gradual spatial reduction on the encoder side and cross-attention on the decoder. The proposed approach is tested on the Weather4Cast2021 weather forecasting challenge data, which requires the prediction of 8 hours ahead future frames (4 per hour) from an hourly weather product sequence. The dataset was normalized to 0-1 to facilitate using the evaluation metrics across different datasets. The model results in an MSE score of 0.4750 when provided with training data, and 0.4420 during transfer learning without using training data, respectively.
