Title
《SpanBERT: Improving Pre-training by Representing and Predicting Spans》
Abstract
We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text. Our approach extends BERT by (1) masking contiguous random spans, rather than random tokens, and (2) training the span boundary representations to predict the entire content of the masked span, without relying on the individual token representations within it. SpanBERT consistently outperforms BERT and our better-tuned baselines, with substantial gains on span selection tasks such as question answering and coreference resolution.
SpanBert相比于Bert更够更好的表示和预测文章的范围。
我们的方法在Bert的基础上mask了连续随机的范围，而非随机的token， 而且训练范围的表示去预测整个文本的masked span，为不是依赖于单个token。
Introduction
