Chen Yang, Peiyu Liu, Zikang Li, Xinyu Tang, Yifan Li, Ruiyang Ren, Jinhao Jiang, Zhipeng Chen, Yushuo Chen, Wayne Xin Zhao, Yifan Du, Zican Dong, Junjie Zhang, Beichen Zhang, Yingqian Min, Yupeng Hou, Xiaolei Wang, Tianyi Tang, Junyi Li, Kun Zhou
Chen Yang, Peiyu Liu, Zikang Li, Xinyu Tang, Yifan Li, Ruiyang Ren, Jinhao Jiang, Zhipeng Chen, Yushuo Chen, Wayne Xin Zhao, Yifan Du, Zican Dong, Junjie Zhang, Beichen Zhang, Yingqian Min, Yupeng Hou, Xiaolei Wang, Tianyi Tang, Junyi Li, Kun Zhou
This survey highlights the complexity of language and the challenge of developing AI algorithms capable of understanding and generating language. Over the past two decades, language modelling has evolved from statistical models to neural models, with recent advances in pre-trained language models (PLMs) that use Transformer models and large-scale corpora for improved language understanding and generation.