Yupeng Hou, Zhipeng Chen, Yingqian Min, Jinhao Jiang, Beichen Zhang, Ruiyang Ren, Wayne Xin Zhao, Junjie Zhang, Yifan Li, Kun Zhou, Zican Dong, Xinyu Tang, Junyi Li, Yifan Du, Zikang Li, Tianyi Tang, Chen Yang, Peiyu Liu, Xiaolei Wang, Yushuo Chen
Yupeng Hou, Zhipeng Chen, Yingqian Min, Jinhao Jiang, Beichen Zhang, Ruiyang Ren, Wayne Xin Zhao, Junjie Zhang, Yifan Li, Kun Zhou, Zican Dong, Xinyu Tang, Junyi Li, Yifan Du, Zikang Li, Tianyi Tang, Chen Yang, Peiyu Liu, Xiaolei Wang, Yushuo Chen
This survey highlights the complexity of language and the challenge of developing AI algorithms capable of understanding and generating language. Over the past two decades, language modelling has evolved from statistical models to neural models, with recent advances in pre-trained language models (PLMs) that use Transformer models and large-scale corpora for improved language understanding and generation.