Linly Chinese Large Language Model is based on TencentPreTrain, the world's first modular large model pre-training framework (downloads 100,000/month, published in ACL 2023, one of the top natural language processing conferences), and is the first Chinese open source 7B, 13B, and 30B large model.

Its core technologies include: (1) LLaMA Chinese pre-training, instruction fine-tuning and question-answering; (2) language transfer learning with adaptive data sampling; (3) English → Chinese and English parallel corpus training; (4) adaptive adjustment of training ratio to solve the problems of knowledge forgetting and transfer.