PAPER DIGEST
Most Influential EMNLP 2023 Paper · 2026-03 edition

Large Language Models Can Self-Improve

Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, Jiawei Han

Venue
Conference on Empirical Methods in Natural Language Processing (EMNLP) 2023
Recognition
Most Influential EMNLP 2023 Paper (Rank No. 7)
Edition
2026-03
Impact factor
7
Certificate ID
2957a658f1d550c4

Abstract

Large Language Models (LLMs) have achieved excellent performances in various tasks. However, fine-tuning an LLM requires extensive supervision. Human, on the other hand, may improve their reasoning abilities by self-thinking without external inputs. In this work, we demonstrate that an LLM is also capable of self-improving with only unlabeled datasets. We use a pre-trained LLM to generate �high-confidence� rationale-augmented answers for unlabeled questions using Chain-of-Though (CoT) prompting and self-consistency, and fine-tune the LLM using those self-generated solutions as target outputs. We show that without any ground truth label, our approach improves the general reasoning ability of a 540B-parameter LLM (74. 4%?82. 1% on GSM8K, 90. 0%?94. 4% on OpenBookQA, and 63. 4%?67. 9% on ANLI-A3) and can also be adapted to extreme low-resource cases where even training questions and CoT prompts are limited. We conduct ablation studies and show that fine-tuning on diverse reasoning paths is critical for self-improvement.

Download PDF certificate