With Self-Refine, LLMs can generate feedback on their work, use it to improve the output, and repeat this process.

Table of Contents

• Updates
• Setup
• Getting Started with Acronym Generation
• Dialogue Response Generation
• Code Readability Improvement
• Commongen
• GSM-8k
• Yelp
• PIE
• General setup
• Citation

• Nov 2023 : Added visual self-refine examples and colabs . Use GPT4-V to write tikz code for diagrams, and improve them iteratively.

Stokes' Theorem Example

Unicorn Example

• We use prompt-lib for querying LLMs. You can install prompt-lib by running the following commands:

git clone https://github.com/reasoning-machines/prompt-lib
pip install prompt-lib/

Depending on your default settings, you may want to use the following to set a PYTHONPATH:

export
 PYTHONPATH=
"
.:../:.:src:../:../../:.:prompt-lib
"

python -u src/acronym/run.py 
"
Using language models of code for few-shot commonsense
"

• Since we sample with temperature 0.7, you'll get different results each time you run the above command.

0
 INIT
>
 Using language models of code for few
-
shot commonsense

0
 GEN
>
 CLoCK
0
 SCORES
>
 *
 Ease of pronunciation
:
 CLoCK is 
pronounced
 "
clah-k
"
.
 This is 
easy
 acronym to pronounce
.
 4
/
5
 
*
 Ease of spelling
:
 CLoCK is 
easy
 to spell
.
 5
/
5
 
*
 Relation to title
:
 CLoCK stands for 
"
Code Language Models of Commonsense Knowledge
"
 which is 
related
 to the title
.
 5
/
5
 
*
 Positive connotation
:
 CLoCK is 
a
 positive acronym
.
 It
 implies accuracy and efficiency
.
 5
/
5
 
*
 Well
-
known
:
 CLoCK is 
not
 a well
-
known acronym
.
 2
/
5


*
 Total score
:
 21
/
25


1
 GEN
>
 CMNSC
1
 SCORES
>
 *
 Ease of pronunciation
:
 CMNSC is 
pronounced
 "
see-min-es-see
"
.
 This is 
a
 difficult acronym to pronounce
.
 1
/
5
 
*
 Ease of spelling
:
 CMNSC is 
difficult
 to spell
.
 1
/
5
 
*
 Relation to title
:
 CMNSC stands for 
"
Commonsense
"
 which is 
related
 to the title
.
 4
/
5
 
*
 Positive connotation
:
 CMNSC is 
a
 neutral acronym
.
 3
/
5
 
*
 Well
-
known
:
 CMNSC is 
not
 a well
-
known acronym
.
 1
/
5


*
 Total score
:
 10
/
25


2
 GEN
>
 COMMIT
2
 SCORES
>
 *
 Ease of pronunciation
:
 COMMIT is 
pronounced
 "
com-mit
"
.
 This is 
an
 easy acronym to pronounce
.
 4
/
5

*
 Ease of spelling
:
 COMMIT is 
easy
 to spell
.
 5
/
5

*
 Relation to title
:
 COMMIT stands for 
"
Commonsense Model of Code for Intuitive Training
"
 which is 
related
 to the title
.
 5
/
5
 
*
 Positive connotation
:
 COMMIT is 
a
 positive acronym
.
 It
 implies commitment and dedication
.
 5
/
5

*
 Well
-
known
:
 COMMIT is 
not
 a well
-
known acronym
.
 2
/
5


*
 Total score
:
 21
/
25

PYTHONPATH=
"
.
"
 python -u src/responsegen/run.py --output 
<
OUTPUT FILE
>
 --size 
<
DATA SIZE
>

• Use size 0 for running on all test instances

• Note: Please unzip 'data/tasks/codeclean/code_readability/codenet-python-train.jsonl.zip' before running the following commands!

• Running:

PYTHONPATH=
"
.
"
 python -u src/readability/readability.py --output 
<
OUTPUT FILE
>

• Evaluation:

PYTHONPATH=
"
.
"
 python -u src/readability/{count_comment
|
count_function
|
count_meaningful_var}.py --file 
<
INPUT FILE
>

• We use a hard version of commongen. The data is located in data/prompt/commongen . You can download the data by running the following commands:

python -u src/commongen/run.py cmd stair bubble team dryer puppy aliens cat 

• To run the GSM-8k task:

python -u src/gsm/run.py 

• The outputs will be saved in data/tasks/gsm/gsm_outputs.jsonl

• To evaluate the outputs:

python src/gsm/gsm_selfref_eval.py --path  data/tasks/gsm/gsm_outputs.jsonl

• The evaluation script will also generate a report ( data/tasks/gsm/gsm_outputs.jsonl.reports.txt ) showing examples of wrong generations, feedback, and refined feedback generations.

• To run the Yelp task:

python -u src/sentiment_transfer_sr/run.py data/tasks/yelp/yelp-extreme.jso
nl 4 none

• The outputs will be saved in data/tasks/yelp/

• To run the PIE task:

python -u src/pie/run.py --slow_programs_file data/tasks/pie/codenet-python-test-1k.jsonl --max_attempts 4 --outfile data/tasks/pie/output --feedback_type rich

• For evaluation details, please see docs/pie_eval.md .

• Each task has three different types of prompts:

1. Init : used to initialize the task. This is how the initial output is generated.

2. Feedback : used to get feedback from the model on the intermediate results.

3. Iterate : used to get the next iteration from the model, based on the feedback.

• Every task has a run.py that initializes the prompts and runs the task.

• As an example, the prompts for commongen are as follows:

1. Init prompt:

python src/commongen/task_init.py

2. Feedback prompt:

 python src/commongen/feedback.py

3. Iterate prompt:

python src/commongen/task_iterate.py

You can also see these prompts on our website .

@misc{madaan2023selfrefine,
      title
=
{Self
-
Refine: Iterative Refinement with Self
-
Feedback}, 
      author
=
{Aman Madaan 
and
 Niket Tandon 
and
 Prakhar Gupta 
and
 Skyler Hallinan 
and
 Luyu Gao 
and
 Sarah Wiegreffe 
and
 Uri Alon 
and
 Nouha Dziri 
and
 Shrimai Prabhumoye 
and
 Yiming Yang 
and
 Sean Welleck 
and
 Bodhisattwa Prasad Majumder 
and
 Shashank Gupta 
and
 Amir Yazdanbakhsh 
and
 Peter Clark},
      year
=
{
2023
},
      eprint
=
{
2303
.
17651
},
      archivePrefix
=
{arXiv},
      primaryClass
=
{
cs
.
CL
}
}

flowchart LR
    Generator -->|Initializes| Unrefined
    Critic_1 --> Critique_fb
    ... --> Critique_fb
    Critic_k --> Critique_fb
    Critique_fb --> Unrefined{Output to Refine}
    Unrefined --> Refiner
    Refiner --> |R: y_t, x, fb| Refined_Output{Refined Output}
    Refined_Output --> |Stopping Criteria Not Met| Unrefined