Refinement pass for concision

The synthesis node produces a solid first draft, but it often runs long and repeats itself. A second LLM pass, configured as an editor, trims the draft without touching the citations. Two shorter prompts usually outperform one giant mega-prompt because each pass has a clearer job.

mini-perplexity.ipynb
python
def refinement_node(state):
    state["current_step"] = "refining"

    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.1, max_tokens=1000)

    refinement_prompt = ChatPromptTemplate.from_messages([
        ("system", """You are an editor that refines answers to be more concise, clear, and well-structured.

Requirements:
1. Maintain all citations [1], [2], etc.
2. Improve flow and readability
3. Remove redundant information
4. Ensure factual accuracy
5. Make the answer engaging"""),
        ("human", """Please refine this answer:

QUESTION: {question}

ORIGINAL ANSWER:
{original_answer}

Please provide the refined version:""")
    ])

    try:
        chain = refinement_prompt | llm | StrOutputParser()
        response = chain.invoke({
            "original_answer": state["synthesized_answer"],
            "question": state["question"],
        })
        state["final_answer"] = response
    except Exception as e:
        state["final_answer"] = state["synthesized_answer"]
        print(f"Refinement failed, using original answer: {e}")

    return state

If refinement fails for any reason (rate limit, model error, whatever), we fall back to the synthesized draft. Graceful degradation is the difference between a demo and a pipeline people actually use.

Synthesis and refinement as two passes

Each pass has a narrower job than a single mega-prompt.

The cost is usually less than double because the refinement input is just the draft, not the full scraped context. The input tokens on pass two are a small fraction of pass one. For a research tool where quality matters more than a few cents per query, the tradeoff is almost always worth it. If you need to optimize later, cache draft-to-refined mappings for repeated queries.

Quiz: Quiz

Loading practice…