@blu3mo: Is there a tool where users can set evaluation criteria and LLM generates text based on those criteria, then reflects and regenerates?

  • @blu3mo: I usually do it manually, but after hearing about Constitutional AI, I feel like automating it.

  • Desired Features:

    • Self-refine
      • Users can set evaluation criteria, and LLM generates text based on those criteria, then reflects and regenerates.
    • Human selection
      • Generate multiple variations and allow users to choose the preferred expression.
  • Should I make it? (blu3mo)(blu3mo)(blu3mo)

Self-Refine: Iterative Refinement with Self- Feedback

  • There was a demo of the paper, but it’s just a demo and quite basic.