• By having a Large Language Model (LLM) read multiple existing papers, it is possible to generate fictional follow-up research. I thought that by repeating this process and expanding the directed acyclic graph of papers, researchers could simulate the process of reading papers and generating new research.

  • I constructed this simulation using an algorithm similar to MCTS / UCB Tree Search. The output is evaluated by another LLM acting as a reviewer, using the evaluation scores to expand descendants in a positive direction while pruning bad branches.

  • The top node (yellow) represents an existing paper, while the other nodes are generated by the LLM.

    • imageimageimage
  • The results are somewhat cherry-picked, so there is not complete confidence in the conclusions. However, the increase in evaluations for each node indicates that the utilization of good nodes is working well.

    • image
  • It would be interesting to see how changing the definition of “good papers” given to the LLM affects the growth patterns.

  • By altering the parameter c of UCB, changes in growth can also be observed.

    • When placing emphasis on exploration (c=0.3):
      • image
    • When placing emphasis on exploitation (c=0.1):
      • image

Some potential directions:

  • It could be interesting to incorporate human feedback (e.g., gaze attention) to guide the graph’s growth in relevant directions.
  • The same methodology could be applied to other brainstorming tasks, such as generating a game idea or a story plot.
  • Creating multiple agents with different “interests” and allowing them to collaboratively expand the graph could be an intriguing approach.

Repository: https://github.com/blu3mo/simulacra-of-academia