(Information Science Expert)

  • Using the decoder of the Transformer model.
  • Developed by Open AI, the Language Model (v3 of GPT) is extremely large.
  • It utilizes the Common Crawl corpus, books, and Wikipedia, among others.
    • With such a massive corpus, it can prevent the repetition of the same sentences, allowing for training without memorizing them.
  • The Language Model can be applied to various other tasks (how?).
    • Types of tasks include:
      • Zero-shot type (specifying tasks only in natural language)
      • Few-shot type (specifying tasks and providing a few examples)
    • News article generation
    • Sentence completion
    • Machine translation (!?) (it can achieve high-quality translation using only a small portion (7%) of non-English information in the corpus)
    • Weakness: Agreement recognition (determining if A can be inferred from B), as it lacks training in comparing two sentences.
  • #naturallanguageprocessing