T5

From AI Dungeon Wiki
Jump to navigation Jump to search

T5 is short for Text-to-Text Transfer Transformer and is a Transformer developed in 2019 by Google researchers to explore the limits of transfer learning. Transfer Learning is a term used by AI researchers in regards to applying the knowledge a model gains from one task to many other tasks. It is similar to GPT, and is used by AI Dungeon for its quest system. Its largest size is 11 billion parameters. T5 was trained on a curated Common Crawl dataset, but it is usually finetuned for most use cases.