Editor's Synopsis
Andrej Karpathy argues that the term "Large Language Models" (LLMs) is a misnomer, as these systems have little to do with language in a fundamental sense. He notes that the name is merely a historical artifact and that LLMs are better understood as a highly general-purpose technology for statistical modeling of token streams. Karpathy suggests that a more accurate name would be something like "Autoregressive Transformers," which better reflects the underlying mechanism rather than tying the technology to language alone.
This observation highlights an important conceptual point: by framing these models as "language" models, we may be underestimating or obscuring their broader capabilities across diverse domains beyond natural language processing.