Wei Xu, a professor at Georgia Tech, shares the reading list from her Fall 2024 course CS 8803 "Large Language Model". The course covered three major topic areas: learning from human preferences (including techniques like PPO, DPO, SimPO, CPO, RRHF, ORPO, and CTO), real-world LLM applications (covering Llama-3, Aya, and Arena benchmarks), and efficient LLM methods (such as MoMa, LoRA, QLoRA, and LESS).

This post serves as a useful reference for anyone looking to understand the current landscape of LLM research and training methodologies, particularly around reinforcement learning from human feedback variants and parameter-efficient fine-tuning approaches that have become central to modern LLM development.

MIT Sloan professor Rama Ramakrishnan outlines three practical methods businesses can use to leverage large language models (LLMs) like GPT-4, Llama, and Claude. The simplest approach is prompting, where users directly ask an LLM to perform tasks requiring only common sense and everyday knowledge — such as classifying product reviews to identify potential defects. When tasks require more current or proprietary information, companies can employ retrieval augmented generation (RAG), which pairs a prompt with relevant company documents or data, making it particularly effective for applications like customer service chatbots trained on specific return policies.

For more specialized domains involving technical jargon or complex knowledge — such as medical diagnosis or legal document analysis — instruction fine-tuning is the recommended approach. This method further trains an LLM with application-specific question/answer pairs, modifying the model itself to improve accuracy in niche areas. While more labor-intensive, companies can offset this effort through synthetic data generation, using LLMs to create training data. Ramakrishnan emphasizes that organizations don't need to choose a single method; instead, they should deploy a mix based on specific use cases, starting with prompting as the lowest-effort option and scaling up to fine-tuning when greater accuracy and domain expertise are required.