Paper notes
- LoRA: Low-rank matrices for fast and efficient fine-tuning
LoRA: Low-Rank Adaptation of Large Language Models
- Chain-of-thought prompting: Adding reasoning steps to enhance LLM reasoning
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- InstructGPT: Aligning language models with human feedback
Training language models to follow instructions with human feedback
- GPT-3: Parameter scaling for better zero-shot, one-shot, and few-shot performance
Language Models are Few-Shot Learners
- GPT-2: From narrow experts to competent generalists
Language Models are Unsupervised Multitask Learners
- Sentence representation with BERT
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Transformer is all you need
Attention Is All You Need
- Mitigating performance degradation using Residual Networks
Deep Residual Learning for Image Recognition
- Adversarial Training for Generative Models
Generative Adversarial Nets
- Neural Machine Translation with Attention
Neural Machine Translation by Jointly Learning to Align and Translate
- Word representation using vectors: modeling syntactic and semantic word relationships
Efficient Estimation of Word Representations in Vector Space
- The challenge of internal covariates: how distribution shifts hurt training—and batch normalization
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Beyond model context, processing arbitrarily long prompts
Recursive Language Models
- The genesis of a Neural Probabilistic Language Model
A Neural Probabilistic Language Model