1. What the problem to solve?
春节回去的时候正好碰上DeepSeek发布新的模型，一时间各路媒体讨论的沸沸扬扬，几乎上升到国运的高度。讨论的最重要的应 …
Prediction in decoder and KV-Cache
1. Prediciton in Decoder
在前面GPT summary里面对GPT的模型有一个综合的介绍，这里用一个fake example来解释一步步GPT是怎么做的，self attention是怎么计算的，KV cache是怎么回事。
GPT是decoder only的模型，根据前面的token来预测下一个token。比如有一个句子 "it is sunny today."，现在有初始输入 …
Image Generation 2: Latent Diffusion model / Stable Diffusion
In the previous blog we introduced diffusion model (DDPM) which is to learn the step (time \(t\)) and the noise function (NN model) by adding Gaussian noise to an image step by step and reversing the process by denosiing from Gaussian noise to an image. Diffusion model is the most …
Image Generation 1: Diffusion model
The previous notes introduced the text generation models (GPT family). This reading note is about image generator papers.
Similar to text generator which generate the next token, OpenAI has image-GPT which is a large transformer trained on next pixel prediction in which the pixels are concated into a vector to …
GPT-1, GPT-2, GPT-3, InstructGPT / ChatGPT and GPT-4 summary
1. GPT-1
Improving Language Understandingby Generative Pre-Training
What the problem GPT-1 solve?
Before GPT-1, NLP was usually a supervised model. For each task, there are some labeled data, and then develop a suoervised model based on these labeled data. There are several problems with this approach: First, labeled data is …
Recommendation System 05 - Bayesian Optimization
Recommendation System 04 - Gaussian process regression
职场话题——关于职场communications skill的一点感想 (转载)
职场话题——职场常见8大场景沟通(转载)
Recommendation System 03
multi-armed bandit and how to use it in CTR optimization