Speculative Decoding with vLLM using Gemma
Improving LLM inferences with speculative decoding using Gemma
Hey, I'm Hannes.
Product-focused machine learning engineer. Talks, and writes about Machine Learning, MLOps, and Natural Language Processing. I share thoughts, wins and failures.
Deploying Google's Gemma on Vertex AI
A comprehensive guide to deploying Google's Gemma language model on Vertex AI using vLLM, covering model registration, endpoint creation, and production deployment best practices.
Speculative Decoding with vLLM
Improving LLV inferences with speculative decoding
How to Profile TensorFlow Serving Inference Requests with TFProfiler
Determining bottlenecks in your deep learning model can be crucial in reducing your model latency
Receiving Google Open Source Peer Bonus Award 2022
Receiving Google Open Source Peer Bonus Award 2022
Notes on deploying models with TFServing
A collection of useful links with information about the inner working of TFServing
Notes on Reinforcement Learning for Human Feedback
Reinforcement Learning for Human Feedback (RLHF) is the concept with powers recent models like ChatGPT
Notes on Model Performance Profiling
A collection of useful links with information about model performance profiling
Notes on GPT4
A collection of useful links with information about the inner working of TFServing