Speculative Decoding with vLLM
Improving LLV inferences with speculative decoding
Hey, I'm Hannes.
Product-focused machine learning engineer. Talks, and writes about Machine Learning, MLOps, and Natural Language Processing. I share thoughts, wins and failures.
How to Profile TensorFlow Serving Inference Requests with TFProfiler
Determining bottlenecks in your deep learning model can be crucial in reducing your model latency
Receiving Google Open Source Peer Bonus Award 2022
Receiving Google Open Source Peer Bonus Award 2022
Notes on deploying models with TFServing
A collection of useful links with information about the inner working of TFServing
Notes on Reinforcement Learning for Human Feedback
Reinforcement Learning for Human Feedback (RLHF) is the concept with powers recent models like ChatGPT
Notes on Model Performance Profiling
A collection of useful links with information about model performance profiling
Notes on GPT4
A collection of useful links with information about the inner working of TFServing
Highly interesting Twitter threads to revisit from time to time
I find myself revisiting highly interestign Twitter threads. Here is a list of the most interesting threads sorted by topics ...