Langfuse Academy
Welcome to Langfuse Academy, your comprehensive resource for learning about LLM engineering, observability, and evaluation techniques with Langfuse.
Langfuse Academy will provide structured learning paths, tutorials, and best practices to help you build better LLM applications. The content is currently under development. Check back soon for more resources!
Reading List
We are often asked about the resources we can recommend to learn more. This is a non-exhaustive list of what we have found useful/interesting.
Prompting
- Learn Prompting, docs page
- The Prompt Report: A Systematic Survey of Prompting Techniques, paper on arxiv, summary in tweets
- How to prompt o1 (o1 isn’t a chat model – and that’s the point), blog post, by Ben Hylak
Agents
- Building Effective Agents, blog post, by Anthropic
- Hugging Face Agents Course, course, by Hugging Face
- How We Built Ellipsis (or: Lessons from 27 months building LLM coding agents), blog post, by Nick Bradford
LLMOps
- What We Learned from a Year of Building with LLMs, by Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu and Shreya Shankar, Part 1, Part 2, Part 3
- Traceability and Observability in Multi-Step LLM Systems, webinar by Marc Klingen
- Data Flywheels for LLM Applications, blog post, by Shreya Shankar
- Latency optimization, cookbook, by OpenAI
- The OSS LLMOps Stack, page by LiteLLM and Langfuse
Evaluation
- Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge), blog post, by Eugene Yan
- AI Agent Observability & Evaluation, course, by Hugging Face
- Your AI Product Needs Evals, blog post, by Hamel Husain
- Creating a LLM-as-a-Judge That Drives Business Results, blog post, by Hamel Husain
- Evaluating Voice AI Agents, blog post and video, by Marc Klingen and Brooke Hopkins
LLM 101
Beginner-level, non-technical resources
News
- AI News newsletter, daily roundup of top AI discussions from Discord, Reddit, and X/Twitter
- Last Week in AI podcast, weekly summary AI news and research
- Latent Space podcast, deep dives and interview episodes
- Stratechery newsletter/podcast, tech/business strategy deep dives and news, many episodes related to AI/Labs, e.g. DeepSeek FAQ, Deep Research and Knowledge Value
Libraries & Tools
While there are lots of libraries and tools out there, these are the ones we see most often among Langfuse users
- Vercel AI SDK, TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js, and more.
- LiteLLM, use 100+ LLMs as a drop in replacement for OpenAI GPT models, either library or proxy.
- Instructor, library for structured outputs with LLMs.
- Popular frameworks to get started quickly with RAG/Chains/Agents: LangChain, LlamaIndex, Haystack
- Search: Exa, Firecrawl
- Browsing: Browserbase, Browser-use
- Code sandboxes: E2B