327 Building with LLM APIs [1.5d]

Many applications now call an LLM for features that would otherwise be impossible or expensive: summarizing, extracting data from images, recommendations. In this card you build such features into MovieDB with the RubyLLM gem. Your mentors provide an API key.

Important

Work on this lesson in builder mode.

Learning goals

• You can explain how a chat-completion API works — messages with roles, a model, a response — and what tokens and context windows mean for cost and limits.
• You keep instructions and user input in separate message roles: the system prompt is yours, user content goes into a user message and is never interpolated into your instructions (as you learned in LLM and agent security).
• You can call an LLM from Ruby with a gem like RubyLLM, including image input and structured output.
• You treat model output as untrusted data and validate it against your own records before acting on it.
• You can test LLM-calling code without hitting the API in your test suite, using the techniques from the previous card.
• You treat the API key as a secret: provided by your mentors, stored outside the repository.

Resources

Read what's new to you, skim what's familiar, skip what you already master. Stop when you can meet the learning goals.

Your agent can also generate an overview, a tutorial or an explanation for anything here, tailored to what you already know. Just ask.

• 📄 RubyLLM Show archive.org snapshot — the gem: chat, system prompts ("instructions"), image attachments, structured output with schemas
• 📄 Anthropic: System prompts Show archive.org snapshot — what belongs in the system prompt and what doesn't
• 📄 Card 324 LLM and agent security — the security background for everything you build here

Exercises

Recommendations

• In the Code reviews card you built a favorites feature. Use it: send the LLM two lists — the user's favorite movies and the other movies on record (title, year, and whatever else you track) — with a system prompt that defines the task and user-role content carrying the data.
• Ask for structured output: a list of recommended movie IDs, each with a one-sentence reason.
• Show the recommendations on the user's profile.
• Decide how to handle an answer that references a movie that doesn't exist — it will happen. Validate before you render.

Poster intake

• In Basic file uploads you let users upload movie posters. Build on it: when a poster is uploaded, send the image to the model and extract the movie's title and year as structured output.
• Prefill the movie form with the extracted values as a suggestion the user confirms — never save model output without a human or a validation in between.

Tame the suite

• Your specs must not talk to the LLM API. Use the techniques from Consuming external APIs with Ruby: record a VCR cassette or stub with WebMock, and make sure the rest of the suite needs no network at all.