Chat Completions
16 min
A chat completions API call is stateless — the server has no memory
of anything sent in a previous request. Every call must include the
full conversation so far as a list of role-tagged messages: a system
message (instructions/persona), then alternating user/assistant
turns, ending with the newest user message the model should respond
to.
def build_messages(system_prompt, history, new_user_message):
messages = [{"role": "system", "content": system_prompt}]
for role, content in history:
messages.append({"role": role, "content": content})
messages.append({"role": "user", "content": new_user_message})
return messages
Every single call resends the WHOLE history -- which is exactly why long conversations cost more tokens per turn as they grow, and exactly why LLM Fundamentals' context-windows module (truncating/chunking old history once it gets too long) matters for any chat application that runs for a while.
Write `build_messages(system_prompt, history, new_user_message)`: build the messages list a chat completions API expects — a `system` message first, then each `(role, content)` pair from `history` in order, then a final `user` message with `new_user_message`. Each message is a dict with `'role'` and `'content'` keys.
Why does a chat completions API require the ENTIRE conversation history on every single request, rather than the server remembering previous turns?
You can construct a properly-ordered messages list for a stateless chat completions API, and understand why the full history must be resent on every call.