Quiz: What is an LLM — For Experienced Developers

Why does dividing by sqrt(d_k) matter in the attention formula Attention(Q, K, V) = softmax(QK^T / sqrt(d_k)) * V?

A chatbot maintains a 40-message conversation. The user sends message 41. How does the LLM 'remember' the earlier messages?

This function estimates whether a prompt fits in the context window. There's a bug that will cause it to under-count tokens and potentially blow the context limit. Spot it:

```python
def fits_in_context(prompt: str, max_tokens: int = 128000) -> bool:
    # Rough estimate: 1 token ≈ 4 characters
    estimated_tokens = len(prompt) / 4
    return estimated_tokens < max_tokens
```

You set temperature=0.7 and top_p=0.9 for an API call. The model's forward pass produces logits where one token has 98% probability. How many tokens are in the candidate set for sampling?