How this workload behaves
Document summarization workloads are input-heavy due to the need to ingest large volumes of text (e.g., articles, reports, or legal documents) before generating concise summaries. The cost driver here is the input token volume, as models must process the entire document to extract key points. Output costs are secondary but still significant, especially if summaries are detailed or require structured formatting. The context window size (200K tokens) is critical for handling long documents without truncation, but larger windows increase input costs linearly. Accuracy and coherence in summaries often justify higher costs, but efficiency gains are possible with smarter preprocessing or model selection.
Compare against 5 alternatives
| Model | Vendor | Monthly cost | vs primary | Δ |
|---|---|---|---|---|
| Claude Opus 4.7 | Anthropic | $300.00 | 5.00× | +$240.00 |
| Claude Haiku 4.5 | Anthropic | $20.00 | 0.33× | $40.00 |
| Mistral Small | Mistral | $3.20 | 0.05× | $56.80 |
| GPT-4o mini | OpenAI | $2.70 | 0.04× | $57.30 |
| Gemini 1.5 Flash | $1.35 | 0.02× | $58.65 |
Recommendation
For document summarization, **Gemini 1.5 Flash** is the best choice due to its exceptional cost efficiency (97.75% cheaper than Sonnet 4.6) while maintaining strong summarization capabilities. The model’s 1M token context window and optimized architecture for long-context tasks make it ideal for processing large documents. Even if quality is slightly lower than Sonnet 4.6, the savings of $58.65 per month (or ~$700 annually) justify the trade-off for most use cases. If quality is paramount, **Claude Haiku 4.5** offers a balanced middle ground at 67% cheaper than Sonnet 4.6 with Anthropic’s reliable performance.
Cost-saving tactics for this workload
- Use **prompt caching** to store document embeddings or preprocessed text, reducing repeated input costs for similar documents.
- Implement **batch processing** to summarize multiple documents in a single API call, leveraging models like Mistral Small or GPT-4o mini for cost-effective throughput.
- Apply **context compression** (e.g., extractive summarization or chunking) to reduce input token volume before passing documents to the LLM.
- Route simple summaries to **Gemini 1.5 Flash** and reserve **Claude Sonnet 4.6** for complex, high-stakes summaries requiring nuanced reasoning.
- Cache **frequently requested summaries** in a low-cost database (e.g., Redis) to avoid reprocessing the same documents repeatedly.