How this workload behaves
Document summarization workloads are typically input-heavy, with large documents being processed to extract concise summaries. The cost driver here is the input token volume, as the model must read the entire document before generating a shorter output. Output tokens are relatively low since summaries are compact, but the input cost dominates due to the sheer volume of tokens processed. Quality of summarization is critical, as poor summaries require manual review or regeneration, indirectly increasing costs.
Compare against 6 alternatives
| Model | Vendor | Monthly cost | vs primary | Δ |
|---|---|---|---|---|
| GPT-4o | OpenAI | $45.00 | 5.63× | +$37.00 |
| GPT-4o mini | OpenAI | $2.70 | 0.34× | $5.30 |
| GPT-4 Turbo | OpenAI | $160.00 | 20.00× | +$152.00 |
| Mistral Small | Mistral | $3.20 | 0.40× | $4.80 |
| Claude Haiku 4.5 | Anthropic | $20.00 | 2.50× | +$12.00 |
| Gemini 1.5 Flash | $1.35 | 0.17× | $6.65 |
Recommendation
For document summarization, **Gemini 1.5 Flash** is the optimal choice, offering the lowest total cost at $1.35/month (83% cheaper than the primary) while maintaining strong summarization capabilities. Its 1.35M output tokens are sufficient for typical summaries, and its efficiency in handling long contexts makes it ideal for this workload. The $6.65 monthly savings compared to GPT-3.5 Turbo justify the switch, especially at scale.
Cost-saving tactics for this workload
- Use **prompt caching** for repeated summarization of similar documents to avoid reprocessing the same input tokens.
- Batch smaller documents together to reduce per-request overhead and leverage batch API pricing if available.
- Implement **context compression** by truncating or summarizing less critical sections of documents before sending them to the model.
- For multi-stage workflows, use a **smaller model (e.g., Mistral Small)** for initial summarization and reserve larger models only for edge cases or complex documents.
- Monitor output token usage closely—if summaries are consistently longer than expected, refine prompts to encourage brevity.