How this workload behaves
Document summarization workloads are input-heavy, as they require processing large volumes of text (e.g., articles, reports, or logs) to generate concise summaries. The cost is primarily driven by input token consumption, as each document must be fully ingested before summarization can occur. Output costs are secondary but still significant, especially for longer or more detailed summaries. The 128K context window of Mistral Large is well-suited for this task, as it allows for processing entire documents in a single pass without truncation. However, the high input token volume (10M/month) makes cost optimization critical.
Compare against 5 alternatives
| Model | Vendor | Monthly cost | vs primary | Δ |
|---|---|---|---|---|
| Mistral Small | Mistral | $3.20 | 0.10× | $28.80 |
| Codestral | Mistral | $3.20 | 0.10× | $28.80 |
| GPT-4o mini | OpenAI | $2.70 | 0.08× | $29.30 |
| Claude Haiku 4.5 | Anthropic | $20.00 | 0.63× | $12.00 |
| Gemini 1.5 Flash | $1.35 | 0.04× | $30.65 |
Recommendation
For document summarization, **Gemini 1.5 Flash** is the clear winner, offering a 95.8% cost reduction ($30.65/month saved) compared to Mistral Large while maintaining strong summarization capabilities. Its ultra-low input cost ($0.90/M) and competitive output cost ($0.45/M) make it ideal for high-volume summarization tasks. Mistral Small and Codestral are also viable budget-friendly alternatives (90% savings), but Gemini 1.5 Flash’s superior performance-to-cost ratio and Google’s robust ecosystem integration edge them out.
Cost-saving tactics for this workload
- Use **prompt caching** (if supported) to cache document summaries or common summarization templates, reducing repeated input costs for identical or similar documents.
- Batch summarization requests where possible—process multiple documents in a single API call to amortize context loading and reduce per-document overhead.
- Implement **context compression** (e.g., extractive summarization or chunking) to reduce input token count before feeding documents to the LLM, especially for long or redundant texts.
- For non-critical summaries, consider **Mistral Small or Codestral** (10% of primary cost) if quality trade-offs are acceptable, or use them as a routing layer to filter documents before passing to a higher-cost model.
- Leverage **Gemini 1.5 Flash’s 1M context window** to process entire documents in one pass, avoiding multi-stage summarization that could inflate costs.