Updated: 9 May 2026
AI models for Google review replies: methodology summary
13 AI models tested against 53 real Swiss Google reviews in 6 languages. This one-page summary covers the setup, the results, and the criteria.
Key findings
- 9 of 13 models cleared the rule-based threshold (0 forbidden phrases, 0 em-dashes across 53 reviews).
- mistral-large disqualified: em-dashes in 42% (22/53) of replies.
- gpt-5 (Chat Completions) added operational details not present in the review, a real liability for a public reply tool.
- Latency of compliant models: gpt-4.1-mini (1.2s P50) to gemini-2.5-pro (13.3s P50).
- We weight safety and grounding above raw speed. The production choice is based on the complete safety, quality, and latency profile.
The 13 models tested
| Model | Latency P50 | Latency P95 | Forbidden | Em-dash | Status |
|---|---|---|---|---|---|
| gpt-4.1-mini | 1.2s | 1.9s | 0 | 0 | clean |
| gpt-4.1 | 1.3s | 3.0s | 0 | 0 | clean |
| claude-haiku-4-5 | 1.6s | 2.0s | 0 | 1 | em-dash in DE |
| mistral-large | 1.9s | 6.0s | 0 | 22 | em-dash spam (42%) |
| gpt-5-codex | 2.3s | 4.0s | 0 | 0 | clean |
| gpt-5.2-codex | 2.3s | 5.1s | 0 | 0 | clean |
| claude-sonnet-4-6 | 3.1s | 4.2s | 0 | 0 | clean |
| claude-opus-4-7 | 3.3s | 5.4s | 0 | 0 | clean |
| gemini-2.5-flash | 4.7s | 7.0s | 1 | 0 | forbidden phrase in DE |
| o4-mini | 5.3s | 12.3s | 0 | 0 | clean |
| gemini-3-pro-preview | 10.1s | 14.7s | 0 | 0 | clean |
| gpt-5 | 11.6s | 22.7s | 0 | 0 | fabricated details |
| gemini-2.5-pro | 13.3s | 19.9s | 0 | 0 | clean |
Selection criteria (in this order)
- Safety (no fabrication, no admission of liability)
- Grounding in the review
- Swiss multilingual depth (DE-CH / FR-CH / IT-CH)
- Tone fit by industry
- Latency
- Cost
We do not publicly disclose which model we run in production. Our methodology, criteria, and data are public.
Cite this study
StarReview (2026). Methodology: 13 AI models tested for Google review replies. https://www.starreview.ch/en/methodology/study-summary
Free to use with attribution (CC BY 4.0).