Beta Peter Bajnoci s.p.
Slovenian language & voice evaluation · est. 2008
Full Evaluation Report
Customer-Service Chatbot — Slovenian Language Evaluation
Overall assessment
Good foundation overall, with two recurring, systemic issues worth fixing before wider rollout — both are the kind of thing a native-speaker review catches that automated grammar checking generally misses.
Detailed findings
Five representative examples from the full set of 100 reviewed responses.
User: “Kje je moje naročilo?” (Where is my order?) AI: “Vaše naročilo je bilo poslan včeraj.”
Issue: "naročilo" (order) is a neuter noun, so the past participle must agree as "poslano," not the masculine "poslan." A native reader notices this instantly — it reads as clearly non-native, even though the meaning is still understood.
Suggested: “Vaše naročilo je bilo poslano včeraj.”
User: “Ali lahko dobim vračilo denarja?” (Can I get a refund?) AI: “Žal mi je za nevšečnosti. Prosimo, kontaktirajte nas za nadaljnje korake.”
Issue: Reads as translated, not natively written. "Žal mi je" is a personal, singular apology ("I'm sorry") which clashes with the corporate "we" voice the rest of the response implies with "Prosimo" and "nas." A Slovenian brand would apologize as a company, not as an individual.
Suggested: “Iskreno se opravičujemo za nevšečnosti. Za nadaljnje korake se obrnite na našo podporo.”
User: “Se lahko pogovarjam s človekom?” (Can I speak to a human?) AI: “Seveda, samo počakaj trenutek, povežem te z našim skrbnikom.”
Issue: Switches to informal "ti" address ("počakaj," "te") mid-conversation, while the brand's other responses use formal "vi." Slovenian customer service defaults to formal address unless a brand has deliberately chosen otherwise — mixing the two reads as unprofessional and is a common tell of AI-generated Slovenian built from inconsistent training data.
Suggested: “Seveda, počakajte trenutek, povezal vas bom z našim skrbnikom.”
User: “Ali imate ta izdelek na zalogi?” (Do you have this product in stock?) AI: “Trenutno nimamo ta izdelek na zalogi.”
Issue: After the negated verb "nimamo" (we don't have), Slovenian requires the genitive case — "tega izdelka," not the nominative "ta izdelek." This is a recurring pattern (see below), typical of models that handle English word order well but under-apply Slovenian case marking.
Suggested: “Trenutno nimamo tega izdelka na zalogi.”
Context: closing line at the end of a support conversation AI: “Hvala, ker ste izbrali nas! Se vidimo naslednjič.”
Issue: "Se vidimo naslednjič" ("see you next time") reads as casual, peer-to-peer phrasing lifted from an English template — not how Slovenian customer support typically signs off. Minor on its own, but noticeable across repeated interactions.
Suggested: “Hvala za zaupanje. Vabljeni, da nas znova obiščete!”
Recurring patterns
- Formality inconsistency. Formal "vi" and informal "ti" address appear within the same conversation flow in multiple responses — this looks systemic, not accidental, and is worth fixing at the template or prompt level rather than response-by-response.
- Case declension on repeated nouns. Product-related terms frequently appear in the wrong case after negation or prepositions — consistent with a model that has less exposure to Slovenian morphology than to English.
- Translated-sounding phrasing. Several responses follow English sentence structure and idiom rather than natural Slovenian phrasing, suggesting the underlying templates may have originated in English and been machine-translated rather than natively written.
Recommendations
- Standardize on one form of address (formal "vi" is the safer default for Slovenian customer service) across every response template, and audit for mixed usage.
- Build a short reference list of frequently used product and service terms with their correct case forms, to catch the most common declension errors before launch.
- Have a native speaker write (or rewrite) the core response templates — refunds, delivery, escalation, sign-off — natively in Slovenian, rather than translating from English source templates.
- Re-test after changes with a smaller follow-up sample to confirm the fixes hold across new responses, not just the ones flagged here.
This evaluation reflects manual, native-speaker review of the AI's Slovenian output — not automated grammar-checking or machine scoring. It looks at accuracy, natural phrasing, register consistency and cultural fit the way an actual Slovenian user would experience them, not just whether the grammar checker flags anything.