Hey: Why doesn't AI speak my language?

Why doesn’t AI speak in my language (dialect)? is an experimental project exploring the linguistic diversity in AI-driven conversations. Despite the growing presence of AI in daily life, dialects and minority languages remain underrepresented in speech recognition and text-to-speech technologies. This study investigates how well AI can handle dialect-based conversations and whether dialectal AI fosters a stronger sense of cultural identity and emotional connection for users.

The project utilized a prototype called ‘Hey’, an AI-powered system designed to engage users in dialect-based conversations. The experiment involved four participants from Korea and Italy, each speaking distinct regional dialects. Through interactive dialogue and a workshop discussion, the study examined how AI-generated dialect responses affected user engagement, trust, and intimacy in human-AI communication.

Personal Project

My role: C#, Unity Developer

Tools

Unity – Designed the prototype interface for the AI conversation system.
OpenAI Whisper – Used for Speech-to-Text (STT) processing to convert dialect speech into text.
ChatGPT (via OpenAI API) – Handled text-based responses with real-time prompt engineering.
ElevenLabs – Implemented Text-to-Speech (TTS) for AI-generated spoken dialect responses.
Custom Prompt Engineering – Tailored AI responses to match dialectal tones and expressions.

Experiment Design & Results

1. Experiment Process

Step 1: Individual Conversation – Each participant engaged in a casual conversation with the AI in their dialect, selecting from five topics: hobbies, dreams, travel, food, and childhood.
Step 2: Workshop Discussion – Participants gathered to share their experiences and evaluate the AI’s dialect comprehension, response accuracy, and emotional impact.

2. Key Findings

Dialect-Based AI Created Shared Context – Participants felt a deeper connection when AI responded in their regional dialects. One Korean participant remarked, “Did you go there too?” when AI recalled a childhood-related place in dialect.
TTS Sound Quality Varied – Italian participants found AI-generated speech more natural than Korean participants, who felt it sounded artificial and exaggerated regional stereotypes.
Unnatural Sentence Structure – AI responses often adjusted only the sentence endings, which felt unnatural to native dialect speakers.
Increased Familiarity with Dialectal AI – Despite its imperfections, all participants felt dialectal AI was more engaging and personal than standardized AI, making it preferable for casual conversations over formal queries.

Page updated

Google Sites

Report abuse