Voice interfaces are becoming increasingly popular, offering a natural and convenient way for users to interact with applications. Combining the power of AI language models like GPT with voice input and output can create truly engaging and accessible experiences. This tutorial will guide you through integrating **AI Voice Search** into your React application, allowing users to speak their queries and hear AI-generated responses.

We'll leverage the browser's native Web Speech API for speech-to-text (STT) and text-to-speech (TTS), and connect to the OpenAI API via a Node.js backend for intelligent conversational capabilities.

Core Components of AI Voice Search

• Speech-to-Text (STT): Converts spoken language into written text. We'll use the browser's `SpeechRecognition` API.
• AI Language Model: Processes the text query, understands its intent, and generates a relevant text response. We'll use OpenAI's GPT models via a secure Node.js backend proxy.
• Text-to-Speech (TTS): Converts written text into spoken language. We'll use the browser's `SpeechSynthesis` API.
• React Frontend: Provides the user interface, handles voice input/output, and communicates with the backend.

Prerequisites

• A working React Application: You can create one with `npx create-react-app my-voice-app`.
• An OpenAI Account and API Key: (Refer to previous blog posts for setup).
• A Node.js Backend Proxy: (As set up in the "Build a Personal AI Assistant" blog post) to securely handle OpenAI API calls. Ensure it's running and accessible from your React app.
• Modern web browser (Chrome, Firefox, Edge) that supports Web Speech API.

Step-by-Step Integration Guide

Step 1: Set Up Your React Project

If you don't have a React project, create one:

npx create-react-app my-voice-assistant
cd my-voice-assistant
npm start

This will start your React development server, usually at `http://localhost:3000`.

Step 2: Ensure Your Node.js Backend Proxy is Running

Make sure the Node.js backend proxy (from the "Build a Personal AI Assistant" tutorial) is running. This server will handle the secure communication with the OpenAI API. Its URL (e.g., `http://localhost:3001/chat`) will be used in your React app.

Step 3: Create the Voice Assistant Component in React

We'll create a new React component, `VoiceAssistant.js`, to encapsulate the voice interaction logic.

src/VoiceAssistant.js:

// src/VoiceAssistant.js
import React, { useState, useEffect, useRef } from 'react';

// IMPORTANT: Replace with your backend proxy URL
const BACKEND_URL = 'http://localhost:3001/chat';

// Check for Web Speech API compatibility
const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
const SpeechSynthesis = window.speechSynthesis;

function VoiceAssistant() {
    const [listening, setListening] = useState(false);
    const [spokenText, setSpokenText] = useState('');
    const [aiResponse, setAiResponse] = useState('Hello! How can I assist you today?');
    const [loading, setLoading] = useState(false);
    const recognitionRef = useRef(null);
    const sessionIdRef = useRef(localStorage.getItem('voiceAssistantSessionId') || crypto.randomUUID());

    useEffect(() => {
        localStorage.setItem('voiceAssistantSessionId', sessionIdRef.current);

        // Initialize SpeechRecognition
        if (SpeechRecognition) {
            recognitionRef.current = new SpeechRecognition();
            recognitionRef.current.continuous = false; // Listen for a single utterance
            recognitionRef.current.interimResults = false; // Only return final results
            recognitionRef.current.lang = 'en-US'; // Set language

            recognitionRef.current.onstart = () => {
                setListening(true);
                setSpokenText('Listening...');
                setAiResponse(''); // Clear previous AI response
            };

            recognitionRef.current.onresult = (event) => {
                const transcript = event.results[0][0].transcript;
                setSpokenText(transcript);
                setListening(false);
                sendToAI(transcript); // Send spoken text to AI
            };

            recognitionRef.current.onerror = (event) => {
                console.error('Speech recognition error:', event.error);
                setSpokenText(`Error: ${event.error}`);
                setListening(false);
                setLoading(false);
                speakText('Sorry, I didn\'t catch that. Please try again.');
            };

            recognitionRef.current.onend = () => {
                setListening(false);
                // If no speech was detected, or an error occurred before sending to AI
                if (spokenText === 'Listening...' && !loading) {
                    setSpokenText(''); // Clear listening status if nothing was said
                }
            };
        } else {
            setSpokenText('Speech Recognition not supported in this browser.');
            console.warn('Web Speech API (SpeechRecognition) not supported.');
        }

        // Initial welcome message spoken by AI
        speakText(aiResponse);

        return () => {
            if (recognitionRef.current) {
                recognitionRef.current.stop();
            }
            SpeechSynthesis.cancel(); // Stop any ongoing speech
        };
    }, []); // Run once on component mount

    const startListening = () => {
        if (recognitionRef.current && !listening) {
            setSpokenText(''); // Clear previous spoken text
            setAiResponse(''); // Clear previous AI response
            recognitionRef.current.start();
        }
    };

    const stopListening = () => {
        if (recognitionRef.current && listening) {
            recognitionRef.current.stop();
        }
    };

    const speakText = (text) => {
        if (SpeechSynthesis) {
            SpeechSynthesis.cancel(); // Stop any current speech
            const utterance = new SpeechSynthesisUtterance(text);
            utterance.lang = 'en-US'; // Set language for speech
            SpeechSynthesis.speak(utterance);
        } else {
            console.warn('Web Speech API (SpeechSynthesis) not supported.');
        }
    };

    const sendToAI = async (text) => {
        if (!text.trim()) {
            setAiResponse('Please say something.');
            speakText('Please say something.');
            return;
        }

        setLoading(true);
        setAiResponse('Thinking...');

        try {
            const response = await fetch(BACKEND_URL, {
                method: 'POST',
                headers: {
                    'Content-Type': 'application/json',
                },
                body: JSON.stringify({ message: text, sessionId: sessionIdRef.current }),
            });

            if (!response.ok) {
                const errorData = await response.json();
                throw new Error(errorData.error || `HTTP error! status: ${response.status}`);
            }

            const data = await response.json();
            const reply = data.reply;
            setAiResponse(reply);
            speakText(reply);

        } catch (error) {
            console.error('Error communicating with AI backend:', error);
            const errorMessage = 'Sorry, I am unable to connect to the AI right now. Please check the backend server.';
            setAiResponse(errorMessage);
            speakText(errorMessage);
        } finally {
            setLoading(false);
        }
    };
   return (
        <div style={{
            display: 'flex',
            flexDirection: 'column',
            alignItems: 'center',
            justifyContent: 'center',
            padding: '25px',
            backgroundColor: '#ffffff',
            borderRadius: '12px',
            boxShadow: '0 6px 20px rgba(0, 0, 0, 0.1)',
            maxWidth: '500px',
            width: '100%',
            margin: 'auto',
            border: '1px solid #ddd',
        }}>
            <h2 style={{ color: '#007bff', marginBottom: '20px' }}>Voice Assistant</h2>
            <div style={{
                marginBottom: '20px',
                padding: '15px',
                backgroundColor: '#e0f7fa',
                borderRadius: '8px',
                width: '100%',
                minHeight: '80px',
                display: 'flex',
                alignItems: 'center',
                justifyContent: 'center',
                textAlign: 'center',
                border: '1px solid #b2ebf2',
                color: '#00796b',
                fontWeight: 'bold',
            }}>
                {loading ? 'AI is processing...' : (aiResponse || 'Ready for your command.')}
            </div>
            <div style={{ marginBottom: '20px', fontSize: '0.9em', color: '#555' }}>
                {listening ? 'Listening... Speak now.' : (spokenText || 'Press the button to speak.')}
            </div>
            <button
                onClick={listening ? stopListening : startListening}
                disabled={loading || !SpeechRecognition}
                style={{
                    backgroundColor: listening ? '#dc3545' : '#007bff',
                    color: '#fff',
                    border: 'none',
                    padding: '15px 30px',
                    borderRadius: '30px',
                    cursor: 'pointer',
                    fontSize: '1.1em',
                    fontWeight: 'bold',
                    transition: 'background-color 0.3s ease, transform 0.1s ease',
                    boxShadow: '0 4px 10px rgba(0, 123, 255, 0.3)',
                }}
            >
                {listening ? 'Stop Listening' : 'Start Voice Search'}
            </button>
            {!SpeechRecognition && (
                <p style={{ color: '#dc3545', marginTop: '15px', fontSize: '0.9em' }}>
                    Your browser does not support Speech Recognition. Please use Chrome, Firefox, or Edge.
                </p>
            )}
        </div>
    );
}


export default App;

Step 4: Run Your React App

Ensure your backend server is running (`node server.js` in your backend directory). Then, in your React project directory, run:

npm start

Open your browser to `http://localhost:3000`. Click the "Start Voice Search" button, grant microphone permissions, and speak your query. The AI should respond both visually and audibly!

Best Practices and Next Steps

• Error Handling & UI Feedback: Provide clear messages for microphone access issues, network errors, or API failures. Use visual cues (e.g., button state, loading spinners) to indicate when the assistant is listening, thinking, or speaking.
• Conversation History: For a more robust assistant, display the full conversation history in a scrollable chat window, similar to the previous AI Assistant tutorial.
• Advanced STT/TTS: For higher accuracy, more natural voices, or specific language support, consider cloud-based services like:
  • AWS Transcribe (STT): For converting audio to text.
  • AWS Polly (TTS): For highly natural-sounding voices.
  • OpenAI Whisper (STT) / OpenAI TTS: High-quality alternatives for both.
  These would require sending audio to your backend and then to the respective cloud service, returning text/audio.
• Wake Word Detection: Implement a "wake word" (e.g., "Hey Assistant") to activate listening without needing to click a button. This typically requires a client-side library.
• Deployment: Deploy your React frontend to a static hosting service (AWS S3, Amplify) and your Node.js backend to a server (EC2, Lambda/API Gateway, Elastic Beanstalk).
• Accessibility: Ensure your voice assistant is accessible to users with disabilities, providing alternative input methods (typing) and clear visual feedback.